There has never been a good way to compile Classic Mac OS apps on modern OS X – for the most part, you were stuck using ancient tools, either Apple’s MPW or CodeWarrior, running in a VM of some sort. CodeWarrior, of course, is not free, and MPW only runs on Classic Mac OS, which is unstable at the best of times and downright nightmarish when trying to use it for development in an emulator like SheepShaver.
Enter ‘mpw’ (which I will refer to in lowercase throughout as something distinct from Apple’s MPW toolset).
mpw is an m68k binary translator/emulator whose sole purpose is to try and emulate enough of Classic Mac OS to run MPW’s own tools directly on OS X. MPW is unique in that it provided a shell and set of commandline tools on Classic Mac OS (an OS which itself has no notion of shells or commandlines) – this makes it particularly suited to an emulation process like mpw attempts to provide, as emulating a commandline app is a lot easier than one built for UI.
At the time I came across the project, the author himself had never attempted using mpw to build a Classic MacOS app – only commandline tools and Apple II-related stuff. Naturally, building a UI app was the first thing I’d try.
The Experiment I started off by writing code just to see how well mpw emulated the MPW compilers, and over time managed to write a working shim of an app that could run on System 1.1g. This in itself was a learning process, not only in code but in piecing together the build process. All the sample code and documentation of the time was in Pascal, so I had to translate that to C – not so difficult, it turns out (technically, first I had to transcribe it from a PDF…).
Eventually I had something that worked, built a few sample projects, uploaded some to Github and left the classic Mac stuff for a while.
More recently, towards the end of 2014, mpw added support for the PowerPC tools, so I immediately set out to update my build processes to support that – a trivial effort.
However, now that it was possible, I really wanted to try Carbonization.
Why Carbon? I vaguely knew what Carbon was from having lived through the OS 9 -> OS X transition, and that knowledge came with a certain amount of bias. “Carbon is that thing that badly ported OS 9 apps used, right?” It always felt ‘off’ in OS X, in the same way that cross-platform UI toolkits invariably feel off.
I knew I wanted to understand the process better, however, and see what would be involved in porting my sample projects to Carbon (and thusly, OS X). I read some books, and set to work.
The actual porting process didn’t take much time at all, and for the most part I ended up with fewer lines of code than where I started. Most of the changes involved #ifdef-ing out lines of code that weren’t necessary anymore, and changing anything that directly accessed system structs to using accessor functions – a trivial amount of work (for an admittedly trivial set of projects).
What interested me the most is how so much of the API remained identical – I was still using only functions that existed on System 1.0 in my app, but they were working just the same as ever in a Carbonized version. The single built binary ran on OS 8.1 all the way to 10.6 (care of Rosetta).
My mind wandered to Carbon as it exists in 10.10. While Apple decided not to port it to 64-bit (for all the right reasons), the 32-bit version of Carbon is still here in the latest release of OS X – I wondered how much of it was intact.
Turns out the answer is: all of it.
The only change I had to make was to point my header includes at the right place, but after that the whole app came to life exactly as it did on Classic Mac OS.
With the same source file, and only a handful of #ifdefs, I could build the same app for 1984’s System 1.0 all the way up to the current release of OS X, Yosemite.
The Sample Project
Just to provide an example for this post, I put together a trivial drawing app called BitPaint. It isn’t very interesting, but it should illustrate a few things:
What’s involved in bringing a trivial classic Mac app to Carbon
How the Classic Mac OS build process works
How much source compatibility exists between 1984’s Toolbox and Carbon today
Carbon, redux
The more I dug into it, the more I came to the conclusion that Carbon was probably one of the most important things Apple did in building OS X. Even today it provides source compatibility for a huge chunk of the classic Mac OS software base. It kept the big companies from ditching Apple outright when they were needed the most, and gave them a huge runway – 16 years to port perhaps millions of lines of code to OS X while still being able to iterate and improve without spending thousands of man-years upfront starting from scratch. Over time, of course, Carbon has improved a lot and you can mix/match Carbon & Cocoa views/code to the point where you can’t realistically tell which is which. I appreciate what a monumental effort Carbon was, from a technical standpoint. That Cocoa apps always felt ‘better’ is more to Cocoa’s credit than Carbon being a bad thing – it’s a lot easier to see that in hindsight.
Final Thoughts
I am incredibly psyched about mpw. Its developer, ksherlock, has been very responsive to everything I’ve come up against as I stress test it against various tools and projects.
Right now it’s a fully usable tool that makes Classic Mac OS compilation possible and easy to do on modern versions of OS X, without requiring emulators or ancient IDEs or the like. To my knowledge, this is the first time this has been possible (excluding legacy versions of CodeWarrior).
I have used this toolset to build all kinds of things, including fun ports of my own apps. I’m sure I’ll be coming back to it for a long time to come.
I ran into a few things along the way that are worth noting, mostly because information about them either doesn’t exist or is difficult to find on the web – check BitPaint’s makefile for context on any of these:
Pascal Strings You want to tell clang to enable Pascal-style strings (-fpascal-strings).
-mmacosx-version-min=10.4 If you specify -mmacosx-version-min=10.4, your Intel binary will work all the way back to 10.4, otherwise it will crash on launch trying to use invalid instructions.
PICTv1 Systems 1-6 support only one picture format for resources, and that’s PICTv1. Helpfully, it seems like nothing on Earth supports the creation of PICTv1 files anymore, so I wrote a very suboptimal one (but it works well): https://github.com/steventroughtonsmith/image2pict1
OS X Packages When you package a Carbon binary into a .app folder structure, as necessary for OS X, you’ll find it won’t be able to find its resource fork anymore, despite the fact that running it from the commandline will work fine. Instead, you can put the resource fork into a data file inside the bundle’s Resources folder and it will work as expected.
‘SIZE’ If you accidentally your SIZE resource, your app will launch on OS X but appear to hang, unresponsive, in the background. I ran into this more than once.
‘carb’ From what I can tell, including a ‘carb’ resource in your binary will stop it from launching on System-7.x, but be fine on System 1-6 and 8-9.2.2. Not sure if this is an MPW problem or a me problem, but I lost quite a bit of time to “This version of MPW is not compatible with your system” alerts from my apps before realizing this.
Packaging! Those who knew Classic Mac OS will be well accustomed to type/creator codes and resource forks; those who did not will be absolutely baffled by trying to figure out why they can’t open their files/disk images/binaries. I run SetFile on my disk images after creation so that DiskCopy will be able to see/open them, and I binhex encode the disk images so I can safely transfer them to a real Mac using Internet Explorer without losing the resource fork. Neither Samba (as used in VMWare’s Shared Folders) or FAT32 support resource forks, so they will get stripped and render your files unusable. SheepShaver’s external folder support does indeed support resource forks, so you’re totally fine there.
Rez MPW includes a version of the Rez tool (which compiles your resource forks for you), but currently mpw is unable to emulate it successfully. Fortunately, Xcode still ships with Rez and today’s Rez seems almost unchanged from the version included with MPW all those years ago. Pass it the Classic Mac OS set of includes and it’s happy to spit out resource forks compatible with System 1.0.
Starting off, we have the typical issue that tips me off to a questionable profile. This character stands at 6’8. Bad roleplayers have a tendency to make their character unnaturally tall. This will be a common complaint, have no doubt. I don’t know if people simply believe having someone super tall makes them more badass or if it is an attempt to say they are stronger because of it.
So lets take a look at some of the death knight concept art:
And some TCG art:
Hey these guys look pretty big!
“More iron bulwark than man.” Plate wearers seem to be more prone to absurd shit like this than other armor wearers. They talk about their armor/metals in it to sell the fact that you can’t hurt them. The talk about frost is likely another allusion to defending against heat based attacks like fire.
Actually it’s probably a way to mention that the person is a Frost DK without saying it outright. “Show, don’t tell” and all of that nonsense about good writing.
The second paragraph is more stuff that simply does not need to be in a description. Seeing him would not tell you this. It is also telling you how to feel about his character. People like this want to tell you how to react, instead of letting you play your own character. Beyond that, he goes further into his absurd armor. The description about the armor being a weapon itself and the type of metal are his justification when the inevitable fight comes and he can simply ram you to hurt you. Even disarmed, he is supposedly fully capable. The weapon is another absurd item. A greatsword nearly seven feet tall? But people like this will never admit that all that layers of heavy metal plate and absurdly large sword would begin to slow them down at all.
Yeah a greatsword that is nearly as tall as the death knight wielding it that–
would be–
oh never mind they’re undead anyway! It’s not like they couldn’t be super strong or something.
Well what about the ridiculous spiked armor?
Oh.
This profile has less detail than many, but hits most of the general features of a bad roleplayer. Over-focus on armor, giant weapons, described mastery of combat, and defense against most magics (less explicit, but due to his being a death knight).
30. “I hate seeing you unhappy so now I’m going to do everything I can to cheer you up. Even if it means making a fool of myself.”
——
Varaelian hated the silence. The pain might have finally lessened, but he found he preferred the long days that he’d drifted through half-lucid nightmares, anchored by agony. Certainly it was better than being awake and numb.
And alone. Not that Kirian and Elleynah had been able to help it. Kirian had broken his seemingly endless vigil once Varaelian had finally awoken for good, disappearing for brief business, and Elley had been needed to help run the affairs of the estate while its master was on the mend. They would return, of course they would. The two people he loved most in the world would never abandon him.
But waiting… was awful. Varaelian felt even more acutely that his life was changing in a direction that left him floundering helplessly. First Jeyse, his eye, and now this—too many short years tumbling one disaster over another, and Varaelian found himself struggling to keep up.
He still couldn’t look at the bandaged stump where his left arm used to be, even though it’d been almost a month. He couldn’t even touch it without his stomach turning to ice, his throat closing up with fresh disbelief. Kirian was doing all he could to ease Varaelian through the transition, but the blood elf ultimately had to walk this path alone.
A firm knock came from the door to his room, and Varaelian didn’t even have time to mutter an invitation before the door swung open. A behemoth of a man entered, heavy plate boots thumping noisily on the wooden floorboards, a long, tattered fur cloak billowing behind him. His skin was sallow, lips pressed into a permanent line, and eyes that burned with the frigid judgment of arctic ice held an unblinking gaze beneath locks of inky black.
Truly his spike-and-skull-covered body would send onlookers into fearful retreat, but Varaelian only sighed and rolled his eye.
“Tev, what are you doing here?”
The death knight answered him with an abrupt twist of squealing metal and snapping bones, and a moment later, Tevruden’s left arm landed on the medic’s bed.
Varaelian recoiled away from the macabre offering, his good eye wide. “Tev, what the fuck?!”
“I am lending it to you,” the undead elf responded, deep voice rumbling. “Just stitch it back on my shoulder when I return in a few hours.”
“I can’t use this!”
Tevruden shrugged, and his massive shoulderplates clinked almost musically over the chainmail curving around his biceps. “That isn’t my problem.”
The medic grimaced, ready to complain more, but the longer he stared at his friend, the more he found him to look absolutely… ridiculous. Standing there with one arm like it was the most normal, boring thing in the world.
Varaelian hesitantly touched the severed limb. “If I didn’t know any better, I’d think you were trying to cheer me up,” he said slowly.
Tevruden’s expression remained impassive. “You are a medic, Nilhandril, and while thus bedridden, you are useless to all of us.”
He left then, and though his words had been cold and logical, Tevruden hadn’t denied him.
And Varaelian found that his mood was no longer quite so defeatist and dour.
Just a little under 31 years ago, I played a key role in a conspiracy theory that grew up around a passenger plane downed by a Russian missile. Trust me, I did not mean to be involved.
On September 1, 1983, Korean Airlines flight 007, a Boeing 747 with 269 passengers, was shot down over the Sea of Japan. At about 6am that morning, I arrived at my summer job at the American Embassy in Tokyo where my task was usually to start up the computer which had been turned off over night. But on this morning, I realized the system was already engaged and that a surprisingly large number of workstations had been left on over night. While rare, I had seen this pattern before when a Washington deadline for information was looming.
Not long after I arrived in my office, I received a call from a secretary in the Agriculture Department who liked to play a computer game before her workday started. Her favorite game had a bug that regularly froze her workstation. This was the “bad old days” of computers and the only way to reset her station was from my central console.
On this day, I highlighted her workstation and hit the F6 key to reset. But my screen went temporarily black and then seemed to be starting again. I realized that I had mistakenly hit F7 and reset all the workstations in the embassy. This realization didn’t bother me much, because no one except the Agriculture section secretary was usually on the computer system this early in the morning.
But then all hell broke lose.
My boss, a Japanese computer engineer named Itoh, poked his head in the door. This was a shock because I had never seen Mr. Itoh before 10am ever. My job was to come in early and leave early and he arrived late and stayed late to shut down the system each night. He asked me what had happened. I told him I had shut down the system by mistake. He shook his head and ran down the hall.
Next, the head administrator, who I had only seen once in the computer room, walked in. He asked where Mr. Itoh was. I pointed down the hall. And he ran that direction as well.
More than an hour later, the Administrative Director returned to my office to explain what had happened. He told me about the Korean Airline disaster and that no one really knew what was going on, but that most of the information available was coming in from Japanese sources—first from Japanese fishing ships in the area and later from Japanese defense forces who were being dispatched to look for debris. A team of translators and US diplomats had been readying the first report for President Reagan at the time I turned off the computer systems. As this was a very early computer with limited backup capability, hours of work of dozens of experts had been lost when I inadvertently closed down the computer.
I, naturally, felt terrible and was, appropriately, fired.
It was only weeks later that I began to comprehend the effects of this single keystroke mistake. President Reagan was criticized in the press for his administration’s delayed announcement of the tragedy. But more troublesome, the reports that were being compiled in the US Embassy at the time of my error were meant to be shared with the South Korean government. As the team in Tokyo went back to rewriting the report—with clear evidence that the plane had been downed in the Sea of Japan—the South Korean government, working from flawed data, announced that the airliner had simply been forced to land in Russian territory and that all passengers and crew were safe.
That Korean announcement and the slow response by the US President—both caused by delayed real information—caused decades of conspiracy theories. Until the fall of the Soviet Union in the early 1990s, many Koreans clung to the hope that their loved ones were still alive and well in some Siberian prison camp.
So today, in the face of a Malaysian Airline crash in the Ukraine—and with all the associated speculation of 24-hour news organizations and the Tweetosphere, my advice is to take a deep breath, count to ten, and know that there is a very good chance that truth in the matter will be forthcoming very soon. And let’s hope that there is no stupid 23-year-old with his finger on an important keyboard in this information chain.
Ama has the facial marks/tattoos similar to what many scourge/cult members had, though death has faded them somewhat. (I think Thassarian has them too). She also has a scourge insignia rather blatantly on her right forearm and she keeps it covered. It glows- along with unholy runes on her body- when she casts specific spells. She’s tried to get rid of it but there hasn’t been much success.
Iris has multiple earrings and she swaps them around purely for vanity’s sake. She’s got a stupid amount of scars but no tattoos as of yet.
Noralla has runic glyphs and wards mainly concentrated on her arms and hands to assist in spellcasting. She also has earrings and wears a fair amount of jewelry.
Icthlarin has a ton of earrings in her ears and she’s carved a few of them herself; there’s a lot of shamanistic charms and stuff there, and she likes the aesthetic of them because they’re unusual. She’d probably have more than one nose piercing if she had a nose, but she’s a Broken so that’s sort of out. 🙁
Gideon is a forsaken and is missing a lot of skin, but he used to have a ton of tattoos while he was alive. A lot of them were placed there to cover the names of girlfriends he had (he didn’t learn the first or third time). He had an earring as well.
Due to his family being slightly sadistic, Varaelian has runic tattoos that stretch from his upper back to his hip then on the front from his shoulder his hip again. It was given to him after a sort of rite of passage when he turned a specific age and had to take over the family. He has several other smaller wards along his hands to help him heal, a gift from a friend… sort of.
No really crazy piercings to mention, just the two on his lobes and the cuff at the top.
Vaelios is decorated with a runic pattern that stretches from the back of his neck to the backs of his feet that is otherwise invisible until active. It’s a sort of ward that protects from the shadows but also calls them when in need.
He only has one piercing and it’s just a normal ear one.
Tevruden is covered in runic tattoos that both help reanimate him and focus his power when he uses it. They usually glow dimly but will brighten as he exerts more effort. His ears are pierced but he doesn’t care enough to replace the earrings (they were lost sometime between the time he was reanimated and when he was freed from the Scourge.)
Waking up to a string of missed calls is rarely a harbinger of good, and this time would prove no different. Upon returning the calls I was greeted with a simple “You’ve been hacked.” Great.
It turned out someone with my username had spent the morning causing a ruckus on 4chan. At first other users and moderators assumed I’d been drinking—at 7:00AM no less—but quickly concluded it was the work an intruder.
I immediately hopped out of bed and onto a call with our developer and a moderator to establish a timeline of events, and set about digging through error logs. It didn’t take long to piece together what had happened.
First, the intruder was able to enumerate files on a domain hosting moderator tools. He used an off-the-shelf script, and had the benefit of having leaked source code in his possession to build a site-specific wordlist.
Mistake #1: No rate limiting or HTTP auth dialog was present on the domain.
Most of the enumerated files did him no good since they made use of a PHP auth check, but one file was different: instead of displaying an error message, it simply bounced him to our front page. He manually set a 4chan admin username cookie (the cookie name gleaned from the source code), and voila, he was no longer redirected.
Mistake #2: The PHP auth check for this particular file was broken.
The page in question was a once-off used to generate statistics about reported posts. “0 days” was displayed on the page, and he guessed it might be a query parameter. He was correct, and as it turned out this one parameter was vulnerable to SQL injection. After some more testing, he realized he was able to return information from our database using the error messages displayed.
Mistake #3: Unescaped SQL query, and not disabling MySQL errors in production.
Using the injection vulnerability, he was able to exfiltrate information from our backend database. Thankfully we have a record of all information accessed (thanks to those handy error messages being logged), and it appears he mostly poked around before settling on accessing moderator credentials. He took my username and password, set them in his cookies, confirmed they worked by visiting an admin panel on the main site, and began lurking.
Mistake #4: Boneheaded cookie auth—we simply stored the bcrypted password from the database in a cookie, which was all that was required to pass PHP auth.
After lurking for a few days, the intruder decided to make himself known by publicly shaming a user under my username. He later stated this was his motivation for gaining entry all along—to out a user he disliked. Using admin panels, he was able to obtain and leak information about other users, as well as moderator names and IP addresses.
We quickly patched the vulnerability, forced a password reset for all moderator accounts, and have spent the past two weeks reviewing our code and servers to apply fixes and make improvements where possible.
The day didn’t end there though. This next bit concerns DrawQuest.
I loaded my e-mail for the first time that morning, and at the top of the list was a message from Amazon: “Unauthorized activity in your AWS account”
To make an already long story short, when leaving the company a few months ago, one of our developers asked to open source a project he had worked on. You can probably already guess where this is going, but suffice it to say he did so by flipping the existing repository to public.
Despite having overwritten our AWS keys, they were still available in the commit history and likely recovered by a bot. An intruder was able to use them to create an administrator account, giving them access to our AWS dashboard, and used it to spin up a hundred extra-large instances—probably for Bitcoin mining. They also would have had access to all of our S3 buckets, EC2 and RDS instances, DNS, etc.
Mistake #5: Not creating a fresh repo for a newly open sourced project, or at least scrubbing commit history.
Mistake #6: Using a highly-privileged key where a lesser-privileged one would have sufficed.
For DrawQuest, this was catastrophic. Despite our team going their separate ways a few months ago, we’d hoped to continue operating the app using the company’s remaining capital. However without any full-time employees left on staff, after the breach we felt the only responsible path forward was to shutter the service completely since we lack the time and resources to do a complete audit in order to ensure the integrity of our servers and Amazon account.
On one hand I’m frustrated we made such simple mistakes that resulted in very real consequences, but also grateful that it provided us an opportunity to learn from those mistakes, and share them with the world.
There’s no silver bullet when it comes to security, and the only way to stay ahead of it is constant vigilance. Don’t rely on any one method to protect your service, assume the methods you already have in place don’t work, adhere to best practices, and make it a point to revisit security on a regular basis—not just when something goes terribly wrong.
To that end and in keeping with our ongoing commitment to security, I’m pleased to announce the launch of 4chan’s Vulnerability Disclosure Program. It’s my hope that by embracing responsible disclosure and providing an officially sanctioned way for security researchers to submit such reports, we’ll be in a better position to avoid or at least mitigate future incidents.
In the end, I accept full responsibility for both breaches. I wasn’t ever involved on the technical side for DrawQuest and don’t actively write code for 4chan any longer, but know it was ultimately my responsibility as founder and CEO to ensure the security of both. I’m very sorry to both communities that I failed in that capacity.
If I’ve learned anything from this experience, it’s that if you don’t treat security as a top priority, it will bite you. It’s not a matter of if—only when.
Today I want to tell my story of how drawing helps me write better code.
No more than 1.5 years ago I didn’t know how to draw anything more complex than a human-like figure made from 5 lines and 1 circle. Nor did I believe that I ever could or will. I was wrong.
If you can draw this, you can draw anything.
One day Hacker News had a nice article about books that help you improve in unusual ways, self-help books without intentionally being about self-help. The article had a nice pick of titles, but the most promising one was ‘Drawing on the Right Side of the Brain’, because the idea that drawing is actually easy, pitched right through my mind.
And when I finally got it and started to read… magic happened. This book is one of the best ‘how-to’ books ever written, and it does its job in a really special way. It doesn’t show you drawing techniques and it doesn’t want you to draw simple shapes as you would think. It starts by showing that you can draw with simple exercises, and reinforcing your confidence about your drawing ability forever. And that was all I ever needed.
It really comes as a revelation.
I went through the book, finished almost every exercise, and stopped. I stopped because the book already fulfilled my desire – desire to know that I am not hopeless in a field of drawing. I didn’t know what to do with my newfound skill, so I switched on to what I’ve done in my spare time before – coding the game this blog is about. And I didn’t draw. Until about 3 months ago.
You see, when you’re working on a video game, you naturally play and analyze a lot of other video games, just to be a better video game designer. And when you’re playing other video games, especially indie ones, your mind sometimes comes with the notion: ‘Wow, this is really nice art, I wish I could draw like that’, and then instantly to: ‘But there is nothing that stops me, because I know I can draw after that awesome book’. And, after a couple of strokes like this, I just couldn’t keep myself from pencil and paper anymore.
I started sketching again. At first I did it after job hours, in my spare time, but then I noticed that, after acquiring some basic knack, I can draw simple sketches quickly, so I tried to have drawing breaks when I got stuck with a new coding problem in my head. And to my surprise, my productivity rose.
Every software engineer worth a dime knows that programming is more about thinking, than typing code (and if you do not agree, you should probably go do copywriting or something). When you work on a hard problem, you think, think, think, read an article on your current topic, think, maybe do some tinkering here and there, think again, get an ‘AHA’ moment, and then, only then do typing.
But there is a subtle problem in this approach, at least for me. I can procrastinate between parts. Because focused thinking is hard, and checking email and twitter feed is easy. It’s a known problem in the field, and I consider myself in a constant state of war with my slackier self, employing usefulweapons, which sadly do not address the core of the problem, but help to focus a lot nevertheless. And drawing is a latest weapon in my armory.
So now I do one or two daily drawing breaks, when I feel tired and in a need of some mental replenishment. I draw simple sketches, copying images I like, or just doodling around. I give myself 20 minutes max, and that is more than enough in most cases. And I feel better after that.
2 breaks x 20 minutes = this pic and a less tired mind
I do not know why it is working for me, but I think main two reasons are:
Drawing doesn’t break the workflow. Drawing is work too, just a different kind. Maybe even symmetrically different to logical work as programming. Reading twitter feed however can break your workflow faster than a sledgehammer breaks a bulls skull.
Drawing ‘uses’ different parts of the brain than programming, and the brain kind of sorts out your previous thoughts while you draw. This is absolutely unscientific observation, and you probably shouldn’t believe me. But I still think it does.
Recreation is not the only reason I draw, but it’s certainly a big one. And it helps other causes. Hope you enjoyed the read!
