How it all started

In this post I will tell how I became involved with the ScummVM project.
The story begins in the early 90s, when I played my first point and click adventure game. It might have been Gobliiins or Indiana Jones or the last Crusade. I am not sure; that was a long time ago and my memory is a bit fuzzy.

Je vous parle d'un temps

Que les moins de vingt ans ne peuvent pas connaître

Montmartre en ce temps-là accrochait ses lilas

Jusque sous nos fenêtres

La ville évoque en moi

des souvenirs de café, de musique...

et de mort

 George Stobbart*

* I am just kidding. The first four lines are lyrics from La Bohême, a song written by Jacques Plante and Charles Aznavour.

I then played several other Lucas Arts game (I love Day of the Tentacles!), and a bit later Les Chevaliers de Baphomet (I love it!) and other games followed. I had tasted the forbidden apple, and several years later, when the forbidden changed to an X and the G3 became an Intel, I had to find new ways to replay those games.

I starting using ScummVM around 2003 to play some old LucasArts games (did I tell you how much I love Day of the Tentacles?). And when support for new games was added, that sometimes meant I could play more of my old games (Gobliiins! Yeah! Thanks Doc! I love that er... no). But while I was already working as a software developer at the time (nothing to do with games), it is only several years later that I started to contribute to ScummVM.

That part of the story starts in October 2006 when I bought an Intel Mac to replace my G3 Mac. Suddenly games I could play by booting on MacOS 9 (or using the Classic mode on OS X) stopped working. One of those games was Les chevaliers de Baphomet (literally The Baphomet Knights). So I started to search on Google for a way to play it. That's when I discovered that the English name for the game was Broken Sword. And that I had seen that name before. That game was supported by ScummVM! Yeah! Except it only supported the Windows version.

I then decided to start looking at the engine source code for the broken sword engine in ScummVM, and with a bit of luck supporting the Mac version would turn out to be easy. And it was. I proposed a patch less than a month later. How did I proceed to write this patch? With a lot of educated guesses based on my rudimentary knowledge of the engine and on the data files.

I did not have the data files for the Windows version (otherwise I would not have bothered adding support for the Mac version! My primary goal was to play the game again, not to have a brain meltdown). So I could not compare the files between the Mac and Windows version. But I could guess what the file format was from the source code and look at the Mac version data files to see if they were similar. They seemed to be very similar.

But first lets look at the list of files. The Windows version has the following files:

• Cluster files (with the CLU extension)
• swordres.rif (a catalog file that provides the offset of the resources in the cluster files)
• Music (WAV files)
• Cutscenes (SMK files)

I had the same files on my Mac version, except that the extensions where sometimes different. Here is the list:

• Cluster files (with the CLm extension, except for the two speech files that have the CLU extension)
• swordres.rif
• Music (AIFF files)
• Cutscenes (SMK files)

Music is simple to work out. WAV and AIFF are two well known file formats. Cutscenes are equally simple. They are smacker files and likely to be identical between the two versions. For the other files I made the guess the ones with the same extension are identical, and the ones with a different extension (CLm instead of CLU) are big endian versions of the same files (the Windows file are in little endian). That proved to be almost correct. And it also turned out I was lucky. More on that later.

Small reminder on what is big endian and little endian: this is the convention used to interpret the bytes making up a data word (more at http://en.wikipedia.org/wiki/Endianness).  For example:
42 in hexa is 2A, or when using two bytes 002A
when using the big endian convention, this would be stored as 00 2A
but when using the little endian convention this would be stored as 2A 00

The Intel and AMD CPUs use the little endian convention to store numbers in memory. This is thus no surprise that the Windows version store its data using this same convention. That way the data bytes do not need to be reordered in memory after being read from disk.
The Motorola 680x0 and Motorola/IBM PPC CPUs use the big endian convention. Since the game was released for mac at a time when they were using big endian CPUs, assuming that the data file may have been modified to use the big endian convention was a natural guess to make.

So I started modifying the code. First I added detection for the mac version and modified the code to take into account the different extensions. I try running the game then, just to make sure it did not work. And sure enough it crashed right away. Then I starting looking at all the places where the engines read the data files to modify the code to read big endian data (i.e. assume a different byte order) in the case of the mac version. This was easy. Since ScummVM works on little and big endian computers, when playing the little endian Windows version on a big endian computer it already needed to reorder the bytes. So I looked at all the places where it did so and modified them to also reorder the bytes when playing the mac version on a little endian system. I did not modify the read of the swordres.rif and speech.clu files however. Since the extension had not been changed, I assumed they were kept in little endian in the mac version.

Then I played the game. And oh joy! It worked! After the intro with the explosion in the cafe I would see George standing up in the terrace and... then it crashed.

Time to debug. It turned out that even in the CLm file a few resources had been kept in little endian. This was the case of the mouse cursor, and as a result as soon as the cursor was supposed to appear on screen it crashed. So I started to play the game, debug the crashes, and revert progressively back some of my changes for resources still stored using little endian convention in the mac version.

As for the swordres.rif file and speech file. It turned out I was right. They were also still stored in little endian. So why did I write I was also lucky? You will have to read my next blog post to find out.

Quickly enough I had a fully working game. So I made a patch and proposed it to the ScummVM team. In that first patch, I ignored the music. It could be played by reencoding it to MP3 for example, so I did not bother adding support for the AIFF format (sev later wrote the initial code for that using the demo). Support for the mac version was added in the ScummVM 0.10.0 release. But that was not the end of the story...

Disclaimer: All this happened a long time ago and while I did have some notes on my computer, most of this post was written from what I remember. I am not responsible for approximations or errors resulting from the occasional loss of neurones that happened in the past 8 years.