Working with video in Full Throttle Remastered

Full Throttle logo" src="https://habrastorage.org/getpro/habr/post_images/40f/c00/6dd/40fc006dd06eebb1329bdf47f34f0f3b.png">

Full Throttle is a point-and-click adventure game released by LucasArts back in the twentieth century, April 30, 1995. In addition to the standard visual style of the gameplay is point-and-click of those times (the character sprites moving on a static background) the game was also attended by FMV-video cutscenes in non-interactive and interactive modes of gameplay. This technology has strengthened the FMV story aspect of the genre, because it allowed the creators to use computer-rendered sequence of frames that do not require large load on the processor. Besides a short FMV takes up less space and to fit on the destination media — CD-ROM with a capacity of around 650 megabytes.

In the classic version of Full Throttle was used in video technology developed by an employee of LucasArts Vince Lee called INSANE. INSANE stands for INteractive Streaming ANimation Engine ("engine interactive streaming animation"). It was first created and used in Star Wars: Rebel Assault. Technology was improved and used in later games:

the

Star Wars: Rebel Assault (November 1993) the

• Full Throttle (April 1995)

Star Wars: Rebel Assault II (November 1995)

The Dig (November 1995)

Outlaws (March 1997)

This system play video has allowed the creators to compress large chunks of data to a smaller size, which, together with other game data could fit on a CD-ROM. At that time the speed of the I / o drive and increased computing power was more than sufficient for streaming, CD-ROM, compressed FMV, unpacking them into video frames and display on the screen. The FMV of the Full Throttle encoded with 320x200 resolution and 10 frames per second. In addition, before sending to the audio device in real-time, processed and mixed by the built-in video audio tracks.

For encoding and compressing video into a file .SAN (Smush ANimation) has been used a tool called SMUSHFT.EXE (SMUSH for Full Throttle). Unlike conventional playback systems video with one video track and one audio track file .SAN can contain one or multiple video tracks and multiple audio tracks for playback to the user depending on the state of the game. The figure below shows the SMUSHFT.EXE source file of the project. You may notice that when you scroll through the timeline green stripes indicate video frames, and dark yellow stripes — audio. Also there are gray bars that represent text strings corresponding to the information of the subtitles.

The most important aspect of our approach in the remastering of the classic adventure LucasArts was to preserve the integrity of the original game along with improving the old content. We wanted the game felt and played exactly the same as in the 90s, but with new graphics, much better sound and support for new, more convenient input methods. To do this, we first decided to port the original game on modern hardware. Then when you run the game for the user we each frame was a queue of States of the game and received a substantial amount of data about game state, to provide the user with more modern content. That is, the game still works exactly the same way as I did back then in 1995.

This means that we have adapted all the technologies used by the game, including the INSANE engine. To complete remastering of all the video content, we had to start from the original video game from 1995, and use them as the blueprint for the updated content. The end result frame same as original but with higher resolution and much improved sound.
the

Extracting

The first task was getting the original video file (files .SAN) and removing all the audio and visual information in the format upon which it is possible to make a "scheme". We already had code that was used to decode these files, so we wrote a C# tool which was the original interface to the decoder via compiled native DLL. This tool, named SanExtract, parses the file .SAN and share audio and video content into a convenient directory structure with which to work.

---

If you have the original game and you can get access to the files on the CD, try to extract the video alone! Here is a link to the tool SanExtract: SanExtract.zip

MD5 [ac951d858cd8d252222d31fee61b1ce4]

SHA-1 [2915295a260306292f979788fe047a208e5f4e3b]

Just run SanExtract.exe at the command prompt, using as input the original files (they most likely are in [CD_DRIVE]:/RESOURCE/VIDEO/ in [CD_DRIVE]:/RESOURCE/DATA/).

---

The resulting data will be one or more streams of frames line video, and all audio used during playback. And I was not wrong: the engine was dynamic enough to provide enable or disable elements (audio and video) at run time. Details I'll tell you about it later. Extract was just the beginning of this process. We needed to know how to create the original footage, so we could recreate a similar workflow for the project of remastering. For example, certain frames or sequences contain a combination of multiple sources of graphics. Look at the animation from the original game:

Maybe this and say, but in the present frame and drawn by hand and rendered three-dimensional elements. Motorcycle Ben, the limo and the shadow of a motorcycle Ben, leaning on a limo, taken from the rendered 3D sequence. Gold plated broken ornament on the hood is a separate hand-drawn animated sequence. The end result is a composition of 3D-render-drawn animations and a static background image of the clouds.

Below shows all the parts used to create the finished video. The left sequence is rendered manually decoration of the hood. Right — the mask used for compositing 3D renders (of the motorcycle and the limousine) with clouds in the background. And in the center is shown a sequence of 3D render + background of clouds. Then hand painted and rendered 3D elements are put together to create the above snippet.

To maintain the level of aesthetics of the original we couldn't just take each frame of video (approximately 16 000 frames) and manually draw them. We needed to correctly repeat the solid surface of motorcycles and cars from the original, including the shadows cast in the scene. In the end we need to find the original 3D models and hand-drawn animation, and then use a remastering of these separate parts, to connect them with compositing.

So, I mentioned that we need to find the original 3D files to hand-drawn animation. At first we thought "Oh, maybe there is some metadata that is used to overlay the original component parts of the video that can be learned". But actually the files are renamed and information was lost. Metadata doesn't work, and even when looking in the archives we find many files with the same names but with different dates and different contents. We have no idea which of the files originally used to generate the finished frame, so we had to find a solution to this problem.
the

Archives

Our archive consisted of about 67 gigabytes of data, and useful data pertaining to the original FMV, were scattered throughout this volume. After studying the schema of the data in the backup, we can conclude that by the end of the project all data files from all computers were copied to the backup folder. The result is a mess, similar to the style of storage from Indiana Jones. We have a lot of discs containing copies of hard drives of all employees. Upon further investigation, turned out to be. that there is one disk which contains all the needed data. Helpful information scattered throughout the archives. Furthermore, the frequent duplication of names of files containing different content, different dates, changes, etc. to move forward, we needed a way to index all the duplicate files, and compare the CRC to identify all unique files.

We decided to find a file extensions that were used in the process of working with FMV and create a manifest for the following file types: .ANM, .CEL, .FLC, .FLI .GIF, .MAX .NUT, .OBJ .PRO. We have written a tracking duplicates the Python script to crawl all the available archives and the creation of the XML map each file name with the desired extension. In the end we got a cleaned catalog of useful files that make it easier to understand and look for. Here is an example of the node in the manifest xml file:

the

<file>
<name>b_cm1</name>
<most_recent>C:\Backup\Lleslle backup\THROT-C\RESTSTOP\STUFF\B_CM1.ANM</most_recent>
<unique_files count="2">
<file_item>
<path>C:\Backup\Lleslle backup\THROT-C\RESTSTOP\STUFF\B_CM1.ANM</path>
<modify_date>1994/07/30</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>2D31393738373530333931</crc_value>
</file_item > 
<file_item>
<path>C:\Backup\PC2\FUL528_Davi_DV2\BEN\RESTOP\FLICS\B_CM1.ANM</path>
<modify_date>1994/07/13</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>32303432363131303937</crc_value>
</file_item > 
</unique_files>
<all_files count="4">
<file_item>
<path>C:\Backup\Lleslle backup\THROT-C\RESTSTOP\STUFF\B_CM1.ANM</path>
<modify_date>1994/07/30</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>2D31393738373530333931</crc_value>
</file_item > 
<file_item>
<path>C:\Backup\Lleslle backup\THROT-C\RESTSTOP\STUFF\B_CM1.ANM</path>
<modify_date>1994/07/30</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>2D31393738373530333931</crc_value>
</file_item > 
<file_item>
<path>C:\Backup\cleslie's-backup\THROT-C\RESTSTOP\STUFF\B_CM1.ANM</path>
<modify_date>1994/07/30</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>2D31393738373530333931</crc_value>
</file_item > 
<file_item>
<path>C:\Backup\PC2\FUL528_Davi_DV2\BEN\RESTOP\FLICS\B_CM1.ANM</path>
<modify_date>1994/07/13</modify_date>
<size_bytes>115829</size_bytes>
<crc_value>32303432363131303937</crc_value>
</file_item > 
</all_files>
</file>

The above XML fragment shows that the archive found two unique files named b_cm1. These manifest files allowed us to narrow the search space when studying the dependencies of the frames.

the

Sorted according to

Even finding all of these dependencies in the archives, we could not determine exactly which dependencies were used in the finished FMV. We can make assumptions and use some fuzzy logic, but in the end decided to create a special tool which can engage people and sort all the necessary resources visually. This tool, called SanWrangler allowed us to download the source sequence of frames, and then view all the archived data in search of an intermediate component parts used to create this frame.

Looking at the example below SanWrangler, you can see on the left page in the form of a Bank of 36 individual streams of sequences of images that can learn the user. This array sequence is all footage found in the archives, with the exception of the binary duplicates. To the right of the archive of the array at the top is shown the original sequence, extracted from .SAN the above-described process. The user can press Shift+Click on the threads from the backup array to the left to move them in the right array. This right array of the selected sequence of frames represents all the threads that contain parts used to generate the finished sequence of frames.

SanWrangler allows you to search the following files from the backup array:

the

the
• .ANM files are animations created in our own tool LucasArts. These files contain hand-drawn cell animation. Later, if necessary connected with the other files.
the
• .FLC/.FLI files Autodesk Animator and Autodesk Animator Pro
the
• .NUT — they contain a linear sequence of uncompressed frames transmitted in SMUSHFT.EXE and later encoded into a compressed format that is written to the file .SAN
the
• .MAX — source file of 3D meshes created in 3D Studio 2.0. We were able to run a batch process to render sequences used in SanWrangler, that we might know what is contained in binary files .MAX. These images were crucial to understanding what file .MAX used in a specific sequence FMV.

So, we've reached the point where immediate starts remastering the old files. SanWrangler able to save the selected list of file dependencies in a cleaner format. From this point the working process of remastering is divided into three branches: remastered hand-drawn frames, remastered 3D models, and remastered sound.

Wait for the second part of the article! In the next part I will talk about how automation and transformation, used by us to build anew all content.

Article based on information from habrahabr.ru