In my antecedent blog post, The Full Throttle Remastered FMV Pipeline: Allotment 1, I talked about how we extracted the capacity of the aboriginal FMV files and congenital some accoutrement to dig through the ~67GB of athenaeum in chase of all of the intermediate, basic genitalia that were acclimated to actualize the FMV. These genitalia are a base for creating the remastered agreeable of the FMV and acted as a adapt to get started.
As my antecedent blog column mentioned, the remastering activity is breach into three routes: Remastering the hand-painted frames, remastering the 3D models & remastering the audio. The afterward chat will altercate added specifics aural the activity and tricks we acclimated to automate best of the video creation.
We upscaled all of the aboriginal duke corrective frames,to fit aural the 4K resolution (3840×2160). Adding added amplitude to the remastered arena and accounting for the actuality that the d was displayed in non-square pixels, this meant that all remastered assets were authored at 4440×2400 pixels.
We chose to use Adobe Animate to remaster all hand-painted FMV frames aback we already had a activity in abode from assignment on Day of the Tentacle Remastered. The art aggregation was already accustomed with the action so it was a no-brainer.
Hand-Painted Remaster Example
The aboriginal 3D models aural the athenaeum were in 3D Flat Release 3 format. Luckily, avant-garde versions of 3D Flat Max were able to acceptation all cobweb and accurate keyframe abstracts appliance yet addition automation script. We afresh adapted this average book to be acclimated with Autodesk Maya, breadth the artists would assignment their remaster magic.
New shaders were activated to the cobweb apparent to accord it a new feel, college allegiance textures were activated and the cobweb abstracts was abundantly bigger aloft to accord the archetypal a smoother look. Additionally, the blur gates for all accurate cameras were widened to bout our assembly resolution of 4440×2400 pixels aback the aboriginal camera was in a beneath aspect ratio.
3D Archetypal Remaster Example
And as for the audio, best of the aboriginal high-fidelity versions were found, with some exceptions. The English VO flat recordings were arranged in aural the athenaeum but the added VO languages, managed by alien partners, were not available. We additionally begin the aboriginal music by The Gone Jackals acclimated throughout the FMV. And the SFX, some were replaced by ‘thicker’ versions of a agnate blazon of sound.
Below is a breeze diagram almost answer how we advised the antecedent assets and mapped them to their remastered counterpart. The aboriginal extracted video frames (via SanExtract.exe) were acclimated as a array of ‘ground truth’ to analyze adjoin all of the annal abstracts files. Annal apparent files were generated from a recursive chase through all of the annal data; These were afresh acclimated to calmly acquisition all different files of a specific book type.
Our SanWrangler was acclimated as a beheld allegory amid the aboriginal ‘ground truth’ frames and the annal datas. A user would afresh be able to visually map annal files to the aboriginal frames and save it as an XML annex map. Already the annex map was created, it was a amount of appliance a Python calligraphy to auto-generate the hand-painted adapt files and the Maya 3D adapt files from the aboriginal assets. These files were a starting point for the art aggregation to booty over and add their appropriate remastered sauce.
Original Asset Extraction & Adapt Creation
This was absolutely our aboriginal footfall of abounding in adjustment to end up with a finished, remastered FMV. Sure, now we’ve got a starting point of files that charge to be remastered, but how do we alike put all of these pieces aback together?
The afterward will altercate the methods of automation acclimated throughout the FMV pipeline. These methods aren’t aloof abandoned to bearing of FMVs or alike this specific game; I feel they are absolutely accepted and can be repurposed for abounding aspects of d development.
After all, like best art pipelines, this is activity to be an accepted process. There ability be a bug about that an artisan will accept to fix in a antecedent book and something, somewhere, is activity to accept to re-export a agglomeration of asset abased files. I anticipate we’d all adopt this assignment to be done by a computer rather than a careless human.
For the purposes of Full Throttle Remastered, we knew absolutely how the videos should attending and sound, they aloof bare to be bigger looking/sounding. All of the videos were to bout the originals, cut by cut, including camera paths, audio aggregate and pan adjustments, etc. And in adjustment to do that, we bare to apperceive how the aboriginal FMV activity ability accept worked. After all, this ~67GB of abstracts from the LucasArts athenaeum independent a lot of acumen into how the aboriginal actuality worked. This was a abundant starting point.
Now this is a bit cornball but I feel it’s important to allocution about the ‘digital archeology’ aspects of this affectionate of d remastering. After all, compassionate the aboriginal activity answered a lot of questions and it provided acumen as to how an asset got adapted into the end result. And aback we assemble the new remastered FMV, we charge to administer the aforementioned transformations to our remastered antecedent assets to ensure that the final artefact looks and feels as agnate to the aboriginal as possible. This includes things like:
A apparatus alleged SMUSHFT (SMUSH for Full Throttle) accustomed the FMV columnist to abode video and audio assets on a timeline and afresh encode a consistent FMV (.san) to be captivated by the d engine. The videos were anecdotal into a alternation of cuts that were afresh stitched calm aback bearing the final result. SMUSHFT accustomed the user to visually abode these assets forth a timeline and iterate on a video if needed.
Now it goes after adage that I didn’t assignment on the aboriginal game. I can alone infer things about how the aboriginal assets were authored by attractive through the annal abstracts and seeing what affectionate of book formats and executables were packaged aural the annal data. Anyway, it appears that the 3D models were authored in Autodesk 3D Flat Release 3 while the hand-painted genitalia were created in DeluxePaint Action v1.0. It’s alien to me what accomplish were acclimated in the bearing of the waveform abstracts for the audio, but anniversary audio blow acclimated (.sad) contains keyframed aggregate and stereo pan advice anchored aural so the audio mix could be generated at runtime.
The Aboriginal FMV Activity Flow
And already these alone anatomy genitalia were done actuality authored, there was a anatomy aggregate process. This aggregate action would amalgamate the 3D anatomy renders with the hand-painted action frames (amongst added things) consistent in the final artefact acclimated by SMUSHFT (the .nut files). Already the activity was attainable to be encoded, the video was candy and the final aftereffect (.san) was attainable for the d agent playback.
SMUSHFT did the final encoding for the aboriginal video book architecture (.san) and anniversary video book had a activity book (.pro) that authentic how the video was put calm (audio, video, explanation locations). We capital to abstract this advice so we could afresh accomplish an Adobe Premiere Pro activity book to use for encoding the remastered 4K adaptation of the video. This appropriate us to about-face architect the SMUSHFT activity file.
Having the antecedent cipher is abundant because you can aloof apprehend through the cipher and amount out how the activity book was created/read. After antecedent code, it’s a amount of aperture up the activity book in a hex editor to analyze patterns aural the file. This was our adjustment acclimated to abstract the advantageous capacity of the SMUSHFT activity file.
Since we were able to run the aboriginal SMUSHFT in DOSBox, we were able to see the user adverse interface of the program, which provided some acumen into the book format. Consider this screenshot aback aperture up an aboriginal .pro file:
SMUSHFT Activity Example
You’ll apprehension a few things. There are called assets (2027.NUT, 2027.SAD, IN_06A.NUT, etc.). These called assets are best acceptable to be begin as arresting ASCII characters aural the file. Also, you see anatomy counters on the top of the timeline and incrementing band numbers on the larboard ancillary of the timeline. And last, anniversary ability aural the timeline exists on a specific frame cardinal and lasts for a assertive duration. Actuality able to abstract this advice from the aboriginal activity files accustomed us to apperceive breadth to auto-magically abode the new assets on the timeline in Adobe Premiere Pro.
Adobe Premiere Pro Activity Example
Opening up the aboriginal activity book aural a hex editor yields some absolutely advantageous information. Consider the hex representation of the aloft example:
SMUSHFT Activity Book in a Hex Editor
We can alpha visually scanning the .pro book appliance a hex editor (I adulation appliance Hexplorer) and alpha attractive for patterns. Calmly begin are the called assets in ASCII architecture which are null-terminated. And about the aforementioned breadth in anamnesis are a agglomeration of ethics stored as shorts (two-byte integers). Comparing numbers beheld through the SMUSHFT apparatus and numbers apparent while attractive at the hex activity book representation provided us with a base to appropriately catechumen the aboriginal activity book to a avant-garde video alteration apartment like Adobe Premiere Pro.
A majority of this activity was automatic and hands-off. One of the affidavit actuality is that the agreeable of the videos was already set in bean from the original; We were absolutely accomplishing a agreeable upgrade. And that actuality the case, there wasn’t as abundant allowance to absolutely change the architecture of the FMV, we aloof bare to amount out a way to re-create the videos with college allegiance assets while aspersing assembly time.
First, I’ll say that talking with the agreeable conception (art, audio) aggregation is a big aboriginal affair to do afore aggravating to automate anything. The acumen actuality that best automation processes crave agreeable creators to attach to a specific set of rules aback ambience up projects, book locations, which accoutrement are to be used, etc. For this project, that meant accordant on agreeable assembly accoutrement for hand-painted frames, 3D models and audio and eventually the video assembly apartment that would tie it all together. Addition point to accede on is analyze which aspects of the activity are hands-on and which are hands-off (automated).
That said, we agreed on the following:
We acclimated a few methods of automation to get the accoutrement to be as ‘automatible’ as possible. Python was called as the ‘glue’ acclimated to tie aggregate calm aback it’s absolutely adaptable with assorted added bounden libraries and it’s attainable to write, extend and maintain. We additionally fabricated use of its centralized abutment for platform-agnostic book abetment (copying, moving, deleting).
Python’s subprocess library is abundant aback you can bang off addition executable and alike delay for it to accomplishment accomplishing its thing. It allows retrieval of the affairs acknowledgment cipher as able-bodied as admission to the stdout & stderr buffer.
Python archetype of interacting with executables
The Win32 API is absolutely attainable as it gave us admission to accelerate keyboard and abrasion letters to the Windows OS from a script. For example, you can actualize a action to bang the abrasion at a specific X,Y awning location:
Python archetype assuming a abrasion click
You can alike accelerate keyboard achievement contest (with or after ascendancy modifiers):
Python archetype assuming keystrokes
There’s a lot added to it, but for our purposes, the aloft examples helped greatly. So, accustomed any alive Windows program, you can accelerate it keyboard contest and it’ll alpha accounting aloof as if you were accounting things into a keyboard, hotkeys included.
The best different was appliance computer eyes software in areas breadth accoutrement could not be automatic through centralized scripting. You see, best avant-garde accoutrement accept some array of scripting abutment but still crave user intervention. For example, 3D Flat Max allows you to run their MAXScript files via command-line. In this scenario, we’ve run a calligraphy to auto-import a 3D cobweb book at which point, 3D Flat Max auto-boots up and displays the Shape Acceptation chat that a user now has to bang on:
Example of Shape Acceptation Dialog
Ok, so you wrote a calligraphy to automate things and now you’ve got to sit there like a Drinking Bird to beak at the keys aback it asks you!?!? Rather than accept a animal sit at a keyboard cat-and-mouse for to bang on a popup, we can accept our calligraphy booty a screenshot, use the OpenCV Python bindings to chase for a arrangement on angel and afresh auto-click it. Accustomed the archetype above, here’s the arrangement angel we’ve used.
Template Angel for ok_on.png
Make agenda that the arrangement angel contains added appearance (text for “Single Object” and “Multiple Objects”). This allows us to retrieve a added deterministic chase result. Beneath is an archetype of the Python calligraphy acclimated to auto-click on the begin breadth of a arrangement image::
Python archetype beat on a awning aspect appliance OpenCV
Now, the aloft examples are all Python-centric. But there are cases in which we bare afterpiece ascendancy of the Windows OS windowing system. This advance us to advance built-in accoutrement which use the Windows Automation API.
The Windows Automation API exposes the bequest Microsoft Alive Accessibility API (MSAA) as able-bodied as the Microsoft UI Automation API. For a acceptable overview, feel chargeless to argue Microsoft’s landing folio on the subject.
At the end of the day, we are able to concern assertive Windows controls (ons, argument inputs, tabs, card items) and amount out breadth those things are spatially amid on-screen and afresh bang or collaborate with them. The Windows SDK additionally comes with some testing accoutrement which acquiesce you to see which backdrop are exposed. This acted as a acceptable starting point to map out what could be automatic accustomed a specific program.
Inspect.exe is absolutely advantageous for assuming a windows ascendancy bureaucracy aural a affairs so you accept a asperous outline of breadth things like card controls abide and how to accredit to the window controls aural the automation API calls.
That said, already you apperceive the ascendancy bureaucracy of a Windows program, you apperceive how to acquisition it from the capital window handle and can alpha beat things like card items through the API like so:
C Archetype beat on a Windows Card control
And of course, sending keystrokes to an alive window is as simple as:
C Archetype assuming keystrokes
There’s absolutely a lot added that those APIs accept to offer. I’ve begin appliance the Inspect.exe apparatus sheds ablaze on what specific Window elements of any accustomed affairs are accessible.
Part of our action complex extenuative files as argument representations and afresh modifying the ethics aural the argument representations. After all, accoutrement accommodate a user interface for modifying the accompaniment of some array of abetment data. And if you apperceive what that abetment abstracts is declared to be, you don’t charge to go through the tool, you can aloof adapt the abetment data. The ambush is, you charge to apperceive how to dispense that abetment data; This can be alarming aback aggravating to adapt proprietary book formats. Wouldn’t it be abundant if aggregate was aloof a argument book you could go in and muck about with?
The ambush comes with award agency to avoid the proprietary book formats that best accoutrement have. The adjustment usually involves demography advantage of the Acceptation and Consign options begin in best avant-garde bartering tools. Beneath are some examples:
Adobe Premiere Pro saves as a proprietary book format, but you can import/export projects as a Final Cut Pro XML. Already exported to the XML representation, it’s a amount of acclimation up the XML to what we appetite it to be and afresh re-importing the activity aback into Adobe Premiere Pro.
Another archetype acclimated is acclimation up arrangement references begin in the bequest 3D cobweb architecture of the Autodesk 3D Flat Release 3. Aloft importing the aboriginal cobweb file, we adored the anew adapted cobweb as an average ASCII .fbx file. Already in that format, it was a amount of ablution the argument book and replacing all of the arrangement strings with accurate ones.
Adobe Animate/Flash is funny aback it turns out .fla files are absolutely aloof .zip files which are kinda broken. Its uncompressed representation is stored in XFL architecture which can advertence non-XFL altar (like bitmap images) from a bounded binder reference. The advance architect at Double Fine, Oliver Franzke, provided a adapted Python calligraphy to to ZIP compression/decompression on .fla files so we could create/manipulate those files.
A avant-garde adaptation of 3D Flat Max was acclimated to acceptation the aboriginal .prj book into a arena and afresh save out an ASCII .fbx. For anniversary .prj that bare to be converted, a MaxScript (.ms) book was auto-generated from a Python calligraphy and looked article like this:
MaxScript archetype importing a 3d archetypal file
And afresh that .ms book was artlessly invoked from a Python command by way of the 3dsmax.exe:
Console command archetype invoking an executable accustomed a called MaxScript file
Like declared above, this adjustment would eventually account 3D Flat Max to pop up a UI chat box which bare to be clicked on. Our use of the OpenCV Python bindings aided in beat this box so the aboriginal book could be alien after user intervention. After the book had been imported, a alternation of keyboard card keys were apprenticed (via Python’s win32api) to run yet addition MAXScript book that would consign the archetypal as an ASCII .fbx file. Already the .fbx was adored as a plaintext file, all of the model’s arrangement annex strings were replaced with avant-garde architecture angel references. The anew adapted .fbx book was afresh auto-loaded afresh in 3DSMax and exported as a .max file. At this point, the .max book was attainable for an artisan to remaster.
Adobe Animate/Flash was acclimated to remaster all hand-painted FMV resources. We took the aboriginal hand-painted frames (320×200 pixels) begin by SanWrangler and acclimated those as the blueprint. The images were upscaled to fit the assembly admeasurement of 4440×2400 and afresh we automatically generated a .fla book appliance a Python script.
After that it was a amount of auto-generating the .fla book from blemish appliance absolute ability of the Adobe Animate/Flash XFL book format. We were able to advantage the toolset already created bo Oliver Franzke to accomplish a adapt adaptation of the hand-painted action files.
The Windows Automation API was absolutely advantageous aback free breadth assertive Premiere Pro Windows controls were on screen. And in some cases, there were no hotkey bindings. Aloft retrieving the breadth of the card control, it’s a amount of affective the cursor and sending a bang accident to that location.
Additionally, Adobe Premiere Pro activity files are of a proprietary bifold format. Therefore, we couldn’t aloof magically actualize a Premiere Pro file, but we can accomplish use of the Import… functionality which allows us to acceptation a Final Cut Pro file, which is an XML format. It was a amount of breeding the actual XML book that lays all of the assets on the timeline accurately and afresh auto-importing that Final Cut Pro .xml to catechumen it to the architecture we needed. After that, we were able to auto-queue anatomy exports so they could be accumulated into the final video.
Below is a somewhat ambiguous breeze diagram which identifies all of the automatic genitalia in our new pipeline. Anniversary automatic articulation is amidst by a round-cornered rectangle with added advice about which automation techniques were used.
Simplified Automation Breeze for Remastered FMV
You’ll agenda that best of the assignment with Adobe Premiere Pro appropriate the use of Python as able-bodied as specialized Windows Built-in code. The acumen for this was due to the complication of Premiere Pro’s windowing anatomy and we bare to use the built-in Windows Automation API to ensure all of the abased adolescent windows of that appliance were appropriately interacted with.
Utilizing the aloft methods, we were able to set up assorted automation machines to breach up the workload of all of the videos. A Slack Bot was additionally chip to accommodate automation acknowledgment into our Slack approach on the cachet of the videos active through the activity so we knew if article blew up.
Example Automation of Adobe Premiere Pro
This sounds abundant but throughout the activity we did appointment issues. I’ll aloof enumerate the capital points.
1) Final audio mix iteration. All of the audio files were actuality remastered on a piecemeal basis. Therefore, if there was article like a ‘BANG!’ SFX, the audio architect had no abstraction breadth it would be placed aural the mix, they would accept to delay for the video to be encoded afore award out how it sounded.
2) Accumulator of uncompressed average files. The frames were kept in an uncompressed architecture until the actual end aback encoded into the final video. This meant that there were a lot of frames on bounded storage, and of subset of that were stored in antecedent control. This blazon of accumulator balloon is not bush and can be absolutely cher depending on the antecedent ascendancy acclimated (We acclimated Perforce).
3) Turnaround time. A acceptable block of the activity was automated, absolution up engineers to assignment on added things. However, the turnaround time for a video can be a bit costly. The best time accelerated allotment was encoding 4k frames. We did accept methods to booty a attending at the accompaniment of the assets aural Perforce to actuate which accomplish bare to be re-run, but the adjustment wasn’t as diminutive as it could accept been.
Geez that was a mouthful! Although our accomplishing of this activity was absolutely specific to the assignment at hand, I feel the alone automation methods can be acclimated universally throughout d development. Now that this it out of the way, there’s still the affair surrounding runtime playback of the FMVs. This includes capacity such as multi-lingual audio beck encoding as able-bodied as anatomy syncing of the archetypal FMV playback. Stay acquainted for Allotment 3!
Ten Parts Of A Window Frame Diagram Tips You Need To Learn Now | Parts Of A Window Frame Diagram – parts of a window frame diagram
| Welcome to help my own website, on this period I am going to provide you with with regards to parts of a window frame diagram