Recompiler

Current release

Note that it still requires Microsoft Visual Studio 2015 or newer to be installed. Moving to clang/llvm soon :)

Porting Xbox360 executables to Windows

The idea is simple: what if you could take the Xbox360 game and run it on your PC? Is this even possible in principle? I was pondering this question few years ago and that should not come as a surprise that there are some obvious technical difficulties in getting this done:

• Different CPUs - Xbox360 uses PowerPC based CPU, our PCs are based on x86 architecture. They are different in so many ways that I don't even know where to start :) PowerPC is RISC based, has shitloads of registers but very simple instructions. x86 is totally different on the other hand - not so many registers and many more instructions that are more complicated (addressing modes...). It's obvious that a simple transcription is not feasible.

• Memory Layout - Xbox360 uses BigEndian byte ordering, x86 CPUs use LittleEndian. To be compatible with incoming data that is being read from files and read/written into the memory all memory based operands must be byteswapped. This may pose a significant performance issue.

• Encrypted executable image - Yup, for various reasons the executables on Xbox360 are encrypted. There are some cleaver guys in Russia though that figured how :)

• Different and outdated GPU architecture - If we want to see any graphics rendered the GPU needs to be emulated. There are two hard nuts to crack: first, the shaders we see will be complied into the GPU compatible format, no HLSL on input, sorry. Those shaders will have to be reverse engineered as well. Secondly, the Xbox360 GPU was using ~10MB of internal memory called EDRAM that was serving as a temporary storage of render target for the duration of rendering. Although some cards today still use similar concept this is never exposed directly to the user. Since there a lot of different ways people used the EDRAM on Xbox this part has to be emulated. To be honest probably differently for every game.

• Inlining of graphics/kernel functions - Some of the functions used while compiling the executable were inlined directly into the compiled code making it much harder to write a simple API level wrapper. This kills the dream of making "function level" wrapper where we could just go and wrap the "d3d->DrawPrimitive" call directly. Nope, this is not going to happen.

Fortunately, every problem is solvable and the answer is YES in principle. If you want to know how, keep reading :)

Current state of the project

Currently the published branch of the project allows to run simple Xbox360 demo apps (samples). I've not yet attempted to run it with any real game as it probably would not work with anything big and serious. Also, on the legal side, this is a fine line because getting anything bigger is tricky as it requires going basically to the Torrent Sites and digging through old Xbox Live Arcade content or pirated game. Xbox360 is not yet abandonware :) For the same reason there are no source executables given, you need to get one "from somewhere". Sorry :(

Stuff currently implemented:

Backend (offline processing):

• XEX image loading, decryption and decompression
• PowerPC instruction disassembly
• Program blocks reconstruction
• Generation C++ equivalent code for whole executable ("recompilation")
• 96% of PowerPC CPU instructions implemented

Runtime (host):

• Loading and running recompiled image (as DLL)
• Basic kernel with threads, synchronization
• Basic IO with file support
• GPU command queue bootstrap
• Tracing functionality (offline debugging)

GPU:

• AMD Microcode shader disassembly and recompilation into DX11 HLSL shaders
• Command buffer parser and executor
• Very simple EDRAM simulator
• Trace dump functionality

Debugging tools

• Basic IDE that allows to view the disassembly
• Basic offline (trace based) debugger that allows to inspect every executed instruction
• Basic GPU trace viewer that allows to inspect internal GPU state at each point
• Time Machine tool that makes it possible to find previous instruction that touched given register or memory

How to run it ?

(NOTE: You require Visual Studio 2015 to build and use this project - at present it is entirely Windows-specific with regards to the code and the build process.)

• Get wxWidgets 3.1.0 and compile it to obtain the x64 DLLs. Place them in dev\external\wxWidgets-3.1.0\.
• Open the solution recompile.sln under dev\src with Visual Studio. This is the top-level VS solution that we will use to build the project.
• Compile the whole solution. You should now have binaries and their associated symbols under _bin\Debug in your project root directory.

Now you have two ways to use a binary XEX file and obtain x86-64 code from it. The first way is to use the GUI, which can be launched using recompiler_tools. This way is straightforward - create a new project (or open an existing .rep project), add an image to it (this is your XEX file), and then use the toolbar to build the imported image. You will see a log window pop up and at the end of the process, you should be able to click on the Run button to run your newly recompiled binary.

The second way is to use the command-line APIs as follows:

• Run recompiler_api -command=decompile -in=imagePath -out=outputPath to decompile your image. imagePath should point to your XEX file, and outputPath should point to the folder where you want the output (uncompressed XEX, with the filename output.pdi) to go to.
• Run recompiler_api -command=recompile -in=pdiPath -out=outputPath -generator=cpp_msvc to recompile the PDI output from the previous step into native code. At the end of the process you should have the recompiled binary, code.bin in outputPath.
• Run xenon_launcher -fsroot=dataPath -image=binPath to launch the recompiled binary. dataPath refers to the folder where the assets of the binary can be found (example: projects\xenon\dolphin for the file dolphin.xex), and binPath is the location of the .bin file from the previous step.

To exit the app (or end the xenon_launcher process), simply close the GPU output window.

References

• XEX information
• PowerPC ISA
• Free60 description of the XEX
• Sourcecode of the Free60 project
• Radeon R600 ISA
• Radeon microcode decompiler
• Radeon R6xx/R7xx Acceleration
• Radeon Linux Driver
• Radeon Driver parts
• Actual open source Radeon Driver
• Mesa source code
• Radeon Evergreen 3D Register Reference Guide
• Radeon R200 OpenGL driver
• Radeon R600 driver parts
• Legacy rendering formats
• Crunch texture decompression
• CRC32 calculation
• CRC calculation
• Slicing-by-8 CRC algorithm
• Kernel objects on Windows Vista
• More Windows kernel stuff
• APC on Windows
• RtlFillMemoryUlong reference
• Dashboard for Xbox360
• Tiled rendering patent
• Deflate compression

XEX

First, the XEX (Xbox Executable) format must be ripped apart and the actual code has to be extracted. XEX is a Xbox360 specific executable packing/encryption format. It's not very complicated and quite good description can be found here: Free60. There are also some old references here. Inside the XEX there are some platform specific headers (like file certificate, media/region information, file encryption key, etc) but also there's a normal PE style executable, unfortunately it's packed and encrypted.

Decryption of any actual executables requires knowing the secret AES key that is used internally by the loader to compute another AES key that is actually used to decrypt the file content. I found it on a Russian site few years ago but could not retrace my steps any more, most likely the site is down gone for good. The rest of the XEX format suggests strongly that it was basically built on top of existing PE image loader that exist