Porting Notes by Ericson2314

Making Ken Silverman's "Voxlap" voxel graphics engine run on any platform with SDL and C++.

Build Prerequisites:

• Build Tools:
  1. Option 1
     • MSVS utilities (nmake, ml, cl)
     • Windows SDK (libs)
  2. Option 2
     • MinGW/GCC
     • Nasm in %MinGW%/bin
• Graphics
  1. Option 1
     • DirectX SDK (libs)
  2. Option 2
     • SDL DEV-MinGW/GCC and RUNTIME (libs)

Guide to the various branches (they don't all exist yet)

I used to have an outline of things to do. But it became horrendously out of date. This should serve the same purpose, and actually reflects what I want to do. Keep in mind that to some extent all branches are developed simultaneously.

Conservative

This is what master is now. In short, every block of MSVC inline asm has a block of gcc inline asm to go with it. Nothing else semantically should be changed. I doubt this will work, as sometimes Voxlap used persistent registers.

no_fatbin

Currently Voxlap actually contains both SSE2 and 3DN inline assembly, and uses some run-time stuff to chose which routines to use. This is really weird, and makes replacing with C slightly more complicated as some functions like movps_3dn are no longer used. I should remove the "cputype" routine, and make the choice of architecture compliantly compile time. This should be rebased off conservative. ("Fatbin" refers to the fact that cputype is used to compile something analogous to a fat binary.)

no-asm

This is no-asm, a rush to get something that works and also uses as little compiler-specific code as possible. I have used ken's C alternatives even when vector built-ins would be more efficient, or said alternatives don't even work right (kv6draw). all the point4d vector arithmetic functions should be moved to a separate header. This should be rebased off no_fatbin.

SIMD

This is where I want to end up. This will be completely architecture independent. Pure C is used where appropriate, but if something is better done with SIMD/vector built-ins I use those instead. By the time I get to this, no-asm should be bug free. And if I haven’t bothered to redo v5.?asm in no-asm, I should re-do it now with vector-built-ins. Probably a lot of the little inline functions used for assembly can be manual inlined and discarded at this point. I may drop MSVC support here, as I don't really want to have to redo all the SIMD stuff for two compilers. Also I might do the minimal pre-processor stuff needed to allow for building as Pure C if I have not done so already (this involves making a few static struct members into global variables, so it will change the API). This should be rebased off no_fatbin, not no-asm.

Introduction:

So, the basic idea is to get Ken Silverman's Voxlap voxel graphics engine to run on just about any platform that supports C++ and SDL (DirectX is still kept operational for windows). This all boils down to 2 fundamental things: getting the code to compile in GCC, and making sure that now windows libraries are in use.

Compiler issues

The compiler part is pretty easy, except for one thing: the assembly code. Voxlap uses both a single file of routines in assembly, and copious amounts of inline assembly within it's C++ files. Now, most of this assembly should be perfectly portable, but every compiler collection has a different format for both external and internal assembly.

I say format, because converting from one to another is a fairly trivial task, but not quite trivial enough where I can confidently do it with my limited assembly knowledge in an error-free manor. I have tried various scripts to convert assembly (inline and external), but the bottom line is that I don't really trust them. Most significantly, v5.asm's NASM port won't work with MSVC (though it should), so why should I trust it with GCC? Dealing with the inline assembly is even simpler, but the conversion tools are correspondingly more primitive, and I can't test them anyways until v5.asm works.

Therefore I HAD opted to dispense with assembly code altogether. Voxlap5 (the main Voxlap code) is littered with commented out replacement C for various assembly routines. Not all of it works as well as the assembly -- I think Ken mainly used it as a guide for writing the assembly and didn't debug it as the program evolved -- but it's certainly better than nothing. As to the external assembly, there is a macro that's supposed to let the program run without it, but it leaves in a few missing linkages -- it was probably also ignored as the program evolved. For better or worse, the inline assembly and external assembly often relies on each other. On one hand, this means taking out one can help get rid of the other. On the other hand, it means it's very difficult to replace the assembly incrementally and debug as you go.

I learned of another assembly, JWASM, on IRC which is both cross-platform and Supports NASM syntax as close as possible. This gave me the opportunity to go back to my original plan of porting all the assembly. This of course came with the added benefit of leaving the original program as intact as possible, and utilising the inline assembly that was admittedly still faster than the C. For ARM ports it will still have to go but this seems to be the easier solution for now. Now my plan is to work along both methods. I don't know whether converting or removing inline assembly will get to a finished port faster, so I've made a git branch for each method to work on both concurrently.

I was able to quickly fix the few syntax errors that existed, and get a working Game.exe with MSVC and JWASM. But JWASM in itself did nothing to prevent the link errors with MSVC external assembly together is often made easier. The cacophony of link errors, was a bit hard to decipher, and in order to better separate assembly-caused linking errors from other issues, I decided to get the whole thing to be compiled as C. C, unlike C++, has no name mangling, and in addition C-derived objects files made by different compilers have a better chance of linking correctly (especially on windows). I figured this would open up new doors testing.

I now have everything compiling and linking on Windows and Linux, MSVC and gcc, and SDL on both platforms. Unfortunately however, when running a gcc build on any platform some text is displayed on the screen. When I don't use the -ffast-math flag, it will in addition crash due to arithmetic overflows. Because this is thus strictly a compiler-caused issue, my guess is there is still some mistake in the inline-assembly I converted to C. It is now clear to me that in order to get this working, I will have to remove all inline assembly. Because in many cases the C is less efficient, I will try to add it back in after I have fixed most bugs. But the only way to rigorously cross- reference the MSVC and gcc builds/bugs, I must avoid as much compiler-specific syntax as I can. I imagine once I reach the point where it is safe to again experiment with assembly, asm and no-asm versions will be developed with some independence, and the benchmarks will state which branch to focus on.

Library issues

I can't really dive into this until the former step is complete, but it seems that again a good bit of the work is already done.

Voxlap itself is a surprisingly self-contained beast, primary because it's a software render so it only uses DirectX to copy frame buffers and other trivial stuff. The back-end it currently uses Winmain, is not Voxlap-Specific, but rather used by a couple of Ken's things (maybe even the build engine).

On the flip side, the example game, and probably other games based around Voxlap don't just use voxlap's header, but also directly use Sysmain.h, Winmain's interface. So to have a genuinely useful Voxlap port, seemingly unused features of Sysmain.h must also be ported.

Thankfully, available on Ken's site is SDLmain, a non-Voxlap-Specific port of Winmain. I'm not sure how complete it is, but I'd hope it has the most basic stuff that Voxlap itself requires available. The more progress I've made, the more I been able to validate the SDLmain basically works as advertised. Now that I have the Voxlap building with SDLmain and MSVC, I am almost sure it will hardly need modification.

The other example programs besides 'game' use some GDI and other Windows- specific APIs not abstracted in Winmain, so I will not focus on porting them for the time being. I did recently however add Slab6 to my code base, as it shares both Winmain code that later ended up in Voxlap. To fully incorporate it I will need to deal with these other windows dependencies, so doing so is definitely on the agenda.

What Next:

A true port should change as little of the old program as possible in order to maintain maximal compatibility. At the same time, porting is a perfect time to introduce deep architectural changes into a program. I believe a few obvious improvements that would not effect existing programs ability to interface with Voxlap to much extent.

Voxlap is a library, nothing more, nothing less. As such it should be able to be both statically and dynamically linked with another program. Currently, only static linking is tested to work, but with a couple trivial changes, dynamic linking should be too.

Voxlap is C++, but in name only. In practice it contains little-to-none Object-Oriented components, and only a few C++ idiosyncrasies prevent it from being compiled as C. In fact, it even contains a couple of preprocessor conditionals to test for C or C++ completion. Ideally it sh