Blink16

This is an 8086 real mode only branch of Blink, useful for emulating ELKS and MSDOS executable programs. This is an early work in progress, with Blink's x86_64 VM replaced with a tiny 8086 emulator and disassembler. The system calls are also replaced for ELKS and DOS support, with currently just a few syscalls implemented.

The Blink16 branch is implemented in the blink16/ directory, using portions of Blink from the original blink/ directory and master branch.

To build:

cd blink16
make

To run sample binaries:

cd blink16
./blink16 banner ELKS   # ELKS binary
./blink16 test.exe      # DOS binary

To demo booting a prebuilt ELKS kernel from 0:7c00 (use s/s/c/^C/C/C/D to step through, and mousewheel on disassembly to show execution history):

make elks

Screenshot of Blink16 running 'banner':

A big thank you to @jart for the wonderful Blink Project!!

Screenshot of Blink running GCC 9.4.0:

Blinkenlights

This project contains two programs:

blink is a virtual machine that runs x86-64-linux programs on different operating systems and hardware architectures. It's designed to do the same thing as the qemu-x86_64 command, except (a) rather than being a 4mb binary, Blink only has a ~177kb footprint; and (b) Blink goes 2x faster than Qemu on some benchmarks such as emulating GCC. The tradeoff is Blink doesn't have as many features as Qemu. Blink is a great fit when you want a virtual machine that's extremely small and runs ephemeral programs much faster. For further details on the motivations for this tool, please read https://justine.lol/ape.html.

blinkenlights is a TUI interface that may be used for debugging x86_64-linux programs across platforms. Unlike GDB, Blinkenlights focuses on visualizing program execution. It uses UNICODE IBM Code Page 437 characters to display binary memory panels, which change as you step through your program's assembly code. These memory panels may be scrolled and zoomed using your mouse wheel. Blinkenlights also permits reverse debugging, where scroll wheeling over the assembly display allows the rewinding of execution history.

Getting Started

We regularly test that Blink is able run x86-64-linux binaries on the following platforms:

• Linux (x86, ARM, RISC-V, MIPS, PowerPC, s390x)
• MacOS (x86, ARM)
• FreeBSD
• OpenBSD
• NetBSD
• Cygwin (no JIT on Windows yet)

Blink depends on the following libraries:

• libc (POSIX.1-2017)

Blink can be compiled on UNIX systems that have:

• A C11 compiler (e.g. GCC 4.9.4+)
• Modern GNU Make (i.e. not the one that comes with XCode)

The instructions for compiling Blink are as follows:

$ make -j4
$ o//blink/blink -h
Usage: o//blink/blink [-hjms] PROG [ARGS...]
  -h        help
  -j        disable jit
  -0        to specify argv[0]
  -m        enable memory safety
  -s        print statistics on exit

Here's how you can run a simple hello world program with Blink:

o//blink/blink third_party/cosmo/tinyhello.elf

Blink has a debugger TUI, which works with UTF-8 ANSI terminals. The most important keystrokes in this interface are ? for help, s for step, c for continue, and scroll wheel for reverse debugging.

o//blink/blinkenlights third_party/cosmo/tinyhello.elf

Testing

Blink is tested primarily using precompiled x86 binaries, which are downloaded automatically. You can check how well Blink works on your local platform by running:

make check

To check that Blink works on 11 different hardware $(ARCHITECTURES) (see Makefile), you can run the following command, which will download statically-compiled builds of GCC and Qemu. Since our toolchain binaries are intended for x86-64 Linux, Blink will bootstrap itself locally first, so that it's possible to run these tests on other operating systems and architectures.

make check2
make emulates

Alternative Builds

For maximum performance, use MODE=rel or MODE=opt. Please note the release mode builds will remove all the logging and assertion statements and Blink isn't mature enough for that yet. So extra caution is advised.

make MODE=rel
o/rel/blink/blink -h

For maximum tinyness, use MODE=tiny. This build mode will not only remove logging and assertion statements, but also reduce performance in favor of smaller binary size whenever possible.

make MODE=tiny
strip o/tiny/blink/blink
ls -hal o/tiny/blink/blink

You can hunt down bugs in Blink using the following build modes:

• MODE=asan helps find memory safety bugs
• MODE=tsan helps find threading related bugs
• MODE=ubsan to find violations of the C standard
• MODE=msan helps find uninitialized memory errors

Compiling and Running Programs under Blink

Blink can be picky about which Linux executables it'll execute. For example the host system page size may cause problems on non-Linux platforms like Apple M1 (16kb) and Cygwin (64kb). On such platforms, you may encounter an error like this:

I2023-01-06T18:12:51.007788:blink/loader.c:91:47550 p_vaddr p_offset skew unequal w.r.t. host page size

The simplest way to solve that is by disabling the linear memory optimization (using the blink -m flag) but that'll slow down performance. Another option is to try recompiling your executable so that its ELF program headers will work on systems with a larger page size. You can do that using these GCC flags:

gcc -static -Wl,-z,common-page-size=65536,-z,max-page-size=65536 ...

However that's just step one. The program also needs to be using APIs like sysconf(_SC_PAGESIZE) which will return the true host page size, rather than naively assuming it's 4096 bytes. Your C library gets this information from Blink via getauxval(AT_PAGESZ).

If you're using the Blinkenlights debugger TUI, then another important set of flags to use are the following:

• -fno-omit-frame-pointer
• -mno-omit-leaf-frame-pointer

By default, GCC and Clang use the %rbp backtrace pointer as a general purpose register, and as such, Blinkenlights won't be able to display a frames panel visualizing your call stack. Using those flags solves that. However it's tricky sometimes to correctly specify them in a complex build environment, where other optimization flags might subsequently turn them back off again.

The trick we recommend using for compiling your programs, is to create a shell script that wraps your compiler command, and then use the script in your $CC environment variable. The script should look something like the following:

#!/bin/sh
exec cc \
  -g \
  -Os \
  -no-pie \
  -fno-pie \
  -static \
  "$@" \
  -U_FORTIFY_SOURCE \
  -fno-stack-protector \
  -fno-omit-frame-pointer \
  -mno-omit-leaf-frame-pointer \
  -Wl,-z,common-page-size=65536 \
  -Wl,-z,max-page-size=65536

Those flags will go a long way towards helping your Linux binaries be (1) capable of running under Blink on all of its supported operating systems and microprocessor architectures, and (2) trading away some of the modern security blankets in the interest of making the assembly panel more readable, and less likely to be picky about memory.

If you're a Cosmopolitan Libc user, then Cosmopolitan already provides such a script, which is the cosmocc and cosmoc++ toolchain. Please note that Cosmopolitan Libc uses a 64kb page size so it isn't impacted by many of these issues that Glibc and Musl users may experience.

• cosmopolitan/tool/scripts/cosmocc
• cosmopolitan/tool/scripts/cosmoc++

If you're not a C / C++ developer, and you prefer to use high-level languages instead, then one program you might consider emulating is Actually Portable Python, which is an APE build of the CPython v3.6 interpreter. It can be built from source, and then used as follows:

git clone https://github.com/jart/cosmopolitan/
cd cosmopolitan
make -j8 o//third_party/python/python.com
blinkenlights -jm o//third_party/python/python.com

The -jm flags are helpful here, since they ask the Blinkenlights TUI to enable JIT and the linear memory optimization. It's helpful to have those flags because Python is a very complicated and compute intensive program, that would otherwise move too slowly under the Blinkenlights vizualization. You may also want to press the CTRL-T (TURBO) key a few times, to make Python emulate in the TUI even faster.

Some other programs you can try, are SQLite and Antirez's Kilo editor.

git clone https://github.com/jart/cosmopolitan/
cd cosmopolitan
make -j8 o//third_party/sqlite3/sqlite3.com
blinkenlights -jm o//third_party/sqlite3/sqlite3.com
make -j8 o//examples/kilo.com
blinkenlights -jm o//examples/kilo.com

For further details, please read Getting Started with Cosmopolitan Libc which is a blog post explaining how you can write your own programs in the cosmopolitan mono-repo, which naturally will be guaranteed to work really well under Blink and Blinkenlights.

Technical Details

blink is an x86-64 interpreter for POSIX platforms that's writ