Billion Star 3D Rendering Engine

bsrender is a 3D rendering engine for large star databases such as the ESA's mission Gaia DR3 data set with over a billion stars. It generates images from any position inside or outside the galaxy and can be run from the command line or from a web server in CGI mode. Direct rendering is used for every star without the use of pre-rendered frames or low-res previews.

Resource | Link
---|--- Project website|bsrender.io Live demo|bsrender.io/demo/ Source code|github.com/kevinloch/bsrender Sample binary data files|bsrender.io/sample_data/1.0-dev/

Key features

Generate images in JPG, PNG, AVIF, HEIF, or OpenEXR formats with multiple bit depths depending on format. AVIF encding is very slow. HEIF is supported in cli mode only (not CGI/web interface).
HDR is supported on all formats using an experimental oepn source Rec. 2100 PQ ICC profile (PNG, JPG), or in-header signaling (EXR, AVIF, HEIF). HDR is known to work with Chrome browser on M1/M2 Macbooks. HDR images on some unsupported viewers/hardware may appear very washed out or very dark
3D translations/rotations of camera position/aiming using ICRS equitorial or Euclidian coordinates. Camera can be placed anywhere in the Universe
Customizable camera resolution, field of view, sensitivity, white balance, color saturation, and gamma
Several raster projection modes are supported: lat/lon (equirectangular), Spherical (forward hemisphere centered), Spherical (front/rear hemispheres), Hammer, Mollewide
Stars can be filtered by parallax quality (parallax over error), distance from camera or target, and apparent temperature
Camera color is modeled with a Planck spectrum for the apparent temperature of each star and customizable bandpass filters for each color channel
Apparent star temperature (color) is derived from Gaia bp/G and/or rp/G flux ratios for most stars
Multithreading support with customizable number of threads

Optional features

After aiming at target, separate controls are provided to pan and tilt away from target for maximum flexibility with aiming. Camera can also be rotated about it's view axis for desired orientation
Support for user-supplied stars. A sample external database is provided with the Sun and all stars brighter than magnitude 3 that are not included in the Gaia dataset
Airy disks provide photorealistic renderings of individual stars and clusters when enabled
Gaussian blur and/or Lanczos output scaling. This allows high resolution renderings to be smoothed and downsampled on a server before downloading
Anti-aliasing, helpful when combining multiple frames into videos or simulating DSLR/MILC images
Skyglow for simulating views through Earth's atmosphere
A sample html/javascript interface includes presets for a few camera targets and several common Hubble bandpass filter settings (along with typical LRGB). Also allows copy/paste settings URL for sharing links to your rendering settings

Memory requirements

Using the full Gaia dataset with 1.4B stars requires at least 64GB of ram to run as fast as possible. This is for the operating system to cache the 46GB dataset in memory in addition to ram used by bsrender. Larger resolutions and/or use of blur or output scaling will increase memory requirements. Full 64-bit support allows for extremely large resolutions, limtied only by available ram and CPU time. 128000x64000 downsampled to 3200x16000 has been rendered with 512GB ram.

Installation

This program is written in C and requires gcc, GNU make, libpng, libjpeg, libavif, libheif, and zlib to compile. You can disable compiling in specific output formats by commenting out '#define BSR_USE_<format>' in bsrender.h and removing the associated -l<library> flag from BSR_LIBS in Makefile.

On Linux or Mac w/Xcode, go to the 'src' directory and type:

make

On FreeBSD and other systmes with non-GNU make as default:

gmake

There is no 'make install' feature yet so you will have to manually copy the executables to /usr/local/bin or wherever you want them to live. For example (on some systems this my need to be done as root):

cp bsrender /usr/local/bin; chmod 755 /usr/local/bin/bsrender
cp mkgalaxy /usr/local/bin; chmod 755 /usr/local/bin/mkgalaxy
cp mkexternal /usr/local/bin; chmod 755 /usr/local/bin/mkexternal
cp ../scripts/getgalaxydata.sh /usr/local/bin; chmod 755 /usr/local/bin/getgalaxydata.sh
cp ../scripts/gaia-dr3-extract.sh /usr/local/bin; chmod 755 /usr/local/bin/gaia-dr3-extract.sh

To show the version and all command line options run:

bsrender --help

Configuration file (optional)

Most options can also be set with a configuration file. By default bsrender looks for 'bsrender.cfg' in the same directory it is run from. A sample configuration file is provided in the source distribution. The -c command line option can be used to specify an alternate configuration filename/location. The compiled-in defaults are the same as those in the sample bsrender.cfg.

Data files (required)

The Gaia archive .csv files are not suitable for direct 3D rendering. They must be processed into a binary data format that includes the 3D position, apparent temperature, and normalized intensity (relative to Vega at one parsec) of each star. The fastest and easiest way to use bsrender is to download pre-generated data files from bsrender.io/sample_data/1.0-dev/ (46GB).

By default bsrender expects these files to be located in the subdirectory 'galaxydata' relative to where bsrender is run from. The -d command line option or data_file_directory config file option can be used to specify an alternate filename/location. To create the subdirectory and download the sample data files to it:

mkdir galaxydata
cd galaxydata
getalaxydata.sh

Alternatively, you can create a symlink to the data file directory. For example if you put the binary data files in /data and you are currently in the directory you want to run bsrender from:

ln -s /data galaxydata

Generating a custom external data file (optional)

An "external" star database of user-supplied stars is supported. By default bsrender expects galaxy-external.dat to be in the data files directory but this can be disabled with the use_external_db=no configuration option. You can use a custom generated galaxy-external.dat with the sample Gaia data files downloaded from bsrender.io, it is not necessary to recreate the Gaia data files just to use a custom galaxy-external.dat. It is also possible to use only the external data file by setting use_external_db=yes and use_Gaia_db=no. In this mode bsrender can be used for any arbitrary star database instead of the Gaia dataset and there is no need to download or create the Gaia data files.

A sample external.csv source and binary galaxy-external.dat are provided in the source distribution and in the sample data files downloaded by getgalaxydata.sh. The sample file includes the Sun and all other stars that are too bright to be included in the Gaia dataset, plus a few fainter stars obscured by bright stars. To customize, edit external.csv to add/delete/modify any stars you want and then run mkexternal to generate galaxy-external.dat:

mkexternal

Be sure to copy the new galaxy-external.dat to your data files directory.

Generating the Gaia data files manually (optional)

Generating your own Gaia data files requires 805GB of disk space and approximately 24 hours of CPU time. You might want to do this if you want to use non-default settings of mkgalaxy, or you are using bsrender on a system architecture incompatible with the sample data files which are in little-endian (x86/arm) format.

Download the entire 'gaia_source' directory (644GB) from the Gaia archive website http://cdn.gea.esac.esa.int/Gaia/gdr3/. Note that this version of bsrender will only work with gdr3. Do not uncompress the downloaded csv.gz files.

The script 'gaia-edr3-extract.sh' will extract the columns bsrender uses from the compressed csv source files and create a single uncompressed 'gaia-edr3-extracted.csv' (115GB). From the directory above gaia_source containing the compressed csv files run the script:

gaia-edr3-extract.sh

The utility 'mkgalaxy' is used to process gaia-edr3-extracted.csv into the binary data files used by bsrender. There are a few command line options for mkgalaxy that can be shown with:

mkgalaxy --help

To generate the Gaia binary data files run mkgalaxy in the same directory as gaia-edr3-extracted.csv:

mkgalaxy

This may take up to 24 hours to complete, depending on system and disk speed. Be sure to copy the new files to your data files directory. Note that binary data files created on similar but different systems may not be identical due to different non-significant bits of floating point values. This has no effect on the precision or operation of bsrender.

Operation

By default it will render a 360 degree lat/lon (equirectangular) projected panorama of the entire sky from the sun with the default settings in the sample bsrender.cfg. Descriptions of configuration options and their function are provided in the sample configuration file. Options are set in the following sequence:

Compiled-in defaults
Command line options -c and -h
Configuration file options
Other command line options
CGI options from environment variable QUERY_STRING, if in CGI mode

Some options are privileged and cannot be set by CGI users.

Helpful hints

Reducing the 'camera_fov' (zooming in) will generally require increasing 'camera_pixel_limit_mag' (pixel intensity limit in the web interface) which makes camera more sensitive to maintain the same subjective iamge brightn

Bsrender

Install / Use

README