Polyphorm

Polyphorm is an interactive tool to analyze intergalactic gas and dark matter filaments (together known as 'Cosmic web') using the Monte Carlo Physarum Machine (MCPM) algorithm inspired by the foraging behavior of Physarum polycephalum 'slime mold'.

Polyphorm has two tightly coupled main components: simulation and visualization.

The simulation component implements the MCPM algorithm to reconstruct an optimal transport network given a set of point data in 3D space. Such data can represent the distribution of galaxies or dark matter halos, typically on the scales of 100s of megaparsecs. MCPM uses a swarm of millions of particle-like agents to explore the simulation domain. These agents are attracted to each other, as well as the input data. After a number of iterations, the simulation reaches a dynamic equilibrium: the agents still move, but the totality of their trajectories is stable. The output of the algorithm at this stage is a 3D spatio-temporal density of the agents which we interpret as an estimate of the Cosmic web.

The visualization component facilitates analysis tasks of the estimated network. Thanks to the interactive nature of Polyphorm we can observe changes of the estimate in response to changing MCPM parameters. The main concern is whether the reconstruction fits the input data (i.e. all the input points are contained in it) as well as the plausibility of the obtained filamentary structures.

You can see Polyphorm in action in the following video. A selection of more videos demonstrating the software in different reconstruction and visualization scenarios is available here.

System Requirements

• Decent GPU, especially in terms of available VRAM (minimum 2GB, recommended 4GB; tested on NVIDIA TitanX 12GB, RTX Titan 24GB, RTX 2070 Mobile 8GB, AMD Radeon RX5500 8GB)
• Windows OS (tested on 10)
• DirectX 11 installed on system
• Visual Studio (tested on 2017 or 2019, probably might be adapted to older versions)
• Visual Studio Build Tools installed (can simply be selected during the VS install process, or alternatively, installed separately)

Installation Instructions

• Clone the DirectXTex repository
• Build the DirectXTex library
  • Open one of the premade .sln files (corresponding to your platform), select the 'Release' build configuration, and build the solution
  • If the build doesn't go through, make sure you have VS Build Tools installed (v141 for VS 2017, and v142 for VS 2019)
• Clone the Polyphorm repository
• Setup the bundled build tool (./builder/)
  • Make sure that the paths in build.bat point to existing vcvarsall.bat (depends on your VS installation version, usually somewhere in Program Files)
  • Add the absolute path to ./builder/bin/ into the PATH environment variable
• Check that polyphorm.build points to the correct directories (especially wrt. the DirectXTex library installation)
• Run build run polyphorm.build --release from the root (the build process will produce a short log - if not then something went wrong, make sure that all the paths are configured correctly and the required dependencies exist)

Troubleshooting checklist: 1] system requirements all met; 2] DirectXTex library successfully built (in Release mode); 3] paths in build.bat and polyphorm.build point to existing valid folders; 4] there's enough video memory available (if not, decrease 'Grid Resolution' in the config.polyp file).

Quick Manual

The software is launched simply by running ./bin/polyphorm.exe. The ./bin/config.polyp plaintext file holds most of the settings related to the performance of the application (screen resolution, number of agents, resolution and margins of fitting grids).

Input Data

The supplied sample dataset is a catalog of 37.6k galaxies from the SDSS catalog described in Burchett et al. 2020: Mapping the Dark Threads of the Cosmic Web (see Publications below).

A new dataset can be added the preamble of main.cpp by defining a new REGIME_* and specifying the path to the source data file, colormaps for the visualization, and initial parameter values for MCPM. The input data consists of two files: a binary file specifying the target 3D point data, and a plaintext metadata file:

• The data file must contain serialized 4-vectors [XYZW] of float32 values, each 4-vector storing the 3D position XYZ of a data point and its weight W. One way to produce such a file is through the Python function nparray.tofile().
• The metadata file must specify the number of data points, their XYZ spatial extrema, and the average mean value of the points' weights W.

An example preprocessing script is provided in the root directory under the name pack_data_celestial.py. This script parses a simple CSV file containing the source points (defined in polar coordinates) and produces the two input files for Polyphorm as specified above. It can be easily modified to load your custom data.

Outupt Data

The main data product of Polyphorm is a 'trace' grid: a 3D array of float16 scalar values representing the spatiotemporal MCPM agent density. This can be exported at any point during fitting by pressing 'F6' (allow a few seconds for the operation). The file and its metadata will be saved in the ./bin/export/ folder.

Along with the trace grid, a 'deposit' grid with the same dimensions will be exported. This represents the data-emitted marker and can be used as a baseline reference comparison, since it's equivalent to a weighted kernel density estimate that uses a Gaussian kernel.

A notebook illustrating how to load these datasets is provided in the root directory under the name OpenPolyphorm.ipynb.

Controls

Most of Polyphorm's controls are a part of the UI, including changing the visualization modality and its parameters. The rest is mapped as follows:

• Left/right/middle mouse: rotate/pan/zoom camera
• F1/Esc: toggle/terminate UI
• F2/F3: reset/toggle the simulation
• F4: autorotating camera
• F5: export agent data (to ./bin/export/)
• F6: export trace and deposit field data (to ./bin/export/)
• F7/'1': activate continuous/single screen capture (stored in ./bin/capture)
• F8: flush the trace data (but maintain agents' and simulation state)
• F9/F10: save/load current visualization state (camera + visual settings)

Simulation

<img src="docs/energy.png" align="right"> Immediately upon launching, Polyphorm starts fitting to the input data. The state of the fitting is captured in the red energy plot: the higher the energy value, the more are the simulation agents aligned with the input data. The spatio-temporal agent density is captured in the green histogram: healthy fits typically have a bell-shaped log-density distribution with a slight positive skew.

Visualization

Polyphorm supports multiple visualization modalities to facilitate the analysis of the reconstructed Cosmic web estimate.

• Particle mode directly renders the input data (red) and MCPM agents (white) as discrete points. While the data points are static, the agents flow dynamically through space.<p> <img src="docs/mode_particles.png" width="80%" height="80%">

</p>

• Trace mode uses direct volume rendering to visualize the spatio-temporal agent density field. The density is mapped to a configurable color palette and rendered using the emission-absorption volumetric medium model. <p> <img src="docs/mode_trace.png" width="80%" height="80%">

</p>

• Overdensity mode segments the trace field into three configurable intervals (low/medium/high density) and renders each with a different color (blue/green/red) to better understand the spatial distribution of the agents. <p> <img src="docs/mode_segmentation.png" width="80%" height="80%">

</p>

• Highlights mode renders the trace (purple) superimposed on the deposit - the volumetric 'footprint' of the input data (bright yellow). This modality additionally supports the highlighting of a selected density band. <p> <img src="docs/mode_combined.png" width="80%" height="80%">

</p>

• Trimming function allows axis aligned slicing of the 3D space to allow for more detailed/uncluttered inspection. This works for every visualization modality. <p> <img src="docs/trim_particles.png" width="24%" height="24%"> <img src="docs/trim_trace.png" width="24%" height="24%"> <img src="docs/trim_segmentation.png" width="24%" height="24%"> <img src="docs/trim_combined.png" width="24%" height="24%">

</p>

• Proxy objects, such as sightlines and spherical shells, can be manually placed in the volumetric modalities to serve as visual aides. Computationally, these are handled as implicit surfaces or volumes evaluated directly in the corresponding shader. <p> <img src="docs/proxy_sightline.png" width="55%" height="55%"> <img src="docs/proxy_shells.png" width="42%" height="42%">

</p>

• Path tracing is an experimental feature. This mode uses a progressive implementation of volumetric Monte Carlo path tracing to compute physically plausible global illumination in the reconstructed data. Multiple custom coloring schemes are being developed. <p> <img src="docs/pt_cool_blue.png" width="35%" height="35%"> <img src="docs/pt_aniso_red.png" width="32%" height="32%"> <img src="docs/pt_pointlight.png" width="29.5%" height="29.5%">

</p>

Publ