Northstar

Northstar is an embedded container runtime prototype for Linux.

Table of content

• Northstar
  • Table of content
  • About
    • Containers
    • Processes
    • Comparison
  • Quickstart
  • SquashFS tools
  • Configuration
    • Repositories
  • Console
  • Debugging
  • cargo-npk
  • Integration tests
  • Integration
  • Container launch sequence
  • Manifest Format
    • Mounts
    • Seccomp
    • Capabilities
    • Custom
  • Roadmap
  • Questions and Help
  • Contributing
  • License
  • Contact
  • Acknowledgements

About

Northstar is an open source embedded container runtime optimized for speed and resource usage. Northstar combines several standard Linux process isolation and sandboxing features to gain a medium level of isolation between containers/processes. The Northstar runtime consists out of two parts: The container handling and process spawning. To build the most efficient and robust solution, Northstar is completely developed in Rust, a language designed to afford the performance of C/C++ without their footguns.

The Northstar - Embedded Container Runtime whitepaper describes the motivation and initial concepts and ideas of Northstar.

Security is a strong concern and hard to achieve. Northstar uses tons of OS interfaces with room for mistakes security and stability wise. This report describes the results of a security assessment of the Northstar Embedded Linux Container Runtime. Carried out by Cure53 in August 2022, the project included a penetration test and a dedicated audit of the source code. Outcome and actions take are documented in the projects issue tracker label audit_0822.

Containers

<!-- markdown-link-check-disable -->

Northstar containers are called NPK. The NPK format is heavily inspired by the Android APEX technology. A Northstar container contains:

<!-- markdown-link-check-enable -->

• Root filesystem in a Squashfs file system image (optionally compressed)
• Northstar manifest with process configuration and container meta information

Northstar containers can be created with the Northstar utility sextant or the cargo-npk subcommand.

Processes

Started Northstar containers are Linux processes. The attributes and environment for a spawned container is described in a container manifest which is included in a NPK. The container manifest allows to configure the following Linux subsystems and features:

• Arguments passed to the containers init
• Environment variables set in the container context
• User and group id
• Mount namespace
• PID namespace
• IPC namespace
• UTS namespace
• Cgroups memory (optional)
• CGroups CPU (optional)
• Additional bind mounts (optional)
• Capabilities (optional)
• Stdout/stderr handling (optional)
• Seccomp configuration (optional)

Comparison

• Northstar containers are not portable and are tailored to a known system (uid/gid/mounts...)
• TODO

Quickstart

Northstar is written in Rust. The minimum supported Rust version (MSRV) is 1.74.0. Rust is best installed and managed by the rustup tool. Rust has a 6-week rapid release process and supports a great number of platforms, so there are many builds of Rust available at any time. rustup manages these builds in a consistent way on every platform that Rust supports, enabling installation of Rust from the beta and nightly release channels as well as support for additional cross-compilation targets.

Building Northstar is limited to Linux systems and runs on Linux systems only! The Northstar build generates bindings for various system libraries and uses the mksquashfs command line tool for NPK creation.

Install build dependencies on Debian based distributions by running

sudo apt-get install build-essential libclang1 squashfs-tools

The squashfs-tools package is required in version 4.6 or higher.

Northstar comes with a set of examples that demonstrate most of the Northstar features. Building the example binaries and packing its corresponding NPKs is done via:

./examples/build.sh

Building and starting the example runtime main is triggered by a

cargo run --bin northstar

The Northstar workspace configuration configures a cargo runner that invokes the runtimes example main binary with super user rights.

Use the northstar-nstar utility to inspect and modify the runtimes state e.g.

cargo build --release --bin northstar-nstar
...
./target/release/northstar-nstar --help
...
> ./target/release/northstar-nstar -j start hello-world
{"Response":{"Err":{"StartContainerStarted":{"name":"hello-world","version":"0.0.1"}}}}
> ./target/release/northstar-nstar -j kill hello-world
{"Response":{"Ok":null}}

northstar-nstar is developement tool and not considered for any production usecase. See console for details why.

SquashFS tools

Sadly northstar-sextant relies on a mksquashfs binary available on the host. Here's a recipe to build a mksquashfs binary from source with gzip support only:

git clone https://github.com/plougher/squashfs-tools.git
cd squashfs-tools/squashfs-tools
git checkout 4.6.1
sudo make install

For different compression algorithms, install the corresponding dependencies (Debian based distributions):

sudo apt install help2man libz-dev liblzo2-dev liblz4-dev libzstd-dev
git clone https://github.com/plougher/squashfs-tools.git
cd squashfs-tools/squashfs-tools
git checkout 4.6.1
sudo CONFIG=1 LZO_SUPPORT=1 LZ4_SUPPORT=1 ZSTD_SUPPORT=1 XZ_SUPPORT=1 XATTR_SUPPORT=1 make -j $(nproc) install

Configuration

The example executable northstar reads a configuration file that represents northstar_runtime::config::Config. Example with comments here.

Repositories

A repository is an entity that is able to store NPK's at runtime. Repositories are configured at initialization time of the runtime. The repository configuration cannot be changed at runtime. Each configured repository has a unique identifier. Currently two types of repositories exists: fs and mem.

The fs type repositories are backed by file system storage. The configured directory (dir) is used to store NPK's. If this directory is read only, no additional install requests can be performed at runtime. If a fs repository configuration contains a key field, the repository is treated as "verified". The configured key is used to verify the signature of the containers manifest and it's verity root hash. When the container is mounted, the verity root hash is used to configure a device mapper verity devices that is mounted instead of the contained Squashfs image.

Repositories without a key are treated as trustful sources. No signature checks are performed. The root filesystem is mounted without verity. A possibly present verity root hash with in the NPK is ignored. Trusted repositories should be verified and read only file systems.

Set the mount_on_start flag of a fs repository to true to make the runtime mount all containers present at startup. The mount operations are done in parallel.

The mem type repositories use memfd for their storage. No data is persistently stored during an installation of a container. Obviously it's not possible to have NPK's preinstalled in a mem repository at runtime startup. The mem repositories are mainly used for testing.

Console

Northstar uses JSON to encode the messages shared with clients. The messages are newline delimited. This is a common approach that facilitates clients being implemented in any programming language. However, Northstar as a library, provides a convenient Client type that can be used for a simpler client implementation using Rust.

Details about the Northstar console are here. A good starting point is to run the northstar-nstar tool with the -j flag that. This instructs northstar-nstar to display the raw json payload exchanged with the runtime.

Details about the model used as json payload are found here.

Debugging

Debugging containers can be hard. Most useful information can be obtained by configuring various debug commands that are spawned with the container e.g strace. Check the debug section of the runtime configuration for