Forklift
A configurable bill-of-materials (CBOM) system for declaratively composing and upgrading/downgrading your hardware-specific embedded Linux operating systems
Install / Use
/learn @forklift-run/ForkliftREADME
A configurable bill-of-materials (CBOM) system for declaratively composing and upgrading/downgrading your hardware-specific embedded Linux operating systems.
Note: while Forklift's architectural design and configuration schema have mostly stabilized, Forklift's Go packages and command-line interface will undergo some backwards-incompatible changes. Forklift is already used in production as lower-level infrastructure in the rpi-imswitch-os operating system for openUC2 devices, but other possible ways of using Forklift are not yet officially documented or supported for production operations.
Introduction
Forklift is a software deployment and configuration system providing a simpler, easier, and safer mechanism for provisioning, updating, reconfiguring, recomposing, and extending browser apps, network services, and system files on single-computer systems (such as a Raspberry Pi or a laptop). The design of Forklift makes tradeoffs specific to the ways in which many open-source scientific instruments need to be deployed and operated (e.g. intermittent internet access, independent administration by individual people, decentralized management & customization). For a quick three-minute overview of the motivation for Forklift and how it can be used, please refer to this demo video: https://www.youtube.com/watch?v=4lHh_NDlFKA
For open-hardware project developers, Forklift enables Linux-based devices and appliances (especially those based on the Raspberry Pi) to have the "interesting" parts of their software programs and configurations to be specified, composed, deployed, and reversibly upgraded/downgraded as version-controlled changes to a configurable bill-of-materials (CBOM) of software modules. openUC2 GmbH, which develops and sells an open-source modular microscopy and modular optics platform, uses (and funds the development and maintenance of) Forklift as low-level infrastructure for software releases, deployment, and extensibility in openUC2's rpi-imswitch-os, a hardware-specific operating system based on the Raspberry Pi OS; and Forklift was designed specifically to solve the unique combination of OS/software maintenance, customization, and operations challenges specific to open-source instrumentation projects like openUC2 and PlanktoScope.
For end-users operating open-source instruments with application services (e.g. network APIs or browser-based interfaces) and/or system services (for e.g. data backups/transfer, hardware support, computer networking, monitoring, etc.), Forklift aims to provide an experience for installing, updating, and uninstalling software and OS customizations similar what is achieved by app stores for mobile phones - but with more user control. Forklift also simplifies the process of keeping software up-to-date and the process of rolling software back to older versions if needed; this reduces the need to (for example) re-flash a Raspberry Pi's SD card with a new OS image just to update the application software running on the instrument while ensuring the validity of the resulting state of the system.
For indie software developers and sysadmins familiar with DevOps and cloud-native patterns, Forklift is just a GitOps-inspired no-orchestrator system which is small and simple enough to work beyond the cloud - using Docker Compose to avoid the unnecessary architectural complexity and overhead of even minimal Kubernetes distributions like k0s for systems where a container workload orchestrator is unnecessary; and bundling app deployment with the deployment of system files, executables, and systemd units from configuration files version-controlled in Git repositories. Thus, Forklift allows hassle-free management of software configurations on one or more machines with only occasional internet access (or, in the future, no internet access at all!) and no specialized ops or platform team.
For people who are Very Into Linux, Forklift is a way to bring a partial subset of the architectural benefits of atomic/"immutable" OSes (such as ChromeOS, Fedora Atomic Desktops, Universal Blue, Fedora CoreOS/IoT, NixOS, Flatcar Container Linux, GNOME OS, etc.) into more traditional non-atomic Linux distros (such as Raspberry Pi OS, Debian, etc.), by enabling atomic, composable, and reprovisionable changes for a sufficiently-interesting subset of the OS. The specific scope of what Forklift manages depends on how Forklift gets integrated with the OS, but Forklift is intended to enable sysadmins to incrementally reduce reliance on system packages, and to incrementally reduce any responsibilities of the base OS beyond acting as a container host with hardware drivers (e.g. for Wi-Fi).
To learn more about the design of Forklift, please refer to Forklift's design document.
Project Governance
Currently, design and development of Forklift prioritizes the needs of openUC2 together with the Forklift project's values for Forklift as infrastructural software. Thus, for now decisions will be made by the Forklift project's maintainer (currently @ethanjli) as a "benevolent dictator"/"mad scientist" employed by openUC2. This will remain the governance model of Forklift while it's still only used for openUC2's rpi-imswitch-os operating system, in order to ensure that Forklift develops in a cohesive way consistent with the values mentioned above and with intended use of Forklift for openUC2 devices. If/when Forklift starts being used for delivering/maintaining variants of rpi-imswitch-os, for integration of third-party apps by users of openUC2 devices, or for software configuration by ordinary users, then governance of the github.com/forklift-run/forklift repository will transition from benevolent dictatorship into a consensus-based proposals process. In the meantime, we encourage anyone who is interested in using/adapting Forklift to fork this repository for experimentation and/or to create new discussion posts in this repository, though we can't make any guarantees about the stability of any APIs or about our capacity to address any external code contributions or feature requests.
If other projects beyond openUC2 decide to use Forklift as part of their software delivery/deployment infrastructure, we can talk about expanding governance of Forklift beyond openUC2 - feel free to start a discussion in this repository's GitHub Discussions forum.
Usage
For a more guided demo showing off Forklift's capabilities, you can try out Forklift on a Raspberry Pi or in a VM:
- Raspberry Pi:
github.com/ethanjli/rpi-forklift-demo provides an
SD card image of the Raspberry Pi OS with Forklift already integrated into it, so that Forklift
can be used to provision/update/deprovision Docker Compose apps as well as system config files,
scripts, and binaries in the
/etcand/usrdirectories. - VM: github.com/ethanjli/ublue-forklift-sysext-demo provides an ISO installer of Bluefin (i.e. a custom image of Fedora Silverblue) with Forklift already integrated into it, so that Forklift can be used to provision/update/deprovision Docker Compose apps as well as systemd system/configuration extensions.
If you don't want to try out Forklift in either of the above ways, the below instructions will help you to set up Forklift on your own system to Docker Compose apps on your system:
Download/install forklift
First, you will need to download the forklift tool, which is available as a single self-contained
executable file. You should visit this repository's
releases page and download an archive
file for your platform and CPU architecture; for example, on a Raspberry Pi 4, you should probably
download the archive named forklift_{version number}_linux_arm64.tar.gz (where the version number
should be substituted). You can extract the forklift binary from the archive using a command like:
tar -xzf forklift_{version number}_{os}_{cpu architecture}.tar.gz forklift
Then you may need to move the forklift binary into a directory in your system path, or you can
just run the forklift binary in your current directory (in which case you should replace
forklift with ./forklift in the example commands listed below), or you can just run the
forklift binary by its absolute/relative path (in which case y
