torte: reproducible feature-model experiments à la carte 🍰

torte is a declarative experimentation platform for reproducible feature-model analysis research. It integrates 10+ configurable software systems, various extractors and transformers, and 700+ automated solvers.

Why torte? Take your pick:

"Tseitin or not Tseitin?" Evaluator
CNF Transformation Workbench
KConfig Extractor that Tackles Evolution
Towards Reproducible Feature-Model Transformation and Extraction
That's an Obviously Reverse-Engineered Tool Name!
KConfig = 🍰 config ∧ 🍰 = torte ∎

torte can be used to

extract feature models from KConfig-based configurable software systems (e.g., the Linux kernel),
transform feature models between various formats (e.g., FeatureIDE, UVL, and DIMACS), and
solve feature models with automated reasoners to evaluate the extraction and transformation impact,

all in a fully declarative and reproducible fashion backed by reusable containers. This way, you can

draft experiments for selected feature models first, then generalize them to a larger corpus later,
execute experiments on a remote machine without having to bother with technical setup,
distribute fully-automated reproduction packages when an experiment is ready for publication, and
adapt and update existing experiments without needing to resort to clone-and-own practices.

To demonstrate the capabilities of torte, we provide a dashboard and demo video.

Getting Started: The Quick Way

torte provides four major setup options:

This one-liner will get you started with the default experiment (Docker or Podman required).
```
curl -sL https://elias-kuiter.de/torte/ | sh
```
Choose this option if you do not intend to customize torte. <details> <summary><strong>What will this do?</strong></summary> This one-liner will: <ul> <li>download the default experiment to <code>$PWD/default.sh</code></li> <li>clone this repository to <code>$PWD/torte</code></li> <li>build all required Docker/Podman images under the <code>torte_</code> prefix</li> <li>execute the experiment and store all data in <code>$PWD/stages</code></li> </ul> To fully undo the effects, run: <pre>torte/torte.sh uninstall && rm -rf default.sh torte stages</pre> If you do not want to pipe into <code>sh</code> for security reasons, consider one of the other setup options. </details>
You can also clone this repository and run torte directly.
```
git clone --recursive https://github.com/ekuiter/torte.git
cd torte
./torte.sh
```
Choose this option if piping into a shell is no option for you, or if you want to contribute to torte.
A third option is to manually download and set up a release of torte. This increases the total download size, but it also allows you to skip building any Docker images. Choose this option if reproducibility matters a lot, or images fail to build with the other options above.
Finally, torte can be set up from our reproduction package available on Zenodo. This is effectively identical to the third option, but we release there less frequently.

By default, any of these options will store results in the stages directory. Read on if you want to know more details (e.g., how to execute other experiments).

Getting Started: In Detail

To run torte, you need:

an x86_64 or arm64 system [^28] with Linux, macOS, or Windows with WSL
Git, curl, GNU tools (bash, coreutils, make, grep, and sed)
Docker (preferably in rootless mode on Linux) or Podman

[^28]: On arm64 systems (e.g., Windows tablets and Apple Silicon Macs), torte cross-compiles some Docker images to ensure that precompiled binaries (e.g., JavaSMT, Z3, and all solvers) function correctly. This may negatively impact performance on some systems (e.g., ARM-based Windows tablets), although recent Macs should not be affected due to Rosetta. (If you encounter errors like this one, try to disable "Use Rosetta for x86_64/amd64 emulation on Apple Silicon" in the Docker settings. This setting can be re-enabled after the Docker images have been built.) Executing torte from within a virtual machine has only been confirmed to work with Linux guest systems on x86_64 host systems. Despite our efforts, some functionality involving precompiled binaries is still known to cause problems on arm64 systems. If such functionality is required, the easiest solution is to switch to an x86_64 system (e.g., with SSH).

Experiment files in torte are self-executing - so, you can just create or download an experiment file (e.g., from the experiments directory) and run it.

The following instructions will get you started on a fresh system. By default, each of these instruction sets will install torte into the torte directory. All experiment data will then be stored in the stages directory in your working directory.

Ubuntu 22.04

# install and set up dependencies
sudo apt-get update
sudo apt-get install -y curl git make uidmap dbus-user-session

# install Docker (see https://docs.docker.com/desktop/install/linux-install/)
curl -fsSL https://get.docker.com | sh
dockerd-rootless-setuptool.sh install

# download and run the default experiment
curl -sL https://elias-kuiter.de/torte/ | sh

macOS 14

# install and set up dependencies (this will replace macOS' built-in bash with a newer version)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
(echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> $HOME/.zprofile
eval "$(/opt/homebrew/bin/brew shellenv)"
brew install bash coreutils gnu-sed grep

# install Docker (see https://docs.docker.com/desktop/install/mac-install/)
curl -o Docker.dmg https://desktop.docker.com/mac/main/arm64/149282/Docker.dmg
sudo hdiutil attach Docker.dmg
sudo /Volumes/Docker/Docker.app/Contents/MacOS/install --accept-license
sudo hdiutil detach /Volumes/Docker
rm Docker.dmg
open /Applications/Docker.app

# download and run the default experiment
curl -sL https://elias-kuiter.de/torte/ | sh

Windows 11

# install WSL (see https://learn.microsoft.com/windows/wsl/install)
powershell
wsl --install

# install Docker (see https://docs.docker.com/desktop/install/windows-install/)
Invoke-WebRequest https://desktop.docker.com/win/main/amd64/149282/Docker%20Desktop%20Installer.exe -OutFile Docker.exe
Start-Process Docker.exe -Wait -ArgumentList 'install', '--accept-license'
Remove-Item Docker.exe

# restart your computer, start Docker, then install and set up dependencies
wsl
sudo apt-get update
sudo apt-get install -y curl git make

# download and run the default experiment
curl -sL https://elias-kuiter.de/torte/ | sh

Executing Experiments

Above, we run the default experiment, which extracts, transforms, and solves the feature model of BusyBox 1.36.0 as a demonstration. To execute another experiment with the one-liner, run curl -sL https://elias-kuiter.de/torte/ | sh -s - <experiment> (information about predefined experiments is available here). You can also write your own experiments by adapting an existing experiment file.
As an alternative to the self-extracting one-line installer shown above, you can clone this repository and run experiments with ./torte.sh <experiment> (e.g., `./torte.sh busybox

Torte

Install / Use

README