Tips for running linux containers (LXC) on ChromeOS via Crostini

Crostini is a system for running Linux in LXC containers on the Chrome OS operating system. LXC containers provide the user with a full operating environment, including a command-line shell; and optionally, a graphical user interface. Crostini was designed with a focus on insulating Chrome OS from security threats coming from inside the containers. This is great for developers who want a secure, managed, operating system; while also having access to a linux development environment.

LXC containers are similar to virtual machines, except that LXC containers are less demanding on resources (CPU, memory) because they share the system kernel amongst multiple containers. Whereas virtual machines virtualize an entire physical system, including the kernel. LXC containers differ from Docker containers, in that Docker is mainly used to virtualize single applications, like a web-server, rather than a whole interactive operating environment.

Crostini's architecture is complex. There are several layers of virtualization technologies nested within each other; as well as control channels that traverse layers. The Chrome OS setup instructions are sufficient for basic operation. However, understanding the architecture and terminology is helpful for making use of the more advanced features, and troubleshooting in the event something breaks.

Chrome OS vs Chromium OS and Open Source Software

Chromium OS is an Open Source Software project founded by Google. Software contributors, including Google employees in their professional capacities, contribute to Chromium OS. Google then pulls from the Chromium OS source tree to build their commercial Chrome OS products. The Chromium web browser operates on an identical model, being the underlying source code for Google's Chrome browser. For simplicity, we will refer to Chrome OS and Chrome browser.

Quickstart Guide

Crostini runs well without any major configuration. If you just want to get started running Linux on your Chrome OS device, the instructions from Google are good. Chrome OS devices from 2019 onwards generally support Crostini. If your device is older, you can check if it is supported. If your Chrome OS device is controlled by an educational organization, Crostini may have been disabled.

Before following the Google install directions, one small improvement is to go into the Chrome browser and type chrome://flags#crostini-multi-container in the address bar, and enable this feature. As of Chrome version 98, on the Beta channel, this enables some additional options in the user interface to manage multiple linux instances.

Crostini Architecture Overview

This architecture diagram from the Crostini Developer Guide is helpful for understanding Crostini's various subsystems and how they interact. It should be read in tandem with Running Custom Containers Under Chrome OS, where the Overview sections gives a concise description of each of the subsystems. The following description is intended as a reference you can come back to, and may make more sense after getting some experience with running LXC linux containers.

On this chart, virtualization boundaries are represented with rounded corners (CrOS, Termina VM, Debian Container). Services and daemons are represented with square boxes. Communication channels are represented with solid lines. The dashed lines appear to represent a launch or initialization rather than a persistent communication channel. Note maitred pid 1 is the only service/daemon with a dashed outline. This appears to reflect its unique position as an "agent", which can be discussed later.

CrOS is Chrome OS, the host's operating system (light blue background). Termina VM (pink background) is a special, read-only, virtual machine instance that runs the LXD software that manages the LXC containers. Note it is the name of a virtual machine instance, rather than a service. In traditional terms it can be referred to as a "guest operating system." Debian Container (light yellow background) is an LXC container. The default LXC container is named Penguin. User-launched containers would be represented alongside Debian Container, as independent instances .

Two important pieces, not pictured, are crosh and crosvm. Crosh is a limited command-line shell that runs directly on Chrome OS. Crossvm is a "virtual machine monitor". In more traditional terms it can be thought of as a type-2 hypervisor. Crossvm is the execution environment for the special Termina virtual machine.

The nesting hierarchy of the various virtualization environments can be confusing. This tree shows the relationships.

Chrome OS (Host Operating System)
  ├── Chrome (Browser process)
  ├── Crosh (Command-line Shell)
  └── CrosVM (Hypervisor)
      └── Termina VM (Guest Virtual Machine instance)
          └── LXD (Daemon)
              ├── Penguin LXC (Container)
              └── LXC 2 (Optional user-specified container)

Garcon, Sommelier,and Vshd (yellow) are daemons that run inside LXC containers. They are installed with the cros-continer-guest-tools package. They cross the isolation boundary of the LXC container and the virtual machine. This enables users to use the Chrome OS user interface to control LXC containers.

This should be enough of an architecture understanding for intermediate-level running and troubleshooting of LXC containers. The following advanced section is included so that there is a complete explanation of the architecture diagram in one place.

Crostini Architecture Overview (Advanced)

There are a couple of flavors of communication channel: D-Bus, gRPC, REST API over unix socket, and Vsock. Most of the messages passing through these channels are protobuf format.

Seneschal uses the Plan 9 Filesystem Protocol to share a Chrome OS user's files (i.e. Downloads) with virtual machines (i.e. Termina), which are then mounted into a user's home directory in the LXC container.

Concierge is responsible for starting, stopping, and tracking virtual machines. It communicates to Chrome OS over d-bus.

Cicerone is responsible for communicating with containers. It talks to the [Garcon] service, installed by cros-guest-tools in LXC containers. It communicates to Chrome OS over d-bus

Vmlog_forwarder and vm_syslog work together to capture logs from virtual machines and store them in Chrome OS.

Tremplin Translates gRPC between Cicerone on the host and LXD in the VM.

Directly Managing LXC Containers

Now that you understand the basic Crostini architecture, directly managing LXC containers will make more sense. The basic workflow is:

• Open Crosh, with the Ctrl+Alt+T shortcut. Or by navigating to chrome-untrusted://crosh in Chrome.
• At the crosh> shell prompt type vmc launch termina
• Your shell prompt should change to (termina) chronos@localhost, and you are now logged into the virtual machine named termina.
• From this termina shell prompt you have access to the lxc command, where you can create and manage LXC containers.
• From the crosh shell prompt you can launch lxc containers that you have created via vmc termina <container name>
• For best integration into Chrome OS, it is strongly recommended that you install the cros-continer-guest-tools

Crostini is under active development as of Feb 2022. There are ways to get the LXC containers into bad states. For example if you are starting and stopping containers via one interface (LXC or graphical interface), their state may not be picked up by the other interfaces.

Using the Multi-Container User Interface to Manage LXC Containers

Depending on your version of Chrome OS