PiBuilder

<a name="introduction"></a>

Introduction

This project documents my approach to building Raspberry Pi operating systems to support SensorsIot/IOTstack.

Design goals:

1. To have a reliable, repeatable and auditable approach to building Raspberry Pi OS, primarily as a support platform for SensorsIot/IOTstack.
2. As far as possible, to favour speed over any need for interaction during the build process.
3. All work done "headless" via ssh.
4. To have all IOTstack prerequisites satisfied so there is no need for the menu or other IOTstack scripts to be installing docker, docker-compose, and so on.

PiBuilder can't possibly be a "one size fits all" for all possible Raspberry Pi configurations. Of necessity, the scripts and supporting files implement my decisions and assumptions. You will almost certainly need to make adjustments for your own situation. I have tried to make allowances for that by providing a patching system that is flexible and extensible.

I have tested PiBuilder on:

• Raspberry Pi 3B+, 4B and Zero W2 hardware
• 32-bit versions of Raspberry Pi OS (aka Raspbian) Buster and Bullseye
• 64-bit version of Raspberry Pi OS Bullseye
• 64-bit version of Debian Bullseye running in an AMD64 guest system under Parallels
• 64-bit version of Debian Bookworm running in an AMD64 guest system under Proxmox VE (see tutorial)
• 64-bit version of Ubuntu Bookworm running in an AMD64 guest system under Proxmox VE

The scripts are written in bash and there is a reasonable chance that they will work on any Debian- or Ubuntu-based system. Any steps which have a specific dependency on the Raspberry Pi are skipped. Please see Running on other platforms for more information.

<a name="toc"></a>

Contents

• About placeholders

• Definitions

• The Build Process

  • Download Raspberry Pi Imager

  • Download Raspberry Pi OS image

  • Transfer Raspberry Pi OS to SD or SSD

  • Boot your Raspberry Pi

  • Connect to your Raspberry Pi using SSH

  • Clone PiBuilder onto your Raspberry Pi

  • Run the PiBuilder scripts in order

    • Script 01
    • Script 02
    • Script 03
    • Script 04
    • Script 05

  • Your system is ready

• Resources

<a name="replaceme"></a>

About placeholders

Guillemets (the characters <kbd>«</kbd> and <kbd>»</kbd> ) are used throughout this documentation to indicate placeholders where you should substitute actual values. The most common patterns are:

• without other forms of surrounding quote mark. Example:

  $ ssh «user»@«hostname»
  

  This syntax means you should replace the entire placeholder, including the guillemets, with the actual value. Example:

  $ ssh pi@raspberrypi.local
  

• with other forms of surrounding quote mark. Example:

  $ git add "«filename»" 
  

  This means you should replace the entire placeholder, including the guillemets, with the actual value, while retaining the surrounding quote marks:

  $ git add "resolvconf.patch@iot-hub" 
  

<a name="definitions"></a>

Definitions

• "your Raspberry Pi" means the Raspberry Pi device for which you are building an operating system using PiBuilder.
• "«hostname»" is a placeholder meaning "the name you chose for your Raspberry Pi".
• "«username»" is a placeholder meaning "the account name you use to login to your Raspberry Pi".
• "your support host" means the system from which you connect to your Raspberry Pi using SSH. It will usually be a Mac or PC.

<a name="buildProcess"></a>

The Build Process

This process is intended for first-time users. No configuration is required. The end point is a system with IOTstack and all dependencies installed. You can either start building a Docker stack using the IOTstack menu or restore an IOTstack backup.

<a name="downloadImager"></a>

Download Raspberry Pi Imager

Follow the instructions at Raspberry Pi Imager to download and install the Raspberry Pi Imager application onto your support machine. If you launch the Imager and it prompts you to update, you should do that.

<a name="chooseOSForImager"></a>

Download Raspberry Pi OS image

Although Raspberry Pi Imager can download current Raspberry Pi OS images dynamically, I strongly recommend downloading images to your support host beforehand.

1. You can be certain which image you are starting from. The alternatives displayed on the releases page (URL below) are laid out sensibly in groups, whereas Raspberry Pi Imager's <kbd>CHOOSE OS</kbd> dialog both co-mingles and distributes the alternatives between menu levels.
2. You can retain downloaded image files for as long as you need them and, providing you also save the hashes, you can be 100% certain they have not changed. This means you repeat a build as many times as necessary (eg audit requirements, testing various scenarios, building multiple Pis alike) without any risk of the base operating system changing underneath you. Although it is possible to find historical images at the Raspberry Pi OS downloads page, you have to know exactly what you are looking for, you have to take it on faith that the image hasn't changed, and you still have to download the desired image anyway.

The most recent Raspberry Pi OS can always be found at:

• https://www.raspberrypi.com/software/operating-systems

Unless you have specialised requirements, there is no point installing any 32-bit system. I always start from "Raspberry Pi OS (64-bit)" so that is what I recommend. You are building a server class device (rather than a general purpose desktop class device) so your practical options are:

1. <a name="desktopOS"></a>Raspberry Pi OS with Desktop. This retains the ability to enable the Desktop (a screen connected to the HDMI port displays a Graphical User Interface) or enable connection over VNC.

   PiBuilder disables both by default. See why PiBuilder disables RealVNC and Desktop if you want to understand the pros and cons of re-enabling the Desktop and/or VNC.

2. <a name="liteOS"></a>Raspberry Pi OS Lite. Choosing this version essentially commits you to a console (a screen connected to the HDMI port displays a Command Line Interface), and you won't have the ability to connect over VNC without doing extra work. However, you get a lean, mean system which is well-optimised for running IOTstack.

Either image will work so, ultimately, it's your Raspberry Pi and your choice.

Images for the Raspberry Pi are downloaded as .xz files (previously these were .zip files). You have the choice of:

1. downloading the image directly via the <kbd>Download</kbd> button; or
2. downloading the image indirectly by clicking the "Download torrent" link.

It is always a good idea to check the SHA256 signature on each image. It gives you assurance that the image has not been tampered with and wasn't corrupted during download. The magic incantation is:

$ SIGNATURE=«hash»
$ IMAGE=«pathToDownloadedFile»
$ shasum -a 256 -c <<< "$SIGNATURE *$IMAGE"

You get the «hash» by clicking the Show SHA256 file integrity hash link. Here's an example run. It assumes the .xz file:

$ SIGNATURE=ffe864dec3f6ee183f1fa553c496753febd6ed5f7393cbe24ae00f5fcba7b7d1
$ IMAGE=2024-03-12-raspios-bookworm-arm64-lite.img.xz
$ shasum -a 256 -c <<< "$SIGNATURE *$IMAGE"
2024-03-12-raspios-bookworm-arm64-lite.img.xz: OK

If you don't see "OK", start over!

If your first attempt was a direct download of the image, consider trying the indirect method using a torrent.

Tip:

• Once you have verified that a hash matches an image, consider saving the hash in a text file alongside the image. That way, you always have the means to re-verify that the image is unchanged.

<a name="burnImage"></a>

Transfer Raspberry Pi OS to SD or SSD

The steps are:

1. Connect your media (SD or SSD) to your support host (eg Mac/PC).

2. Launch Raspberry Pi Imager.

3. Click <kbd>CHOOSE DEVICE</kbd>, then

   • Select your hardware platform.

4. Click <kbd>CHOOSE OS</kbd>, then

   • Scroll down and choose "Use custom".
   • Select the .xz (or .zip) you downloaded earlier.

5. Click <kbd>CHOOSE STORAGE</kbd>, then

   • Select the media connected in step 1. Be careful with this step!

6. Click "NEXT".

7. Click "EDIT SETTINGS". This opens the "OS Customisation" panel at the "GENERAL" tab <!--A-->🄰:

   

   Unless you have good reasons to do otherwise, I recommend:

   • Enable <!--B-->🄱, then enter a name for your host at <a name="firstBootHostName"></a><!--C-->🄲.

     I strongly recommend using the name "raspberrypi" (one word, all lower-case) in this panel, and then changing the name later when you run the first PiBuilder script. These instructions assume you accept this advice.

     However, if you decide to choose a unique name for your host in this panel, then:

     1. You need to follow the rules for domain names:

        • letters ("a".."z", "A".."Z") but lower case is recommended
        • digits ("0".."9")
        • hyphen ("-") not underscore

     2. You will have to remember to come back into this panel each time you run Raspberry Pi Imag