Freifunk Berlin Firmware

https://wiki.freifunk.net/Berlin:Firmware

This is the build system for the firmware of Freifunk Berlin. The firmware is based on vanilla OpenWrt with some modifications (to fix broken stuff in OpenWrt itself or for example LuCI) and additional default packages/configuration settings.

Contact / More information

More user-relevant information on the firmware can be found on the wiki page at: https://wiki.freifunk.net/Berlin:Firmware. There you can also find the

Release Notes
a tutorial (en / de) on router configuration

If you have any questions, send an email to berlin@berlin.freifunk.net, open a discussion with @freifunk-berlin/firmware or come to our weekly meetings. If you find bugs, please report them at: https://github.com/freifunk-berlin/firmware/issues

Development

Info

For the Berlin Freifunk firmware we use vanilla OpenWrt with additional patches and packages. The Makefile automates the build of the firmware (applies patches, integrates custom packages and uses the ImageBuilder of OpenWrt).

The idea is to download OpenWrt via git, patch it with all the patches in the patches/ folder and configure it according to the configuration snippets in configs/. Then use the OpenWrt build system to build all packages and the ImageBuilder. The packages that will be assembled into the final firmware image are taken from the image flavors defined in packagelist/.

Build prerequisites

Please take a look at the OpenWrt documentation for a complete and up-to-date list of packages for your operating system.

Alpine:

apk add asciidoc bash bc binutils bzip2 cdrkit coreutils diffutils \
findutils flex g++ gawk gcc gettext git grep intltool libxslt \
linux-headers make ncurses-dev openssl-dev patch perl \
python3 rsync tar unzip util-linux wget zlib-dev

Arch / Manjaro:

# Essential prerequisites
pacman -S --needed base-devel bash bzip2 git libelf libxslt ncurses \
openssl python time unzip util-linux wget zlib
 
# Optional prerequisites, depends on the package selection
pacman -S --needed asciidoc help2man intltool perl-extutils-makemaker

CentOS / Fedora:

sudo dnf --skip-broken install bash-completion bzip2 gcc gcc-c++ git \
make ncurses-devel patch perl-Data-Dumper perl-Thread-Queue python2 \
python3 rsync tar unzip wget perl-base perl-File-Compare \
perl-File-Copy perl-FindBin diffutils which

Debian / Ubuntu:

sudo apt install build-essential flex gawk gettext git libncurses5-dev \
libssl-dev libxml-perl python3 quilt time unzip wget zlib1g-dev

Gentoo:

echo \
app-arch/{bzip2,sharutils,unzip,zip} sys-process/time \
app-text/asciidoc \
dev-libs/{libusb-compat,libxslt,openssl} dev-util/intltool \
dev-vcs/git net-misc/{rsync,wget} \
sys-apps/util-linux sys-devel/{bc,bin86,dev86} \
sys-libs/{ncurses,zlib} virtual/perl-ExtUtils-MakeMaker \
| sed "s/\s/\n/g" \
| sort \
| sudo tee /etc/portage/sets/openwrt-prerequisites \
&& sudo emerge -DuvNa "@openwrt-prerequisites"

On openSUSE:

sudo zypper install --no-recommends asciidoc bash bc binutils bzip2 \
flex gawk gcc gcc-c++ gettext-tools git git-core intltool \
libopenssl-devel libxslt-tools make ncurses-devel patch \
perl-ExtUtils-MakeMaker python-devel rsync sdcc unzip util-linux \
wget zlib-devel

Building the firmware for all targets

To get the source code and build the firmware locally, use:

git clone https://github.com/freifunk-berlin/firmware.git
cd firmware
make

The build will take some time. You can improve the build time with build options such as -j <number of cores>. With V=s more detailed error messages are output.

An internet connection is required during the build process. A good internet connection can improve the build time.

You will need about 10 GB of disk space for the build.

Building individual packages

To develop on a single package or to compile a special package that is not available by default on OpenWrt or Freifunk-Berlin, you can use the SDK. The precompiled SDK of the firmware you are using can be found in the root directory TARGETS.

To build your own package using the SDK, follow these steps:

(cd /tmp; wget https://firmware.berlin...SDK*.tar.xz)
git clone https://github.com/freifunk-berlin/firmware.git
cd firmware
make setup-sdk SDK_FILE=/tmp/<SDK-file from above>
cd sdk-<target>

This folder represents the environment that was used to create the firmware, including any patches. You can customize the environment, install feeds and packages, and modify the existing code. To build a single package, use the normal OpwnWrt command:

make package/freifunk-berlin-ffwizard/compile

Directory layout

The actual firmware images generated by the ImageBuilder (and the ImageBuilder itself) can be found in firmwares. The layout looks like this:

firmwares/
    TARGET/
        backbone/
           images..
        default/
           images..
        ...
        OpenWrt-ImageBuilder-....tar.xz
        OpenWrt-SDK-....tar.xz
        initrd/
           images..
        packages/
           packages/<ARCH>
              base/*.ipk
              luci/*.ipk
              packages/*.ipk
              packages_berlin/*.ipk
              routing/*.ipk
           targets/MAINTARGET/SUBTARGET/packages/
              *.ipk

As you notice, there are several different image variants ("notunnel", "manual", etc.). These different package lists are defined in packagelists/. For a description of the purpose of each packagelist, have a look at the Freifunk-Wiki. With the "OpenWrt-ImageBuilder" you can assemble your own image variant with your package lists without having to compile everything yourself. The "OpenWrt-SDK" is the fastest way to build your own packages or programs without compiling OpenWrt yourself. The "initrd" directory contains some initrd images for netboot, which are used on some boards for the initial installation of OpenWrt.

Customizing make

By default make uses TARGET and PACKAGES_LIST_DEFAULT which are defined in config.mk. You can customize this by overriding them:

make TARGET=mpc85xx PACKAGES_LIST_DEFAULT=manual

Additionally, you can create your own image from a pre-built ImageBuilder by doing something like:

make images IB_FILE=<file> TARGET=... PACKAGES_LIST_DEFAULT=...

The default target is ath79-generic. For a complete list of supported targets, see the configs/ directory for target-specific configurations. Each of these targets needs a matching file in profiles/ with the profiles (boards) to be built with ImageBuilder.

Build system structure

Where can I change something? What are these files for?

config.mk                    - Generic build parameters, default target to build, choose which
                               package lists should be built
modules                      - Defines all external repositories to be used
                               (see https://gluon.readthedocs.io/en/latest/dev/build.html#feed-management)
configs/                     - Target-specific configuration snippets that are passed to the OpenWrt build system
  common.config              - OpenWrt configuration parameters for all targets
  $(TARGET).config           - Target specific configuration. Will be added to common.config.
                               Options from target.config override those from common.config.
  common-autobuild.config    - Added on top when building with Makefile.autobuild.
patches/                     - Patches against OpenWrt / individual feeds
  openwrt                    - Patches for OpenWrt-core
  packages/$(FEED)           - Patches for each feed used (closely related to the definitions in modules).
packagelists/                - Package lists 
  profile-packages.txt       - Allows you to specify packets on a per-router basis. Allows adding and removing
                               packages defined in the default OpenWrt list and in our package list.
profiles/                    - List of router profiles for each target - profile names correspond to OpenWrt
                               board definition
Makefile                     - Does all the stuff. Cloning and updating OpenWrt, patching, running make.
Makefile.autobuild           - Slightly reduced Makefile for CI builds (more granular steps, ...)
scripts/assemble_firmware.sh - Script to run the ImageBuilder for a target

Continuous integration / GitHub Actions

The firmware is built automatically via a GitHub Actions workflow. A build is triggered by any PR or commit to the master branch. On each CI-run some artifacts are kept which are linked to the related build. The artifacs include:

buildtools - SDK and imagebuilder
bins - the contens of the openwrt/bin/ folder (all packages and "pure" (according to built in package-selection) firmware-images)
logs - the contens of the openwrt/logs/ folder (the log that would have been printed when building with V=s)
packageslist - the final firmware images build via imagebuilder (each packagelist type creates a separate artifact)

All artifact filenames are prefixed by their target, e.g. ath79-gerneric_, ramips-mt7621_.

Creating a release

Each release has a [semantic version number] (https://semver.org/); each major release has its own code name. We name our releases after important female computer scientists, hackers, etc. For inspiration, please see the related [ticket](ht

Firmware

Install / Use

README