Switchboard
Communication framework for RTL simulation and emulation.
Install / Use
/learn @zeroasiccorp/SwitchboardREADME
Switchboard
Introduction
Switchboard (SB) is a framework for communication between distinct hardware models, such as RTL simulations, RTL implemented on FPGAs, and fast SW models. This makes it possible to simulate large hardware systems in a distributed fashion, using whatever models are available for the different components.
In such a simulation, each hardware model has one or more SB ports. Each is unidirectional: it may act as an input or an output, but not both. In addition, each SB connection is single-producer, single-consumer (SPSC): an output port may not drive more than one input port, and an input port may not be driven by more than one output port.
Here's an example of what a switchboard connection topology might look like:
<img width="318" alt="image" src="https://user-images.githubusercontent.com/19254098/225485548-ff127b2e-d959-46c0-af1d-2c4bbe3f119d.png">The method for adding a switchboard port depends on the language that a HW model is implemented in. For RTL-based models, SB ports are instantiated as Verilog models, whereas for C++ and Python-based models, these ports are instantiated as objects. We provide both a low-level interface for moving data directly between SB ports, as well as a higher-level interface for running UMI transactions over SB connections.
Under the hood, communication happens through shared-memory queues, where an SB output port is driving packets into the queue, and an SB input port is reading from that queue. This standardization is what allows any two kinds of models to talk to each other. A shared-memory SPSC queue is an appealing common interface because it is one of the fastest interprocess communication techniques, with latencies on the order of hundreds of nanoseconds; no system calls are required to transmit and receive data. At the same time, this type of queue is straightforward to implement for FPGA platforms, with queue read and write operations only requiring a handful of memory transactions.
For an in depth explanation of the theory and philosophy behind Switchboard, we recommend reading the paper Switchboard: an Open-Source Framework for Modular Simulation of Large Hardware Systems".
If used for research, please cite Switchboard by the following publication:
@misc{herbst2024switchboardopensourceframeworkmodular,
title={Switchboard: An Open-Source Framework for Modular Simulation of Large Hardware Systems},
author={Steven Herbst and Noah Moroze and Edgar Iglesias and Andreas Olofsson},
year={2024},
eprint={2407.20537},
archivePrefix={arXiv},
primaryClass={cs.DC},
url={https://arxiv.org/abs/2407.20537},
}
Installation
The fastest way to install this package is from PyPI:
pip install switchboard-hw
However, if you want to run the examples below (or if you're a switchboard developer), clone this repository and install the Python package in-place:
git clone https://github.com/zeroasiccorp/switchboard.git
cd switchboard
git submodule update --init
pip install --upgrade pip
pip install -e .
Examples
Various examples demonstrating the features of switchboard are in the examples folder. If you'd like to run them yourself, please run this command first:
pip install -r examples/requirements.txt
This clones some additional repositories that are needed by the examples.
A good starting point is the python example, where a Python script sends packets to and receives packets from a Verilator RTL simulation. The configuration is simple: there is a small RTL simulation that accepts an SB packet, increments the data payload, and transmits the result on its SB output port. On the other side, a Python script sends an SB packet to the simulation, and checks that the packet it gets back has been incremented.
<img width="311" alt="image" src="https://user-images.githubusercontent.com/19254098/225485672-1793521d-a9db-4c18-ad61-c22a605f8720.png">To run this example, you'll need verilator (sudo apt install verilator for Ubuntu, brew install verilator for macOS). You can then run the example by changing directory to examples/python and then typing make. That should produce output similar to the following:
*** TX packet ***
dest: 123456789
last: 1
data: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
24 25 26 27 28 29 30 31]
*** RX packet ***
dest: 123456789
last: 1
data: [ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32]
- ../verilog/testbench.sv:72: Verilog $finish
PASS!
To get a sense of how this works, open the Python script examples/python/test.py. The core logic is essentially:
from switchboard import PySbPacket, PySbTx, PySbRx
...
tx = PySbTx("to_rtl.q", fresh=True)
rx = PySbRx("from_rtl.q", fresh=True)
...
txp = PySbPacket(...)
tx.send(txp)
...
rxp = rx.recv()
In other words, we create an SB output port (tx) and an SB input port (rx). An SB packet is then created (txp) and sent via the output port. Finally, a new SB packet is received from the input port.
To get a sense of how switchboard is used in RTL, have a look at the Verilog part of this example in examples/python/testbench.sv. The core logic is the instantiation of queue_to_sb_sim (SB input port) and sb_to_queue_sim (SB output port), along with the initialization step to define the name of each SB connection. Notice that the Python output port is matched to the Verilog input port (to_rtl.q) and similarly the Python input port is matched to the Verilog output port (from_rtl.q).
`include "switchboard.vh"
...
`SB_WIRES(to_rtl, DW);
`QUEUE_TO_SB_SIM(to_rtl, DW, "to_rtl.q");
...
`SB_WIRES(from_rtl, DW);
`SB_TO_QUEUE_SIM(from_rtl, DW, "from_rtl.q");
Using the same name for two ports is what establishes a connection between them. You can use any name that you like for a SB connection, as long as it is a valid file name. The reason is that SB connections are visible as files on your file system. After this example runs, it will leave behind files called to_rtl.q and from_rtl.q. It's convenient to name SB connections in a way that is amenable to pattern matching, so that you can do things like rm *.q to clean up old connections.
We encourage you to explore the other examples, which demonstrate simulation with Icarus Verilog and switchboard's C++ library (minimal), bridging SB connections via TCP (tcp), and switchboard's UMI abstraction (umiram).
Build automation
We also provide build automation powered by SiliconCompiler that makes it easy to build RTL simulations with switchboard infrastructure (queue_to_sb_sim, sb_to_queue_sim, etc.). This is mainly important because Verilog DPI and VPI are used under the hood, requiring certain flags to be passed to the RTL simulator during the build. Using our build automation lets you focus on specifying RTL sources, without having to deal with these details.
As an example, we return to examples/python. The basic logic for a Verilator build is:
from switchboard import SbDut
dut = SbDut('name-of-top-level-module')
dut.input('path/to/file/1')
dut.input('path/to/file/2')
...
dut.build()
dut.simulate()
In other words, create an SbDut object, input() files, build() it to compile the Verilator simulator, and use simulate() to start the simulator. SbDut is a subclass of siliconcompiler.Chip, which allows you to invoke a range of features to control the simulator build, such as specifying include paths and `define macros. More information about siliconcompiler.Chip can be found here.
Packet format
An SB packet is a simple data structure with three parts, defined in switchboard/cpp/switchboard.hpp.
- A 32-bit
destination. - A 32-bit
flagsbit vector. Currently only bit "0" is used, providing thelastflag. - A 416-bit data payload. This width was chosen to accommodate a UMI packet with a 256 bit payload, 64-bit source and destination addresses, and a 32-bit command. In the future, we may support parameterizable data widths for switchboard connections.
destination and flags control how the packet is routed. destination indicates the intended recipient of the packet as a flat, unsigned 32-bit integer. This provides a mechanism where a packet can be routed through multiple hops before reaching its final destination.
For example, consider using switchboard to build a simple topology in which packets can be sent from one HW block to one of two other blocks. One could indicate which block should receive the packet using the destination field, with a router transmitting the packet to the right one.
The last
