DiffTest

DiffTest (差分测试): a modern co-simulation framework for RISC-V processors.

Usage

DiffTest supports the following run-time command-line arguments. This list is not complete as we are still working on improving the documentation.

• -i, --image=FILE for the workload to be executed by the design-under-test (DUT)

  • DiffTest supports linear (binary, gz, zstd, ELF) and footprint input formats, controlled by build-time and run-time arguments.

  • By default, the image is loaded as a binary file with a linear (continuous) address space starting at 0x8000_0000. This behavior is overrided if an advanced image format (gz, zstd, ELF) is detected.

  • Compressed binary files in gz or zstd formats are supported, determined by leading magic numbers of the image file. Once detected, they will be first decompressed and then loaded into the linear memory. Use IMAGE_GZ_COMPRESS=0 or NO_ZSTD_COMPRESSION=1 to disable their support at build-time.

  • ELF files are supported, determined by leading magic numbers of the image file. Use IMAGE_ELF=0 to disable it at build-time.

  • Footprint inputs are supported by the run-time argument --as-footprints. Basically, every time when a new address is accessed by the CPU, a new data block is read from the image file and put at the accessed address.

For more details on compile-time arguments, see the Makefiles. For more details and a full list of supported run-time command-line arguments, run emu --help.

Example: Generate Verilog for DiffTest Interfaces

DiffTest interfaces are provided in Chisel bundles and expected to be integrated into Chisel designs with auto-generated C++ interfaces.

We strongly recommend using Chisel as the design description language when using DiffTest. It will greatly benefit the verification setup since we are providing some advanced features only in Chisel, such as datapath optimizations for higher simulation speed on emulation platforms.

If you are using DiffTest in a non-Chisel environment, we still provide examples of the generated Verilog modules. You may configure the test interfaces in src/test/scala/DifftestMain.scala based on your design details. The generated Verilog and C++ files will match (in type and count) what you have described about your use case. After running the following command, files will be generated at build.

make

We support the DiffTest Profile as a configuration file for DiffTest to record and reconstruct DiffTest interfaces through a json file.

Example Chisel Usage: Connecting Your Own Design with DiffTest

We are supporting Chisel 7.3.0 (the latest stable version supporting MLIR FIRRTL Compiler). Note that we have removed the support for Chisel 3.6.1 and older versions.

Here are the detail instructions on integrating DiffTest to your own project.

1. Add this submodule to your design.

In your Git project:

git submodule add https://github.com/OpenXiangShan/difftest.git

In Mill build.sc:

import $file.difftest.build

object difftest extends millbuild.difftest.build.CommonDiffTest {
  override def millSourcePath = os.pwd / "difftest"
}

In Makefile:

emu: sim-verilog
	@$(MAKE) -C difftest emu WITH_CHISELDB=0 WITH_CONSTANTIN=0

2. Add difftest modules (in Chisel or Verilog) to your design. All modules have been listed in the APIs chapter. Some of them are optional.

import difftest._

val difftest = DifftestModule(new DiffInstrCommit, delay = 1, dontCare = true)
difftest.valid  := io.in.valid
difftest.pc     := SignExt(io.in.bits.decode.cf.pc, AddrBits)
difftest.instr  := io.in.bits.decode.cf.instr
difftest.skip   := io.in.bits.isMMIO
difftest.isRVC  := io.in.bits.decode.cf.instr(1, 0)=/="b11".U
difftest.rfwen  := io.wb.rfWen && io.wb.rfDest =/= 0.U
difftest.wdest  := io.wb.rfDest
difftest.wpdest := io.wb.rfDest

3. Call val difftesst = DifftestModule.finish(cpu: String) at the top module whose module name should be SimTop. The variable name difftest must be used to ensure DiffTest could capture the input signals.

An optional UART input/output can be connected to DiffTest. DiffTest will automatically DontCare it internally.

val difftest = DifftestModule.finish("Demo")

// Optional UART connections. Remove this line if UART is not needed.
difftest.uart <> mmio.io.uart

Alternatively, you can skip the optional UART connections by using an overloaded version of DifftestModule.finish(cpu: String, createTopIO: Boolean) with the 2nd parameter createTopIO set to false. This overloaded version can be used in non-module context (e.g. in App class) as following.

object Main extends App {
  // ...
  DifftestModule.finish("Demo", false)
}

4. Generate verilog files for simulation.

5. make emu and start simulating & debugging!

We provide example designs, including:

• XiangShan
• XiangShan (2nd Gen, Nanhu)
• XiangShan (1st Gen, Yanqihu)
• NutShell
• Rocket

If you encountered any issues when integrating DiffTest to your own design, feel free to let us know with necessary information on how you have modified your design. We will try our best to assist you.

APIs (DiffTest Interfaces)

Currently we are supporting the RISC-V base ISA as well as some extensions, including Float/Double, Debug, and Vector. We also support checking the cache coherence via RefillTest.

| Probe Name | Descriptions | Mandatory | | ---------- | ------------ | --------- | | DiffArchEvent | Exceptions and interrupts | Yes | | DiffInstrCommit | Executed instructions | Yes | | DiffTrapEvent | Simulation environment call | Yes | | DiffCSRState | Control and status registers (CSRs) | Yes | | DiffPhyIntRegState<br>~~(DiffArchIntRegState)~~ | Physical general-purpose registers<br>(Act as DiffArchIntRegState when numPhyRegs = 32) | Yes | | DiffPhyFpRegState<br>~~(DiffArchFpRegState)~~ | Physical floating-point registers<br>(Act as DiffArchFpRegState when numPhyRegs = 32) | No | | DiffPhyVecRegState<br>~~(DiffArchVecRegState)~~ | Physical vector registers<br>(Act as DiffArchVecRegState when numPhyRegs = 64) | No | | DiffArchIntRenameTable | Rename table for general-purpose registers<br>(Only instantiated when numPhyRegs > 32) | No | | DiffArchFpRenameTable | Rename table for floating-point registers<br>(Only instantiated when numPhyRegs > 32) | No | | DiffArchVecRenameTable | Rename table for vector registers<br>(Only instantiated when numPhyRegs > 64) | No | | DiffFpCSRState | CSRs for the Floating-point extension | No | | DiffVecCSRState | CSRs for the Vector extension | No | | DiffHCSRState | CSRs for the Hypervisor extension | No | | DiffTriggerCSRState | CSRs for the Trigger support | No | | DiffDebugMode | Debug mode registers | No | | DiffArchIntDelayedUpdate | Delayed general-purpose writeback | No | | DiffArchFpDelayedUpdate | Delayed floating-point writeback | No | | DiffStoreEvent | Store operations | No | | DiffSbufferEvent | Store buffer operations | No | | DiffLoadEvent | Load operations | No | | DiffAtomicEvent | Atomic operations | No | | DiffL1TLBEvent | L1 TLB operations | No | | DiffL2TLBEvent | L2 TLB operations | No | | DiffRefillEvent | Cache refill operations | No | | DiffLrScEvent | Executed LR/SC instructions | No | | DiffNonRegInterruptPengingEvent | Non-register interrupts pending | No | | DiffMhpmeventOverflowEvent | Mhpmevent-register overflow | No | | DiffCriticalErrorEvent | Raise critical-error | No | | DiffSyncAIAEvent | Synchronization of AIA | No | | DiffSyncCustomMflushpwrEvent | custom CSR mflushpwr | No | | DiffUncacheMMStoreEvent | Uncache buffer main memory store operations | No | | DiffCMOInvalEvent | Commit of cbo.inval | No |

The DiffTest framework comes with a simulation framework with some top-level IOs. They will be automatically created when calling DifftestModule.finish(cpu: String).

• LogCtrlIO
• PerfCtrlIO
• UARTIO

These IOs can be used along with the controller wrapper at src/main/scala/common/LogPerfControl.scala.

For compatibility on different platforms, the CPU should access a C++ memory via DPI-C interfaces. This memory will be initialized in C++.

You may also use macro DISABLE_DIFFTEST_RAM_DPIC to remove memory DPI-Cs and use Verilog arrays instead.

val mem = DifftestMem(memByte, 8)
when (wen) {
    mem.write(
    addr = wIdx,
    data = in.w.bits.data.asTypeOf(Vec(DataBytes, UInt(8.W))),
    mask = in.w.bits.strb.asBools
    )
}
val rdata = mem.readAndHold(rIdx, ren).asUInt

To use DiffTest, please include all necessary modules and top-level IOs in your design. It's worth noting the Chisel Bundles may have arguments with default values. Please set the correct parameters for the interfaces.

Plugins

There are several plugins to improve the RTL-simulation and debugging process.

LightSSS: a lightweight simulation snapshot mechanism

After the simulation aborts, we require some debugging information to assist locating the root cause, such as waveform and DUT/REF logs. Traditionally, this requires a second run for the simulation with debugging enabled for the last period of simulation (region of interest, ROI). To avoid such tedious stage, we propose a snapshot mechanism to periodically take snapshots for the simulation process with minor performance overhead. A recent snapshot will be restored after the simulation aborts to reproduce the abortion with debugging information enabled. To understand the technical details of this mecha