Simutrace

Simutrace is a tracing framework for full system simulators and has been conceived with full length, no-loss tracing of high-frequency events such as executed CPU instructions and main memory references in mind.

The framework places no restriction on the type and number of captured events and employs a scalable storage format, which easily handles traces of hundreds of gigabytes in size. Simutrace has been particularly extended to facilitate efficient memory tracing by incorporating aggressive, but fast memory trace compression.

Simutrace is a research project of the operating systems group at Karlsruhe Institute of Technology (KIT).

Table of Contents

• Architecture
• Getting Started
  • Supported Platforms
  • Installation
  • Building from Source
  • Starting Simutrace
  • Writing and Reading Traces
• Documentation
  • Building the Documentation
• Bugs and Feature Requests
  • Known Issues
• Versioning
• Contributing
• Copyright and Licenses
• 3rd Party Components
• Authors

Architecture

Simutrace uses a client-server architecture, where a client such as an extension in a full system simulator collects events (e.g., memory accesses) and a storage server receives the data, performs trace compression and storage as well as provides easy access to the data for later analysis and inspection. The server can be situated on the same as well as on a remote host.

When connecting to the server, the client is assigned a session and may create a new or open an existing trace data store. Within a store, trace events are organized into streams. Each stream stores events of a single type (e.g., memory writes). A client may create an arbitrary number of streams using built-in or custom event types. Writing and reading trace data with Simutrace thus follows common streaming semantics.

For a thorough introduction to Simutrace and its capabilities please refer to our white paper Efficient Full System Memory Tracing with Simutrace.

Binary Files

Simutrace consists of four libraries and one executable:

<b>simustore(.exe)</b>: The server executable, containing all trace processing and storage functionality.

<b>libsimutrace(.so/.dll)</b>: This is the client library offering the public C interface to Simutrace. This library needs to be used by all applications that want to use Simutrace. See http://simutrace.org/documentation for a documentation of the API.

<b>libsimutraceX(.so/.dll)</b>: This is the client extensions library, which provides additions on top of the standard Simutrace API such as a stream multiplexer and helper functions.

<b>libsimubase, libsimustore</b>: These are static libraries that encapsulate common code for the client library and the storage server. You won't see these as build output as they are statically linked into the client library and storage server binary, respectively.

Getting Started

In this section, we first describe the supported platforms, prerequisites and the environment for building Simutrace from source. You may download pre-compiled binaries at http://simutrace.org/downloads. The section continues by giving a short introduction on how to start Simutrace.

Supported Platforms

Simutrace supports Windows Vista+, Linux, and MacOSX 10.9+. Since trace processing is a memory intensive task, we only support 64 bit operating system versions. We tested Simutrace on x86-64 hardware, only.

For best performance, we recommend running the server on a system with at least 8 CPU cores (4+4) and 12 GiB of RAM. The server very well operates a 24 core dual-socket Xeon system at full load when reading a memory trace with 700 MiB/s. The hardware requirements for the client are significantly lower as all heavyweight processing is done in the server process. The client runs perfectly fine on a single core with 1 GiB of RAM. When running the client and server on the same machine, the memory of the client is shared with the server and does not add to the server's requirements.

Installation

This section describes the installation of one of the pre-compiled versions of Simutrace and its prerequisites. If you cannot find a version of Simutrace on http://simutrace.org/downloads suitable to your environment (e.g., OS), you can build Simutrace from source.

Note: The installation does not currently include extensions for a full system simulators to generate trace events from a simulation. For information on simulator extensions, please refer to http://simutrace.org.

Step 1: Visual C++ Redistributable Package 2015 (Windows-only)

The Visual C++ Redistributable package installs run-time components that are required to run Simutrace. Download the package from Microsoft and follow the instructions given on the download page.

<i>If you have Visual Studio 2015 installed, the required components are already on your system.</i>

Step 2: Simutrace

Download the latest version of Simutrace from http://simutrace.org/downloads. For Windows we provide MSI installation packages. For rpm-based and deb-based Linux distributions you can add repositories to your packaging software from Launchpad and openSUSE Build Service. For all other supported platforms, we currently do not provide binary packages. You have to build from source for these systems.

Building from Source

If you want to make changes to Simutrace or want to run Simutrace on a platform that is not officially supported, you can build the software from source.

Step 1: Setting up the Build Environment

Simutrace uses CMake (2.8 or newer) to generate build files for the development tools available on your system.

On <b>Windows</b>, download the latest CMake version from http://cmake.org. To build Simutrace, we use the latest version of Microsoft Visual Studio (Visual C++). A free (express) version can be downloaded from Microsoft.

On <b>Linux</b>, you have to install cmake, the basic development tools such a C++11 compatible GCC (version 4.7 or higher) and make. Some distributions provide a package that will install the required tools. On Ubuntu Linux 14.04 LTS you can enter:

$ sudo apt-get install cmake build-essential

Step 2: Cloning the Source Repository

Before you can clone the source repository you need to install git. On <b>Windows</b> you may install it from https://msysgit.github.io/. On <b>Linux</b> you will most probably find a binary package in your system's package management system. On Ubuntu 14.04 LTS you can enter:

$ sudo apt-get install git

After installing git, you clone the source repository by entering:

$ git clone https://github.com/simutrace/simutrace.git

The repository encompasses the server, the C client interface library to write simulator extensions and analysis clients, sample code, documentation sources, and more.

The repository has the following structure:

<pre> . |-- <b>3rdParty</b> | `-- libconfig-1.4.9 <i>libconfig to parse configuration files</i> | |-- <b>samples</b> | |-- csharp.memreplay <i>Memory replay sample in c#</i> | |-- csharp.simple <i>Sample on how to use C# binding</i> | |-- storemon <i>Small monitoring app, which displays live stats</i> | | <i>on the streams in a specified store</i> | |-- simple <i>Very basic sample client in C</i> | `-- parallel <i>Sample client in C, which writes and reads</i> | <i>to/from a trace using multiple threads</i> |-- <b>setups</b> <i>Setup and packaging projects and metadata</i> | `-- <b>simutrace</b> |-- bindings <i>Bindings for Simutrace</i> |-- documentation <i>Documentation source and theme.</i> |-- <b>include</b> <i>Header files, use SimuTrace.h.</i> | |-- simubase <i>Internal headers for libsimubase.</i> | `-- simustor <i>Internal headers for libsimustor.</i> |-- libsimubase <i>Internal platform abstraction library.</i> |-- libsimustor <i>Internal tracing core library.</i> |-- <b>libsimutrace</b> <i>Public client library.</i> |-- <b>libsimutraceX</b> <i>Public client extensions library.</i> `-- <b>storageserver</b> <i>Trace storage and access server.</i> `-- simtrace <i>simtrace (*.sim) storage format provider</i> | <i>and type-specific trace encoders/decoders.</i> `-- VPC4 <i>Prediction-based memory trace compressor.</i> </pre>

Step 3: Building

Before you can build the source, you have to generate the build files with CMake. If you are using <b>Windows</b> you will most probably want to generate a Visual Studio solution. On <b>Linux</b>, you can generate makefiles with cmake and build Simutrace by running the following commands from the root directory of the source tree:

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make

The final executables will be located in <i>bin/</i> and can be installed using:

$ make install

Debug Version

To build the debug version of Simutrace you have to choose the appropriate solution configuration in Visual Studio (<i>Debug</i>) or on Linux generate the makefiles with:

$ cmake -DCMAKE_BUILD_TYPE=Debug ..

Starting Simutrace

Simutrace uses a client-server architecture. Starting Simutrace thus constitutes starting the server on the one side and starti