PEBIL (pebil) - README 2.2 Binary Instrumentation Packages for Gathering Computational Trace The PMaC Prediction Framework relies on binary instrumentation to gather information about the computational work done during an application run. The information in a computational trace is the floating point operation count, memory operation count, and simulated cache hit rates (stored per basic block for each level of the caches in the memory hierarchy) of the target machine(s). Each basic block in an application is given a unique ID that is assigned to it in a consistent manner during each instrumentation pass. The unique ID for every basic block in the application is consistent as long as the executable is kept the same . The simulated cache hit rates are computed on-the-fly during the trace by inputing the address stream of the instrumented running application into a cache simulator. The specifications for the cache or caches to be simulated are specified in the CacheStructures.h and describe in more detail in section 2.2.1.3. To gather computation traces, PMaC provides two instrumentation tools. PEBIL is a binary editing tool for X86/Linux systems (described in this document) and PMaCInstrumentor is a binary editing tool for IBM Power/AIX systems. 2.2.1 PEBIL PEBIL is a binary instrumentation toolkit that operates on ELF binaries on Linux for x86/x86_64 processors. PEBIL has a C++ API that provides the means to inject code and data into a binary file. It also provides several pre-implemented tools for basic block counting and cache simulation for a set of memory hierarchies. 2.2.1.1 Installation for Application Tracing Host Systems: Any X86/Linux The source code for the PEBIL library and instrumentation tools is available at through PMaC Laboratories by e-mailing help@pmaclabs.com. The directory structure of distribution is as follows: $ cd /home/userid/tools/PEBIL $ ls bin/ include/ instcode/ Makefile scripts/ testapps/ tools/ docs/ etc/ lib/ src/ The include, src, tools, and instcode directories contain the source code for instrumentation related libraries and instrumentation tools. The bin and lib directories contain the executables and shared libraries after building the distribution. The scripts directory contains additional PERL scripts to assist application tracing for high level instrumentation activities. After retrieving the source, users can change to the top directory of the distribution and follow the instructions in the INSTALL to build the distribution. After building the executables and shared libraries, it is also advised to link or copy the scripts in the scripts directory to the bin directory of the distribution so that the scripts will be found in user's path as well. The following is an example of this procedure: $ cd /home/userid/tools/PEBIL $ gmake clean $ gmake $ cd bin $ ln -s ../scripts/selectSimBlocks.pl selectSimBlocks.pl $ ln -s ../scripts/traceHelper.pl traceHelper.pl The Makefile at the top directory /home/userid/tools/PEBIL iterate over sub directories and run the appropriate make commands. Alternatively, users can follow the same steps manually. Instead of gmake at the top directory, the users can run the following. Note that the order of making src should come before the order of making tools directory. For an installed distribution to work, minimal set of steps is to make src, tools and instcode directories. $ cd /home/userid/tools/PEBIL $ cd ./src $ gmake $ cd ../tools $ gmake $ cd ../instcode $ gmake Manually following the make steps is particularly useful when the default configuration (configuration that comes with the source code) is not used for building the instcode directory. There are various ways of building the code under this directory and the available compilation macros are described in details in Section 2.2.1.2. To use PEBIL, users need to include the bin directory of distribution to their path. Thus, if a particular distribution needs to be used, the user needs to set the path variables accordingly before using that distribution as the name of the executable and additional PERL scripts would be the same. When installed properly, the PEBIL executable, pebil, should be in user's path. It can be checked using the “which” command of Unix from a different directory (you may need to source .cshrc/.tcshrc or .profile/.bashrc files for paths to be adjusted). The --help option will print a brief usage information. $ source ~/.cshrc $ which pebil /home/userid/tools/PEBIL/bin/pebil $ pebil --help usage : pebil --typ (ide|dat|bbt|cnt|jbb|sim|csc) --app <executable_path> --inp <block_unique_ids> <-- valid for sim/csc [--lib <shared_lib_topdir>] [--ext <output_suffix>] [--dtl] [--lpi] <-- valid for sim/csc [--phs <phase_no>] <-- valid for sim/csc [--dfp <pattern_file>] <-- valid for sim/csc [--help] Brief Descriptions for Options:

    --typ : required for all. Instrumentation type.
    --app : required for all. Executable path.
    --lib : optional for all. Instrumentation shared library top
            directory. default is $PEBIL_LIB
    --ext : optional for all. File extension for output files.
            default is (typ)inst, such as jbbinst for type jbb.
    --dtl : optional for all. Enable detailed .static file with line numbers
            and filenames. default is no details.
    --inp : required for sim/csc. File including list of block ids to instrument
    --lpi : optional for sim/csc. Loop level block inclusion for
            cache simulation. default is no.
    --phs : optional for sim/csc. Phase number. defaults to no phase,
            otherwise, .phase.N. is included in output file names
    --dfp : optional for sim/csc. dfpattern file. defaults to no dfpattern file,

2.2.1.2 Installation for MultiMAPS Tracing Host Systems: Any X86/Linux In the framework, the MultiMAPS data for a target system needs to be augmented with the cooresponding hit rates. To accomplish this, MultiMAPS needs to be traced with the target system’s cache structure. The default PEBIL installation, which comes configured for application tracing, uses address stream sampling for cache simulation. Rather than using every address in the address stream of an application run, it uses only a subset of addresses in the address stream and simulates them for the target memory subsystems. This method of sampling the address stream has been used to reduce the overhead of instrumentation. For MultiMAPS tracing, however, the entire address stream needs to be consumed for cache simulations, therefore sampling should not be used. Hence, separate installations of PEBIL are needed for tracing MultiMAPS for cache hit rates. To install PEBIL for MultiMAPS tracing, you need to edit the instcode/Makefile file under the PEBIL installation directory (a different directory than the one installed for application tracing) and repeat the installation process described in Section 2.2.1.1 for this new directory. To install pebil for MultiMAPS tracing, user needs to replace the EXTENDED_SAMPLING compilation macro in instcode/Makefile with the NO_SAMPLING_MODE compilation macro. Assuming the source code for PEBIL for MultiMAPS tracing is under /home/userid/mmaps user needs to; $ cd /home/userid/mmaps/PEBIL $ vi instcode/Makefile

change extended sampling mode to no sampling mode

#EXTRA_CFLGS = -DEXTENDED_SAMPLING -DPER_SET_RECENT -DVICTIM_CACHE EXTRA_CFLGS = -DNO_SAMPLING_MODE -DPER_SET_RECENT -DVICTIM_CACHE $ gmake 2.2.1.3 Using a Different Set of Memory Hierarchies (Caches) The PEBIL source is distributed with a set of memory hierarchies to be used in cache simulation runs. However, it also is flexible enough to easily allow a different set of memory hierarchies with the installation. PEBIL provides a PERL script under the scripts directory of the distribution, scripts/generateCaches.pl to do this. This script takes an input file of memory hierarchy specifications for a set of systems and generates a C header file that contains these same specifications as C code. The user can also specify the stride system to use from one of the caches given in the input file if stride information will be collected. Otherwise, the default action of the script is to add a 2MB direct cache that is used for gathering stride information. Information on how to use the generateCaches.pl script can be seen by using the --help flag. The following example shows the --help flag: $ /home/userid /PEBIL/scripts/generateCaches.pl --help usage : scripts/generateCaches.pl [--sys-file <cache_desc_file>] <-- defaults to CacheDescriptions.txt [--out-file <outfile>] <-- defaults to CacheStructures.h [--stride-sys <system idx>] <-- defaults to 0, 2 MB direct cache [--nostd] ? static allocation of cache structures and use of safe lib functions [--puniq] ? print unique cache information. [--help] [--readme] A sample input to generateCaches.pl script is given in the text box below. Everything after “#” sign is assumed to be a comment and is ignored. Each line in the input file defines a memory hierarchy by listing the sysid, number of cache levels and the specifications of each cache. The sysid needs to be greater than 0 and unique. It is used by the prediction database to deferentiate different systems and cache structures. Each cache is defined by 4 attributes: the cache size (which can be given in bytes or in KB or MBs), the associativity, the line size in bytes and the replacement policy. The replacement policy can be lru, lru_vc, dir or ran where lru is the LRU pseudo implementation, lru_vc is the LRU pseudo implementation for victim caches, dir is the direct address