[cargo-]flamegraph

A Rust-powered flamegraph generator with additional support for Cargo projects! It can be used to profile anything, not just Rust projects! No perl or pipes required <3

Built on top of @jonhoo's wonderful Inferno all-rust flamegraph generation library!

[!TIP] You might want to also try samply, which provides a more interactive UI using a seamless integration with Firefox's Profiler web UI. It is also written in Rust and has better macOS support.

Quick Start

Install it, and run

# Rust projects
cargo flamegraph

# Arbitrary binaries
flamegraph -- /path/to/binary

How to use flamegraphs: what's a flamegraph, and how can I use it to guide systems performance work?

Installation

[cargo-]flamegraph supports

• Linux: relies on perf
• MacOS: relies on xctrace
• Windows: native support with the blondie library; also works with dtrace on Windows

cargo install flamegraph will make the flamegraph and cargo-flamegraph binaries available in your cargo binary directory. On most systems this is something like ~/.cargo/bin.

Linux

Note: If you're using lld or mold on Linux, you must use the --no-rosegment flag. Otherwise perf will not be able to generate accurate stack traces (explanation). For example, for lld:

[target.x86_64-unknown-linux-gnu]
linker = "/usr/bin/clang"
rustflags = ["-Clink-arg=-fuse-ld=lld", "-Clink-arg=-Wl,--no-rosegment"]

and for mold:

[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-Clink-arg=-fuse-ld=/usr/local/bin/mold", "-Clink-arg=-Wl,--no-rosegment"]

Debian (x86 and aarch)

Note: Debian bullseye (the current stable version as of 2022) packages an outdated version of Rust which does not meet flamegraph's requirements. You should use rustup to install an up-to-date version of Rust, or upgrade to Debian bookworm (the current testing version) or newer.

sudo apt install -y linux-perf

Ubuntu (x86)

sudo apt install linux-tools-common linux-tools-generic linux-tools-`uname -r`

Ubuntu (aarch)

sudo apt install linux-tools-common linux-tools-generic linux-tools-`uname -r`

Note: The perf binary is not packaged for all kernel versions. A check and workaround is below.

# Check if the perf binary is missing:
ls -l /usr/lib/linux-tools/`uname -r`/ | grep perf

# If it is there, you can stop here.
# If is missing, check if you have the tools for another kernel version installed:
ls -l /usr/lib/linux-tools/

# If you do, check it contains perf:
# (replace <FROM_KERNEL>)
ls -l /usr/lib/linux-tools/<FROM_KERNEL>/

# If you do, symlink it. This has been tested with 6.14.0-1014-aws running perf from 6.8.0-85-generic
# (replace <FROM_KERNEL> as before)
sudo ln -s /usr/lib/linux-tools/<FROM_KERNEL>/perf /usr/lib/linux-tools/`uname -r`/perf

Ubuntu/Ubuntu MATE (Raspberry Pi)

sudo apt install linux-tools-raspi

Pop!_OS

sudo apt install linux-tools-common linux-tools-generic

Windows

Blondie Backend

This is enabled by default. Windows is supported out-of-the-box, thanks to Nicolas Abram's excellent blondie library.

DTrace on Windows

Alternatively, one can install DTrace on Windows. If found, flamegraph will always prefer using dtrace over the built-in Windows support.

Shell auto-completion

At the moment, only flamegraph supports auto-completion. Supported shells are bash, fish, zsh, powershell and elvish. cargo-flamegraph does not support auto-completion because it is not as straight-forward to implement for custom cargo subcommands. See #153 for details.

How you enable auto-completion depends on your shell, e.g.

flamegraph --completions bash > $XDG_CONFIG_HOME/bash_completion # or /etc/bash_completion.d/

Examples

# if you'd like to profile an arbitrary executable:
flamegraph [-o my_flamegraph.svg] -- /path/to/my/binary --my-arg 5

# or if the executable is already running, you can provide the PID via `-p` (or `--pid`) flag:
flamegraph [-o my_flamegraph.svg] --pid 1337

# NOTE: By default, perf tries to compute which functions are
# inlined at every stack frame for every sample. This can take
# a very long time (see https://github.com/flamegraph-rs/flamegraph/issues/74).
# If you don't want this, you can pass --no-inline to flamegraph:
flamegraph --no-inline [-o my_flamegraph.svg] /path/to/my/binary --my-arg 5

# cargo support provided through the cargo-flamegraph binary!
# defaults to profiling cargo run --release
cargo flamegraph

# by default, `--release` profile is used,
# but you can override this:
cargo flamegraph --dev

# if you'd like to profile a specific binary:
cargo flamegraph --bin=stress2

# if you want to pass arguments as you would with cargo run:
cargo flamegraph -- my-command --my-arg my-value -m -f

# if you want to use interesting perf or dtrace options, use `-c`
# this is handy for correlating things like branch-misses, cache-misses,
# or anything else available via `perf list` or dtrace for your system
cargo flamegraph -c "record -e branch-misses -c 100 --call-graph lbr -g"

# Run criterion benchmark
# Note that the last --bench is required for `criterion 0.3` to run in benchmark mode, instead of test mode.
cargo flamegraph --bench some_benchmark --features some_features -- --bench

cargo flamegraph --example some_example --features some_features

# Profile unit tests.
# Note that a separating `--` is necessary if `--unit-test` is the last flag.
cargo flamegraph --unit-test -- test::in::package::with::single::crate
cargo flamegraph --unit-test crate_name -- test::in::package::with::multiple:crate
cargo flamegraph --unit-test --dev test::may::omit::separator::if::unit::test::flag::not::last::flag

# Profile integration tests.
cargo flamegraph --test test_name

Usage

flamegraph is quite simple. cargo-flamegraph is more sophisticated:

Usage: cargo flamegraph [OPTIONS] [-- <TRAILING_ARGUMENTS>...]

Arguments:
  [TRAILING_ARGUMENTS]...  Trailing arguments passed to the binary being profiled

Options:
      --dev                            Build with the dev profile
      --profile <PROFILE>              Build with the specified profile
  -p, --package <PACKAGE>              package with the binary to run
  -b, --bin <BIN>                      Binary to run
      --example <EXAMPLE>              Example to run
      --test <TEST>                    Test binary to run (currently profiles the test harness and all tests in the binary)
      --unit-test [<UNIT_TEST>]        Crate target to unit test, <unit-test> may be omitted if crate only has one target (currently profiles the test harness and all tests in the binary; test selection can be passed as trailing arguments after `--` as separator)
      --bench <BENCH>                  Benchmark to run
      --manifest-path <MANIFEST_PATH>  Path to Cargo.toml
  -f, --features <FEATURES>            Build features to enable
      --no-default-features            Disable default features
  -r, --release                        No-op. For compatibility with `cargo run --release`
  -v, --verbose                        Print extra output to help debug problems
  -o, --output <OUTPUT>                Output file [default: flamegraph.svg]
      --open                           Open the output .svg file with default program
      --root                           Run with root privileges (using `sudo`)
  -F, --freq <FREQUENCY>               Sampling frequency in Hz [default: 997]
  -c, --cmd <CUSTOM_CMD>               Custom command for invoking perf/dtrace
      --deterministic                  Colors are selected such that the color of a function does not change between runs
  -i, --inverted                       Plot the flame graph up-side-down
      --reverse                        Generate stack-reversed flame graph
      --notes <STRING>                 Set embedded notes in SVG
      --min-width <FLOAT>              Omit functions smaller than <FLOAT> pixels [default: 0.01]
      --image-width <IMAGE_WIDTH>      Image width in pixels
      --palette <PALETTE>              Color palette [possible values: hot, mem, io, red, green, blue, aqua, yellow, purple, orange, wakeup, java, perl, js, rust]
      --skip-after <FUNCTION>          Cut off stack frames below <FUNCTION>; may be repeated
      --flamechart                     Produce a flame chart (sort by time, do not merge stacks)
      --ignore-status                  Ignores perf's exit code
      --no-inline                      Disable inlining for perf script because of performance issues
      --post-process <POST_PROCESS>    Run a command to process the folded stacks, taking the input from stdin and outputting to stdout
  -h, --help                           Print help
  -V, --version                        Print version

Then open the resulting flamegraph.svg with a browser, because most image viewers do not support interactive svg-files.

Enabling perf for use by unprivileged users

To enable perf without running as root, you may lower the perf_event_paranoid value in proc to an appropriate level for your environment. The most permissive value is -1 but may not be acceptable for your security needs etc...

echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid

Improving output when running with --release

Due to optimiz