Betterleaks

Betterleaks is a tool for detecting secrets like passwords, API keys, and tokens in git repos, files, and whatever else you wanna throw at it via stdin. If you wanna learn more about how the detection engine works check out this blog: Regex is (almost) all you need.

Betterleaks development is supported by <a href="https://www.aikido.dev">Aikido Security</a> <br><a href="https://www.aikido.dev"><img src="docs/aikido_log.svg" alt="Aikido Security" width="80" /></a>

➜  ~/code(master) betterleaks git -v


  ○
  ○
  ●
  ○  Betterleaks v1.0.0

  Finding:     "export BUNDLE_ENTERPRISE__CONTRIBSYS__COM=cafebabe:deadbeef",
  Secret:      cafebabe:deadbeef
  RuleID:      sidekiq-secret
  Entropy:     2.609850
  File:        cmd/generate/config/rules/sidekiq.go
  Line:        23
  Commit:      cd5226711335c68be1e720b318b7bc3135a30eb2
  Author:      John
  Email:       john@users.noreply.github.com
  Date:        2022-08-03T12:31:40Z

Wait wtf this isn't Gitleaks. You're right, it's not but it's built by the same people who maintained Gitleaks and ships with some cool new features.

What's New?

A couple things:

• Secrets Validation (using CEL) — check if a detected secret is live by firing an HTTP request
• Token Efficiency Filter
• Parallelized Git Scanning (--git-workers=8)
• Optimized Recursive Decoding (for catching those nasty SHA1-HULUD variants)
• Misc optimizations
• Regex engine switching w/ (--regex-engine=stdlib/re2 or BETTERLEAKS_REGEX_ENGINE=stdlib)
• MORE RULES! Ahhh finally!

Benchmarks

Scan times compared against gitleaks on real-world repos (lower is better):

What's Coming?

Great question. Check out what we're cookin in the v2 branch.

Installation

# Package managers
brew install betterleaks
brew install betterleaks/tap/betterleaks

# Fedora Linux
sudo dnf install betterleaks

# Containers
docker pull ghcr.io/betterleaks/betterleaks:latest

# Source
git clone https://github.com/betterleaks/betterleaks
cd betterleaks
make betterleaks

Usage

Betterleaks scans code, past or present, for secrets

Usage:
  betterleaks [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  dir         scan directories or files for secrets
  git         scan git repositories for secrets
  help        Help about any command
  stdin       detect secrets from stdin
  version     display betterleaks version

Flags:
  -b, --baseline-path string          path to baseline with issues that can be ignored
  -c, --config string                 config file path
                                      order of precedence:
                                      1. --config/-c
                                      2. env var BETTERLEAKS_CONFIG or GITLEAKS_CONFIG
                                      3. env var BETTERLEAKS_CONFIG_TOML or GITLEAKS_CONFIG_TOML with the file content
                                      4. (target path)/.betterleaks.toml or .gitleaks.toml
                                      If none of the four options are used, then the default config will be used
      --diagnostics string            enable diagnostics (http OR comma-separated list: cpu,mem,trace). cpu=CPU prof, mem=memory prof, trace=exec tracing, http=serve via net/http/pprof
      --diagnostics-dir string        directory to store diagnostics output files when not using http mode (defaults to current directory)
      --enable-rule strings           only enable specific rules by id
      --exit-code int                 exit code when leaks have been encountered (default 1)
      --experiments string            comma-separated list of experimental features to enable (e.g. "validation")
  -i, --gitleaks-ignore-path string   path to .betterleaksignore or .gitleaksignore file or folder containing one (default ".")
  -h, --help                          help for betterleaks
      --ignore-gitleaks-allow         ignore betterleaks:allow and gitleaks:allow comments
  -l, --log-level string              log level (trace, debug, info, warn, error, fatal) (default "info")
      --match-context string          context around match: L (lines), C (columns/characters). e.g. 10L, 100C, -2C,+4C (see Match Context)
      --max-archive-depth int         allow scanning into nested archives up to this depth (default "0", no archive traversal is done)
      --max-decode-depth int          allow recursive decoding up to this depth (default "5")
      --max-target-megabytes int      files larger than this will be skipped
      --no-banner                     suppress banner
      --no-color                      turn off color for verbose output
      --validation                      enable validation of findings against live APIs (default true)
      --validation-status string        comma-separated list of validation statuses to include: valid, invalid, revoked, error, unknown, none (none = rules without validation)
      --validation-timeout duration     per-request timeout for validation (default 10s)
      --validation-extract-empty        include empty values from extractors in output
      --validation-full-response        include full HTTP response body on validated findings
      --redact uint[=100]             redact secrets from logs and stdout. To redact only parts of the secret just apply a percent value from 0..100. For example --redact=20 (default 100%)
  -f, --report-format string          output format (json, csv, junit, sarif, template)
  -r, --report-path string            report file
      --report-template string        template file used to generate the report (implies --report-format=template)
      --timeout int                   set a timeout for betterleaks commands in seconds (default "0", no timeout is set)
  -v, --verbose                       show verbose output from scan
      --version                       version for betterleaks

Use "betterleaks [command] --help" for more information about a command.

Commands

Git

The git command lets you scan local git repos. Under the hood, betterleaks uses the git log -p command to scan patches. You can configure the behavior of git log -p with the log-opts option. For example, if you wanted to run betterleaks on a range of commits you could use the following command: betterleaks git -v --log-opts="--all commitA..commitB" path_to_repo. See the git log documentation for more information. If there is no target specified as a positional argument, then betterleaks will attempt to scan the current working directory as a git repo.

If you want faster git scans you can enable parallelized git log -p with --git-workers={int}.

Dir

The dir (aliases include files, directory) command lets you scan directories and files. Example: betterleaks dir -v path_to_directory_or_file. If there is no target specified as a positional argument, then betterleaks will scan the current working directory.

Stdin

You can also stream data to betterleaks with the stdin command. Example: cat some_file | betterleaks -v stdin

Creating a baseline

When scanning large repositories or repositories with a long history, it can be convenient to use a baseline. When using a baseline, betterleaks will ignore any old findings that are present in the baseline. A baseline can be any betterleaks report. To create a report, run betterleaks with the --report-path parameter.

betterleaks git --report-path betterleaks-report.json # This will save the report in a file called betterleaks-report.json

Once as baseline is created it can be applied when running the detect command again:

betterleaks git --baseline-path betterleaks-report.json --report-path findings.json

After running the detect command with the --baseline-path parameter, report output (findings.json) will only contain new issues.

Load Configuration

Betterleaks supports both betterleaks and gitleaks naming conventions for backwards compatibility. The betterleaks variants take precedence.

The order of precedence is:

1. --config/-c option:

   betterleaks git --config /home/dev/customconfig.toml .
   

2. Environment variable BETTERLEAKS_CONFIG or GITLEAKS_CONFIG with the file path:

   export BETTERLEAKS_CONFIG="/home/dev/customconfig.toml"
   betterleaks git .
   

3. Environment variable BETTERLEAKS_CONFIG_TOML or GITLEAKS_CONFIG_TOML with the file content:

   export BETTERLEAKS_CONFIG_TOML=`cat customconfig.toml`
   betterleaks git .
   

4. A .betterleaks.toml or .gitleaks.toml file within the target path:

   betterleaks git .
   

If none of the four options are used, then the default config will be used.

Configuration

Betterleaks offers a configuration format you can follow to write your own secret detection rules:

# Title for the Betterleaks configuration file.
title = "Custom Betterleaks configuration"

# You have basically two options for your custom configuration:
#
# 1. define your own configuration, default rules do not apply
#
#    use e.g., the default configuration as starting point:
#    https://github.com/betterleaks/betterleaks/blob/master/config/betterleaks.toml
#
# 2. extend a configuration, the rules are overwritten or extended
#
#    When you extend a configuration the extended rules take precedence over the
#    default rules. I.e., if there are duplicate rules in both the extended
#    configuration and the default