Octocov

<p align="center"> <img src="https://github.com/k1LoW/octocov/raw/main/docs/logo.png" width="200" alt="octocov"> </p>

octocov is a toolkit for collecting code metrics (code coverage, code to test ratio, test execution time and your own custom metrics).

Key features of octocov are:

• Useful both as a CI tool and as a CLI tool
• Support multiple coverage report formats.
• Support multiple code metrics.
• Support for even generating coverage report badges.
• Have a mechanism to aggregate reports from multiple repositories.

Getting Started

On GitHub Actions

:octocat: GitHub Actions for octocov is here !!

First, run test with coverage report output.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

And generete .octocov.yml to your repository.

$ octocov init
.octocov.yml is generated

And set up a workflow file as follows and run octocov on GitHub Actions.

# .github/workflows/ci.yml
name: Test

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      -
        uses: actions/checkout@v3
      -
        uses: actions/setup-go@v4
        with:
          go-version-file: go.mod
      -
        name: Run tests with coverage report output
        run: go test ./... -coverprofile=coverage.out
      -
        uses: k1LoW/octocov-action@v1

Then, octocov comment the report of the code metrics to the pull request.

It is also possible to add reports to GitHub Actions Job Summaries by editing .octocov.yml.

It can also be inserted into the body of a pull request.

Note that only pull requests from the same repository can be commented on (Reporting to GitHub Actions Job Summaries is permitted). This is because the workflow token of a forked pull request does not have write permission.

On Terminal

octocov acts as a code metrics viewer on the terminal.

For example, in case of Go language, add -coverprofile=coverage.out option as follows

$ go test ./... -coverprofile=coverage.out

And run octocov ls-files , octocov view [FILE...] and octocov diff [REPORT_A] [REPORT_B]

Usage example

Comment report to pull request

By setting comment:, comment the reports to pull request.

# .octocov.yml
comment:
  hideFooterLink: false # hide octocov link

octocov checks for "Code Coverage" by default. If it is running on GitHub Actions, it will also measure "Test Execution Time".

If you want to measure "Code to Test Ratio", set codeToTestRatio:.

comment:
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'

By setting report: ( report.path: or report.datastores ) and diff: ( diff.path: or diff.datastores ) additionally, it is possible to show differences from previous reports as well.

comment:
  updatePrevious: true
report:
  datastores:
    - artifact://${GITHUB_REPOSITORY}
diff:
  datastores:
    - artifact://${GITHUB_REPOSITORY}

Check for acceptable score

By setting coverage.acceptable:, the condition of acceptable coverage is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
coverage:
  acceptable: 60%

$ octocov
Error: code coverage is 54.9%. the condition in the `coverage.acceptable:` section is not met (`60%`)

By setting codeToTestRatio.acceptable:, the condition of acceptable "Code to Test Ratio" is specified.

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
codeToTestRatio:
  acceptable: 1:1.2
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'

$ octocov
Error: code to test ratio is 1:1.1, the condition in the `codeToTestRatio.acceptable:` section is not met (`1:1.2`)

By setting testExecutionTime.acceptable:, the condition of acceptable "Test Execution Time" is specified (on GitHub Actions only) .

If this condition is not met, the command will exit with exit status 1.

# .octocov.yml
testExecutionTime:
  acceptable: 1 min

$ octocov
Error: test execution time is 1m15s, the condition in the `testExecutionTime.acceptable:` section is not met (`1 min`)

Generate report badges self.

By setting *.badge.path:, generate badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg

# .octocov.yml
codeToTestRatio:
  badge:
    path: docs/ratio.svg

# .octocov.yml
testExecutionTime:
  badge:
    path: docs/time.svg

You can display the coverage badge without external communication by setting a link to this badge image in README.md, etc.

# mytool

![coverage](docs/coverage.svg) ![coverage](docs/ratio.svg) ![coverage](docs/time.svg)

Push report badges self.

By setting push:, git push report badges self.

# .octocov.yml
coverage:
  badge:
    path: docs/coverage.svg
push:

Store report to datastores

By setting report:, store the reports to datastores and local path.

# .octocov.yml
report:
  datastores:
    - github://owner/coverages/reports
    - s3://bucket/reports

# .octocov.yml
report:
  path: path/to/report.json

Supported datastores

• GitHub repository
• GitHub Actions Artifacts
• Amazon S3
• Google Cloud Storage (GCS)
• BigQuery
• Local

Central mode

By enabling central:, octocov acts as a central repository for collecting reports ( example ).

# .octocov.yml for central mode
central:
  root: .                                  # root directory or index file path of collected coverage reports pages. default: .
  reports:
    datastores:
      - bq://my-project/my-dataset/reports # datastore paths (URLs) where reports are stored. default: local://reports
  badges:
    datastores:
      - local://badges                     # directory where badges are generated.
  push:                                    # enable self git push

Merging reports from multiple GitHub Action jobs

This example illustrates how to merge coverage reports from multiple jobs. It is useful when you are testing code branches for multiple targets (e.g. test sharding, runtime version, dependency version, OS, etc).

# .github/workflows/ci.yml
name: merge coverages from multiple jobs

on:
  push:
    branches:
      - main
      - master
  pull_request:

jobs:
  test:
    strategy:
      fail-fast: true
      matrix:
        shard: [1, 2, 3]
    runs-on: ubuntu-latest
    permissions:
      contents: read # to checkout
    steps:
      - uses: actions/checkout@v4

      - run: YOUR TEST HERE

      - name: Upload coverage to workspace
        uses: actions/upload-artifact@v4
        with:
          path: lcov.info # adjust this path to your coverage file
          name: octocov-${{ matrix.shard }}
          if-no-files-found: error

  coverage-aggregation:
    runs-on: ubuntu-latest
    permissions:
      contents: read # to checkout
      pull-requests: write # for octocov to post comments
      actions: read # for octocov to parse test execution time
    needs:
      - test
    steps:
      - uses: actions/checkout@v4

      - uses: actions/download-artifact@v5
        with:
          pattern: octocov-*

      - uses: k1LoW/octocov-action@v1

# .octocov.yml
codeToTestRatio:
  code:
    - '**/*.go'
    - '!**/*_test.go'
  test:
    - '**/*_test.go'
coverage:
  paths:
    # Name of artifacts uploaded by the test-running jobs.
    # lcov.info (or equivalent) will be downloaded in the directory named after the below.
    - "octocov-1"
    - "octocov-2"
    - "octocov-3"
comment:
  if: is_pull_request
  hideFooterLink: true
  deletePrevious: true
summary:
  if: true
report:
  if: is_default_branch
  datastores:
    - artifact://${GITHUB_REPOSITORY}
diff:
  datastores:
    - artifact://${GITHUB_REPOSITORY}

Supported datastores

• GitHub repository
• GitHub Actions Artifacts
• Amazon S3
• Google Cloud Storage (GCS)
• BigQuery
• Local

View code coverage report of file

octocov ls-files command can be used to list files logged in code coverage report.

octocov view (alias: octocov cat) command can be used to view the file coverage report.

Configuration

repository:

The name of the repository.

It should be in the format owner/repo.

By default, the value of the environment variable GITHUB_REPOSITORY is set.

In case of monorepo, code metrics can be reported to datastore separately by specifying owner/repo/project-a or owner/repo@project-a.

repository: k1LoW/octocov

timeout:

Timeout for octocov