Monocle

Monocle is designed for development teams to provide:

analytics on project changes
boards to display changes by criteria

It helps teams and individual to better organize daily duties and to detect anomalies in the way changes are produced and reviewed.

Monocle supports GitHub, GitLab and Gerrit.

How to get started with Monocle:

Explore the website and blog: changemetrics.io
Try on the demo instance: demo.changemetrics.io
Chat with us in the project Matrix room: #monocle:matrix.org
Ask and Discuss about Monocle on the Discussions page
Run your own instance: Read the installation guide
Hack on it: Read the contributing guide
Checkout who use Monocle

Screenshots of Monocle from the demo instance

The activity view:

The developer board:

The peers strength view:

Installation

The process below describes how to index changes from GitHub repositories and then how to start the web UI to browse metrics.

The deployment is based on Docker via a Docker compose definition. This compose definition can also be used by Podman compose, then all docker-compose commands from this documentation could be replaced by podman-compose.

Alternatively, the following deployment method are supported:

Checkout the code

git clone https://github.com/change-metrics/monocle.git
cd monocle
# Init a .secrets file with a default API key for the crawler process
echo CRAWLERS_API_KEY=$(uuidgen) > .secrets

Create the config.yaml file

The config.yaml file is used by the crawler and api services.

To crawl GitHub public repositories, you must generate a personal access token on GitHub (w/o any specific rights) at https://github.com/settings/tokens.

Then create the config file etc/config.yaml. Here is an example your could start with. Make sure to write GITHUB_TOKEN=<github_token> in the .secrets file:

---
workspaces:
  - name: monocle
    crawlers:
      - name: github-tektoncd
        provider:
          github_organization: tektoncd
          github_repositories:
            - operator
            - pipeline
        update_since: '2021-01-01'

To crawl the full tektoncd GitHub organization then remove the github_repositories entry from the file. Check the section Workspaces for a complete description of the configuration.

Start docker-compose

The docker-compose file is set to use the container image of the latest stable release of Monocle (1.11.2). It is adviced to use the latest stable Monocle version. However, as our CI publishes the latest (devel) container image then it is possible to run the very last version. To do so set COMPOSE_MONOCLE_VERSION to latest in the .env file.

Please refer to the Environment variables section.

Start Monocle:

docker-compose up -d

Ensure services are running and healthy:

docker-compose ps

Inspect services logs:

docker-compose logs -f

You should be able to access the web UI at http://localhost:8080.

See Troubleshooting section if needed.

Configuration

The Monocle configuration is splitted between a configuration file and various environment variables.

Environment variables configure system settings and secrets. Any changes require a restart of the Monocle API.

The configuration file contains non sensitive data and could be exposed in a Git repository. A CI/CD flow could be enabled on top of the Git repository to allow Monocle' users to propose configuration changes. The Monocle API automatically reload the configuration when it changes.

Environment variables

For a local deployment, default settings are fine.

The following settings are available in the .env file (See .env.sample for a sample file):

COMPOSE_ES_XMS and COMPOSE_ES_XMX to change the ElasticSearch JVM HEAP SIZE. By default 512m.
COMPOSE_MONOCLE_VERSION to use a specific version. By default it uses latest stable release version.
COMPOSE_MONOCLE_API_ADDR to set binding address where the Monocle API is exposed by the container.
COMPOSE_MONOCLE_API_PORT to set binding port where the Monocle API is exposed by the container.
COMPOSE_MONOCLE_PUBLIC_URL to configure the public URL to access the UI and API. The URL is required for user redirection during the authentication. By default it is http://localhost:8080
COMPOSE_MONOCLE_WEBAPP_TITLE to change the title of the web application. By default it is Monocle.

The following settings are available in the .secrets file:

MONOCLE_JWK_GEN_KEY to set the local JWT issuer key. The key size must be 64 characters minimum. By default the key is automatically generated.
MONOCLE_OIDC_<PROVIDER_NAME>_CLIENT_SECRET to set the secret used by Monocle to request an ID Token (Unset by default).
MONOCLE_ADMIN_TOKEN to set the token to access admin endpoints. By default not set and endpoints deactivated.

Configuration file

The Monocle configuration file is used by the API and crawlers processes. The format of the file is YAML.

The file configures the following:

Monocle workspaces
The About WEB Application endpoint
The Authentication system

You might want to use Dhall to manage it or to better understand the schema (dhall-monocle).

Workspaces

A workspace uses a dedicated ElasticSearch index. A workspace defines:

crawlers - details
projects - details
identities - details
groups - details
search aliases - details

Crawlers

Monocle provides two kinds of crawlers:

Change: A crawler to fetch Changes proposed to a repository. Monocle supports Gerrit (Reviews), GitHub (Pull-Requests), GitLab (Merge-Requests).
TaskData: A crawler to fetch task data related to a repository. Monocle supports GitHub (issues), and BugZilla (Bugs).

The .secrets file is used to store credentials and API keys used by crawlers to authenticate on providers.

The crawlers value is a list of crawler. Each crawler is composed of:

name: an abitrary name used to identify the crawler.
update_since: the crawler will fetch changes that has been created/updated since that date.
provider: provider settings

workspaces:
  - name: demo
    crawlers:
      - name: spinnaker
        update_since: "2020-01-01"
        provider: {}

Change

A GitHub provider settings

  provider:
    github_organization: spinnaker
    # Optional settings
    github_repositories:
      - pipeline
    github_url: https://github.com/api/graphql
    github_token: GITHUB_TOKEN

github_organization is the only mandatory key. If github_repositories is not specified then the crawler will crawl the whole organization repositories. If specified then it will crawl only the listed repositories. To crawl repositories from a personnal GitHub account, you need to set github_organization to the account name and list repositories under the github_repositories key.

github_url might be specified in case of an alternate url. Default is "https://github.com/api/graphql".

github_token might be specified to use an alternate environment variable name to look for the token value. Default is "GITHUB_TOKEN"

Regarding the Github tokens (classic):

To crawl public repositories, no specific scope are needed (no checkbox selected).
To crawl privates repositories, you must set the "repo" scope.

Regarding the Github fine grained tokens (new):

To crawl public repositories, keep the "Public Repositories (read-only)" checkbox selected. Nothing else is needed.
To crawl privates repositories, select "All repositories" or "Only select repositories", then in "Repository permissions" select "Pull Requests", "Contents" as "Read-only".

The GitHub provider can also be configured to crawl Pull-Requests created by specific GitHub users. For instance the following crawler's provider will fetch Pull-Requests and related events for users john and jane:

  provider:
    github_users:
      - john
      - jane
    # Optional settings
    github_url: https://github.com/api/graphql
    github_token: GITHUB_TOKEN

A Gerrit provider settings

  provider:
    gerrit_url: https://review.opendev.org
    gerrit_repositories:
      - openstack/nova
      - openstack/neutron
    # Optional settings
    gerrit_login: monocle
    gerrit_password: GERRIT_PASSWORD
    gerrit_prefix: opendev/

gerrit_url is mandatory and must be the url of the Gerrit provider. gerrit_repositories is mandatory and is the list of repositories from which the crawler will fetch Reviews from.

gerrit_login might be specified to authenticate on the provider API. gerrit_passwordmight be specified to use an alternate environment variable name to look for the password. Default is "GERRIT_PASSWOR

Monocle

Install / Use

README

Monocle

Screenshots of Monocle from the demo instance

Installation

Checkout the code

Create the config.yaml file

Start docker-compose

Configuration

Environment variables

Configuration file

Workspaces

Crawlers

Change