GitHub Secret Scanner Auto Remediator (GSSAR)

Welcome to the GSSAR Product! :wave:

Table of Contents

• Overview
• How this works
  • Non-Technical
  • Technical
• Design
  • Design Overview
  • Technologies Used
• Pre-Reqs
• Initial Installation
  • Step One: Create IAM User
  • Step Two: Create and Configure GitHub App
  • Step Three: Create Parameters within AWS Systems Manager (Parameter Store)
  • Step Four: Deployment into AWS
  • Step Five: Update GitHub App to send webhooks to the URL output from Step Five
• Configuring Remediators
• Issues and Feedback
• Contributing
• FAQs

Overview

GSSAR is an open-source initiative helping teams automatically revoke secrets discovered by GitHub's Secret Scanning tool.

Right now, whenever a new secret is discovered, you are notified straight away within the Secret Scanning tab. You can then make a conscious decision over whether to revoke the secret or mark it as a false positive, etc.

With some secrets, it's okay not to take immediate action and only revoke after the right people have reviewed. However, for some other types of secrets (such as AWS Access Keys), more of an immediate (and automated) approach may be required in revoking these secrets. GSSAR is an initiative to solve that use case. :rotating_light:

GitHub Secret Scanner Auto Remediator (GSSAR) is an initiative that revokes certain secret types automatically. GSSAR takes an agnostic approach to the kinds of secrets and leaves it up to each GitHub organization to decide what secret types should and should not be automatically revoked.

How this works

Non-Technical

Whenever a secret is discovered within a repository, a webhook is triggered, and a process kicks off. That process starts by collecting information about that secret. It then looks within the webhook to see what type of secret it has found (e.g. AWS, Azure, GCP, Dropbox, etc.). Then, if the secret type is one you have a remediator for, the secret will automatically be revoked. You will then be notified via a GitHub Issue on the repository where the secret has been found. If the secret type is one you do not have a remediator for, the process will finish.

Technical

Whenever a new secret alert is opened, resolved or reopened, a webhook will fire from a GitHub App and be sent to an API Gateway within AWS. The API's first step is passing the context of the payload to a Lambda Authorizer, ensuring the Webhook has come from GitHub. If valid, the API will pass the event payload (from the webhook) to a Lambda to confirm the secret within the webhook is valid and expected. After both authentication methods pass, an AWS State Machine is kicked off to revoke the secret defined within the alert.

The state machine firstly goes and filters out the resolved and reopened events. The state machine will automatically finish for any secret alert action that != created. If the action type does === created, it kicks off the first lambda, which fetches the value of the secret found. The secret itself doesn't come as part of the webhook payload, so we have to make an API call to get the value.

Once the value has been collected, we then kick off a Choice. This choice will look at the alert.secret_type and, based on the value, will decide to pass it onto a remediator or exit the state machine if the secret type isn't defined within the state machine.

If the secret within the alert.secret_type matches a secret type within the state machine, it will send the payload to a remediator. A remediator is a function that revokes a secret, custom to a type of secret. Once revoked, it will send the result back to the state machine, then mark that secret as closed within GitHub. Finally, once all steps are complete, it will run a final step which notifies people the secret has been remediated. It informs by opening an issue on the repository where the secret was leaked.

Design

Design Overview

The design has been implemented in a way to be very plug and play. Where you, as an organization, only have to focus on building remediators. The architecture can be found below:

This solution is designed in a way where the remediators are the only part you need to add/remove.

What are remediators? As mentioned above, remediators are functions that revoke specific secret types. There should be a 1:1 mapping between secret type and remediator.

For example, if a company wanted to remediate the following secret types automatically:

• Dropbox Access Token
• Amazon AWS Access Key ID
• Google Cloud Private Key ID

There would need to be three remediators. One for Dropbox Access Tokens, one for Amazon AWS Access Keys and one for Google Cloud Private Keys.

The rest of the state machine (outside of remediators) is secret type agnostic. Meaning it will work for any current and future secret types. It is worth understanding how the rest of the design works, but you do not directly need to edit ot change anything outside of the remediators.

Technologies Used

The following technologies are used throughout this solution:

• AWS
  • Lambda is used for compute power.
  • Cloud Formation is used as our IaC (Infrastructure as Code).
  • HTTP API Gateway is used for ingress into AWS.
  • Cloud Watch is used for logging and monitoring.
  • IAM is used to connect resources and allow deployments into AWS from GitHub Actions
  • S3 is used by AWS SAM to deploy the stack, and therefore deploy it into the AWS ecosystem using Cloud Formation.
  • AWS Systems Manager Parameter Store is used to store parameters.
  • AWS Step Functions is used to co-ordinate the end-to-end process
• GitHub
  • GitHub App is used as our egress method out of GitHub.
  • GitHub Actions is used to deploy the solution into AWS.

AWS SAM is used for the lambda & HTTP API Gateway resources.

Note: Even though this solution is deployed to AWS, the code can be changed to work with the likes of Azure and GCP (Azure Function, Google Functions, etc.).

Pre-Req's

1. Access to a cloud environment (AWS would be the quickest to get started)
2. Access to a GitHub environment.
3. A repository where the code for this solution is going to live.

Initial Installation

The below steps show the path of least resistance way of deploying this solution into AWS. There are many different ways to deploy this. Every organization likely has different processes (especially with deploying into AWS), meaning you may have to pivot during these steps to accommodate organization-specific processes. This is okay. Please treat these instructions as an example and reference; if they work end-to-end, great; if not, please adjust to your company policies (and if needed, contribute back!).

If you get an error you cannot get around, please log an issue on this repository.

Step One: Create IAM User

Create an IAM User. The IAM User will need to have the capability to do the following:

• CRUD access over S3 Resources.
• CRUD access over IAM Resources.
• CRUD access over API Gateway Resources.
• CRUD access over Lambda Resources.
• CRUD access over CloudWatch Resources.
• CRUD access over Step Functions

From that user, create an AWS Access key and secret. Once you have both, create a GitHub Enviroment called main and within that environment create two secrets AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY with the relevant information from AWS in. Set the environment to only deploy from the main branch. (This can be changed later at any time).

NOTE: If your organization doesn't allow the use of IAM Users, this isn't a problem. We use the official configure-aws-credentials GitHub action. Meaning you can head to the .github/workflows/deploy.yaml file and swap out the AWS User method to assuming an AWS Role. Or, if you have a custom GitHub Action which authenticates into AWS, remove the configure-AWS-credentials action and swap it out for your custom one.

Step Two: Create and Configure GitHub App

Create a GitHub Application. You will need to be an administrator of your GitHub organization to do this.