<div align="center">

This is the official Kubewatch project, originally by Bitnami, now maintained by Robusta.dev.

Feel free to open issues, raise PRs or talk with us on Slack!

kubewatch is a Kubernetes watcher that publishes notification to available collaboration hubs/notification channels. Run it in your k8s cluster, and you will get event notifications through webhooks.

See the blog post on KubeWatch 2.0 to learn more about how KubeWatch is used.

<img src="./docs/kubewatch-logo.jpeg">

</div>

Fine-Grained Resource Tracking

KubeWatch tracks all changes to Kubernetes resources of given types.

If you need fine-grained filtering of which changes are forwarded, you can use KubeWatch in conjunction with Robusta.

For more details, refer to the Robusta docs on change-tracking.

A diagram showing the two options is shown below. The flow that only uses KubeWatch is purple and the flow that adds Robusta is green.

Latest image

robustadev/kubewatch:v2.9.0

Usage

$ kubewatch -h

Kubewatch: A watcher for Kubernetes

kubewatch is a Kubernetes watcher that publishes notifications
to Slack/hipchat/mattermost/flock channels. It watches the cluster
for resource changes and notifies them through webhooks.

supported webhooks:
 - slack
 - slackwebhook
 - msteams
 - hipchat
 - mattermost
 - flock
 - webhook
 - cloudevent
 - smtp

Usage:
  kubewatch [flags]
  kubewatch [command]

Available Commands:
  config      modify kubewatch configuration
  resource    manage resources to be watched
  version     print version

Flags:
  -h, --help   help for kubewatch

Use "kubewatch [command] --help" for more information about a command.

Install

Cluster Installation

Using helm:

When you have helm installed in your cluster, use the following setup:

helm repo add robusta https://robusta-charts.storage.googleapis.com && helm repo update
helm install kubewatch robusta/kubewatch --set='rbac.create=true,slack.channel=#YOUR_CHANNEL,slack.token=xoxb-YOUR_TOKEN,resourcesToWatch.pod=true,resourcesToWatch.daemonset=true'

You may also provide a values file instead:

rbac:
  create: true
  customRoles:
    - apiGroups: ["monitoring.coreos.com"]
      resources: ["prometheusrules"]
      verbs: ["get", "list", "watch"]
resourcesToWatch:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: true
  pod: true
  job: false
  node: false
  clusterrole: true
  clusterrolebinding: true
  serviceaccount: true
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
  coreevent: false
  event: true
customresources:
  - group: monitoring.coreos.com
    version: v1
    resource: prometheusrules
slack:
  channel: '#YOUR_CHANNEL'
  token: 'xoxb-YOUR_TOKEN'

And use that:

$ helm upgrade --install kubewatch robusta/kubewatch --values=values-file.yml

Using kubectl:

In order to run kubewatch in a Kubernetes cluster quickly, the easiest way is for you to create a ConfigMap to hold kubewatch configuration.

An example is provided at kubewatch-configmap.yaml, do not forget to update your own slack channel and token parameters. Alternatively, you could use secrets.

Create k8s configmap:

$ kubectl create -f kubewatch-configmap.yaml

Create the Pod directly, or create your own deployment:

$ kubectl create -f kubewatch.yaml

A kubewatch container will be created along with kubectl sidecar container in order to reach the API server.

Once the Pod is running, you will start seeing Kubernetes events in your configured Slack channel. Here is a screenshot:

To modify what notifications you get, update the kubewatch ConfigMap and turn on and off (true/false) resources or configure any resource of your choosing with customresources (CRDs):

resource:
  deployment: false
  replicationcontroller: false
  replicaset: false
  daemonset: false
  services: true
  pod: true
  job: false
  node: false
  clusterrole: false
  clusterrolebinding: false
  serviceaccount: false
  persistentvolume: false
  namespace: false
  secret: false
  configmap: false
  ingress: false
  coreevent: false
  event: true
customresources:
  - group: monitoring.coreos.com
    version: v1
    resource: prometheusrules

Working with RBAC

Kubernetes Engine clusters running versions 1.6 or higher introduced Role-Based Access Control (RBAC). We can create ServiceAccount for it to work with RBAC.

$ kubectl create -f kubewatch-service-account.yaml

If you do not have permission to create it, you need to become an admin first. For example, in GKE you would run:

$ kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=REPLACE_EMAIL_HERE

Edit kubewatch.yaml, and create a new field under spec with serviceAccountName: kubewatch, you can achieve this by running:

$ sed -i '/spec:/a\ \ serviceAccountName: kubewatch' kubewatch.yaml

Then just create pod as usual with:

$ kubectl create -f kubewatch.yaml

Working with CRDs

kubewatch can be configured to monitor Kubernetes Custom Resource Definitions (CRDs), allowing you to receive notifications when changes occur. To configure kubewatch to watch custom resources, you need to define the customresources section either in your values file or by using the --set flag with Helm commands.

Include the custom resource configuration in your values file:

customresources:
  - group: monitoring.coreos.com
    version: v1
    resource: prometheusrules

Then deploy or upgrade kubwatch with helm upgrade or helm install

Alternatively, you can pass this configuration directly using the --set flag:

helm install kubewatch robusta/kubewatch --set='rbac.create=true,slack.channel=#YOUR_CHANNEL,slack.token=xoxb-YOUR_TOKEN,resourcesToWatch.pod=true,resourcesToWatch.daemonset=true,customresources[0].group=monitoring.coreos.com,customresources[0].version=v1,customresources[0].resource=prometheusrules'

Custom RBAC roles

After defining custom resources, make sure that kubewatch has the necessary RBAC permissions to access the custom resources you've configured. Without the appropriate permissions, kubewatch will not be able to monitor your custom resources, and you won't receive notifications for changes.

To grant these permissions, you can define custom RBAC roles using customRoles within the rbac section of your values file or by using the --set flag with Helm commands. This allows you to specify exactly which API groups, resources, and actions kubewatch should have access to.

Here’s how you can configure the necessary permissions to monitor your resources:

rbac:
  create: true 
  customRoles:
    - apiGroups: ["monitoring.coreos.com"]
      resources: ["prometheusrules"]
      verbs: ["get", "list", "watch"]

Then deploy or upgrade kubwatch with helm upgrade or helm install

Alternatively, you can pass this configuration directly using the --set flag:

helm install kubewatch robusta/kubewatch --set='rbac.create=true,slack.channel=#YOUR_CHANNEL,slack.token=xoxb-YOUR_TOKEN,customRoles[0].apiGroups={monitoring.coreos.com},customRoles[0].resources={prometheusrules},customRoles[0].verbs={get,list,watch}'

Metrics

kubewatch runs a Prometheus metrics endpoint at /metrics on port 2112 by default. This endpoint can be used to monitor health and the performance of kubewatch.

The kubewatch_events_total metric can help track the total number of Kubernetes events, categorized by resource type (e.g., Pods, Deployments) and event type (e.g., Create, Delete).

You can change the default port (2112) on which the metrics server listens by setting the LISTEN_ADDRESS environment variable. Format is host:port. :5454 means any host, and port 5454

extraEnvVars:
  - name: LISTEN_ADDRESS
    value: ":5454"

Local Installation

Using go package installer:

# Download and install kubewatch
$ go get -u github.com/robusta-dev/kubewatch

# Configure the notification channel
$ kubewatch config add slack --channel <slack_channel> --token <slack_token>

# Add resources to be watched
$ kubewatch resource add --po --svc
INFO[0000] resource svc configured
INFO[0000] resource po configured

# start kubewatch server
$ kubewatch
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-service
INFO[0000] Starting kubewatch controller                 pkg=kubewatch-pod
INFO[0000] Processing add to service: default/kubernetes  pkg=kubewatch-service
INFO[0000] Processing add to service: kube-system/tiller-deploy  pkg=kubewatch-service
INFO[0000] Processing add to pod: kube-system/tiller-deploy-69ffbf64bc-h8zxm  pkg=kubewatch-pod
INFO[0000] Kubewatch controller synced and ready         pkg=kubewatch-service
INFO[0000] Kubewatch