---

name: whizard-notification description: Use when working with WizTelemetry Notification extension for KubeSphere, including installation, configuration, troubleshooting, notification channel setup, alert routing, and silence management.

WizTelemetry Notification

Overview

WizTelemetry Notification is the notification component of the KubeSphere observability platform. It receives alerts, cloud events, and audit logs in a multi-tenant Kubernetes environment and distributes notifications to different channels based on tenant labels (e.g., namespace).

Architecture

┌──────────────────────────────────────────────────────────────────────────────┐
│                             Alertmanager                                     │
│      Prometheus  ──▶ metrics alert  ──▶ notification-manager-svc:19093       │
│      K8s Events  ──▶ events alert   ──▶ notification-manager-svc:19093       │
│      Auditing    ──▶ auditing alert ──▶ notification-manager-svc:19093       │
│      Logging     ──▶ logging alert  ──▶ notification-manager-svc:19093       │
└──────────────────────────────────────────────────────────────────────────────┘
                                    │
                    notification-manager (:19093)
                                    │
         ┌──────────────────────────┼──────────────────────────┐
         │                          │                          │
    [Silence Stage]          [Route Stage]             [Filter Stage]
         │                          │                          │
         ▼                          ▼                          ▼
 [Match Silence CRs]         [Match Router CRs]      [alertSelector filtering]
                                 │
                          [Aggregation Stage]
                                 │
                          [Notify Stage]
                                 │
         ┌────────────────────────┼────────────────────────────┐
         │                        │                            │
    Email/Slack              DingTalk/Feishu            WeChat/Webhook
         │
         ▼
  [History Webhook]
         │
         ▼
┌─────────────────────┐
│  Vector Aggregator  │
└────────┬────────────┘
         │
         │
         ▼
     OpenSearch

Components

| Component | Description | Default | |-----------|-------------|---------| | notification-manager | Core notification management, receives alerts, dispatches to channels | true | | alertmanager (v0.27.0) | Alert routing, deduplication, aggregation, silencing | true | | alertmanager-proxy | Alertmanager API proxy, NodePort 31093 | true | | notification-history | Stores sent notifications via Vector to OpenSearch | true |

Dependencies

| Dependency | Type | Required | |------------|------|----------| | whizard-telemetry | extension | Required | | vector (WizTelemetry Data Pipeline) | extension | Required only notification history is enabled | | opensearch | extension | Required only notification history is enabled |

CRDs

All CRDs belong to notification.kubesphere.io/v2beta2, scope is Cluster.

| CRD | Kind | Description | |-----|------|-------------| | notificationmanagers.notification.kubesphere.io | NotificationManager | Notification manager deployment spec (image, replicas, resources, sidecar, receiver config) | | receivers.notification.kubesphere.io | Receiver | Tenant-level notification receiver, references Config | | configs.notification.kubesphere.io | Config | Credential references (via Secret selectors), independent per channel | | routers.notification.kubesphere.io | Router | Route alerts to receivers based on label matchers | | silences.notification.kubesphere.io | Silence | Time-bounded silencing, supports matchers |

Supported Notification Channels (11 channels)

| Channel | Receiver Field | Config Field | Description | |---------|---------------|--------------|-------------| | Email | spec.email | spec.email | SMTP with TLS/STARTTLS support | | Slack | spec.slack | spec.slack | Slack webhook + token | | DingTalk | spec.dingtalk | spec.dingtalk | ChatBot + group conversation | | WeChat | spec.wechat | spec.wechat | ChatBot + group conversation | | Feishu | spec.feishu | spec.feishu | ChatBot + group conversation | | Webhook | spec.webhook | spec.webhook | Generic HTTP / HTTPS POST |

Config → Receiver Association Mechanism

Config (credentials) --labels--> Receiver (via label selector) --labels--> Router (route matching)
                        │
                        └── namespace for multi-tenant isolation

Config's labels are used by Receiver's *ConfigSelector. Receiver's labels are used by Router's receiverSelector.

Installation

Prerequisites

REQUIRED: Complete all steps in order before generating InstallPlan.

Step 1: Get Latest Version (if not provided by user)

kubectl get extensionversions -l kubesphere.io/extension-ref=whizard-notification -o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V | tail -1

This outputs the latest version (e.g., 2.7.0). Note this down.

Step 2: Create InstallPlan

CRITICAL:

• metadata.name MUST be whizard-notification
• config field is YAML format
• DO NOT add fields not shown in the template
• Replace all placeholders with actual values

Basic installation:

apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-notification
spec:
  extension:
    name: whizard-notification
    version: <VERSION>  # From Step 1
  enabled: true
  upgradeStrategy: Manual

Disable notification history:

apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-notification
spec:
  extension:
    name: whizard-notification
    version: <VERSION>
  enabled: true
  upgradeStrategy: Manual
  config: |
    notification-history:
      enabled: false

Custom index format (monthly):

notification-history:
  sinks:
    opensearch:
      index:
        prefix: "{{ .cluster }}-notification-history"
        timestring: "%Y.%m"

Configuration Parameters

Notification History

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | notification-history.enabled | bool | true | Enable notification history storage | | notification-history.sinks.opensearch.enabled | bool | true | Enable OpenSearch sink | | notification-history.sinks.opensearch.index.prefix | string | {{ .cluster }}-notification-history | Index prefix, supports template | | notification-history.sinks.opensearch.index.timestring | string | %Y.%m.%d | strftime format | | notification-history.ism_policy.enable | bool | false | Enable ISM policy | | notification-history.ism_policy.min_index_age | string | 7d | Minimum index retention |

Alertmanager

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | alertmanager.replicaCount | int | 1 | Replica count | | alertmanager.replicaAutoSet | bool | true | Auto set replicas by node count (<3=1, >=3=3) | | alertmanager.service.nodePort | int | 31093 | NodePort (alertmanager-proxy) | | alertmanager.config.group_wait | string | 30s | Alert aggregation wait time | | alertmanager.config.repeat_interval | string | 12h | Repeat alert interval | | alertmanager.config.route.receiver | string | Default | Default receiver | | alertmanager.config.inhibit_rules | list | see below | Inhibition rules (critical→warning→info) |

Default inhibit_rules:

• critical alerts silence warning and info with same labels
• warning alerts silence info with same labels
• InfoInhibitor silences all info level alerts

Notification Manager

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | notification-manager.notificationmanager.sidecar.tenant.resources.limits.cpu | string | 200m | Sidecar CPU limit | | notification-manager.notificationmanager.sidecar.tenant.resources.limits.memory | string | 512Mi | Sidecar memory limit | | notification-manager.notificationmanager.sidecar.tenant.resources.requests.cpu | string | 200m | Sidecar CPU request | | notification-manager.notificationmanager.sidecar.tenant.resources.requests.memory | string | 256Mi | Sidecar memory request | | notification-manager.operator.containers.operator.resources.limits.cpu | string | 50m | notification manager operator CPU limit | | notification-manager.operator.containers.operator.resources.limits.memory | string | 50Mi | notification manager operator memory limit | | notification-manager.operator.containers.operator.resources.requests.cpu | string | 5m | notification manager operator CPU request | | notification-manager.operator.containers.operator.resources.requests.memory | string | 20Mi | notification manager operator memory request | | notification-manager.notificationmanager.resources.limits.cpu | string | 500m | notification manager CPU limit | | notification-manager.notificationmanager.resources.limits.memory | string | 500Mi | notification manager memory limit | | notification-manager.notificationmanager.resources.requests.cpu | string | 5m | notification manager CPU request | | notification-manager.notificationmanager.resources.requests.memory | string | 20Mi | notification manager memory request | | notification-manager.notificationmanager.replicas | int | 1 | notification manager replicas |

Alertmanager-Proxy

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | alertmanager-proxy.service.nodePort | int | 31093 | NodePort | | alertmanager-proxy.resources.limits.cpu | string | 500m | CPU limit | | alertmanager-proxy.resources.limits.memory | string | 500Mi | Memory limit | | alertmanager-proxy.resources.requests.cpu | string | 100m | CPU request | | alertmanager-proxy.resources.requests.memory | string | 100Mi | Memory request |

Notification API

All APIs are at /kapis/notification.kubesphere.io/v2beta2/, routed through whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80. No authentication token is required.

API Patterns

The API supports two levels of resources:

| Level | URL Pattern | Resources | |-------|-------------|-----------| | Global | /{resources} | notificationmanagers, configs, receivers, routers, silences, secrets, configmaps | | Tenant | /users/{user}/{resources} | configs, receivers, silences, secrets, configmaps (NOT notificationmanagers or routers) |

Supported HTTP methods: GET (list/detail), POST (create), PUT (update), PATCH (patch), DELETE (delete).

Query parameters for list endpoints: name, labelSelector, type, page, limit, ascending, orderBy.

List Global Receivers

curl -X GET "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/notification.kubesphere.io/v2beta2/receivers" \

List Global Receivers (filtered by type)

curl -X GET "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/notification.kubesphere.io/v2beta2/receivers?type=email" \

List Tenant Receivers

curl -X GET "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/notification.kubesphere.io/v2beta2/users/admin/receivers" \

Create Receiver — Email (Global)

curl -X POST "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/notification.kubesphere.io/v2beta2/receivers" \
  -H "Content-Type: application/json" \
  -d '{
    "apiVersion": "notification.kubesphere.io/v2beta2",
    "kind": "Receiver",
    "metadata": {
      "name": "email-receiver",
      "labels": {
        "type": "default"
      }
    },
    "spec": {
      "email": {
        "emailConfigSelector": {
          "matchLabels": { "type": "default" }
        },