StakaterPlatform

Problem Statement

When an individual or a company deploys a kubernetes cluster for workload management. What is the next thing required. Security, Monitoring, Logging, Tracing, Alerting and CI/CD for workloads, a lot of great opensource tools exist for these processes and selecting a tools based on industry's standards and recommendations is quite a hard & duanting task as it requires a lot of research that fulfill the requirements.

Solution

We at Stakater understand the industry's problem and keeping that in mind we have developed a consolidated solution named StakaterPlatform that will help the community to adopt Security, Monitoring, Logging, Tracing, Alerting and CI/CD processes for thier workloads.

Introduction

StakaterPlatform gives a head-start to individuals and companies with a set of seven stacks containing best of the breed opensource tools based on industry's recommendation and best practices for monitoring, logging, security, tracing and alerting of Kubernetes cluster. End-users only need to focus on developing their application while everything else will be managed by StakaterPlatform!

StakaterPlatform uses community managed helm charts and docker images.

For detailed walk-through please visit StakaterPlatform on Stakater Playbook

StakaterPlatform consist of 7 stacks:

• Control
• Delivery
• Logging
• Monitoring
• Security
• Alerting
• Tracing

Table for Stacks tools:

| Control | Delivery | Logging | Monitoring | Security | Alerting | Tracing | |---|---|---|---|---|---|---| | External DNS | Jenkins | Elasticsearch | Prometheus Operator | Keycloak | Uptime Robot | Istio | | Nginx Ingress | Sonatype-Nexus | Fluentd | Metrics Server | Proxy Injector | Slack || | Descheduler | RDLM | Eventrouter ||||| | Forecastle || Kibana ||||| | Reloader || Cerebro ||||| | Sealed Secret Controller || Elasticsearch Curator ||||| | K8S Dashboard || Logrotate ||||| | Gitwebhookproxy || Konfigurator ||||| | Ingress Monitor Controller ||||||| | Xposer |||||||

Prerequisites

• Kubernetes cluster with at least 8 VCPUS & 32 GB of RAM
• A working domain (e.g. stakaterplatform.com )
• SSL Certificate for that domain. Creating & using custom SSL certificates

Tools/Terms

• Flux: You define the entire desired state of your cluster in git and flux ensures that the current state matches the one declared in repo.
• Kubeseal: Required to encrypt base64 encoded secrets to commit-able(git) sealed-secrets

Install from local machine

Video Tutorial

Installation tutorial for local machine can be seen here

Prerequisites

• kubectl (between v1.11 & v1.15.3)
• helm (v2.15.0 or lower)
• kubeseal

1. Duplicate this repository.

2. Update configuration variables in variables.config file and provide the relevant values.

3. [Recommended but optional] To take full advantage of the tool stack configure Additional Variables as well.

4. Ensure that correct context is set for kubectl & helm.

5. run make configure, this will make all required substitutions based on configuration variables in the repository. When prompted commit those changes, don't commit them if you want to run everything from your local machine i.e. No flux hence no GitOps.

6. For deployment there are two options:

   a. Using flux, Add the public SSH key of flux(configs/flux.pub) to your Git repository with write access. For flux run make deploy-flux and provide git credentials on prompt.

   b. Without flux, just don't add flux key and this will remove the process of GitOps(using flux) from your cluster. It removes the need of having your repository available remotely. Run make deploy

7. Estimated time for everything to be up and running is 5-10. Use the printed dashboard token to access the Kubernetes dashboard at dashboard-control.YOURDOMAINNAME

8. Visit https://forecastle-control.YOURDOMAINNAME and you'll be able to view all applications deployed by StakaterPlatform.

Note: Since variables.config file and configs/ directory contains private information those files are not being tracked in git and won't/shouldn't be committed. In case you want to commit those changes run make track-secrets.

Install via GitLab CI Pipeline

1. Duplicate this repository in a GitLab account.
2. Update configuration variables in variables.config file and provide the relevant values.
3. Create a Personal Access Token on GitLab and note down Personal Access Token (Available only when created)
4. Configure CI/CD Environment variables in <Your GitLab Project> -> Settings -> CI/CD.
5. Add Following variables for the pipeline

| Pipeline Variables | Description |
|:---|:---| | KUBE_CONFIG | Base64 encoded KubeConfig of the kubernetes cluster you want to deploy on | | REPO_ACCESS_TOKEN | Personal access token generated in Step 3 | | TARGET | Make file target. Value: pipeline-deploy | | AWS_ACCESS_KEY_ID | (Define only if CLOUD_PROVIDER is aws) AWS Access Key Id. | | AWS_SECRET_ACCESS_KEY | (Define only if CLOUD_PROVIDER is aws) AWS Secret Access Key. |

6. Add the public SSH key(Deploy Keys) to your GitLab account with write access printed at the end of pipeline logs.
7. Once key is added StakaterPlatform will deploy on your cluster (Estimated time: 5-10 minutes). :confetti_ball:
8. Use the printed dashboard token to access the Kubernetes dashboard at dashboard-control.DOMAIN

Flow Diagram for Deployment via GitLab CI Pipeline

<p align="center"> <img src="./images/pipeline-flow.png"> </p>

Verification

Locally

Run make verify to run tests to ensure that all the relevant endpoints are up and running.

GitLab CI

Run pipeline with Pipeline variable: TARGET = verify

StakaterPlatform offers a tool Forecastle to access tools from a single panel. Go to forecastle-control.DOMAIN to access forecastle after complete deployment where you can access all the tools offered by StakaterPlatform and verify if working fine

Default Credentials

See default username and passwords set for tools here

Basic Configuration

| Variables | Description | Default |
|:---|:---|:---:| | CLOUD_PROVIDER | Name of the k8s cloud provider | nil <br> (aws | azure) | | DNS_PROVIDER | Cloud DNS Provider | aws (Route53) | | EXTERNAL_DNS_AWS_ACCESS_KEY_ID | AWS Access Key Id having access to create/delete/update Route53 HostedZone entries | nil | | EXTERNAL_DNS_AWS_SECRET_ACCESS_KEY | AWS Secret Access Key having access to create/delete/update Route53 HostedZone entries | nil | | DOMAIN | Domain to use for StakaterPlatform | nil | | BASE64_ENCODED_SSL_CERTIFICATE_CA_CRT | Base64 encoded Intermediate Certificate value | nil | | BASE64_ENCODED_SSL_CERTIFICATE_TLS_CRT | Base64 encoded Server Certificate value |nil | | BASE64_ENCODED_SSL_CERTIFICATE_TLS_KEY | Base64 encoded Certificate Key value |nil | | STAKATER_PLATFORM_SSH_GIT_URL | SSH URL for your Github repo. | nil<br>(e.g git@github.com/stakater/StakaterPlatform.git. Notice : is replaced with / in the URL ) | | STAKATER_PLATFORM_BRANCH | Branch to use for STAKATER_PLATFORM_SSH_GIT_URL | master |

For generating certificates view: [Creating & using custom SSL certificates](https://pl