This product is no longer actively maintained, but thank you for all those who have used it! We have archived the repo to provide clear guidance on current expectations.

Getting Started • Architecture • Contributing

Multi-Tenancy Extension For Kubernetes

Accounts & Account Users to separate tenants in a shared Kubernetes cluster
Self-Service Namespace Provisioning for account users
Account Limits to ensure quality of service and fairness when sharing a cluster
Namespace Templates for secure tenant isolation and self-service namespace initialization
Multi-Cluster Tenant Management for sharing a pool of clusters (coming soon)

kiosk Demo Video

Why kiosk?
Architecture
- Workflow & Interactions
- Custom Resources & Resource Groups
Getting Started
Upgrade kiosk
Uninstall kiosk
Extra: User Management & Authentication
Contributing

Why kiosk?

Kubernetes is designed as a single-tenant platform, which makes it hard for cluster admins to host multiple tenants in a single Kubernetes cluster. However, sharing a cluster has many advantages, e.g. more efficient resource utilization, less admin/configuration effort or easier sharing of cluster-internal resources among different tenants.

While there are hundreds of ways of setting up multi-tenant Kubernetes clusters and many Kubernetes distributions provide their own tenancy logic, there is no lightweight, pluggable and customizable solution that allows admins to easily add multi-tenancy capabilities to any standard Kubernetes cluster.

The Missing Multi-Tenancy Extension for Kubernetes

kiosk is designed to be:

100% Open-Source: CNCF compatible Apache 2.0 license
Pluggable: easy to install into any existing cluster and suitable for different use cases
Fast: emphasizing automation and self-service for tenants
Secure: offering default configurations for different levels of tenant isolation
Extensible: providing building blocks for higher-level Kubernetes platforms

Architecture

The core idea of kiosk is to use Kubernetes namespaces as isolated workspaces where tenant applications can run isolated from each other. To minimize admin overhead, cluster admins are supposed to configure kiosk which then becomes a self-service system for provisioning Kubernetes namespaces for tenants.

Workflow & Interactions

The following diagram shows the main actors (Cluster Admins and Account Users) as well as the most relevant Kubernetes resources and their relationships.

kiosk Workflow

Click on the following links to view the description for each of the actors and kiosk components:

<details> <summary>Cluster Admin</summary>

Cluster Admins have the permission to perform CRUD operations for cluster-wide / non-namespaced resources (especially RBAC related resources as well as the custom resources Account, AccountQuota, AccountQuotaSet, and Template). Cluster Admins configure kiosk by creating and managing Accounts, AccountQuotas, AccountQuotaSets, and Templates. They can also see and configure all Spaces owned by all Accounts.

</details> <details> <summary>Account</summary>

Every tenant is represented by an Account. Cluster Admins define and manage Accounts and assign Account Users (Users, Groups, ServiceAccounts) to Accounts - similar to assigning RBAC Roles to subjects as part of a RoleBinding configuration.

</details> <details> <summary>Account User</summary>

Account Users perform actions within the Kubernetes cluster via API server requests while using a certain Account. Cluster Admins can assign the same Account User to multiple Accounts. Account Users have access to Spaces that belong to the Accounts they are using. If assigned the default kiosk ClusterRole, every Account User has the permission to list/get/create/delete Spaces for the respective Account, however, this can be changed via RBAC RoleBindings.

</details> <details> <summary>Space</summary>

A Space is a non-persistent, virtual resource that represents exactly one Kubernetes namespace. Spaces have the following characteristics:

Every space can belong up to one Account which is the owner of this Space. Ownerless Spaces are possible.
If a user has rights to access the underlying Namespace, the user can access the Space in the same way. Hence besides Account Users, other actors (User, Group, ServiceAccount) can also access the Space if someone grants this access via additional Kubernetes RBAC.
Every User only sees the Spaces the User has access to. This is in contrast to regular namespaces, where Users can only list all namespaces or none
Space ownership can be changed, by changing the ownership annotation on the namespace
During Space creation (or Space ownership changes) a RoleBinding for the owning Account is created in the corresponding Space namespace. The referenced RBAC ClusterRole can be configured in the account
A Space can be prepopulated during creation with a predefined set of resources by configuring default Templates in the Account. Kiosk will make sure that these resources will be correctly deployed before the user gets access to the namespace.

</details> <details> <summary>Namespace</summary>

A Namespace is a regular Kubernetes Namespace that can be accessed by anyone who has the appropriate RBAC rules to do so. Namespaces are provisioned and managed by kiosk and have a 1-to-1 relationship to the resource Space which is a custom resource of kiosk. By default, Account Users have the permission to operate within all Namespaces that are represented by Spaces which belong to one of their Accounts.

</details> <details> <summary>Template</summary>

Templates are defined and managed by Cluster Admins. Templates are used to initialize Spaces/Namespaces with a set of Kubernetes resources (defined as manifests or as part of a Helm chart). Templates can be created using a different ClusterRole than the Account User uses, so they can be used to create resources that are not allowed to be created by actors of the Space/Namespace, e.g. to set up certain isolation resources (e.g. Network Policies, Pod Security Policies etc.). Cluster Admins can define default Templates within the Account configuration which automatically applies these templates to each Space that is created using the respective Account. Additionally, Account Users can state other non-mandatory Templates that should also be applied when creating a Space.

</details> <details> <summary>TemplateInstance</summary>

When a Template is applied to a Space, kiosk creates a TemplateInstance to keep track of which Templates have been applied to the Space. A TemplateInstance contains information about the Template as well as about the parameters used to instantiate it. Additionally, TemplateInstances can be configured to sync with Templates, i.e. the TemplateInstance will update the resources whenever the Template changes that has been used to create these resources.

</details> <details> <summary>AccountQuota</summary>

AccountQuotas are defined and managed by Cluster Admins. AccountQuotas define cluster-wide aggregated limits for Accounts. The resources of all Spaces/Namespaces that belong to an Account count towards the aggregated limits defined in the AccountQuota. Similar to Namespaces which can be limited by multiple ResourceQuotas, an Account can be limited by multiple AccountQuotas. If the same limit (e.g. total CPU per Account) is defined by multiple AccountQuotas, the Account will be limited according to the lowest value.

</details>

Custom Resources & Resource Groups

When installing kiosk in a Kubernetes cluster, these components will be added to the cluster:

CRDs for Account, AccountQuota, AccountQuotaSet, Template, TemplateInstance
Controller for kiosk Custom Resources (runs inside the cluster)
API Server Extension (runs inside the cluster similar to the Controller)

kiosk Data Structure

kiosk adds two groups of resources to extend the Standard API Groups of Kubernetes:

Custom Resources: config.kiosk.sh
Custom Resource Definitions (CRDs) for configuring kiosk. These resources are persisted in etcd just like any other Kubernetes resources and are managed by an operator which runs inside the cluster.
<details> <summary>Show List of Custom Resources</summary> 
- config.kiosk.sh/Account

Kiosk

Install / Use

README