Multi-Cloud and Multi-Cluster Declarative Kubernetes Cluster Creation

The Cluster API (CAPI) is a Kubernetes project that brings declarative, Kubernetes-style APIs to cluster creation. CAPI does this by using Custom Resource Definitions to extend the API exposed by the Kubernetes API Server, allowing users to create new resources such as Clusters (representing a Kubernetes cluster) and Machines (representing the machines that make up the Nodes that form the cluster). A Controller for each resource is then responsible for reacting to changes to these resources to bring up the cluster. The API is designed in such a way that different infrastructure providers can integrate with it to provide their environment specific logic.

A user communicates with a Kubernetes cluster referred as “Management Cluster”, that contains the Cluster-API machinery and other supporting components. The user can request entire clusters to be created and manage their lifecycle by creating cluster resource objects associated with the cloud providers they want to use and the created clusters are referred as “Workload Clusters”. Management Cluster is also where one or more Infrastructure Providers (like AWS, CGP, Azure, vSphere, Openstack etc.) and Bootstrap Providers (as of now Kubeadm is officially supported, users can implement their custom bootstrap provider using operator framework) run. Information of all cloud provider objects created persists in Management cluster.

Image for post

CAPI — Management Cluster and Workload Clusters

The project maintains multiple infrastructure providers to use CAPI on multitude of cloud providers. Users can implement custom providers by creating controllers and reconciliation using Kubebuilder (operator build framework). Cluster Controller and Machine Controller together form provider controller manager. The machine spec is consumed by the Cluster API component — machine controller. This controller calls machine actuator which is responsible for creation / updation / deletion of machines. Actuators (reconciliation logic) are provider specific.

Image for post

CAPi — Cluster Controller and Machine Controller

Bootstrap providers are used for bootstrapping (installing k8s) the Kubernetes control plane nodes and worker nodes once the machines and other components are created by provider. As of now Kubeadm is the officially supported and maintained bootstrap provider.

Basic Cluster API is the framework used for all providers. It is maintained independently. Cluster API provider implements operation details of different infrastructures for Cluster API framework. With provider, Cluster API abstracts overall management functions of a cluster and its machines out of operation details of different 3rd party infrastructure vendors.

Architecture and Components

Image for post

CAPI — Architecture

Kubernetes manages different resources by objects and controllers. A controller typically manages a resource to make its objects actual state to match the desired state supplied by specification.

The Cluster API consists of a shared set of controllers in order to provide a consistent user experience. In order to support multiple cloud environments, some of these controllers (e.g. Cluster, Machine) call provider specific actuators to implement the behavior expected of the controller. Other controllers (e.g. MachineSet) are generic and operate by interacting with other Cluster API (and Kubernetes) resources. The task of the provider implementor is to implement the actuator methods so that the shared controllers can call those methods when they must reconcile state and update status. When an actuator function returns an error, if it is RequeueAfterError then the object will be requeued for further processing after the given RequeueAfter time has passed.

CAPI comprise multiple controllers and the core components are as follows:


Cluster provides a way to define common Kubernetes cluster configurations, such as pod network CIDR and service network CIDR, as well as provider-specific management of shared cluster infrastructure. Cluster contains the details required by the infrastructure provider to create a Kubernetes cluster (e.g. CIDR blocks for pods, services).

Image for post

CAPI — Cluster


Machine is responsible for describing an individual Kubernetes node. There is only minimal configuration exposed at the common configuration level (mainly Kubernetes version information), and additional configuration is exposed through the embedded ProviderSpec.

Image for post

CAPI — Machine

Machine Deployments

MachineDeployments allow for the managed deployment and rollout of configuration changes to groups of Machines (node-groups), very much like how traditional k8s Deployments work. This allows for rolling out updates to a node configuration or even rolling out version upgrades for worker nodes in the cluster in an orchestrated manner. It also allows for rolling back to the previous configuration. It is important to note that MachineDeployments should not be used for managing Machines that make up the Kubernetes Control Plane for a given managed Cluster, since they do not provide any assurances that the control plane remains healthy when the MachineDeployment is updated or scaled.

Image for post

#kubernetes #azure #cluster-api #aws

Multi-Cloud and Multi-Cluster Declarative Kubernetes Cluster Creation