Introduction

Kubernetes excels at managing containerized applications. When you deploy a service, you declare desired state in a manifest: replica counts, container images, resource limits, and network ports. You submit that manifest to the API server, and controllers reconcile reality. If a pod crashes, a controller replaces it. If a node fails, the scheduler moves pods elsewhere. You declare target state once, and Kubernetes continuously reconciles actual state to match.

Managing infrastructure beneath Kubernetes, however, has historically relied on an entirely different operational model. Provisioning a cluster usually requires a procedural pipeline: Terraform or cloud SDKs to create VPCs and virtual machines, Ansible or cloud-init to install runtimes and packages, and kubeadm to initialize the control plane and join worker nodes.

This procedural approach works well for Day-0 provisioning, but creates friction for Day-2 operations. Scaling worker capacity, rolling out kernel updates, executing Kubernetes version upgrades, and replacing degraded nodes typically require re-running scripts or maintaining custom runbooks. Cluster API (CAPI) brings the declarative controller pattern down to infrastructure, allowing you to manage Kubernetes clusters using the Kubernetes API itself.

Why Cluster API exists

The distinction between point-in-time provisioning and continuous lifecycle management explains why Cluster API exists.

Provisioning is an execution pipeline. A tool issues a sequence of API calls: allocate subnets, launch virtual machines, write certificates, and run bootstrapping scripts. Once the pipeline completes, the tool exits. It does not continuously watch whether those machines stay healthy. If a virtual machine terminates unexpectedly later, the tool does nothing until someone triggers another run.

Continuous lifecycle management requires an active control loop. A production cluster experiences constant change over its lifespan:

  • Worker nodes crash, degrade, or fail hardware health checks.
  • Security updates and kernel patches require rolling node replacements.
  • Traffic changes demand dynamic scaling of compute capacity.
  • Kubernetes version upgrades require rolling changes across control plane members and worker pools without downtime.

Handling these ongoing events with imperative scripts leads to configuration drift and operational overhead. Teams end up writing bespoke tooling to solve problems Kubernetes already solved: state reconciliation, rolling updates, and health monitoring.

Cluster API shifts cluster management into a continuous reconciliation loop. Instead of treating a cluster as external infrastructure created by a pipeline, Cluster API models the cluster as Kubernetes custom resources. Controllers run continuously in an active control plane, watching those resources and making API calls to cloud providers or hypervisors to ensure real-world infrastructure continuously matches your declared specification.

The mental model

To understand Cluster API, map its custom resources directly to the workload primitives you already know.

In standard Kubernetes workload management:

  • A Deployment defines desired state, rolling update strategies, and pod templates.
  • A ReplicaSet ensures the configured count of identical pods is always running.
  • A Pod represents the concrete runtime container instance.

Cluster API mirrors this hierarchy for infrastructure:

  • A Cluster defines global cluster configuration, such as networking CIDR blocks and control plane endpoints.
  • A MachineDeployment defines desired state, versions, and rolling update strategies for worker machines.
  • A Machine represents an individual compute host (a cloud VM or bare-metal server). Once provisioned and bootstrapped, it joins the cluster as a Kubernetes Node.
flowchart TD
    subgraph Workload["Workload Model"]
        D["Deployment"] --> RS["ReplicaSet"]
        RS --> P["Pod"]
    end

    subgraph CAPI["Cluster API Model"]
        C["Cluster"] --> MD["MachineDeployment"]
        MD --> M["Machine"]
    end

Upgrades follow the same pattern as application rollouts. To upgrade a worker fleet from Kubernetes v1.31 to v1.32, you update the version on the MachineDeployment. Behind the scenes, MachineDeployment uses an intermediate MachineSet (equivalent to a ReplicaSet) to roll out new Machine instances, wait for them to register as healthy Nodes, and drain and delete old machines according to configured surge limits.

Because infrastructure varies across clouds and on-premises environments, Cluster API separates generic machine logic from environment-specific operations using providers. Core Cluster API defines generic resources (Cluster, MachineDeployment, Machine). Pluggable providers (such as AWS, Azure, GCP, or vSphere) handle cloud-specific APIs. You write standard declarative manifests; providers handle cloud execution.

Management vs workload clusters

Because Cluster API is built with Custom Resource Definitions (CRDs) and controllers, it requires a Kubernetes control plane to run them. This introduces a foundational architectural distinction:

  • Management cluster: A dedicated Kubernetes cluster that hosts the Cluster API controllers, provider controllers, and custom resources. It serves as the single declarative control plane for managing target clusters.
  • Workload cluster: A target Kubernetes cluster provisioned and managed by the management cluster. Your application workloads and services run here.
flowchart TD
    subgraph Management["Management Cluster"]
        CTRL["Cluster API Controllers"]
        CRD["Cluster & Machine CRDs"]
        CRD --> CTRL
    end

    subgraph Workload["Workload Clusters"]
        W1["Production Cluster\n(User Pods & Services)"]
        W2["Staging Cluster\n(User Pods & Services)"]
    end

    CTRL -->|"Reconciles Infrastructure"| W1
    CTRL -->|"Reconciles Infrastructure"| W2

This separation ensures operational isolation. Workload issues or node outages do not affect the management plane. A single management cluster can also manage multiple workload clusters across different regions, accounts, and cloud providers using consistent declarative manifests.

What to remember

As you move into the deeper architectural layers of Cluster API, carry forward these core concepts:

  1. Continuous reconciliation: Cluster API replaces point-in-time provisioning scripts with continuous control loops that reconcile real-world infrastructure against declared state.
  2. Familiar hierarchy: MachineDeployment manages Machines just as a Deployment manages Pods, applying declarative rolling updates and scaling to nodes.
  3. Management vs workload separation: The management cluster hosts the CAPI controllers and declarative specs; workload clusters host your application workloads.
  4. Provider-based execution: Core CAPI handles generic cluster abstractions, while pluggable providers translate those abstractions into cloud-specific resources.

References