Workshop

Ephemeral Environments
with Crossplane and kube-green

Introduction

In this guided tutorial we are going to see step-by-step how to create self-service ephemeral infrastructure using Crossplane and kube-green.

What is Crossplane?

Crossplane is an open source Kubernetes add-on that enables you to provision and manage cloud infrastructure and services using Kubernetes resources and APIs.

It allows you to define, compose, and orchestrate infrastructure using YAML, bringing GitOps and declarative management to cloud resources.
Official Documentation

How does Crossplane work?

Crossplane works by reconciling Kubernetes Custom Resources that represent infrastructure and services.

The control plane continuously ensures that the desired state (as defined in YAML) matches the actual state in the cloud or external system.
Providers handle the communication and provisioning logic for each type of resource.
How Crossplane Works

What is kube-green?

kube-green is an open-source Kubernetes add-on designed to reduce resource usage and energy consumption in clusters.
It helps organizations save costs and minimize their environmental impact by automatically suspending non-essential workloads during non-working hours.

kube-green.dev

Key Features

Automatic suspension of workloads based on schedules
Easy integration with existing clusters
Customizable rules and schedules

kube-green documentation | GitHub repository

Why Use kube-green?

Save costs by reducing unnecessary resource consumption
Lower your carbon footprint by minimizing energy usage
Improve cluster efficiency with automated workload management

kube-green.dev | Blog & Use Cases

Prerequisites

All cases presented in this workshop are implemented on a local kind cluster.
In this first step, we will install and configure everything needed for the following steps.

Prepare the workspace

Install Kind, Helm, and kubectl on your local machine (Kind guide, Helm guide, kubectl guide).

                $ brew install kind helm
                kubectl
              

Create the cluster

Create a cluster.

                $ kind create cluster --name
                ephemeral-environments-demo
              

Check the cluster initial setup

Check the cluster initial setup.

                $ kubectl get ns
              

You should get this output.

                NAME             STATUS   AGE

                default            Active   4m52s

                kube-node-lease   Active   4m52s

                kube-public        Active   4m52s

                kube-system        Active   4m53s

                local-path-storage Active   4m47s

Add Crossplane

Add Crossplane helm repo.

                    $ helm repo add
                    crossplane-stable https://charts.crossplane.io/stable

                    $ helm repo update
                  

Install Crossplane.

                    $ helm install
                    crossplane --namespace crossplane-system --create-namespace
                    crossplane-stable/crossplane
                  

Check crossplane installation

Check that crossplane is installed correctly.

                    $ kubectl get pods -n
                    crossplane-system
                  

You should get this output.

                    NAME                         READY    STATUS     RESTARTS  AGE

                    crossplane-xxxx                1/1   Running           0   81s

                    crossplane-rbac-manager-xxxx   1/1   Running           0   81s

Install cert-manager

Install the cert-manager:

                $ kubectl apply -f
                https://github.com/cert-manager/cert-manager/releases/download/v1.14.5/cert-manager.yaml
              

Install kube-green

Install kube-green:

                    $ kubectl apply -f
                    https://github.com/kube-green/kube-green/releases/latest/download/kube-green.yaml
                  

Check kube-green installation

Check that kube-green is installed correctly.

                  $ kubectl get pods -n
                  kube-green
                

You should get this output.

                  NAME                                 READY    STATUS   RESTARTS  AGE

                  kube-green-controller-manager-xxxx     1/1   Running          0  85s

1️⃣ Cloud Native PG PSQL Cluster

In this section we will see how to manage the shutdown and restart of a PostgreSQL cluster managed using the CloudNativePG operator and orchestrated by Crossplane.

What is CloudNativePG?

CloudNativePG is an open source Kubernetes operator for managing PostgreSQL clusters in a cloud-native way.

It automates deployment, scaling, backup, failover, and recovery of PostgreSQL databases on Kubernetes.
Documentation | GitHub

Key Features of CloudNativePG

Automated PostgreSQL cluster lifecycle management
High availability and automated failover
Continuous backup and point-in-time recovery
Easy scaling and upgrades
Native integration with Kubernetes resources

What is a Cluster in CloudNativePG?

In CloudNativePG, a Cluster is a custom Kubernetes resource that defines and manages a group of PostgreSQL instances.

The operator ensures high availability, data consistency, and automated failover for all instances in the cluster.
Learn more in the official documentation.

Install the Crossplane Kubernetes Provider

To install the Crossplane Kubernetes provider, create a file named provider-kubernetes.yaml with the following content:

# provider-kubernetes.yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-kubernetes
spec:
  package: xpkg.upbound.io/crossplane-contrib/provider-kubernetes:v0.18.0

What is a Crossplane Provider?

A Crossplane Provider is an extension for Crossplane that enables it to manage resources in a specific cloud, platform, or API.

Providers translate Kubernetes Custom Resources into actions on external systems (e.g., AWS, GCP, Azure, Kubernetes, databases, etc.).
Provider Marketplace

Apply the provider manifest

Apply the provider-kubernetes.yaml manifest:

                  $ kubectl apply -f
                  provider-kubernetes.yaml
                

You should get this output:

                  provider.pkg.crossplane.io/provider-kubernetes created
                

Configure the Crossplane Kubernetes Provider

Create the provider-kubernetes-config.yaml file to configure the Kubernetes Crossplane provider with the following content:

# provider-kubernetes-config.yaml
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  name: kubernetes-in-cluster
spec:
  credentials:
    source: InjectedIdentity

Apply the provider config manifest

Apply the provider-kubernetes-config.yaml manifest:

                    $ kubectl apply -f
                    provider-kubernetes-config.yaml
                  

You should get this output:

                    providerconfig.kubernetes.crossplane.io/kubernetes-in-cluster
                    created
                  

Install the CloudNativePG Operator

Install the CloudNativePG Operator:

                    $ kubectl apply
                    --server-side -f
                    https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.26/releases/cnpg-1.26.0.yaml
                  

Check the operator installation

Check the CloudNativePG Operator installation:

                    $ kubectl rollout status
                    deployment -n cnpg-system cnpg-controller-manager
                  

You should get this output:

                    deployment "cnpg-controller-manager" successfully rolled out
                  

Create the CompositeResourceDefinition (XRD)

To define a new API, create the xrd-postgrescluster.yaml file with the following content:

# xrd-postgrescluster.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xpostgresclusters.demo.crossplane.io
spec:
  group: demo.crossplane.io
  names:
    kind: XPostgresCluster
    plural: xpostgresclusters
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              instances:
                type: integer
                description: "Number of PostgreSQL instances in the cluster."
                default: 1
              hibernation:
                type: string
                description: "Set to 'on' to hibernate the cluster, 'off' to wake it up."
                default: "off"
            required:
            - instances

What is a CompositeResourceDefinition (XRD)?

CompositeResourceDefinition (XRD) is a custom resource in Crossplane that lets you define new API types for your platform.

Declarative API: Create your own Kubernetes resource types (e.g. XPostgresCluster).
Schema: Specify the fields, types, and validation for your resource.
Foundation for Composition: XRDs are the basis for Compositions, which map your custom API to real infrastructure.
Extensibility: Enables platform teams to offer tailored APIs for developers.

Crossplane XRD Documentation

Apply the XRD manifest

Apply the xrd-postgrescluster.yaml manifest:

                    $ kubectl apply -f
                    xrd-postgrescluster.yaml
                  

You should get this output:

                    compositeresourcedefinition.apiextensions.crossplane.io/xpostgresclusters.demo.crossplane.io
                    created
                  

Create the Composition

To map the abstract API to a real resource, create the composition-postgrescluster.yaml file with the following content:

# composition-postgrescluster.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: postgrescluster.demo.crossplane.io
spec:
  compositeTypeRef:
    apiVersion: demo.crossplane.io/v1alpha1
    kind: XPostgresCluster
  resources:
  - name: postgres-cluster-object
    base:
      apiVersion: kubernetes.crossplane.io/v1alpha2
      kind: Object
      spec:
        managementPolicies: ["*"]
        forProvider:
          manifest:
            apiVersion: postgresql.cnpg.io/v1
            kind: Cluster
            metadata:
              namespace: default
            spec:
              storage:
                size: 1Gi
              bootstrap:
                initdb:
                  database: appdb
                  owner: appuser
        providerConfigRef:
          name: kubernetes-in-cluster
    patches:
    - fromFieldPath: "metadata.name"
      toFieldPath: "spec.forProvider.manifest.metadata.name"
    - fromFieldPath: "spec.instances"
      toFieldPath: "spec.forProvider.manifest.spec.instances"
    - type: FromCompositeFieldPath
      fromFieldPath: spec.hibernation
      toFieldPath: spec.forProvider.manifest.metadata.annotations[cnpg.io/hibernation]
      policy:
        fromFieldPath: Optional

What is a Composition?

Composition in Crossplane is a resource that defines how to implement an abstract API (XRD) using real infrastructure resources.

Infrastructure Mapping: Connects your custom API to managed resources.
Reusable Blueprints: Standardized infrastructure patterns for developers.
Patching & Transformations: Map and transform fields from API to resources.
Multi-resource: One Composition can manage multiple resources.

Crossplane Composition Documentation

Apply the Composition manifest

Apply the composition-postgrescluster.yaml manifest:

                  $ kubectl apply -f
                  composition-postgrescluster.yaml
                

You should get this output:

                  composition.apiextensions.crossplane.io/postgrescluster.demo.crossplane.io
                  created
                

Grant cluster-admin permissions to the kubernetes-provider service account

Get the service account name:

                  $ SA=$(kubectl -n
                  crossplane-system get sa -o name | grep provider-kubernetes |
                  sed -e 's|serviceaccount/|crossplane-system:|g')
                

Then run:

                  $ kubectl create
                  clusterrolebinding provider-kubernetes-admin-binding
                  --clusterrole cluster-admin --serviceaccount="${SA}"
                

You should get:

                  clusterrolebinding.rbac.authorization.k8s.io/provider-kubernetes-admin-binding
                  created
                

Create the Database Instance

Create the instance of the database with this file:

# my-postgres-db.yaml
apiVersion: demo.crossplane.io/v1alpha1
kind: XPostgresCluster
metadata:
  name: my-production-db
  namespace: default
spec:
  instances: 1
  hibernation: "off"

Apply the Database Manifest

Apply the my-postgres-db.yaml manifest:

                  $ kubectl apply -f
                  my-postgres-db.yaml
                

You should get the following output:

                  xpostgrescluster.demo.crossplane.io/my-production-db created
                

Verify the Provisioning

Check the Crossplane XR:

                  $ kubectl get
                  xpostgrescluster my-production-db
                

You should get:

                  NAME SYNCED READY COMPOSITION AGE my-production-db True False
                  postgrescluster.demo.crossplane.io 53s
                

Verify the Provisioning

Then, check the CloudNativePG CR:

                  $ kubectl get cluster -n
                  default my-production-db
                

You should get:

                  NAME AGE INSTANCES READY STATUS PRIMARY my-production-db 43s 1
                  1 Cluster in healthy state my-production-db-1
                

Verify the Provisioning

Then, check the database pods:

                  $ kubectl get pods -n
                  default -l cnpg.io/cluster=my-production-db
                

You should get this output:

                  NAME READY STATUS RESTARTS AGE my-production-db-1 1/1 Running
                  0 65s
                

Grant kube-green permission to patch `XPostgresCluster`

Create the file kube-green-rbac-for-postgres.yaml with the following content:

# kube-green-rbac-for-postgres.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-green-xpostgrescluster-patcher
rules:
- apiGroups:
  - "demo.crossplane.io"
  resources:
  - "xpostgresclusters"
  verbs:
  - "get"
  - "list"
  - "watch"
  - "patch"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-green-patch-xpostgrescluster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-green-xpostgrescluster-patcher
subjects:
- kind: ServiceAccount
  name: kube-green-controller-manager
  namespace: kube-green

Apply the Manifest

Apply the kube-green-rbac-for-postgres.yaml manifest:

                  $ kubectl apply -f
                  kube-green-rbac-for-postgres.yaml
                

You should get:

                  clusterrole.rbac.authorization.k8s.io/kube-green-xpostgrescluster-patcher
                  created

                  clusterrolebinding.rbac.authorization.k8s.io/kube-green-patch-xpostgrescluster
                  created
                

Schedule Database Pods Sleeping

Schedule the database pods sleeping with the file schedule-sleep-postgres.yaml:

# schedule-sleep-postgres.yaml
apiVersion: kube-green.com/v1alpha1
kind: SleepInfo
metadata:
  name: sleep-schedule-for-postgres
  namespace: default
spec:
  weekdays: "*"
  sleepAt: "18:47"
  wakeUpAt: "18:49"
  patches:
  - target:
      group: demo.crossplane.io
      kind: XPostgresCluster
    patch: |
      - op: replace
        path: /spec/hibernation
        value: "on"

This SleepInfo resource will stop your db pods at every 5 minutes and restart at every 7 minutes of each hour in the day.

Apply the `SleepInfo`

Apply the schedule-sleep-postgres.yaml manifest:

                  $ kubectl apply -f
                  schedule-sleep-postgres.yaml
                

You should get:

                  sleepinfo.kube-green.com/sleep-schedule-for-postgres created
                

Check the Result

Check the XRD status:

                  $ watch "kubectl get
                  xpostgrescluster my-production-db -n default -o yaml"
                

You should see the XRD initially with the property hibernation at off. When kube-green runs it will modify this property to on.

Check the Result

Then, check the Cluster Object:

                    $ watch "kubectl get
                    cluster my-production-db -n default -o yaml"
                  

You should see the changes done by kube-green reflected here.

Check the Result

Last check if the database pod is up, it shouldn't.

                    $ watch kubectl get pods
                    -n default -l cnpg.io/cluster=my-production-db
                  

Workshop

Ephemeral Environmentswith Crossplane and kube-green

Introduction

What is Crossplane?

How does Crossplane work?

What is kube-green?

Key Features

Why Use kube-green?

Workshop Agenda

Prerequisites

Prepare the workspace

Create the cluster

Check the cluster initial setup

Add Crossplane

Check crossplane installation

Install cert-manager

Install kube-green

Check kube-green installation

1️⃣ Cloud Native PG PSQL Cluster

What is CloudNativePG?

Key Features of CloudNativePG

What is a Cluster in CloudNativePG?

Install the Crossplane Kubernetes Provider

What is a Crossplane Provider?

Apply the provider manifest

Configure the Crossplane Kubernetes Provider

Apply the provider config manifest

Install the CloudNativePG Operator

Check the operator installation

Create the CompositeResourceDefinition (XRD)

What is a CompositeResourceDefinition (XRD)?

Apply the XRD manifest

Create the Composition

What is a Composition?

Apply the Composition manifest

Grant cluster-admin permissions to the kubernetes-provider service account

Create the Database Instance

Apply the Database Manifest

Verify the Provisioning

Verify the Provisioning

Verify the Provisioning

Grant kube-green permission to patch XPostgresCluster

Apply the Manifest

Schedule Database Pods Sleeping

Apply the SleepInfo

Check the Result

Check the Result

Check the Result

Ephemeral Environments
with Crossplane and kube-green

Grant kube-green permission to patch `XPostgresCluster`

Apply the `SleepInfo`