Building a Kubernetes Admission Controller in Go
In this blog post, we will walk through how to write an admission controller in Go. Our admission controller will serve a specific purpose: adding a sidecar to any Pod labeled with "inject-sidecar": true.
First things first, let's understand what an admission controller is.
What are Kubernetes Admission Controllers?
Admission controllers are part of the Kubernetes (K8s) system that help manage and control requests to the Kubernetes API server. They are pieces of code that intercept requests to the Kubernetes API server before the persistence of the object, but after the request is authenticated and authorized.
Kubernetes admission controllers come in various types, each serving a unique purpose. They are part of the larger request-handling pipeline after the request is authenticated and authorized, but before the object is stored in the Kubernetes object store.
There are two types of admission controllers:
- Validating Admission Controllers: These controllers validate the requests. The primary function of these controllers is to ensure that the requests meet certain conditions and specifications. They don't modify the requests, but they can reject requests that don't meet the predefined conditions.
- Mutating Admission Controllers: As the name suggests, these controllers mutate the requests. They can change the requests before hitting the object store. The change could be something as simple as adding a label, annotation, or as complex as injecting a sidecar container in a pod. This is the type of controller we've built in this guide.
It's important to note that if multiple mutating admission controllers are used, they are run sequentially, and each can see the changes made by the previous controllers. In contrast, if multiple validating admission controllers are used, they are run in parallel.
In addition to these two types, there are over 30 different built-in admission controllers that come out-of-the-box with Kubernetes, such as NamespaceLifecycle, ResourceQuota, AlwaysPullImages, PodSecurityPolicy, NodeRestriction, ImagePolicyWebhook, ServiceAccount, and more. You can find the full list of available admission controllers and their specific use-cases in the official Kubernetes documentation.
Admission controllers play a pivotal role in maintaining the integrity of the Kubernetes system. They enforce policies and ensure that all requests meet the preconditions before processing by the API server.
It's also possible to build custom admission controllers, as demonstrated in our guide. For a comprehensive guide on Kubernetes Admission Controllers, refer to this guide from the official Kubernetes blog.
Admission controllers, whether built-in or custom, enable fine-grained control over the resources and operations in a Kubernetes cluster, making them a powerful tool in the hands of Kubernetes administrators and developers.
Writing the Admission Controller
To start, we need to initialize our Go module and add the necessary dependencies.
$ mkdir admission-controller
$ cd admission-controller
$ go mod init github.com/opsarena/admission-controller
$ go get k8s.io/api/admission/v1
$ go get k8s.io/apimachinery/pkg/apis/meta/v1
$ touch main.goNext, we will write the Go server that will handle our admission control logic. The general flow of our server will be:
- Decode the admission review request from the API server.
- Check if the Pod has the correct label.
- If the label exists, add the sidecar to the Pod.
- Encode the response and send it back to the API server.
package main
import (
"context"
"encoding/json"
"net/http"
v1 "k8s.io/api/admission/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
type patchOperation struct {
Op string `json:"op"`
Path string `json:"path"`
Value json.RawMessage `json:"value"`
}
func main() {
http.HandleFunc("/mutate", handleMutate)
http.ListenAndServe(":8080", nil)
}
func handleMutate(w http.ResponseWriter, r *http.Request) {
// Decode the request
var review v1.AdmissionReview
if err := json.NewDecoder(r.Body).Decode(&review); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Check for the right label
var pod corev1.Pod
if err := json.Unmarshal(review.Request.Object.Raw, &pod); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
if pod.Labels["inject-sidecar"] == "true" {
// Define the sidecar
sidecar := `{"name": "my-sidecar", "image": "my-sidecar-image"}`
// Create the patch
patch := []patchOperation{{
Op: "add",
Path: "/spec/containers/-",
Value: json.RawMessage(sidecar),
}}
// Create the response
review.Response = &v1.AdmissionResponse{
Allowed: true,
Patch: patch,
PatchType: func() *v1.PatchType {
pt := v1.PatchTypeJSONPatch
return &pt
}(),
}
} else {
// If the label doesn't exist, allow the request without patching
review.Response = &v1.AdmissionResponse{
Allowed: true,
}
}
// Encode and send the response
if err := json.NewEncoder(w).Encode(review); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}In this server code, we've written a HTTP server in Go that listens to requests at the "/mutate" endpoint. It checks if a Pod has the label "inject-sidecar" set to "true", and if so, it adds a sidecar container to the Pod.
Build the docker image of admission controller
- Create Dockerfile in the root directory of admission controller go
$ touch Dockerfile- Copy the following content to Dockerfile
# Start from a base golang image
FROM golang:1.16 as builder
# Set the working directory
WORKDIR /go/src/app
# Add the go mod and sum files
COPY go.mod go.sum ./
# Download the dependencies
RUN go mod download
# Copy the rest of the application
COPY . .
# Build the application
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# Now, create a minimal image from scratch
FROM alpine:latest
# Set the working directory
WORKDIR /root/
# Copy the binary from the builder stage
COPY --from=builder /go/src/app/main .
# Expose port 8080 for the server
EXPOSE 8080
# Run the binary
CMD ["./main"]- Build and push docker image
$ docker build -t my-admission-controller:1.0.0
$ docker push my-admission-controller:1.0.0
Configuring Kubernetes
With our server written, we now need to configure Kubernetes to use it. We'll create a new service and deployment for our server, and then we'll create a new MutatingWebhookConfiguration to tell the API server to send requests to our server.
apiVersion: v1
kind: Service
metadata:
name: my-admission-controller
namespace: default
spec:
selector:
app: my-admission-controller
ports:
- protocol: TCP
port: 443
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-admission-controller
labels:
app: my-admission-controller
spec:
replicas: 1
selector:
matchLabels:
app: my-admission-controller
template:
metadata:
labels:
app: my-admission-controller
spec:
containers:
- name: my-admission-controller
image: my-admission-controller:1.0.0
ports:
- containerPort: 8080
---
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: my-admission-controller
webhooks:
- name: my-admission-controller.default.svc
admissionReviewVersions: ["v1", "v1beta1"]
sideEffects: None
rules:
- operations: ["CREATE", "UPDATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
clientConfig:
service:
namespace: default
name: my-admission-controller
path: "/mutate"
Remember to replace my-admission-controller:1.0.0 with the actual Docker image of your admission controller server.
Congratulations! 🎉 You now have a Kubernetes admission controller written in Go that adds a sidecar to Pods labeled with "inject-sidecar": true. It's a powerful tool for managing and modifying Kubernetes resources based on your custom needs, all at the API server level before the resources hit the etcd database.
-0-