I was reviewing Golang recently and wrote a web application. After running it locally, I wanted to test it in a k8s cluster. Due to the machine configuration, it was still a bit difficult to build a complete k8s cluster. I remember that a friend said that k8s can also be run in docker, so I tried it.

Today’s protagonist is kind, so what is kind? What can kind be used for?

1. Introduction to kind

kind is the abbreviation of Kubernetes In Docker, a tool that uses docker container nodes to run local kubernetes clusters. Kind is mainly used to test kubernetes itself, suitable for local development or CI.

Kind currently supports almost all official versions of kubernetes. For runtime, it currently only supports docker. In the future, it will gradually support the commonly used runtime in CRI. I believe that in the future, it will slowly support conterned.

In fact, the process of kind managing clusters is using open source components such as kubeadm and kustomize to work.

Without further ado, let’s quickly install and use kind.

2. Kind installation

Since my system is Mac OSX, I will use brew to install it directly:

# brew install kind
==> Downloading https://mirrors.ustc.edu.cn/homebrew-bottles/kind-0.17.0.arm64_monterey.bottle.tar.gz
Already downloaded: /Users/wanzi/Library/Caches/Homebrew/downloads/bcd419997297730492f5cebc36be86fb51f21061a6ddb2e066e1e4d8ad33ddf3--kind-0.17.0.arm64_monterey.bottle.tar.gz
==> Pouring kind-0.17.0.arm64_monterey.bottle.tar.gz
==> Caveats
zsh completions have been installed to: /opt/homebrew/share/zsh/site-functions ==> Summary 🍺 /opt/homebrew/Cellar/kind/0.17.0: 8 files, 8.7MB ==> Running `brew cleanup kind`... Disable this behavior by setting HOMEBREW_NO_INSTALL_CLEANUP. Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`) # kind version kind v0.17.0 go1.19.2 darwin/arm64 ``` For linux installation: ```Shell curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64 chmod +x ./kind sudo mv ./kind /usr/local/bin/kind

Of course, if you are using Windows, you can also use Chocolatey to install; or use the binary method, which is officially supported.

3. Create the first cluster

Before the operation, docker and kubectl are what we need to prepare in advance, and I will ignore the specific installation here.

For the kind help command, we know that kind supports operations such as build, create, delete, get, and load. For subcommands, you can use -h to get more information.

kind creates a cluster. Before creating a cluster, we need to ensure that the ~/.kube directory exists, because during the creation process, kind will write information such as the cluster API address and certificate into kubeconfig.

# kind create cluster
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.25.3) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Have a nice day! 👋

# kubectl get node NAME STATUS ROLES AGE VERSION kind-control-plane Ready control-plane 9m21s v1.25.3 # kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-565d847f94-kp6l7 1/1 Running 0 9m15s kube- system coredns-565d847f94-ml28b 1/1 Running 0 9m15s kube-system etcd-kind-control-plane 1/1 Running 0 9m32s kube-system kindnet-hfsc8 1/1 Running 0 9m15s kube-system kube-apiserver-kind-control -plane 1/1 Running 0 9m31s kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 9m33s
kube-system kube-proxy-m9zp5 1/1 Running 0 9m15s
kube-system kube-scheduler-kind-control-plane 1/1 Running 0 9m33s
local -path-storage local-path-provisioner-684f458cdd-d7f29 1/1 Running 0 9m15s

This is how our first cluster is created using kind.

4. Cluster operation:

1. Customize the cluster name with kind:

# kind create cluster --name ci-cluster
Creating cluster "ci-cluster" ...
✓ Ensuring node image (kindest/node :v1.25.3) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-ci-cluster"
You can now use your cluster with:
``` ### 2. Switch cluster:
```Shell
# kubectl cluster-info --context kind-ci-cluster
Kubernetes control plane is running at https://127.0.0.1:56527
CoreDNS is running at https://127.0.0.1:56527/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Have a nice day ! 👋

3. View all kind clusters

1
2
3

# kind get clusters
ci-cluster
kind

4. Import the image.

The default kind node cannot read the host image and needs to be imported manually. Import to kind

Node:

1
2
3

# kind load docker-image --name ci-cluster --nodes ci-cluster-control-plane traefik:v2.9.5
# docker exec -it ci-cluster-control-plane bash root@ci-cluster-control-plane:/# crictl images |grep traefik
docker.io/library/traefik v2.9.5 a1252ce6bfaaa 132MB

5. Cluster configuration:

1. Multi-node cluster

For the previous introduction, we introduced kind to create a cluster. The default cluster has only one node. Describes how to create multiple nodes.

Create config.yaml with the following content:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker

Recreate the cluster``` Shell

kind create cluster –config config.yaml

kubectl get node

NAME STATUS ROLES AGE VERSION kind-control-plane Ready control-plane 110s v1.25.3 kind-worker Ready 74s v1.25.3 kind-worker2 Ready 88s v1.25.3

kind itself can also support multi-master high availability deployment. I will not briefly introduce it here as it is my main test.

2. Customize the kubernetes cluster version

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
images: kindest/node:v1.24.7@ sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315

3. Mount the host directory to the node container for persistent storage

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts: - hostPath: /Users/wanzi/tools/kind/files
containerPath: /files
- hostPath: /Users/wanzi/tools/kind/other-files/
containerPath: /other-files
readOnly: true
selinuxRelabel: false
propagation: None

4. Port mapping:

Host port 80 is mapped to nodePort 30950, kind node containerPort Need to be the same as the nodePort port of the Service

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30950
hostPort: 80

kind: Pod
apiVersion: v1
metadata: name: foo
labels:
app: foo
spec:
containers:
- name: foo
image: hashicorp/http-echo:0.2.3
args:
- "-text=foo"
ports:
- containerPort: 5678 --- apiVersion: v1 kind: Service metadata: name: foo spec: type: NodePort ports: - name: http nodePort: 30950 port: 5678 selector: app: foo ``` ### 5. Custom label `` `Yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane - role: worker extraPortMappings: - containerPort: 30950 hostPort: 80 labels: tier: frontend - role: worker labels: tier: backend ```

### 6. Kubeadm custom configuration:

kind uses kubeadm to configure the cluster. It will run kubeadm init on the first control plane node. We can customize it:

```Yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "my-label=true"

If you want to do more customization, there are four configuration types to choose from kubeadm init

InitConfiguration
ClusterConfiguration
KubeProxyConfiguration
KubeletConfiguration

For example, we can use kubeadm ClusterConfiguration (spec) to override the apiserver flag:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
extraArgs:
enable-admission-plugins: NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

On each additional node configured in a KIND managed HA cluster, KIND runs kubeadm join and can use JoinConfiguration to customize the configuration:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "my-label2=true"
- role: control-plane
kubeadmConfigPatches:
- |
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "my-label3=true"

7. Configure ingress

Allow local hosts to make requests to the Ingress controller through port 80/443, and configure controller-specific labels

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
-role: control-plane
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingress-ready=true"
extraPortMappings:
-containerPort: 80
hostPort: 80
protocol: TCP
-containerPort: 443
hostPort: 443
protocol: TCP

`1`	`kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml`

6. Complete cluster configuration

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
#Add multi-node cluster and customize the node cluster version
- role: control-plane
images: kindest/node:v1.24.7@sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs: node-labels: "ingress-ready=true" extraPortMappings: - containerPort: 80 hostPort: 80 protocol: TCP - containerPort: 443 hostPort: 443 protocol: TCP - role: worker images: kindest/node:v1.24.7@sha256:577c630ce8e509131eab1aea12c022190978 dd2f745aac5eb1fe65c0807eb315 labels: app: front extraMounts: - hostPath: /Users/wanzi/tools/kind/wwwroot containerPath: /wwwroot - role: worker images: kindest/node:v1.24.7@sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315
labels:
app: backend
extraMounts:
- hostPath: /Users/wanzi/tools/kind/wwwroot
containerPath: /wwwroot
networking:
#Custom configuration APIServer and network
apiServerAddress: "127.0.0.1"
apiServerPort: 6443
podSubnet: "10.244.0.0/16"
serviceSubnet: "10.96.0.0/12"
#disableDefaultCNI: true #The default CNI plugin is kindnetd, and you can also disable other plugins such as calico
kubeProxyMode: "ipvs" #Set kubeproxy mode to lvs, if you want to disable, you can set it to none

So far, using kind to quickly build a cluster has come to an end. You can use it to run tests and verify functions normally.

Reference documents: https://kind.sigs.k8s.io

https://github.com/kubernetes-sigs/kind