How to dynamically add nodes to a kind cluster allows you to run a Kubernetes cluster inside Docker. This is incredibly useful for developing Helm charts, Operators, or even just testing out different k8s features in a safe way. Kind I've recently been working on an operator (built using the ) that manages cluster node lifecycles. Kind allows you to , using a Docker container per node and joining them using a common Docker network. However, the executable does not allow you to modify an existing cluster by adding or removing a node. operator-sdk spin up clusters with multiple nodes kind I wanted to see if this was possible using a simple shell script, and it turns out that it's actually not too difficult! Creating the node Using my favorite diff tool, , to compare an existing kind node's state to a new container's, I experimented with various flags until I got something that's close enough to the kind node. DiffMerge docker inspect docker run docker run \
--restart on-failure \
-v /lib/modules:/lib/modules:ro \
--privileged \
-h $NODE_NAME \
-d \
--network kind \
--network-alias $NODE_NAME \
--tmpfs /run \
--tmpfs /tmp \
--security-opt seccomp=unconfined \
--security-opt apparmor=unconfined \
--security-opt label=disable \
-v /var \
--name $NODE_NAME \
--label io.x-k8s.kind.cluster=kind \
--label io.x-k8s.kind.role=worker \
--env KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER \
kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace Joining the cluster You can join new nodes to a k8s cluster by using the command. In this case, we can use to execute this command on our node after its container has started up. kubeadm join docker exec This command won't work out of the box because kind uses a that does not exist in the node docker image. It is injected into the container by the kind executable. kubeadm.conf Again, using my trusty DiffMerge tool, I compared two files in existing kind nodes and found very few differences. This allowed me to just grab one from any worker node to use as a template. /kind/kubeadm.conf docker exec --privileged kind-worker cat /kind/kubeadm.conf > $LOCAL_KUBEADM From here, I needed to set the node's unique IP in its . We can use to grab any node IP address we need. Since I'm working in bash, I just decided to use a simple sed replacement to replace the template node's IP address with my new node's IP in my local copy of . kubeadm.conf docker inspect kubeadm.conf TEMPLATE_IP=$(docker inspect kind-worker | jq -r '.[0].NetworkSettings.Networks.kind.IPAddress')
NODE_IP=$(docker inspect $NODE_NAME | jq -r '.[0].NetworkSettings.Networks.kind.IPAddress')

ESCAPED_TEMPLATE_IP=$(echo $TEMPLATE_IP | sed 's/\./\\./g' )
ESCAPED_NODE_IP=$(echo $NODE_IP | sed 's/\./\\./g')

sed -i.bkp "s/${ESCAPED_TEMPLATE_IP}/${ESCAPED_NODE_IP}/g" $LOCAL_KUBEADM Now that our is prepared, we need to copy it to the new node: kubeadm.conf docker exec --privileged -i $NODE_NAME cp /dev/stdin /kind/kubeadm.conf < $LOCAL_KUBEADM Finally, we can join our node to the cluster: docker exec --privileged $NODE_NAME kubeadm join --config /kind/kubeadm.conf --skip-phases=preflight --v=6 Node Tags Since you have complete control of the new node's , it is possible to configure many of its properties for further testing. For example, to add additional labels to the new node, you can run something like this: kubeadm.conf sed -i.bkp "s/node-labels: \"\"/node-labels: \"my-label-key=my-label-value\"/g" $LOCAL_KUBEADM This will add the label to the node once it joins the cluster. my-label-key=my-label-value Future Work Based on this script, I believe it's possible to add a subcommand to add a node to an existing cluster. Stay tuned for that... kind create node Also published . here

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Kubernetes in Docker: Adding Nodes Dynamically to a Kind Cluster

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Kubernetes Annotations: Harnessing Power in Operator Development

20,000 Leagues Under Your Shell

47 Stories To Learn About Bash

A Basic Guide to MySQL Tests Automation

A Web Service Written in Pure Bash.

Kubernetes Annotations: Harnessing Power in Operator Development

20,000 Leagues Under Your Shell

47 Stories To Learn About Bash

A Basic Guide to MySQL Tests Automation

A Web Service Written in Pure Bash.

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps