Hackernoon logoKubernetes Tutorial: Merge all the Kubeconfigs! [Part 7] by@jameshunt

Kubernetes Tutorial: Merge all the Kubeconfigs! [Part 7]

image
James Hunt Hacker Noon profile picture

@jameshuntJames Hunt

R&D at Stark & Wayne, finding software solutions to customer problems and changing them into executable best practices.

Once you start managing more than one Kubernetes cluster, you'll start to demand more from your 

$KUBECONFIG
. Early on in my career spinning pods, I kept a separate 
.kubeconfig
 file for each cluster, and wrote a small shell function to set the requisite environment variables. To switch between clusters was as simple as:

→  k is buffalo-lab
kubernetes: buffalo-lab
  (config): /Users/jhunt/.k/buffalo-lab

I did this because the "correct" command was too much to remember:

kubectl config use-context buffalo-lab

See what I mean?

Then I met kubectx and kubens, and they quickly became invaluable.

Now I've got this directory full of all these disparate, discrete configuration files, and I don't want to hand-edit them back together. Thankfully,

kubectl
 itself can merge them for us!

Note: 

kubectl
 will only merge configuration files if you specify them via the 
$KUBECONFIG
 environment variable. You cannot use the
--kubeconfig
 command-line flag, it won't work.

To demonstrate this, I'm going to show you my 

~/.k
 directory:

$ ls -l ~/.k
total 96
-rw-r--r-- 1 jhunt staff  7050 Jan  6 09:56 buffalo-lab
-rw-r--r-- 1 jhunt staff  5641 Jan  1 11:30 jhunt-vcp-prodernetes.yml
-rw-r--r-- 1 jhunt staff  5633 Jan  1 11:30 jhunt-vcp-tinynetes
-rw-rw-rw- 1 jhunt staff  5481 Jan  3 12:32 lke-eval1
-rw-r--r-- 1 jhunt staff 23633 Jan  1 11:56 merged
-rw------- 1 jhunt staff   422 Nov 17 14:25 minikube
-rw-r--r-- 1 jhunt staff  7023 Jan  1 11:30 oss
-rw-r--r-- 1 jhunt staff 25706 Jan  3 12:40 tmp

Using 

kubectl config get-contexts
 commands, we can see that each of them is different:

$ KUBECONFIG=~/.k/buffalo-lab \
kubectl config get-contexts
CURRENT   NAME              CLUSTER           AUTHINFO   NAMESPACE
*         buffalo-lab-k8s   buffalo-lab-k8s   admin      default

$ KUBECONFIG=~/.k/lke-eval1 \
kubectl config get-contexts
CURRENT   NAME                          CLUSTER      AUTHINFO           NAMESPACE
*         [email protected]   kubernetes   kubernetes-admin   k8s-cp-ubuntu

If you put two (or more) files, separated by colons, in 

$KUBECONFIG
, then
kubectl
 will merge them together semantically, and operate on the composite configuration. Any changes you make, like changing namespaces or setting contexts, will update the appropriate member file(s).

$ KUBECONFIG=~/.k/buffalo-lab:~/.k/lke-eval1 \
kubectl config get-contexts
CURRENT   NAME                          CLUSTER           AUTHINFO           NAMESPACE
*         buffalo-lab-k8s               buffalo-lab-k8s   admin              default
          [email protected]   kubernetes        kubernetes-admin   k8s-cp-ubuntu

To write a new, aggregate configuration that contains the results of the merge, we can use 

kubectl config view --raw
. The 
--raw
 flag prevents 
kubectl
 from redacting user secrets, certificates, and keys. To save this combined file, we can employ some shell redirection, along with a bit of caution.

Every UNIX shell I've ever used truncates files that are the target of 

>
-style output redirection, before the command producing the output gets executed. It's the nature of UNIX pipelines. We have to be careful with the order of our operations.

This will fail spectacularly (and most assuredly destroy data in the process!):

# DON'T RUN THIS!
$ KUBECONFIG=a:b:c kubectl config view --raw > a

The 

a
 configuration file will be empty by the time 
kubectl
 gets around to trying to merge it in with the other two. Instead, use a temporary file and a rename operation:

$ KUBECONFIG=tmp:a:b:c kubectl config view --raw > tmp \
    && mv tmp a

Pro Tip: While you are merging all of these configurations together, you can also embed any referenced, external files (like certificates and private keys) by swapping out the 

--raw
 flag for 
--flatten
. This produces "portable" kubeconfigs, which do not have any dependencies on other parts of your filesystem. They can be moved, or copied, without losing their utility.

Also published here.

James Hunt Hacker Noon profile picture
by James Hunt @jameshunt. R&D at Stark & Wayne, finding software solutions to customer problems and changing them into executable best practices. Read my stories

Tags

Join Hacker Noon

Create your free account to unlock your custom reading experience.