How to create Kubernetes YAML files

Introduction

This article is intended as a guide for creating Kubernetes manifest files.

YAML stands for YAML ain’t markup language is used to create Kubernetes manifests. According to the definition:

YAML is a human friendly data serialization language for all programming languages.

Indeed reading YAML is rather straightforward, once you realize that indentation is the way to build data structures. It is especially easy if you are familiar with Python, F# or other indentation-aware languages.

Writing YAML, on the other hand, is much less fun. If you are familiar with schema-less REST API development, think about Kubernetes YAML in the same way as JSON payload for an API. There is no schema to validate against and all the knobs are exposed as data in the payload.

It is relatively straightforward to create a simple YAML file, but for production use, many more settings are required. Here is an example of a Kubernetes Deployment manifest. If you know Kubernetes a bit, most of the settings would make sense when you read the file. Now imagine that you need to create a few of those from scratch (ingress, service, config map, secrets…)

As a developer, you will need to create several different types of manifests. Here is a list of Kubernetes resources, * indicates the resources typically created/maintained by developers. Please note that this only applies if you have a dedicated Kubernetes administrator who can take care of the rest. The list is abbreviated only to common resources and in production, scenarios will likely include various CRDs (Custom Resource Definitions).

If you are not confused, you are not paying attention.

Tom Peters

What you will learn

Let’s try to alleviate some of the confusion around writing YAML files and make the process easier and less painful.

After reading this article, you will learn:

• different ways to create Kubernetes YAML files
• how to automatically detect and correct errors during the development process
• what development tools to use to make creating YAML files easier
• learn about online tools specializing in generating different kinds of YAML files

Prerequisites

If you would like to follow along and get your hands dirty with YAML, you will need:

• VS Code with remote development extension
• docker desktop or any of the alternatives (need to be able to run containers)
• clone this repository

Please note that the repository uses a devcontainer with all the tools and configuration needed to run the examples build in. Because of this, the image is large, so please modify it accordingly before running the container.

Create vs generate

Initially, you might be tempted to generate as much of the boilerplate as possible. My recommendation is, don’t! Especially if you are new to Kubernetes or YAML, experiment, copy-paste from Kubernetes docs, but don’t use generators on day one.

Once you are familiar with the basics, progressively add tools that will make your life easier.

There is good news; you will understand the basics pretty fast.

A good way to know if you are familiar enough with the YAML content of a specific resource if it is getting, well … boring. From here now, you should dive headfirst into the world of generators and helpers to keep your sanity and make your life easier.

#1 YQ

The first tool I want to talk about is yq. Yq is not Kubernetes specific, it’s rather a “jack of all trades” of YAML. Learning this tool will help you query and manipulate YAML files directly from the command line. It helps with tasks, such as:

• filtering YAML file for a specific value, for example retrieving an image name from a deployment file

yq eval '.spec.template.spec.containers[0].image' blazor-deployment-full.yaml

Selecting values from YAML files is useful, but mastering yq will help mostly with bulk operations on multiple files and more complex transformations.

#2 Docker-compose

Do you have a docker-compose.yaml file in your project? Generating Kubernetes manifests from the docker-compose file is possible using a tool called komopse.

Let’s see this in action. We will use a docker-compose file from the awesome-compose repository.

Here is a sample docker-compose file:

Now, let’s generate Kubernetes manifests using kompose:

kompose convert -f docker-compose.yml --controller deployment --out k8s-manifests

This command takes the docker-compose as input and outputs generated Kubernetes YAML into the k8s-manifests folder.

Using kompose is a good option if you already have a docker-compose file. There are often tweaks needed, but it takes you one step closer to having a decent starting point.

#3 CDK8s

Moving on from command line to programming territory. If you have to author a lot of YAML, but happen to know Python, Typescript, JavaScript, Java or Go, you can harness the power of a programming language to make the process of writing YAML much easier.

Introducing CDK8s

cdk8s is an open-source software development framework for defining Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs. cdk8s apps synthesize into standard Kubernetes manifests which can be applied to any Kubernetes cluster.

CDK8s works by exposing Kubernetes resources objects and using an object called constructs to further abstract and automate YAML files creation.

The real power behind this approach is the ability to:

• create reusable components and abstractions that capture your requirements
• use native programming language constructs to automate, test and validate the process of creating YAML

#4 NAML

If you happen to know Go and don’t like YAML at all and want to avoid it at all costs, this project might be something for you!

A very interesting approach designed by Kris Nova is a Go-centric tool called naml which works by creating Kubernetes manifests directly in Go and installing them on the cluster via CLI install command.

This tool can produce YAML similarly to CKD8s but works only with Go.

https://github.com/kris-nova/naml

#5 Online Tools

Now and then a frustrated developer will create a simple Web UI with a form to gather input for generating YAML files. Those projects are usually short-lived and not maintained very well, so my advice is to stay away from online Kubernetes YAML generators.

There is one exception, an online editor by cilium specialized in creating Kubernetes resource called Network Policy

Closing Thoughts

We’ve seen various ways of creating Kubernetes YAML files, some of them using simple command-line tools, auto-generating files. Others exposing full programming languages.

This is great, but how do we make sure that our YAML is correct? What are some of the best practices for validating the content of the file? How can those practices be automated?

Stay tuned for the next article if you would like to know answers to those and similar questions.