My only impression of those spy movies is the scene when agents from NSA eavesdrop on communications outside the United States. But it is not the only job that he does. Apart from that, the NSA also contributes to the cybersecurity community. t National Security Agency NSA made the original , Linux’s optional (MAC) architecture. Also, they wrote s while everyone is working remotely. Recently, , and thus I would like to share these great resources with you and other best practices on K8S security. SELinux mandatory access control guidelines on securing video conferencing, testing, and collaboration tool NSA updated the Kubernetes Hardening Guide Background The first version of the hardening guide was released in August 2021, authored by the National Security Agency (NSA). The guidance was also co-written by the (CISA) to create users’ awareness of critical threats and configurations to minimize risk. Cybersecurity and Infrastructure Security Agency In March 2022, NSA updated its , which is more valuable today. For example, the made several comments on the 1.0 version. Kubernetes Hardening Guide NCC Group NCC saw that the first version’s information about ” because it stated that Kubernetes doesn’t provide an authentication method by default (Fact: K8S natively supports both token and certification authentication.) Kubernetes authentication was “largely incorrect Why Should We Protect K8S? Cloud-Native Survey 2021 According to the by (CNCF): 2021 Cloud-Native Survey Cloud Native Computing Foundation 96% of organizations now use or are evaluating Kubernetes. 5.6 million developers are using Kubernetes, which equals around one-third worldwide. Now, of that vast number of adoption, Notably, Kubernetes does not offer “secure-enough” by default configuration. As Red Hat pointed out: how many do you think are securing Kubernetes properly? Not much. 94% of those surveyed admitted they had experienced a Kubernetes and container environments security incident in the last 12 months. Human errors or misconfiguration is a severe problem. Adversaries know, as well as we do, that we’re now shifting to containers and Kubernetes, and the change is fast. As the , Kubernetes clusters are a prime target for cyberattacks, including: NSA indicates data theft, computational power theft (e.g., for crypto-mining), and denial of service attacks. The NSA and CISA during the release of the hardening guide stated: joint announcement The security of applications running in Kubernetes and their third-party dependencies relies on the trustworthiness of the developers and the defense of the development infrastructure. A malicious container or application from a third party could provide cyber actors with a foothold in the cluster. , followed by hijacking Kubernetes clusters for . As a result, if leaving the Kubernetes installation by default, you probably give hackers access to your data and computing power inside the cluster. Presently, data theft is the number one target crypto-mining Vulnerabilities — Inside Out The Kubernetes project, like any large open-source project, has been subject to several security vulnerabilities. Therefore, it is crucial to be aware of new vulnerabilities and have a planned procedure to patch Kubernetes once the updates are released quickly. Here is the list of known vulnerabilities found in K8S: https://www.cvedetails.com/vulnerability-list/vendor_id-15867/product_id-34016/Kubernetes-Kubernetes.html The NSA and CISA also make a particular point regarding including software and hardware dependencies that could be compromised in the supply chain before deployment — even the tiniest mistake can break the system. supply chain threats, One notable example is the Log4j vulnerability that caused a significant breakdown recently. Log4j is, in fact, not a component in K8S but Java, which is widely adopted in applications. As you can see, the focus is not only on the vulnerabilities of K8S itself but on all components inside the cluster. Nothing Easy Around Kubernetes I know those are basics, but it doesn’t mean they are easy to do. For example, patching in Kubernetes environments is challenging. Besides Kubernetes, numerous other programs run with it in a real-world application. If it is simple, there would not be a place for using K8S. It is also valid for securing it. We all know it isn't good to run applications as root. Yet, many privilege by default. As a result, the applications that run within them inherit as root. It is because: Let’s take a look at root access. Kubernetes container services run as root The root user (UID 0) is the default user inside a container. If you don’t specify a non-root user, the container runs as root. Since the majority of container images in public repositories do not swap out the root user, this situation drags on. The key to hardening guidance is standard best practices. Still, the report also presents an in-depth look at using standard security mitigation measures in complex environments, often in the cloud. NSA’s Recommendations Let’s go through the basics first! According to the hardening guide, some steps should be taken in a production K8S environment. undefined in containers and Pods. [ ] Scan for vulnerabilities or misconfigurations Apply principles in containers and Pods. [ ] least privilege Employ and authorization to limit user access and reduce the attack surface. [ ] strong authentication Use to reduce the exposure if compromised. [ ] network segmentation Use firewalls to l [ ] imit excessive network access. Use to protect confidentiality. [ ] encryption undefined to account for the acceptable security risks. [ ] Review regularly all Kubernetes settings and use vulnerability scans Finally, to alert potential malicious activity. [ ] capture and monitor audit logs These are not all but a starting point for gaining some confidence when putting your containerized applications into production. K8S Security Best Practices Apart from the NSA K8S hardening guide, another helpful resource from NIST is the or SP 800–190. This document outlines the Application Container Security Guide container-related security concerns and offers recommendations accordingly. 0# Containers Security Implementing security practices in the CI/CD pipeline and the entire build, store and deploy process of container images would include: securely storing the container images, vulnerability scanning for container images and; managing/ monitoring the container runtime security. But if we consider K8S alone, we need something more specific. At the same time, the NSA’s guide is a great starting point. And if you follow through with the guide and build your K8S cluster, it will be fair enough and confident enough to put it in production. Below are some other areas that need to highlight. 1# Role-Based Access Control (RBAC) RBAC is the frontline of defense in Kubernetes. It lets the user control who can access the API and their permissions. Although RBAC is enabled by default, it may not be enabled if you upgrade from an old version of Kubernetes. Also, the RBAC settings should be checked and reviewed regularly to ensure no unnecessary permissions are granted. Simply having RBAC enabled is not good enough. There are authorization policies that should be careful with and appropriately used. Always use RBAC with the least privilege principle. That means using it to limit users and groups to only the actions and tasks they require. Confirm that Kubernetes service and other user accounts have the minimal privileges needed. Moreover, we need to make sure not to give cluster-wide permissions and not give anyone cluster-admin privileges unless necessary. (For more details, please refer to the official Kubernetes .) RBAC documentation 2# API is the New Endpoint Despite what I mentioned in #1 and #2, access control and transport security of Kubernetes API, Kubernetes API endpoint is another area we should be careful of. Administrators can configure the API endpoint of a Kubernetes cluster as part of a private or public subnet. It is better to configure the Kubernetes API endpoint as a private endpoint by default. Thus, the master inaccessible from the public internet as the API endpoint inside the control plane has a private IP address. This is essential for fully private clusters that don’t have or expose any public IPs. As a result, there would be no ingress/egress of traffic to and from the internet. (For more information, please refer to the official .) documentation 3# Encryption Keys (Kubernetes Secrets) There are different forms of encryption keys, such as a password, a token, or an SSH key. Kubernetes secrets support the initial communication securely between artifacts and pods. For example, when a pod starts, it will require to access its secrets. Also, whenever a service account is created, a Kubernetes secret holding its authorization token is generated on the fly. In addition, Kubernetes supports encryption at rest. This encryption offers an additional level of protection when backups are not encrypted, or an attacker gets read access to etcd. According to the official , we should also make sure SSL/TLS is applied in the communication between: document from the API server to the kubelets, and users and the API server. Lastly, it is recommended to have a short lifetime for a secret or credential to reduce the time an attacker uses them when they are lost. To achieve that, set short lifetimes on certificates and automate key rotation. 4# OS, Nods, and Pods Security To harden and secure K8S, we cannot ignore the basics — OS, Nodes, and Pods Security. The services running on a node include: First, a K8S is a worker node running in an OS that can be a virtual or physical machine. node the container runtime, kubelet and kube-proxy. To keep the focus on K8S, the is a well-known best practice guide. All the security practices on a server, including vulnerability management, anti-malware, EDR, and patch management, need to be in place for a node. Center for Internet Security (CIS) Kubernetes benchmark In addition to workload security, there is a security risk regarding internal communications. It is recommended that a gateway should be configured for access outside the cluster if needed. Network port access on nodes should be managed via network access control lists. Limiting Secure Shell (SSH) access to the nodes should also be considered. On the other hand, So, initially, we need to use to control the communication for pods within a cluster. To achieve that, we need the network plugin implements network policies, and using them may require a network driver that supports policies. a is a group of one or more containers that run on nodes. pod network policies The pod-level network security posture can be strengthened by a combination of network policies and access control list down to host levels. Also, the of K8S supports the mapping between privilege and access-control settings for containers and pods. (For the complete list of security contexts, please refer to the official guide # . pod security context securitycontext Nevertheless, the core of pod security should be i.e., the power of running containers as privileged containers leveraging the host file system, network, and ports. that let a user control the runtime execution properties of the pods, Pod security policies For more information, please refer to the official Pod security policy document. Final Words Your ultimate security goal is to push it as challenging as possible for hackers to gain access to your systems; secure K8S is the same. And even if that happens, you should be good enough to detect abnormal activity and take action to prevent it from spreading. Securing Kubernetes can be a full-time job. The NSA mentions that there are third-party security programs that can help. But, of course, these also come with security concerns. Security is vital for containers, and it’s something you need to take into account and manage when working with Kubernetes. Reference: https://www.cvedetails.com/product/34016/Kubernetes-Kubernetes.html https://www.nsa.gov/Press-Room/News-Highlights/Article/Article/2716980/nsa-cisa-release-kubernetes-hardening-guidance/ https://kubernetes.io/blog/2016/08/security-best-practices-kubernetes-deployment/ Thank you for reading. May InfoSec be with you🖖.

Chain

Target

Interested in Infosec & Biohacking. Security Architect by profession. Love reading and running.

2021 - HackerNoon Contributor of the Year - CULTURE

2022 - HackerNoon Contributor of the Year - Beginners Guide

2022 - HackerNoon Contributor of the Year - Containers

2022 - HackerNoon Contributor of the Year - Cybersecurity

2022 - HackerNoon Contributor of the Year - Data Privacy

2022 - HackerNoon Contributor of the Year - Privacy

2022 - HackerNoon Contributor of the Year - Security

Support Me on Coil

2021 - HackerNoon Contributor of the Year - PERSONAL-DATA

Nominated for 2022 - HackerNoon Contributor of the Year - Cybersecurity

Nominated for 2022 - HackerNoon Contributor of the Year - Security

Nominated for 2022 - HackerNoon Contributor of the Year - Privacy

Nominated for 2022 - HackerNoon Contributor of the Year - Beginners Guide

Nominated for 2022 - HackerNoon Contributor of the Year - Containers

Nominated for 2022 - HackerNoon Contributor of the Year - Data Privacy

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

How to Harden K8S: Based On the  Recent Updated NSA's Kubernetes Hardening Guide

How to Harden K8S: Based On the Recent Updated NSA's Kubernetes Hardening Guide

Too Long; Didn't Read

Company Mentioned

Coin Mentioned

Zen Chan

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

How to Harden K8S: Based On the Recent Updated NSA's Kubernetes Hardening Guide

Too Long; Didn't Read

Company Mentioned

Coin Mentioned

Zen Chan

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES