Kubecost

kubecost

Scaling cloud resources is easy - so easy, in fact, that many teams end up losing control over their cloud spend. A missed bug or architecture oversight can easily snowball into a huge bill at the end of the month. That’s why teams need a cloud cost monitoring and optimization toolkit that provides detailed visibility, exhaustive reporting, and - in an ideal scenario - automated optimization capable of handling the fast-changing requirements of Kubernetes to generate some serious cost savings. Wondering whether you need something more than just cost reporting and analysis? Here’s a comparison of features delivered by two modern cloud-native solutions, Kubecost and CAST AI. CAST AI - Analysis & Automation Kubecost - Analysis Created by industry veterans, CAST AI is a full-service cloud automation platform providing powerful automation features for optimizing Kubernetes workloads.  Companies across industries such as e-commerce and ad tech are using CAST AI to save from 50% to even 90% on their cloud bills. Kubecost started as an open-source tool that provided developers with more visibility into their Kubernetes costs. Today, Kubecost is a robust cost reporting solution that teams can use to get insights into costs allocation, cost monitoring, and alerts - key tools for teams looking to gain visibility. Kubecost vs. CAST AI - quick feature comparison Feature CAST AI 🥇 Kubecost Supported platforms AWS ✅ ✅ Google Cloud Platform ✅ ✅ Microsoft Azure ✅ (coming soon) ✅ Cost optimization and automation Detailed insights on cluster cost optimization ✅ ✅ Recommendations for optimizing cloud costs ✅ ✅ Real-time alerting functionality ✅ (coming soon) ✅ Horizontal pod autoscaling ✅ ✖ AI-driven instance selection ✅ ✖ Multi-shape cluster construction ✅ ✖ Automated pod scaling parameters ✅ ✖ Automatic bin packing ✅ ✖ Spot Instance automation ✅ ✖ Node autoscaling ✅ ✖ Cluster scheduling ✅ ✖ Cost allocation Detailed cost breakdown ✅ ✅ Allocation by organizational concepts ✖ ✅ Cost view across multi-cloud ✅ ✅ Live customer support ✅ ✖ Full multi-cloud optimization ✅ ✖ Detailed feature comparison of Kubecost and CAST AI 1. Installation and setup CAST AI To start saving on your cloud bill with CAST AI, you need to create an account and connect an existing Kubernetes cluster or create a new one inside the tool. Teams often choose to in read-only mode to get a free detailed report of estimated monthly savings - and then take action by turning automated optimization on. It takes only 15 minutes to get the cost analysis and optimize costs automatically. connect their clusters At the moment, CAST AI supports services from AWS and Google Cloud Platform, with Azure support coming in Q4 2021. Supported platforms: Kubecost To install and operate Kubecost, teams can use the Kubecost helm chart. This installation method brings you all the components for getting started, offering access to Kubecost features and an opportunity to scale to large clusters. Teams can also enjoy a lot of flexibility for configuring Kubecost and its dependencies. Kubecost offers three other installation options, but they require effort and come with less flexibility. Supported platforms: Currently, Kubecost supports cloud services from AWS, Google Cloud Platform, and Microsoft Azure. 2. Cost allocation Detailed cost breakdown offers a cost breakdown and forecasting feature at the level of projects, clusters, namespaces, and deployments. You can analyze costs down to individual microservices and generate a detailed forecast of cluster costs. Moreover, CAST AI delivers insights using universal metrics for any cloud service provider from Grafana and Kibana. CAST AI provides flexible and customizable cost breakdown features as well. You can divide costs by namespace, deployment, service, and more indicators across all the three major cloud service providers. Like in CAST AI, this comprehensive resource allocation leads to generating more accurate showbacks and chargebacks, streamlining the ongoing cost monitoring. Kubecost Allocation by organizational concepts Focusing on automated optimization, offers cost allocation per cluster and per node. CAST AI users can allocate costs to concepts such as teams, individual applications, products, projects, departments, or environments. Kubecost Cost view across multi-cloud Many companies are using the services of more than one cloud provider. Allocating costs across clouds is tricky, but rises to this challenge. It supports teams with a unique , providing universal metrics for any cloud provider. CAST AI full multi-cloud functionality and visibility displays the costs across multiple clusters and multi-cloud environments in a single view or through a single API endpoint. However, Kubecost doesn't help you manage multi-cloud infrastructure - while CAST AI offers a full multi-cloud solution with cost optimization. Kubecost 3. Cost monitoring Cost allocation is the first step to understanding where your cloud bill comes from. Next, you need to keep a close eye on how your resource use translates to costs in real-time. ​ displays the biggest cost driver - compute costs - in the Savings estimator and shows potential savings associated with deployments on Spot Instances. It has a planned feature in the pipeline for , including other dimensions such as control plane, network, egress, storage, and others. CAST AI ongoing cloud cost reporting allows teams can link (CPU, memory, storage, network, etc.) with from the cloud services across AWS, GCP, and Azure - for example, tagged RDS instances, BigQuery warehouses, or S3 buckets. Users get context-aware, cluster-level reports to find an optimal balance between cost and performance matching their service requirements. Kubecost real-time in-cluster costs out-of-cluster expenses 4. Cost optimization and automation Once you allocate costs and monitor them regularly, it’s time to take action and start optimizing your spending. Kubecost and CAST AI support teams on this mission differently. CAST AI: Fully automated cost optimization that beats savings plans - this feature uses business metrics to determine the number of required pod instances. It scales the replica count of your pods up and down - and removes pods if there’s no work to be done. Horizontal pod autoscaling - if your cluster needs extra nodes, CAST AI chooses the best instance types that meet your requirements but still help to save up. AI-driven instance selection - CAST AI delivers an optimized mix of different instance types adapted to your application’s needs. Multi-shape cluster construction - to help teams avoid overprovisioning, CAST AI  sets these parameters automatically and maximizes cost savings. Automated pod scaling parameters - since Kubernetes distributes applications within a cluster evenly, it doesn’t really help teams reduce their cloud spend. CAST AI solves this problem via bin packing for maximum savings. Automatic bin packing - Spot Instances can bring dramatic savings of up to 90% off the On-Demand pricing. You don’t need to worry about a provider pulling the plug on your instance - their replacement is fully automated. Spot Instance automation - this feature makes sure that your nodes match your requirements at all times, scaling nodes up and down automatically. Node auto-scaling - automatically pause and resume clusters to avoid paying for resources your teams aren’t using. Cluster scheduling By turning CAST AI automated optimization on, you can save from 50 to 90% on your cloud spend. Savings: Kubecost: Cluster-level insights and recommendations for engineers to implement Kubecost provides detailed reports and Delivered via Slack or email, these alerts notify teams about budget overruns, anomalous spend patterns, and Kubernetes tenants that fall below the set efficiency levels. Users can set budgets for configurable aggregation levels - for example, team or application. real-time alerting functionality. Savings: Kubecost generates insights DevOps engineers can use to save 30-50% or more. 5. Security Since both Kubecost and CAST AI work with your cloud infrastructure, their security is paramount. offers many security features such as encryption at rest/in transit, secrets management, network security, logging, visibility, and more. Moreover, it provides automatic patching and upgrades to VMs and Kubernetes, so you’re always kept up to date and eliminate the chance of errors in your clusters. CAST AI doesn’t expose private data anywhere, and since users deploy Kubecost in their infrastructure, there’s no need to egress any data to a remote service. You retain, and control access to sensitive cloud spend data at all times. Kubecost 6. Pricing comes with a free Cost Savings report users can run anytime they want to check whether they could save up on their infrastructure. The report generates actionable recommendations. And if you want to add automated optimization into the mix, you can choose between two plans: Growth and Enterprise. In all cases, CAST AI offers CAST AI guaranteed savings of 50%. All plans are free of charge for the first 30 days. Kubecost also offers a free plan where you can monitor and optimize one cluster. To make the most of your paid plan, you’ll need to dedicate time to implement the recommendations provided by Kubecost. This will incur extra charges and doesn’t automatically guarantee savings. Kubecost Summary: When to choose Kubecost vs. CAST AI Both Kubecost and CAST AT are great picks that offer lots of value to Kubernetes teams looking to optimize their cloud bills and streamline processes related to cost monitoring, allocation, and reporting. But if you’d like more than reporting, the automated optimization features of CAST AI are at the top among cloud cost optimization platforms. Bringing Kubecost and CAST AI together will give you an end-to-end solution that will keep your cloud costs in check and generate some pretty impressive savings. BTW. You can use KubeCost together with CAST AI.

Kubecost vs. CAST AI: A Perfect Match for End-to-End Cloud Cost Optimization

Running Kubernetes in the cloud isn’t cheap, and cloud providers understand that their customers want options to reduce the cost of Kubernetes. AWS, GCP, and Azure currently support using “spot” (or “preemptible”) nodes in Kubernetes clusters. generally come from excess compute capacity in a data center and are offered by cloud providers at a discount (up to 90%!)—with the caveat that the provider can take that node away on very short notice. Spot nodes steep The Kubernetes scheduler is adept at moving workloads when available compute resources change (and is getting even better on GCP and AWS, thanks to the feature), making Kubernetes well-suited to using spot nodes for computing. graceful node shutdown However, identifying workloads that are safe to run on-spot—and then selecting the correct node types and quantities to handle the split of spot-ready and non-spot-ready workloads—is no easy task. That’s where Kubecost’s Spot Commander can help! Analyze your cluster’s workloads for spot-readiness First, Kubecost analyzes the workloads running on your cluster to determine their spot-readiness using the Spot Checklist feature. We’ve written in detail in a about how the Spot Checklist works. At a high level, it uses information from the Kubernetes API and heuristics to identify workloads that are spot-ready. previous post Suggest cluster configuration to support spot-ready and non-spot-ready workloads Once Kubecost has identified which workloads on your cluster can run on-spot, it’s time to optimize your cluster configuration. Kubecost can leverage its of (1) your workloads’ usage patterns and (2) your cloud provider’s node capacities and pricing to suggest the least expensive node types and quantities to support both your spot-ready and non-spot-ready workloads. extensive knowledge If you have specific constraints you want to impose on this process as minimum node counts for node pools, a certain resource overhead to have available on all nodes, or a restriction on using shared-core machine types, you can configure these in the UI!! Your overall savings for adopting the new cluster configuration is calculated by comparing the current run-rate of your cluster’s nodes and with the approximate run-rate of the new configuration. For a more in-depth look at how we determine cluster configuration, see our . official documentation Adopting the recommended configuration To realize those estimated savings, you’ll need to update your cluster configuration to include the new spot nodes, reduce the number of on-demand nodes, and also make sure that only spot-ready workloads are scheduled on spot nodes. We’ll walk through the process together in the steps below. First, we have to let Kubernetes know which workloads are spot-ready. To do so, we’re going to use to make the spot-ready workloads “tolerate” running on a spot node. We might add the following toleration to Deployment A: taints and tolerations tolerations: 
- key: "spotnode"
  operator: "exists"
  effect: "NoSchedule" Next, create a new set of spot nodes in your cluster (a “node pool” in GKE or “nodegroup” on EKS). Follow your provider’s documentation! When creating the nodes, make sure to add the equivalent taint so that only workloads with the appropriate spot toleration get scheduled on those nodes. If the tools you are using don’t support tainting at node creation time, you can always use good ol’ : kubectl kubectl taint nodes SpotNode1 spotnode=true:NoSchedule Finally, alter the set of on-demand nodes in your cluster to be in line with Kubecost’s recommendation. You can do this by resizing your current set of on-demand nodes or by creating a new set and deleting the old one. Once you’ve removed some of the current on-demand capacity in the cluster, you should notice the workloads with the new spot toleration getting scheduled onto the new spot nodes! If you’re using GKE, Spot Commander can generate a set of and commands that you can run in your terminal to adopt the recommended cluster configuration. These commands add tolerations to spot-ready workloads, create the new spot and on-demand node pools, and then delete old node pools. gcloud kubectl Start implementing spot nodes using Spot Commander for free Spot Commander is available as part of the free community version of Kubecost, a tool that can help you monitor cluster costs and implement spot node savings for your Kubernetes workloads. Usage data collected by Kubecost never leaves your cluster, allowing for quick adoption without the need for lengthy internal security and data governance reviews. To get started, just install Kubecost with Helm (takes ~2 minutes): helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm upgrade -i --create-namespace kubecost kubecost/cost-analyzer --namespace kubecost --set kubecostToken="QnJkV0psWTI5emRDNWpiMjA9eG0zNDN" installation instructions can be found at https://www.kubecost.com/install.html. Kubecost marking a workload as spot-ready is not a guarantee. A domain expert should always carefully consider the workload before approving it to run on spot nodes. A final note: Particularly complex clusters may need a more involved node configuration than Kubecost currently provides, but we hope our recommendations and the promise of our estimated savings will motivate your infrastructure teams to start adopting spot and realizing those awesome cost reductions. If you run a cluster, Kubecost’s estimated savings may not be accurate, because moving a substantial number of nodes to spot can have a major impact on spot node availability. Do your research if you think you’re in this category! Your provider probably has a rep you can ask. massive

Kubernetes Clusters: Intelligently Adapt To Use Spot Nodes

Over the past two years at Magalix, we have focused on building our system, introducing new features, and scaling our infrastructure and microservices. During this time, we had a look at our Kubernetes clusters utilization and found it to be very low. We were paying for resources we didn’t use, so we started a cost-saving practice to increase cluster utilization, use the resources we already had and pay less to run our cluster. In this article, I will discuss the top five techniques we used to better utilize our Kubernetes clusters on the cloud and eliminate wasted resources, thus saving money. In the end, we were able to cut our monthly bill by more than 50%! 1. Applying Workload Right-Sizing Kubernetes manages and schedules pods are based on container resource specs: : Kubernetes scheduler is used to place containers on the right node which has enough capacity Resource Requests : Containers are NOT allowed to use more than their resource limit Resource Limits Resources requests and limits are container-scooped specs, while multi-container pods define separate resource specs for each container: : apps/v1 : Deployment : : nginx : magalix : : : : nginx : : : : : nginx : :
      - : nginx : : . :
        - : : : : m : Mi : : : Gi apiVersion kind metadata name namespace spec selector matchLabels app replicas 2 template metadata labels app spec containers name image nginx 1.14 2 ports containerPort 80 resources requests cpu 100 memory 100 limits cpu 1 memory 1 Kubernetes schedules pods based on resource requests and other restrictions without impairing availability. The scheduler uses CPU and memory resource requests to schedule the workloads in the right nodes, control which pod works on which node and if multiple pods can schedule together on a single node. Every node type has its own allocatable CPU and memory capacities. Assigning high/unneeded CPU or memory resource requests can end up running underutilized pods on each node, which leads to underutilized nodes. In this section, we compared resource requests, limited against actual usage and changed the resource request to something closer to the actual utilization while adding a little safety margin. 2. Choosing The Right Worker Nodes Every Kubernetes cluster has its own special workload utilization. Some clusters use memory more than CPU (e.g: database and caching workloads), while others use CPU more than memory (e.g: user-interactive and batch-processing workloads) Cloud providers such as GCP and AWS offer various node types that you can choose from. Choosing the wrong node size for your cluster can end up costing you. For instance, choosing high CPU-to-memory ratio nodes for workloads that use memory extensively can starve for memory easily and trigger auto node scale-up, wasting more CPUs that we don’t need. Calculating the right ratio of CPU-to-memory isn’t easy; you will need to monitor and know your workloads well. For example, GCP offers general purpose, compute-optimized, memory-optimized with various CPU and memory count and ratios: Just keep in mind that 1 vCPU is way more expensive than 1GB memory. I have enough memory in the clusters I manage so I try to make sure that when there is a pending pod, this pod is pending on CPUs (which is expensive) so the autoscaler triggers a scale-up for the new node. To see the cost difference between CPU and memory, let us look at the GCP N2 machine price. GCP gives you the freedom to choose a custom machine type: (# vCPU x vCPU price) + (# GB memory x GB memory price) 1 1 It’s clear here that the 1vCPU costs 7.44 times more than the cost of 1GB. 3. Autoscaling Workloads Autoscaling is great because it helps to scale up/down your workloads and shut down nodes to save you money while you sleep. Many cases can benefit from autoscaling: 1. Variable load web applications: A good example would be a web application which receives variable traffic through the day: traffic increases during certain hours of the day and decreases during the night. Kubernetes comes with the Horizontal Pod Autoscaler (HPA) that can scale workload replicas based on their CPU and memory utilization ratio to the resource request. Kubernetes will keep monitoring the target resource in scaleTargetRef and will scale up/down the replica count to keep targetCPUUtilizationPercentage around 75%. In this example, the deployment frontend will be scaled up to 20 replicas during the high load, and 4 replicas during the low load. apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: frontend
  namespace: magalix
spec:
  maxReplicas: minReplicas: scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    Name: frontend
  targetCPUUtilizationPercentage: 20 4 75 2. Event-driven workers: background workers that need to be started in multiple replicas when there are messages in a Kafka topic or a message queue. It can be scaled to zero when there are no messages. Compared to HPA, there is a more advanced Kubernetes Event-driven Autoscaling ( ) that can integrate with Prometheus/PosqreSQL/Kafka/Redis and many more to scale based on more advanced metrics from multiple data sources. In this example, we installed this KEDA ScaledObject custom resource definition to scale the worker deployment eventer replicas. When the Kafka consumer lag changes, it can scale to 0 when there are no messages to consume and can scale up 1 replica for every 10,000 lagged messages up to 8 when the lag is more than 80,000: KEDA apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
  labels:
    deploymentName: eventer
  name: eventer
  namespace: magalix
spec:
  cooldownPeriod: maxReplicaCount: minReplicaCount: pollingInterval: scaleTargetRef:
    deploymentName: eventer
  triggers:
  - metadata:
      metricName: eventer
      query: sum(kafka_consumergroup_lag{consumergroup= })
      serverAddress: http: threshold: type: prometheus 10 8 0 15 "eventer-group" //prometheus-server.monitoring.svc.cluster.local "10000" 4. Autoscaling Worker Nodes After scaling workloads, you will notice the number of running pods is low, but this won’t save you money unless we configure auto-scaling the worker nodes Some Could providers provide node autoscaling out of the box on some node pools. can help you manage worker node autoscaling. Cluster Autoscaler GCP: Kubernetes Engine → Cluster → Node Pool AWS: EKS → Cluster → Node Group Azure: Kubernetes services → Node pools → Scale → Automatic The result of scaling workload and worker nodes together can end with the node count trends: 5. Purchasing Commitment/Saving Plans Running a Kubernetes service is relatively cheap. What’s most expensive is the worker nodes compute cost. offers “Commitment Plans” on a certain number of vCPUs, memory, GPUs, and local SSDs for 1 or 3 years. This can save up to 57% of the compute cost GCP offers “Compute Saving Plans” to commit to using a certain amount of money on compute every month, as well as “Reserved Instances” to commit to using a certain type of machine, both for 1 or 3 years with a possible savings of up to 60% AWS offers “Azure Reserved VM Instances” such as AWS with a possible savings of up to 60% Azure Conclusion As we read in this article, there are multiple factors and considerations when trying to reduce your cluster cost. Going through the whole process can give you huge savings. - and you can do the same! We have managed to reduce our cluster daily cost by 56% Also published here.

Kubernetes: Monitoring, Reducing, and Optimizing Your Costs

101 Stories To Learn About Cloud Infrastructure

The container-native workflow engine for Kubernetes is a tremendously popular open source option for teams looking to run workflows, manage clusters, orchestrate parallel jobs, and more. Argo Pairing Kubernetes and Argo, multiple dev teams can run workloads on centrally-managed clusters, sharing compute resources to increase efficiency and reduce overhead. However, keeping track of each team’s spending—and spending efficiency—in the Kubernetes shared resources environment has become an all-too-familiar challenge. Teams routinely struggle with budgeting projections and waste reduction efforts without visibility into historical resource usage data and the ability to break down cost responsibilities by team. That visibility is crucial to addressing minor inefficiencies before they grow and lead to significant waste as workflows scale. [A completed Argo workflow, its costs completely unknown] Kubecost, a Kubernetes cost monitoring tool with a , enables dev teams to accurately measure Argo workflow costs in real-time, allocate costs to teams, recognize resource efficiencies, and suggest opportunities for optimization. Enter Kubecost, Argo’s spend-conscious companion free community version The following guide demonstrates how teams managing Kubernetes workloads with Argo Workflows can deploy Kubecost to analyze and control their workflow spending. Those interested in beginning to leverage Argo Workflows should follow this on GitHub.   Kubecost itself can be installed using , and should be given its own namespace: simple Quick Start guide Helm Once installed, Kubecost will collect cost data on running Argo Workflows. For this guide, let’s define an Argo Workflow by creating a CRD. We’ll make the workflow mimic high resource usage by having the Docker container make resource-consuming Pods. The workflow will perform its single “stress” stage, running the stress-ng image, along with inputs Argo will handle in the launched workflow.
 
To create the CRD in the Argo namespace, save the following yaml to a file and run . Create an example workflow WorkflowTemplate stress-ng WorkflowTemplate argo template create $filename -n argo The workflow requests 1200m CPU and 40M memory, while the running stress-ng container uses 1 CPU and 30M memory. That gap represents wasteful inefficiency, as the over-provisioned container uses more cluster compute resources than it requires to operate. Because Argo Workflow jobs all feature the metadata label , viewing each workflow’s costs and efficiency metrics within Kubecost is simple. In the Kubecost UI, go to the Cost Allocation page and change the “Aggregate By” field to “Label: workflows_argoproj_io_workflow.” This will display the following per-workflow cost and efficiency data: Measure Kubernetes costs with Kubecost workflows.argoproj.io/workflow Each time a workflow runs, that run receives a unique name based on the template and an added identifier. In the below example, the stress-ng workflow’s first run is named stress-ng-q9wdz. Filtering to this specific workflow run, Kubecost displays that its cost efficiency was just 57%. Clicking the workflow name to open the Details view shows the degree to which CPU and RAM overprovisioning caused this inefficiency: only 55% of requested CPU and 60% of requested RAM actually went to use! Clusters with Kubecost installed are available for inspection using the kubectl-cost CLI tool as well as the UI. Use Krew to install kubectl-cost: In the CLI, workflow costs are available with the command Use the kubectl-cost CLI for quick cost inspection kubectl krew install cost kubectl cost label --label workflows.argoproj.io/workflow : Boost the precision of cost insights The Kubecost tool uses the major cloud providers’ publicly-listed prices by default, but integrating will yield even more precise cost analysis. Doing so can also account for any custom discounts, reserved instances, or other pricing differences that might be relevant to your environment. actual cloud provider billing data

How to Optimize Kubernetes and Argo Workflow Spending

What you always have as a Kubernetes cluster is usually always the bare minimum. Be it a cluster you set up yourself in a cluster of RasberryPi’s or using Amazon EKS or any other form you set up your Kubernetes cluster, it is NEVER production-ready. The keyword here is production-ready. What do we mean when we say production-ready. In very simple terms, it means it is ready to face the world. It is ready to face the good the bad and the ugly traffic that will come to it and still stand strong despite whatever comes to it. Putting this in perspective, it is quite important to find a bench-mark to work with in determining if our cluster is production-ready. The following checklist is my best recommendation to follow to know if your cluster is production-ready, . https://learnk8s.io/production-best-practices In this article, I am looking at tools to have that will make your cluster production-ready. I shall be using the 5 pillars of the to categorize the different types of tools/services that you must have in the cluster to dim it fit for production. Take note, that this list is bare-minimum, the number of services you can deploy in your cluster is endless, but you do not want to deploy so much. AWS Well-Architected Framework The Categories The categories for the different tools are as follows: Reliability and Availability Security Network, Monitoring & Observability Backup/Recovery Cost Optimization Cluster Visualization Reliability and Availability These two terms are actually mean different things; reliability means that a system is doing what is designed to do effectively, while availability is when a system is accessible by the user. These two are very necessary for your applications in your Kubernetes cluster. There are tools that can help your applications to be highly available. Tools Horizontal Pod Autoscaler (HPA) Karpenter (https://github.com/aws/karpenter). Cluster-Autoscaler ( ). https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler Goldilocks ( ). https://github.com/FairwindsOps/goldilocks While HPA is for scaling pods based on CPU and Memory metrics, Karpenter and Cluster-Autoscaler scale nodes are based on the pod capacity of the nodes. Goldilocks helps you recommend the appropriate CPU and Memory metrics for your pods based on how much has been consumed over a period of time. Security The security of your cluster is a very essential part of running workloads in the Kubernetes cluster. Security in Kubernetes is of two folds. First is the security from an external attack on your API server which is usually secured with certificates, but an extra later of putting the API server in a private network reduces the surface area of attack because it is not exposed to the public internet. The other security operations have to do with scanning your cluster for proper configurations to ensure the pods running follow proper security standards. Here are some security tools that can help scan container-based workloads and Kubernetes clusters. Tools Snyk ( ) https://snyk.io Kubelinter ( ) https://github.com/stackrox/kube-linter Kubehunter ( ) https://github.com/aquasecurity/kube-hunter Checkov ( ) https://checkov.io Kube-Scan ( ) https://github.com/octarinesec/kube-scan Network, Monitoring & Observability What is a Kubernetes cluster without its network? The network is used to send and receive traffic within the cluster and outside the cluster. It is important to be able to sketch a flow of traffic within the cluster. It makes it easy to identify bottlenecks like; slow or unresponsive microservices. This is where a comes into play. service mesh ServiceMesh Tools Istio ( ) https://istio.io Linkrd ( ) https://linkrd.io AWS AppMesh (https://aws.amazon.com/app-mesh) Consul ( ) https://www.consul.io/ Monitoring & Observability The availability and Mean Time to Recover (MTTR), of your system is high proportionate to the depth of monitoring and observability in the system. Monitoring helps to collect logs and metrics of worker nodes and pods/applications running within the cluster. To know more about logs and metrics get my book on . There are many monitoring tools out there. But I will pick about three of them that stand out in different scenarios. Observability on the other end is a combination of Application Performance Management (APM), bug tracing and application tracing, with the focus on knowing what is going on within the application. Amazon CloudWatch Monitoring Tools/Services Prometheus (prometheus.io) — for collecting metrics. Grafana (grafana.net) — for data visualization of data from different sources. Amazon CloudWatch Container Insights. Netdata ( ). https://netdata.cloud Observability Tools Amazon X-Ray ( ). https://aws.amazon.com/xray/ Datadog (https://datadog.com). AWS CloudMap ( ). https://aws.amazon.com/cloud-map/ Jaegar ( ). https://www.jaegertracing.io/ Backup/Recovery Backing up all components of your cluster can be a lifesaver in times of disaster or during a migration process. Deployments, Pods, Secrets, ConfigMaps, ClusterRole, ClusterRoleBinding, and all other resources in Kubernetes. The backup of the cluster is stored as a zip file in any of the popular object storage services such as , , . This data can also be retrieved at any time or to an identical cluster with a single command and all resources backed up will be automatically restored. It can also be configured to run a periodic backup of the cluster resources. Amazon S3 Azure Blob Storage Google Cloud Storage Tool(s) ( ) Velero https://velero.io Cost Optimization Most times the cost which is tangible for a Kubernetes cluster in a cloud environment is usually the cost of the Master Node (Control Plane) and the worker nodes and other underlying infrastructure with which they are running on. These costs are clearly displayed in the Billing dashboard of the cloud provider. They can be optimized by choosing better pricing option for the nodes within the cluster. But you can granularly check cost from pod consumption and cascade that cost to numerical dollar value. This can give very interesting insights on application that are costing more to run. Tool(s) Kubecost ( ) https://www.kubecost.com/ Cluster Visualization By deafult, Kubernetes has a dashboard. But most times this dashboard is barely used and it has very limited features which forces most Kubernetes Administrators to learn commands to get what they want. But sometimes, it can be time consumting, trying to find the right command to check for certain resources and perform a comparison or get quick feedback on the behavior of pods or services within the cluster. Some tools actually make managing Kubernetes cluster more fun. kubectl Tool(s) Lens ( ) https://k8slens.dev/ Rancher ( ) https://rancher.com Conclusion There is no perfect Kubernetes cluster, the tools mentioned here are suggestions of tools that you should have in your cluster to get the best out of it. At least one tools from the five sections mentioned is needed to have proper visibility and governance of your Kubernetes cluster. Also published . here

Kubernetes Cluster Must-Haves To Be Production Ready

The cyber security market is always coming up with new technologies and approaches to counter the challenges in the evolving threat landscape. The security threats and challenges have become more complex over the years as organizations have moved most of their applications to the cloud. Moreover, organizations have realized that the typical lift and shift approach to cloud migration is not the silver bullet and can be costly. Hence, they now adopt a careful approach to choosing the right workloads for the cloud and retaining others to their private servers. Though beneficial, this hybrid approach poses significant challenges: ·       Security, compliance, data-locality challenges ·       Hybrid/multi-cloud cost and performance optimization challenges The IT and security teams in most enterprises have tried to upgrade their threat detection and mitigation processes for the complex hybrid and cloud-native environments. New tools, protocols, and playbooks for monitoring, security incident/event management, and threat response have offered varying levels of success. However, cloud security observability has emerged as the most viable approach to solving the above challenges. What is Cloud Security Observability? The cloud security observability approach involves applying the observability concepts to cloud security operations to gain insights into the health of an application by gauging workloads based on their external properties such as metrics, logs, and traces. The keyword here is context; observability builds on existing monitoring technologies but also goes a step ahead, adding context to all sorts of disparate telemetry data from different systems. Again, observability is an attribute of systems; the more observable a system is, the more efficient, secure, and reliable it becomes. Cloud Security Challenges While much has been written and promoted by security vendors on the topics of cloud security and observability, there’s still no definitive answer, and enterprises have to evaluate the pros and cons of different approaches. Most traditional vendors have added a layer of AI to their log management and analysis tools and now claim to offer observability in cloud-native environments. It is important to understand here that log analysis gets significantly complicated in hybrid cloud environments due to the lack of centralization, log format variations, and moving or transient components (containers). Sometimes the logs are overwritten by the time an issue is detected, while at other times, the problem detected via logs becomes difficult to replicate. This is where the AI part of the commercial solutions offers some advantage by simplifying tracing and correlation of log entries. Yet, to make the most of such solutions, development and operations teams need to fully appreciate and understand the complexity of their cloud-native deployments. They might be dealing with hundreds of microservices linked in a service mesh with deployment on Kubernetes, that too on an elastic cloud. An AI-powered cloud observability platform can be useful only when the users understand how their applications are deployed and what datasets can provide the desired insights. The Building Blocks of Cloud Security Observability It involves aggregation and analysis of logs and metrics from the K8s control pane, cluster nodes, application resources (CPU, RAM), and ranking of nodes, workspace, and other resources by usage. While evaluating the solutions, teams should explore the workflows to drill down to details of workloads, pods, containers, etc., to identify issues. Cloud-Native (Kubernetes) Observability: It involves leveraging operational or telemetry data early in the software development lifecycle. Many times, developers lack visibility into real-world resource usage and are only exposed to issues and events. While tools like Terraform automate infrastructure provisioning as per the code, developers usually have no means of visualizing the performance, bugs, etc. Early visibility into operational metrics can allow developers to test and fine-tune their applications and reduce noise/false alarms later in the cycle. DevOps Enhancement: As organizations embrace hybrid and multi-cloud environments, they need to monitor different dashboards to assess security, availability, performance, and costs. As there is usually no unified dashboard, it becomes difficult to get a holistic view of an application’s health and performance. There’s a need to collect data from different clouds for unified analytics and automation. Multi-cloud Observability: AI and machine learning algorithms can help organizations in finding trends and patterns that otherwise remain undetected via standard security policies and threshold and event-based alerts. From analyzing huge volumes of logs and metrics to correlating events across systems, AI can offer significant advantages over traditional methods of incident management. AI-Enabled Insights & Actions: A Smart Approach to Observability It is easy to get overwhelmed by the commercial tools and solutions, with a wide range of overlapping features and capabilities. The chances are that your existing tools and security solutions may offer similar features and capabilities but may seem inadequate due to the lack of integration. Some enterprise teams also dive into the open-source ocean, exploring potential solutions with tools like Nagios, Prometheus, Elastic, FluentD, Grafana, Jaeger, and more. However, there’s an alternate approach to building such capabilities by starting small and building iterative solutions – a bottoms-up approach to improving observability, if you will. Here's an example; for one of our clients using Klera’s no-code application builder platform. Klera offers several out-of-the-box connectors for DevOps tools (Jenkins, Jira, GitHub, SonarQube, TestRail, etc.), cloud platforms (AWS, Google Cloud, Kubecost, Microsoft Azure), and databases. we created a DevOps observability solution With these connectors, we were able to collect data from different sources, apply computations using excel-like custom formulas and Klera’s own formula engine, and build dashboards for unified visibility. We were able to leverage some of Klera’s existing apps and templates from its App Store, which provided a significant head start in the development. Here’s a sneak peek into the solution: The first dashboard in the solution provides a high-level view under the five major parameters, viz. availability, security score, SLA adherence, cloud spend, and KPI index. These are aggregated scores based on several metrics and information from different sub-systems. One can drill down to details of every score and find contextual details down to the code level. The KPI Index, for instance, is based on and two other quality metrics, i.e., average test coverage and quality violations. DORA metrics It is possible to introduce more metrics or adjust their thresholds as per evolving needs. The solution also provides visibility into cloud costs and resource usage: The solution is highly flexible and extensible. Klera’s connectors make it simple to aggregate and centrally visualize data from SIEM tools, vulnerability scanners, threat intelligence feeds, and other existing enterprise sources and develop solutions for your cloud and security operations command center. It offers a practical and simple approach to improving cloud security observability in an iterative manner without getting into lengthy POCs with all-encompassing, complex, and costly solutions.

Cloud Security Observability: Get Set, Go!

Kubernetes has become the de-facto choice for most users when it comes to container orchestration. The popularity of cloud-native developments has brought technologies like containerization and microservices to the forefront. Tools like Kubernetes play a key role by providing a feature-rich, robust and secure orchestration platform for containerization. One aspect that any Kubernetes administrator must look into is managing Kubernetes costs. So, in this article, let’s look into some ways to reduce overall Kubernetes costs. Cluster and Infrastructure Monitoring The primary way to manage costs is to properly monitor the cluster environment, including underlying or dependent resources. Monitoring resource utilization and overall costs are the first steps towards reducing costs regardless of whether you use a managed K8s cluster or a self-hosted one. It allows users to gain a better understanding of computing, storage, network utilization, etc., and how costs are distributed between them. In-built tools and basic monitoring functionality offered by cloud providers can help in this aspect. However, it’s always advisable to utilize tools such as Prometheus, Kubecost, and Replex for a comprehensive understanding. It enables K8s admins to gain a more comprehensive view of the environment and optimize the costs. Pod and Node Rightsizing One of the easiest ways to reduce costs is to manage the resources used by Pods and Nodes. While it is always advisable to have enough headroom, overprovisioning or allowing applications to use unlimited resources can lead to disastrous consequences. For example, let’s assume that a Pod consumes all the available memory from the node due to an application error unnecessarily utilizing resources and making other pods starve for resources. Users can prevent such situations by limiting resource utilization with at a namespace level. Additionally, they can specify resource requests and limits at a container level that enforce how many resources a container can request and the maximum limit. Kubernetes Resource Quotas and Limits Ranges Rightsizing nodes is up to the resources used by Pods. You can safely downsize the nodes reducing costs if you have a workload that utilizes only 50% of the resources available in the node, and you do not see a resource usage increase in the near future. Another consideration is the number of Pods that can be run on a single node. Even though there is no hard-set limit, running a large number of Pods can lead to inefficient resource utilization. Due to this issue, managed K8s service providers like EKS and AKS have limited the number of Pods that can be run on a node. Kubernetes Scheduling After rightsizing pods and nodes, the next step is to ensure that the correct pods get scheduled on the correct nodes. The K8s scheduling process matches pods with nodes, and the default behavior of the scheduler can be customized to benefit users. Assume that you want to put containers with business-critical functionality on a high-performance node and other less critical components on relatively lower-performance nodes. By default, K8s does not have a way to match the correct nodes even if you provision nodes on different performance tiers. This problem can lead to wasted performance and ultimately increased costs if a non-critical pod gets scheduled on a high-performance node. Kubernetes provides features such as nodeSelector, Affinity, and to mitigate this issue and optimize the scheduling. They can be used to fully to match user needs, thus allowing users to efficiently use the resources available throughout the nodes. Taints and Tolerations configure the Kubernetes scheduling process Streamlining Development Every aspect of the development need not be containerized. Some development teams try to containerize applications or workloads for the sake of containerization, which can lead to an unnecessary amount of workloads running on a Kubernetes cluster. This workload can be easily offloaded to other technologies, and more often, it will be comparatively cheaper. A good example is to use serverless functions for event-based functionality and mainly Kubernetes for high availability and mission-critical functionality. Implementing Best Practices Across the Delivery Pipeline Implementing cloud-native best practices across the delivery pipeline can be a tedious and time-consuming task. DevOps has greatly reduced the gap between Dev and Ops aspects of the delivery pipeline and allowed users to create robust and flexible delivery pipelines. The best way to reduce the time and effort to deploy containers to the Kubernetes cluster is to integrate K8s deployments as an automated part of the delivery pipeline. Practices like at an infrastructure level, greatly reducing the deployment workloads of the team. GitOps are tailor-made to facilitate continuous delivery An upfront investment will be needed to properly set up everything from the continuous integration to build, test, and publish containers and continuous delivery to deploy these containers on the cluster. Tools like Jenkins for CI, . A properly integrated delivery pipeline can have far-reaching benefits in the long term to reduce deployment workload and organically introduce best practices and standardization across the development environment. Additionally, it will significantly reduce the chance of misconfigurations or human errors that cause errors within the cluster and disruptions in the application leading to less troubleshooting. This reduced workload will allow teams to focus on more valuable tasks such as functionality development, bug fixes, and improving the security posture of the environment. ArgoCD, and Flux for CD are ideal for creating such delivery pipelines Conclusion One good way to reduce costs is to utilize different cloud providers and set up a multi-cloud environment. This multi-cloud environment enables users to benefit from cost-savings offered by each platform and even move workloads between platforms to the most affordable option without service disruptions or decreases in service quality. Kubernetes will manage all these things. Another option is to use different technologies and offload functionality to the technology or service that best matches the requirement. This option allows greater control over costs while efficiently managing the entire application. CloudPlex is one of the best options for all these needs, allowing users to manage and run workloads across multiple cloud platforms and different technologies using a single platform. CloudPlex offers features such as a drag and drop tool to create K8s manifests or helm charts and comprehensive debugging functionality directly through your IDE. It even provides a complete monitoring environment to monitor your containers throughout their lifecycle. Why not give for your next Kubernetes development needs. CloudPlex a try Asad Faizi Founder CEO CloudPlex.io, Inc asad@cloudplex.io Also published behind a paywall.

Reducing Kubernetes Costs

If you’re terminal-loving engineers using Kubernetes like us, kubectl is your constant companion. That’s why the Kubecost team built a new CLI for interacting with Kubernetes cost data. Today, we’re announcing , an open-source extension that fits directly into your workflow. kubectl cost Here’s an example of it in action: For teams running Kubernetes, gaining visibility into the cost of resources can be hard, especially in dynamic multi-tenant environments. helps teams—developers, managers, executives, and everyone in-between—running Kubernetes understand their cluster spend precisely, all the way down to the pod level. One of our primary goals is to provide cost visibility into tools that teams are already using. Kubecost Kubectl cost is a , a CLI tool designed for application and infrastructure engineers who interact directly with Kubernetes on a regular basis. They would benefit from understanding the costs of their various deployments and other in-cluster resources. With reporting a few keystrokes away on the familiar interface of kubectl, the cost plugin gives engineers quick and easy access to cost and spend efficiency information. At Kubecost, we believe that by making cost metrics more accessible we can help our users build more cost-efficient software systems. Let’s check it out. kubectl plugin Using kubectl cost The basic structure of is built around various aggregations that map to Kubernetes concepts, e.g. namespace, label, deployment, statefulset, etc. It leverages Kubecost APIs which provide users with programmatic access to useful data about workloads and assets in your clusters. kubectl cost Let’s have a look at a few samples: One of the simplest queries, showing a monthly cost projection for each namespace in your cluster based on the past day of activity. This accounts for all workloads and their resources in each label. Total cost, over the past day, broken down by the value of the app label. This accounts for labels applied at both the namespace and workload-level. Monthly rate projection broken down by deployment, including a more detailed look at CPU cost and CPU cost efficiency. How much each namespace cost last month, including CPU cost and efficiency. How much each namespace cost yesterday, with all resource-specific costs and efficiencies displayed. This highlights the granular detail at which Kubecost collects cost metrics. How it works By default, Kubecost integrates with cloud provider billing APIs with support for AWS/EKS, Azure/AKS, GCP/GKE, and on-prem clusters via custom pricing sheets. Kubecost can integrate with your cloud provider billing data directly to accurately reflect enterprise discounts, spot prices, reserved instances, savings plans, and more. Compute and memory costs are calculated using the max of resource usage and resource requests. Network costs are optional and are enabled via the . network-costs daemonset Monthly rate projections are based on average run rate for each individual workload over the measured window. Learning more There’s tons more to discover, check out the and kubectl cost --help to keep going. We’re also going to be adding more functionality in the coming weeks. So or reach out to team@kubecost.com if you have questions or additional features you’d like to see! documentation join our Slack community Previously published at http://blog.kubecost.com/blog/kubectl-cost-kubernetes-monitoring-cli/

Meet kubectl cost: A Tool For Determining Kubernetes Workloads Cost

Welcome to this issue of the Activation Function Newsletter. For those of you who are new, it’s pretty simple - Every other week, I introduce you to a new and exciting open-source backend technology (that you’ve probably only kind of heard about ) and explain it to you in 5 minutes or less so you can make better technical decisions moving forward. In this issue, we’ll explore K8sGPT, an open-source tool that combines expert knowledge and Generative AI to help you identify, understand, and troubleshoot your Kubernetes cluster in plain English. Before you dismiss it as “just another ChatGPT wrapper,” let me assure you that it’s not. It’s much more than that. I think it’s a thoughtful implementation of GenAI that has the potential to bring tangible benefits and make your life a bit easier. Alright, Let’s dig in! Quick Facts: K8sGPT was launched at KubeCon Amsterdam 2023. K8sGPT is open-sourced under an Apache-2.0 license. K8sGPT was accepted to CNCF on December 19, 2023, at the Sandbox maturity level. K8sGPT now has over 4K GitHub stars and 50 contributors. Introduction For years, we’ve used ML for APM and Observability to identify patterns, trends, and anomalies (e.g., behavioral changes in applications or infrastructure), anticipate performance issues, generate early warnings, etc. But so far, it hasn’t been great at actually helping us understand and fix issues. The story goes something like this. You get an alert from your ML-enabled monitoring tool and then spend a lot of time looking through and interpreting complex error messages, trying to communicate those to your team, and iterating through remediation steps until you fix the problem. Enters K8sGPT. Its primary function is to help you quickly triage and remedy issues by giving hints and shortcuts in plain English. It does this by leveraging expert knowledge, Natural Language Processing (NLP), and Generative AI. How K8sGPT Works K8sGPT uses contain codified knowledge, which is essentially a series of rules/checks that an experienced Site Reliability Engineer (SRE) might use to diagnose Kubernetes issues such as pod crashes, service failures, and ingress misconfigurations, etc. Analyzers that It was a good decision to avoid using GenAI at this stage to prevent speculative guesses, hallucinations, and, ultimately, a lot of noise. Thought: Once K8sGpT is connected to your Kubernetes cluster through the API Server, the will start scanning your cluster and looking at logs, error messages, configurations, and pod status to identify potential issues. Analyzers : K8sGPT has a set of , but you will be able to write your own analyzers. Technical Note analyzers that are built in It then tries to correlate issues and run additional checks if necessary to add more context and figure out the root cause before packaging the data and sending it to the AI backend provider to get a simple and descriptive text of the issue and potential remediation steps. As you can see, the use of Gen AI is targeted and practical. Of course, the AI can hallucinate and provide incorrect remediation steps, but that’s a minor inconvenience you can spot and ignore easily. Technical Note: Getting Started with K8sGPT The easiest way to get started is using a by running a brew command on Linux/Mac or downloading the Windows Bianary if you’re on Windows. CLI installation Once installed, you’ll need to , and within a few minutes, you’ll be ready to run your first scan. authenticate your chosen AI backend K8sGPT is designed to work with various AI providers, including , , , , , , and . Technical Note: OpenAI Cohere Amazon Bedrock Amazon SageMaker Azure OpenAI Google Gemini LocalAI You can find how to configure different backends . here Once everything is configured, you can start using K8sGPT directly from your CLI using the following set of commands:‍ Usage:
  k8sgpt [command]

Available Commands:
  analyze     This command will find problems within your Kubernetes cluster
  auth        Authenticate with your chosen backend
  cache       For working with the cache the results of an analysis
  completion  Generate the autocompletion script for the specified shell
  filters     Manage filters for analyzing Kubernetes resources
  generate    Generate Key for your chosen backend (opens browser)
  help        Help about any command
  integration Integrate another tool into K8sGPT
  serve       Runs k8sgpt as a server
  version     Print the version number of k8sgpt

Flags:
      --config string        Default config file (default is $HOME/.k8sgpt.yaml)
  -h, --help                 help for k8sgpt
      --kubeconfig string    Path to a kubeconfig. Only required if out-of-cluster.
      --kubecontext string   Kubernetes context to use. Only required if out-of-cluster.

Use "k8sgpt [command] --help" for more information about a command. You’ll soon be able to debug interactively by asking K8sGPT questions about any issue directly in your CLI. As for writing, this feature isn’t live. [Technical Note] K8sGPT Operator K8sGPT can also be installed as an Operator within your Kubernetes Cluster. By using the K8sGPT operator, you’ll be able to scan your cluster for errors continuously. The works by leveraging Kubernetes Custom Resources and producing reports that it stores in your cluster as YAML manifests. K8sGPT Operator You can cache your results locally or like AWS S3. [Technical Note] offload them to a remote location You can also easily customize the analysis and output of the operator to fit your needs and workflow. For example, you specify a certain namespace or resource and a specific output format like JSON. This makes K8sGPT easy to integrate into your CI/CD pipelines and other workflows. Integrations K8sGPT integrates with observability tools like Grafana and Prometheus, but the really cool thing is that it offers an that allows you to extend scanning and troubleshooting capabilities by writing your own plugins that connect it to other tools. integration feature The first one, contributed by the core team, is the plugging, which enables you to generate vulnerability scans as part of your analysis and leverage AI to help you understand and triage Common Vulnerabilities and Exposures (CVEs). Trivy Security & Privacy Alright! I’m sure this is the first or second thing that came to your mind when hearing that K8sGPT sends your data to an external AI-API. Well, the K8sGPT team thought of this and developed a nifty little feature called anonymize, which masks sensitive data like Kubernetes object names and labels before sending it to the AI backend for analysis. All you need to do is use the anonymize flag: k8sgpt analyze --explain --anonymize During the analysis, k8sGPT retrieves sensitive data and masks it before sending it to the AI backend. Once the solution is returned, the masked data is replaced with the actual Kubernetes object names and labels. Where to go from here? K8sGPT is still a nascent project and is quickly evolving thanks to an active community of contributors. As of writing, an exciting new feature on the horizon is the interactive debugging mode that lets you ask questions directly inside your terminal. Other things on the horizon include Kubernetes auto-remediation with K8sGPT, Karpenter, deep integration into AWS services, and new backends into services such as Hugging Face. If you want to dig deeper into K8sGPT, here are a few resources for you: Documentation & Repos - The main code repository k8sgpt - Documentation Website ( ) docs https://docs.k8sgpt.ai - Community-related information community Tutorials K8SGPT - Kubernetes SRE SUPERPOWERS Using AI with Kubernetes | K8sGPT Operator | DEMO Full Tutorial: K8sGPT -- SRE superpowers through AI Talks AI for Kubernetes with ChatGPT and k8sgpt Unleashing the Power of AI in Kubernetes through K8sGPT | Alex Jones Communities Slack: https://k8sgpt.slack.com Who to Follow Alex Jones (Creator of K8sGPT) on and Linkedin X K8sGPT on and LinkedIn X Thomas Schuetz (Maintainer) on and Linkedin X Matthis Holleville (Maintainer) on and Linkedin X It’s a wrap! I hope this gave you a good overview of K8sGPT and how Gen AI might make your life a bit easier when troubleshooting your Kubernetes clusters. If you want to take K8sGPT for a quick spin, I suggest you check out this . Sandbox tutorial Until next time! Also published . here

Stories, News & Mentions about KUBECOST

Kubecost’s Stories on HackerNoon