US vice president Kamala Harris said concerns about artificial intelligence (AI) safety had to go beyond “existential” fears about threats of cyber attacks or the development of bioweapons.The vice president,

Vice president Harris sets out measures to tackle ‘full spectrum’ of AI risks

More from IndieWire Tom Cruise's 'Mission: Impossible - Dead Reckoning' Inspired President Biden to Bolster Security Against AI Threats 'Joker ... chance it’s already Wednesday.” The original cast members, sans Rachel McAdams, were photographed ...

Turns Out, the ‘Mean Girls’ Cast Reunion Is a Walmart Ad — Watch

In a historic move on Monday, President Biden thrust artificial intelligence into the forefront of national attention by signing a groundbreaking AI executive order. This order, arriving just as the UK’s AI safety summit is set to commence on November 1st,

Tech Enterprises Respond Proactively to Biden’s AI Executive Order

Tarot is a powerful divination tool for gleaming messages from the universe. There are several tarot-related plugins in the ChatGPT library and many websites that utilize AI to read the card, including Tarotoo, MyAiTarot, and Tarotnova.

Which AI for Witches? Best AI Tools for Halloween

The politics of AI are getting messy for Democrats. Biden’s executive order tries to offer something for every faction.

The politics of Biden’s vast new AI order

President Joe Biden signed a wide-ranging executive order on artificial intelligence Monday, setting the stage for some industry regulations and funding for the U.S. government to further invest in the technology.

Biden signs executive order to oversee and invest in AI

The manga Dead Dead Demon's Dededede Destruction is getting a two-parter anime movie that is scheduled to come out next year.

Check Out the Dead Dead Demon’s Dededede Destruction Movie’s Cast

Little Black Book, Asymetric's Nicholas Lam on the transformation of AI from a looming threat into a partner for humankind's success

In Algorithm We Trust: The Battle for AI’s Soul

In an age where multinational corporations proudly tout their commitment to society through corporate social responsibility (CSR) and ethical practices, it's time we cast a discerning eye ... artificial intelligence (AI) exhibits remarkable learning ...

AI’s Social Responsibility Plot Twist: Doing Good by Designing Well

CAST AI

castai

AWS has just made a new open-source tool called Karpenter generally available. I will dig into how it works and will be unavoidably comparing it with what we have built at CAST AI. What exactly is Karpenter, how does its autoscaling mechanism work, and should you use it? Karpenter is a very simple autoscaler with two reactive policies. Its biggest selling point? No node pools. Let’s take a look at how Karpenter works and compare it to the autoscaling mechanisms of CAST AI. Quick feature comparison – CAST AI vs. Karpenter Detailed Feature Comparison of Karpenter and CAST AI Installation and Onboarding After trying Karpenter out, I’d say that onboarding isn’t trivial. Still, it’s not as hard as the Kubernetes Cluster Autoscaler. Karpenter onboarding requires you to have eksctl CLI installed and configured, even if you created an EKS cluster without the help of eksctl. It takes around to get started with Karpenter. 6 to 7 steps CAST AI doesn’t have an eksctl dependency, and you’re looking at a 2-step process your IT manager could complete on their own in a minute or two. Autoscaling Policies Karpenter has only two of the most basic autoscaling policies. At first glance, they are quite similar to two of the policies in CAST AI (Unscheduled Pod Policy and Node Deletion policy). Let’s compare the two. 1. Provisioner In its simplest form (without sub-features), CAST AI’s Unscheduled Pod Policy works in a similar way as Karpenter’s Provisioner. If there are any pods that can’t start because no suitable node has been found by the Kubernetes vanilla scheduler, a new node will be created based on those pending pod constraints like zone, subnet, PodAntiAffinity, capacity preference (spot or on-demand). Karpenter has these constraints available for NodeSelector, NodeAffinity, TopologySpreadConstraints: requirements:
    - key: node.k8s.aws/instance-type #If not included, all instance types are considered
      operator: In
      values: [ , ]
    - key: #If not included, all zones are considered
      operator: In
      values: [ , ]
    - key: #If not included, all architectures are considered
      values: [ , ]
    - key: #If not included, the webhook the AWS cloud provider will to on-demand
      operator: In
      values: [ , ] "m5.large" "m5.2xlarge" "topology.kubernetes.io/zone" "us-east-1a" "us-east-1b" "kubernetes.io/arch" "arm64" "amd64" " karpenter.sh/capacity-type" for default "spot" "on-demand" Karpenter supports GPU and ARM with the following tolerations: , , and . To add a GPU node, one needs to define a specific list of nodes (and we’re back to the node pool idea). nvidia.com/gpu amd.com/gpu aws.amazon.com/neuron CAST AI allows you to specify more pod constraints, like but doesn’t require you to specify the instance types on your own. In case you haven’t checked lately, there are 475 available types on AWS, which makes it easy to get back to implicit Node Pools by accident. get me instances with fast local storage for Spark jobs You can also specify the spot interruption rate per cluster and override per workload in CAST AI. For example, you may want your casual spot workload in a cluster to be on more cost-efficient spot instances – even if that means these instances could be interrupted once in a while. But those huge Jupyter notebook pods with a ton of resources? You want them to run spot instances that are so unpopular that they barely get ever interrupted. Back to the Node Pools against one’s will Running your Kubernetes workloads on burstable T3a instance types is a crime to your business and DevOps. Unfortunately, this is what you will get by default with Karpenter as they are the most affordable instances. Not excited about burstable instances? You’ll have to list all the acceptable instance types that you want your Kubernetes cluster to have in the Karpenter Provisioner CRD. And with that, you have just specified a cherry-picked node list and are implicitly back to Node Pools. And what if these cherry-picked node lists are not available during Black Friday or Christmas holidays? You will need to extend the list with even more alternatives. 2. Delete Empty Node policy As the name implies, Karpenter can and will delete empty nodes. This policy allows you to set the Time To Live for empty nodes ( with a default of 30). ttlSecondsAfterEmpty CAST AI’s Delete Node policy works in the same fashion. The CAST AI engine will not remove nodes if there are workloads running on them (excluding DaemonSets of course). The Kubernetes scheduler by default acts fairly and will always distribute workloads for maximum performance and availability. And if you’re running EKS, you can’t change the Kubernetes scheduler profile. Let’s look at this with an analogy: Imagine a train (Kubernetes cluster) with 10 train cars (nodes); each train car can seat 100 passengers (pods), the train has a capacity to transport 1000 passengers at peak. The train moves from the central station fully utilized with 1000 seats taken and with every stop more people get off the train than on it. Even a few stops before the end of the train journey, if the train is carrying only 15 passengers out of 1000, someone needs to make sure that there are empty train cars. It’s likely that there are 1 to 2 passengers in a train car enjoying their privacy, and the Delete Empty Node train car can’t find anything to remove. So how does Karpenter help with over-provisioned resources for off-peak times when the load is lower? Enter Node Expiry Karpenter offers a Node Expiry feature. If a node expiry time-to-live value ( ) is reached, that node is drained of pods, and the node is deleted. ttlSecondsUntilExpired Great! But wait a minute, what about your workloads that are running on these nodes? Sorry, end of shift. Nodes are going down. Alright, but how would you know what the expiration time should be? If you get it too short, your pods are going down, you then have to wait several minutes for the Karpenter provisioner to add new nodes with refreshed ttlSecondsUntilExpired. If you set the ttlSecondsUntilExpired too long, you’re wasting money. So what we see here is that Karpenter reacts in time to add new capacity but leaves you hanging with reducing any excess capacity. CAST AI, on the other hand, offers Evictor that shrinks the cluster to the minimum number of nodes by bin packing pods (simulating evictions, honoring Pod Disruption Budgets, etc). Once Evictor makes a node empty, it’s deleted automatically. Cloud Vendor Support At the moment, Karpenter only supports AWS. The code has a lot of AWS-specific hardcoded logic. For instance, using EC2 Fleet and Spot Fleet, without a way forward for other clouds. On the other hand, Karpenter has empty interfaces created in the code so the project looks pluggable and open enough if someone were to take the initiative of adding other cloud provider support. CAST AI supports AWS (EKS, kOps), Google Cloud Platform (GKE), and Microsoft Azure (AKS). No Node Pools The promise of being free of node pools sounds great. But the way it is now, Karpenter doesn’t offer users almost any control over the scheduling decisions when using a “naked” Provisioner. Provisioner is Karpenter’s Kubernetes CRD which defines the node creation constraints. For example, if you had workloads that needed a GPU and you wanted the GPU nodes to have a taint (so that only the pods that actually need it are placed on those nodes), you would create a Provisioner like that: apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: gpu
spec:
  requirements: 
  - key: node.kubernetes.io/instance-type
    operator: In
    values:
      - p3 xlarge
      - p3 xlarge
      - p3 xlarge
  taints:
  - key: nvidia.com/gpu
    value: effect: “NoSchedule” .2 .8 .16 true In Karpenter, the engineer still needs to have deep knowledge of the available instance types with all of their nuances. To use the SSDs in a K8s cluster, you would need to create a user-data script that makes a single logical disk out of those local SSDs and creates a symbolic link for the kubelet to use. That requires creating an EC2 Launch Template and referencing it in the Provisioner resource. And, most importantly, you need to keep it up to date if you want to achieve cost savings. In more complicated clusters, you’d have to create many different Provisioners, i.e., node pools: one for GPUs, another one for storage optimized, yet another one for infrequently interrupted Spot instances, etc. Conclusion: Provisioner Conflicts When there are multiple Provisioner resources defined and an unschedulable pod can tolerate/select any of them, the result will be non-deterministic. Here’s an example: apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: team-a
spec:
  labels:
    team: a
  requirements:
    - key: operator: In
      values: [ ]
---
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: team-b
spec:
  labels:
    team: b
  requirements:
    - key: operator: In
      values: [ ]
---
apiVersion: v1
kind: Pod
metadata:
  name: pod
spec:
  containers:
  - image: nginx
    name: pod "node.kubernetes.io/instance-type" "m5.large" "node.kubernetes.io/instance-type" "m5.2xlarge" The nginx pod would be scheduled on either the m5.large or the m5.2xlarge node (chosen based on the sequence the API-Server returns for the records). This is another dimension that the DevOps engineer would need to think about. The Provisioners should clearly define constraints, and all pods should clearly define the selectors to reduce or eliminate randomness. In CAST AI, the selectors/tolerations are predefined, and engineers don’t have to come up with them on their own. Pod Grouping (a.k.a. Bin Packing on Provisioning) Karpenter’s GitHub states that the solution uses bin packing, but only in the nodePlacer sense. That is, it groups pods to be placed on nodes to avoid creating a separate node for each pod. In CAST AI, we use Permutations and Clique functions. Their purpose is to find out the most optimal pod groupings. Karpenter’s algorithm is basically creating a hash of the pod constraints. This means that otherwise compatible pods will end up on different nodes (Karpenter will create multiple nodes). Consider this example: apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  nodeSelector:
    topology.kubernetes.io/zone: containers:
    - image: nginx
      name: pod
---
apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                  - - containers:
    - image: nginx
      name: pod "eu-central-1a" "eu-central-1a" "eu-central-1b" Even though the pods could fit into a single node placed in the eu-central-1a zone, Karpenter will create two nodes (that might even end up in the same zone) because the hash of the pod constraints is different. Cost Optimization When Karpenter bin-packs pods, it produces a list of viable instance types per each pod group. For example, if there’s a pod group that needs 2 CPU and 8 GiB, then a list of all instance types that have at least 2 CPU and 8 GiB is retrieved. The list is sorted by resource weight. Karpenter sorts those instance types using a weight function, where 1 CPU = 1 GiB, favoring nodes with higher CPU and lower memory: func weightOf(instanceType cloudprovider.InstanceType) float64 { euclidean(
		float64(instanceType.CPU().Value()),
		float64(instanceType.Memory().ScaledValue(resource.Giga)), float64(instanceType.NvidiaGPUs().Value())* , float64(instanceType.AMDGPUs().Value())* , float64(instanceType.AWSNeurons().Value())* , )
} // weightOf uses a euclidean distance function to compare the instance types. // Units are normalized such that 1cpu = 1gb mem. Additionally, accelerators // carry an arbitrarily large weight such that they will dominate the priority, // but if equal, will still fall back to the weight of other dimensions. return // 1 gb = 1 cpu 1000 // Heavily weigh gpus x 1000 1000 // Heavily weigh gpus x 1000 1000 // Heavily weigh neurons x 1000 After sorting, only the first 20 nodes are picked. This list might not contain the cheapest instance types because of Karpenter’s particular weighing system. There is no inventory database that can tell exactly how much something costs. After the bin-packing and sorting, the list is passed to the EC2 Fleet API. The mode of choosing the instance type depends on whether it’s an on-demand node or spot instance. On-demand instances In this case, Karpenter passes the “lowest cost” allocation method that makes the EC2 Fleet API choose the cheapest instance type out of the 20 received viable instance types. There might be cheaper instance types, but the API works with what it was given. Spot instances Karpenter passes the “capacity optimized prioritized” allocation method that makes the EC2 Fleet API choose the instance types that have the highest availability, prioritizing the instance types at the top of the list. This has nothing to do with cost optimization and could be considered something similar to the CAST AI “Least interrupted” Spot reliability setting. CAST AI offers an entirely different approach to cost optimization. The platform selects the most cost-efficient instances that match the workload requirements and automatically rightsizes instances to avoid overallocation. CAST AI also comes with features such as cost reporting on any level, information about cluster cost, or potential savings. Karpenter doesn’t have any of those. Karpenter vs. CAST AI – Which Autoscaler is a Better Pick for You? As you can see, Karpenter offers a limited set of functionalities right now. AWS Karpenter is an option if you’re on AWS and considering the native Kubernetes Cluster Autoscaler. Today, , so node pools on which the Kubernetes Cluster Autoscaler works become unmanageable considering the exhaustible instance inventory. This is essentially what Karpenter kind of tries to solve. AWS offers 400+ instances If you’re looking for an autoscaler that can significantly reduce your cloud bill for production workloads without impacting their availability, you need to look elsewhere. CAST AI offers a substantially more robust set of features which makes it a great fit for reliable automated cost reduction on production and development workloads. It also works with all three major cloud providers: AWS, Google Cloud, and Azure. To see what results CAST AI could bring you, you can or book a quick demo by clicking the button below. register and start with a free cluster analysis

AWS' Karpenter Autoscaler and How it Stacks Up Against CAST AI

To deliver on our promise of fully automated cloud cost optimization for Kubernetes, we led by the San Francisco-based , with and other investors participating. raised $10 million in a Series A round Cota Capital Samsung Next Managing cloud deployments effectively to control costs has become a priority for organizations of all sizes. Bobby Yazdani, Cota Capital founder, and partner “We are excited to partner with the CAST AI team, as we believe in the company’s ability to address cloud-native optimization challenges,” he added. And there’s more news we’d like to share. CAST AI platform now supports all three major cloud service providers: Amazon Web Services, Google Cloud Platform, and Microsoft Azure. Cloud Cost Management Industry is Ripe for Disruption In 2020 alone, companies around the world spent $17 billion of unnecessary cloud expenses due to inefficient tool selection and the inability to intelligently scale capacity to match the changing application requirements. Our platform solves this problem by using AI and machine learning to save our clients between 60% and 90% of total annual cloud expenditures for businesses in industries such as e-commerce, adtech, fintech, data analytics, cybersecurity, and SaaS application delivery. The new funding round will accelerate our product development initiatives in this direction. We are thrilled to have Cota Capital lead our Series A round. As we continue on our mission to utilize artificial intelligence for optimizing cloud deployments, we will use the additional funds to further accelerate development and delivery of our platform and provide real and measurable value to our customers. Yuri Frayman, CAST AI co-founder and CEO “The adoption of Kubernetes continues to grow as containers and microservice architecture become the defacto standard to serve modern applications at scale,” said Raymond Liao, Managing Director, Samsung Next. “Our team is thrilled to participate in CAST AI’s latest round, as the company advances its quest to make Infrastructure as a Service (IaaS) more efficient and on budget.” Towards Autonomous Kubernetes Our approach revolves around optimizing and autonomously managing Kubernetes for any cloud service. By connecting existing clusters to our platform with a single click, our users get a free instant savings report and instantly find cost-saving opportunities. CAST AI analyzes and learns more about the company’s cloud environment to identify the resources required for maximum performance. With CAST AI, our applications are using a more efficient combination of cloud services, and we have already reduced our annualized cloud costs by millions of dollars. Mark Weiler, SVP Engineering at Branch.io The core value of CAST AI lies in complete automation that allows users to define their own optimization settings to handle low-level optimizations without any extra work for DevOps engineers. Also published on: https://cast.ai/blog/10m-to-optimize-kubernetes-cast-ai-raises-series-a-and-extends-support-to-top-3-cloud-providers/

CAST AI Extends Support to AWS, Azure, and GCP After Raising $10M 

Automation is the best answer to controlling cloud costs, especially if you work with modern solutions like containers and Kubernetes. But automated optimization solutions come in different forms: they focus on automating different cost-saving tactics and give you various levels of visibility and control. Here’s a comparison of two automation solutions available to teams that work with Kubernetes: CAST AI and Spot.io. Keep on reading to find out which one supports your use cases better. – Full-Service Cloud-Native Platform CAST AI – Automated Spot Instance Use Spot.io Created by industry veterans, CAST AI is a comprehensive cloud automation platform for environments. Companies across e-commerce and ad tech are using CAST AI to . Automation features include spot instances, autoscaling up & down, and AI VM type selection. optimizing Kubernetes save from 50% to even 90% on their cloud bills Developed by NetApp, is a cloud cost management solution that focuses on optimizing cloud infrastructure costs with spot instance automation. Using machine learning and analytics algorithms, helps teams to get spot capacity for their workloads automatically, cutting costs and ensuring high availability. Spot.io Spot.io CAST AI vs. – Quick Feature Comparison Spot.io Feature CAST AI 🥇 Spot.io Supported platforms AWS ✅ ✅ Google Cloud ✅ ✅ Microsoft Azure ✅ ✅ Cost visibility Detailed cluster cost breakdown ✅ ✅ Automated cost forecasting ✅ ✖ Cost view across multi-cloud ✅ ✖ Real-time alerting functionality ✖ (coming soon) ✅ Kubernetes cost optimization and automation Rightsizing Recommendations, for instance, types and sizes ✅ ✅ Automated rightsizing via AI-driven instance selection ✅ ✖ Multi-shape cluster construction ✅ ✖ Autoscaling ✅ ✅ Automated pod scaling parameters ✅ ✅ Horizontal pod autoscaling ✅ ✅ Node autoscaling ✅ ✅ Cluster scheduling ✅ ✅ Automatic bin packing ✅ ✅ Spot Instance automation ✅ ✅ Full multi-cloud optimization ✅ ✖ Detailed Feature Comparison of and CAST AI Spot.io Cost visibility and monitoring Cost optimization and automation Spot instance automation Full multi-cloud optimization Security Pricing 1. Cost visibility and monitoring Cluster cost breakdown & forecasting In , you get a cost breakdown at the project, cluster, namespace, and deployment level. Teams can analyze costs down to individual microservices and then generate a detailed forecast of their cluster costs for better planning. CAST AI uses universal metrics that work with all supported cloud service providers. CAST AI The platform also comes with a cost allocation feature that works per cluster and per node. A feature for ongoing cloud cost reporting is also in the near-future pipeline since the CAST AI team plans to expand cost dimensions beyond compute – such as control plane, network, egress, storage, and others. breaks down the infrastructure costs of clusters and offers insights into each of the layers. Costs are broken down into namespaces and individual workloads within every namespace, with the option to filter them according to your container labels and annotations. For each workload, Spot.io displays compute and storage costs. Spot.io Users can use this data to analyze the application costs, perform chargebacks without extensive resource tagging, and estimate future cloud spend. Cost View Across Multi-cloud Many companies use more than one cloud service today. Allocating costs in a multi-cloud context is challenging, but equips teams with insights for addressing that. Its full multi-cloud functionality comes with visibility via universal metrics from Grafana and Kibana that can be used for any supported cloud service providers. CAST AI currently doesn’t support multi-cloud functionality or visibility across its solutions for containerized applications. Spot.io 2. Cost Optimization and Automation Rightsizing uses AI to choose the best instance types and sizes that meet an application’s requirements but still help teams save cloud costs. Whenever a cluster needs extra nodes, the AI-driven instance selection algorithm selects instances that achieve maximum performance at a minimum cost. Your team doesn’t have to lift a finger because it all happens automatically. CAST AI The platform also comes with multi-shape cluster construction, since choosing the same instance shape for every node in a cluster may easily lead to overprovisioning. That’s why CAST AI delivers an optimized mix of different instance types adapted to an application’s requirements. offers a handy Right-Sizing recommendation mechanism. The tool monitors workload utilization in real-time to provide you with recommendations for adjusting the resource requirements of workloads when they start consuming significantly more or fewer resources than requested. Spot.io The solution provides recommendations per container and summarizes them for the entire workload to enable easier visualization at a high level and faster implementation. Autoscaling Horizontal Pod Autoscaling and Node Autoscaling automates pod scaling parameters to help teams avoid overprovisioning their infrastructures. The platform also includes a Horizontal Pod Autoscaler, which uses business metrics to generate the optimal number of required pod instances. The feature then scales the replica count of your pods up and down, scaling to zero and removing all pods if there’s no work to be done. CAST AI CAST AI also ensures that the number of nodes in use matches the application’s requirements at all times, scaling nodes up and down automatically. continuously checks for unschedulable pods, and if it finds one, it scales up to make sure that all the pods have a place to run. The solution also removes nodes automatically when all pods running on the node can be moved to other nodes in the cluster. To optimize costs, Spot.io prioritizes downscaling the least utilized nodes. Spot.io Cluster Scheduling and Termination automatically pauses and resumes clusters created within the platform to help teams avoid paying for resources they’re not using. CAST AI makes sure that all that pods and nodes are terminated in a case of scale-down or instance replacement. Spot.io Automatic Bin Packing Kubernetes presents a cost challenge to teams because it distributes applications within a cluster evenly, with no attention paid to how cost-effective this setup is. changes the default pod scheduling behavior and uses automated bin-packing to achieve maximum savings in line with your settings. Fewer nodes translate to lower costs. CAST AI uses bin-packing algorithms as well. When the tool identifies instances where workloads may be distributed across the cluster, it triggers a scale-down to drain and terminate the instance. Spot.io 3. Spot Instance Automation Spot Instances can result in dramatic savings of up to 90% off the On-Demand pricing. But there’s a catch – the cloud provider can pull the plug anytime. That’s why automation is so critical for effective spot instance use. makes sure that the replacement of spot instances that were interrupted is fully automated. This means that teams don’t have to worry about their workloads not finding a place to run. The platform is always on the lookout for better instance alternatives and provisions in a split-second to guarantee high availability. CAST AI allows users to run their clusters on spot instances without the need to provision or scale instances. The tool also takes care of bin packing containers on spot instances to optimize their use even further. Spot.io 4. Full Multi-Cloud Optimization As more and more companies use multiple cloud services to access best-in-class services and prevent disasters, the need to monitor, manage, and optimize costs across providers is greater than ever. CAST AI meets this need by offering you the option to set up multi-cloud clusters using a number of : multi-cloud features – CAST AI replicates applications and data across multiple cloud services. If one fails, others keep your workloads afloat, guaranteeing business continuity. Active-Active Multi-Cloud – the platform distributes traffic evenly across all the participating clouds, always choosing the endpoints that are up and healthy. Global Server Load Balancing – CAST AI makes cost allocation across cloud services easier thanks to universal metrics from Grafana and Kibana. Multi-cloud visibility doesn’t support the multi-cloud functionality at the moment. Spot.io 5. Security Since both automation solutions work directly with the cloud infrastructure, their security is essential. Created by cybersecurity experts, offers a number of security features such as encryption at rest/in transit, secrets management, network security, logging, visibility, and more. The platform also comes with automatic patching and upgrades to VMs and Kubernetes to eliminate the chance of errors in your clusters. Additionally, CAST AI doesn’t access any environment variables that are considered sensitive such as secrets, passwords, tokens, etc. CAST AI is ISO-certified and well underway to obtaining SOC 2 certification. CAST AI encrypts account data within the browser and re-encrypts it with a secure algorithm when it reaches its servers. The tool doesn’t store any private keys, passwords, or authentication tokens – authentication is based on the IAM Cross-Account Role & External ID. Spot.io 6. Pricing offers a free Savings report you can run to check whether you could save up on your cloud bill. The analyzes your setup and generates actionable recommendations. You can implement them manually or turn automated cost optimization features – in that case; you can choose between two plans: Growth and Enterprise. CAST AI read-only agent CAST AI offers guaranteed savings of at least 50%. offers a free trial, after which users can choose from two different pricing models: Pay-as-you-Go without the annual commitment and Subscription that comes with a commitment but opens the doors to more discounts and priority support. Spot.io Summary Both Spot.io and CAST AI are great automation solutions for reducing cloud costs and facilitating processes like cost monitoring, management, and optimization. Using spot instances is a smart method for reducing the cloud bill, but it’s just part of the picture. By – for example, ones purchased within an AWS Savings Plan – you can drive your cloud bill down even more. rightsizing other instances While Spot.io offers recommendations, CAST AI does the job for you via automated instance selection. The broad range of optimization features combined with automated cost forecasting and unique multi-cloud functionality brings CAST AI to the top of cloud cost optimization platforms. To accurately estimate the savings that you would be able to get – start with the free AI Cluster Analyzer.

CAST AI vs. Spot.io: Two Approaches to Automated Cloud Cost Optimization

Scaling cloud resources is easy - so easy, in fact, that many teams end up losing control over their cloud spend. A missed bug or architecture oversight can easily snowball into a huge bill at the end of the month. That’s why teams need a cloud cost monitoring and optimization toolkit that provides detailed visibility, exhaustive reporting, and - in an ideal scenario - automated optimization capable of handling the fast-changing requirements of Kubernetes to generate some serious cost savings. Wondering whether you need something more than just cost reporting and analysis? Here’s a comparison of features delivered by two modern cloud-native solutions, Kubecost and CAST AI. CAST AI - Analysis & Automation Kubecost - Analysis Created by industry veterans, CAST AI is a full-service cloud automation platform providing powerful automation features for optimizing Kubernetes workloads.  Companies across industries such as e-commerce and ad tech are using CAST AI to save from 50% to even 90% on their cloud bills. Kubecost started as an open-source tool that provided developers with more visibility into their Kubernetes costs. Today, Kubecost is a robust cost reporting solution that teams can use to get insights into costs allocation, cost monitoring, and alerts - key tools for teams looking to gain visibility. Kubecost vs. CAST AI - quick feature comparison Feature CAST AI 🥇 Kubecost Supported platforms AWS ✅ ✅ Google Cloud Platform ✅ ✅ Microsoft Azure ✅ (coming soon) ✅ Cost optimization and automation Detailed insights on cluster cost optimization ✅ ✅ Recommendations for optimizing cloud costs ✅ ✅ Real-time alerting functionality ✅ (coming soon) ✅ Horizontal pod autoscaling ✅ ✖ AI-driven instance selection ✅ ✖ Multi-shape cluster construction ✅ ✖ Automated pod scaling parameters ✅ ✖ Automatic bin packing ✅ ✖ Spot Instance automation ✅ ✖ Node autoscaling ✅ ✖ Cluster scheduling ✅ ✖ Cost allocation Detailed cost breakdown ✅ ✅ Allocation by organizational concepts ✖ ✅ Cost view across multi-cloud ✅ ✅ Live customer support ✅ ✖ Full multi-cloud optimization ✅ ✖ Detailed feature comparison of Kubecost and CAST AI 1. Installation and setup CAST AI To start saving on your cloud bill with CAST AI, you need to create an account and connect an existing Kubernetes cluster or create a new one inside the tool. Teams often choose to in read-only mode to get a free detailed report of estimated monthly savings - and then take action by turning automated optimization on. It takes only 15 minutes to get the cost analysis and optimize costs automatically. connect their clusters At the moment, CAST AI supports services from AWS and Google Cloud Platform, with Azure support coming in Q4 2021. Supported platforms: Kubecost To install and operate Kubecost, teams can use the Kubecost helm chart. This installation method brings you all the components for getting started, offering access to Kubecost features and an opportunity to scale to large clusters. Teams can also enjoy a lot of flexibility for configuring Kubecost and its dependencies. Kubecost offers three other installation options, but they require effort and come with less flexibility. Supported platforms: Currently, Kubecost supports cloud services from AWS, Google Cloud Platform, and Microsoft Azure. 2. Cost allocation Detailed cost breakdown offers a cost breakdown and forecasting feature at the level of projects, clusters, namespaces, and deployments. You can analyze costs down to individual microservices and generate a detailed forecast of cluster costs. Moreover, CAST AI delivers insights using universal metrics for any cloud service provider from Grafana and Kibana. CAST AI provides flexible and customizable cost breakdown features as well. You can divide costs by namespace, deployment, service, and more indicators across all the three major cloud service providers. Like in CAST AI, this comprehensive resource allocation leads to generating more accurate showbacks and chargebacks, streamlining the ongoing cost monitoring. Kubecost Allocation by organizational concepts Focusing on automated optimization, offers cost allocation per cluster and per node. CAST AI users can allocate costs to concepts such as teams, individual applications, products, projects, departments, or environments. Kubecost Cost view across multi-cloud Many companies are using the services of more than one cloud provider. Allocating costs across clouds is tricky, but rises to this challenge. It supports teams with a unique , providing universal metrics for any cloud provider. CAST AI full multi-cloud functionality and visibility displays the costs across multiple clusters and multi-cloud environments in a single view or through a single API endpoint. However, Kubecost doesn't help you manage multi-cloud infrastructure - while CAST AI offers a full multi-cloud solution with cost optimization. Kubecost 3. Cost monitoring Cost allocation is the first step to understanding where your cloud bill comes from. Next, you need to keep a close eye on how your resource use translates to costs in real-time. ​ displays the biggest cost driver - compute costs - in the Savings estimator and shows potential savings associated with deployments on Spot Instances. It has a planned feature in the pipeline for , including other dimensions such as control plane, network, egress, storage, and others. CAST AI ongoing cloud cost reporting allows teams can link (CPU, memory, storage, network, etc.) with from the cloud services across AWS, GCP, and Azure - for example, tagged RDS instances, BigQuery warehouses, or S3 buckets. Users get context-aware, cluster-level reports to find an optimal balance between cost and performance matching their service requirements. Kubecost real-time in-cluster costs out-of-cluster expenses 4. Cost optimization and automation Once you allocate costs and monitor them regularly, it’s time to take action and start optimizing your spending. Kubecost and CAST AI support teams on this mission differently. CAST AI: Fully automated cost optimization that beats savings plans - this feature uses business metrics to determine the number of required pod instances. It scales the replica count of your pods up and down - and removes pods if there’s no work to be done. Horizontal pod autoscaling - if your cluster needs extra nodes, CAST AI chooses the best instance types that meet your requirements but still help to save up. AI-driven instance selection - CAST AI delivers an optimized mix of different instance types adapted to your application’s needs. Multi-shape cluster construction - to help teams avoid overprovisioning, CAST AI  sets these parameters automatically and maximizes cost savings. Automated pod scaling parameters - since Kubernetes distributes applications within a cluster evenly, it doesn’t really help teams reduce their cloud spend. CAST AI solves this problem via bin packing for maximum savings. Automatic bin packing - Spot Instances can bring dramatic savings of up to 90% off the On-Demand pricing. You don’t need to worry about a provider pulling the plug on your instance - their replacement is fully automated. Spot Instance automation - this feature makes sure that your nodes match your requirements at all times, scaling nodes up and down automatically. Node auto-scaling - automatically pause and resume clusters to avoid paying for resources your teams aren’t using. Cluster scheduling By turning CAST AI automated optimization on, you can save from 50 to 90% on your cloud spend. Savings: Kubecost: Cluster-level insights and recommendations for engineers to implement Kubecost provides detailed reports and Delivered via Slack or email, these alerts notify teams about budget overruns, anomalous spend patterns, and Kubernetes tenants that fall below the set efficiency levels. Users can set budgets for configurable aggregation levels - for example, team or application. real-time alerting functionality. Savings: Kubecost generates insights DevOps engineers can use to save 30-50% or more. 5. Security Since both Kubecost and CAST AI work with your cloud infrastructure, their security is paramount. offers many security features such as encryption at rest/in transit, secrets management, network security, logging, visibility, and more. Moreover, it provides automatic patching and upgrades to VMs and Kubernetes, so you’re always kept up to date and eliminate the chance of errors in your clusters. CAST AI doesn’t expose private data anywhere, and since users deploy Kubecost in their infrastructure, there’s no need to egress any data to a remote service. You retain, and control access to sensitive cloud spend data at all times. Kubecost 6. Pricing comes with a free Cost Savings report users can run anytime they want to check whether they could save up on their infrastructure. The report generates actionable recommendations. And if you want to add automated optimization into the mix, you can choose between two plans: Growth and Enterprise. In all cases, CAST AI offers CAST AI guaranteed savings of 50%. All plans are free of charge for the first 30 days. Kubecost also offers a free plan where you can monitor and optimize one cluster. To make the most of your paid plan, you’ll need to dedicate time to implement the recommendations provided by Kubecost. This will incur extra charges and doesn’t automatically guarantee savings. Kubecost Summary: When to choose Kubecost vs. CAST AI Both Kubecost and CAST AT are great picks that offer lots of value to Kubernetes teams looking to optimize their cloud bills and streamline processes related to cost monitoring, allocation, and reporting. But if you’d like more than reporting, the automated optimization features of CAST AI are at the top among cloud cost optimization platforms. Bringing Kubecost and CAST AI together will give you an end-to-end solution that will keep your cloud costs in check and generate some pretty impressive savings. BTW. You can use KubeCost together with CAST AI.

Kubecost vs. CAST AI: A Perfect Match for End-to-End Cloud Cost Optimization

The idea that you can build an app in one place and then deploy it across multiple clouds is an enticing one. That’s why multi-cloud solutions are on the rise today. They promise to help developers in expanding their cloud portfolios and battle for the industry’s attention. While Google Anthos uses a more complicated approach and you need to create a cluster for each cloud, CAST AI offers a more advanced way to solve this multi-cloud riddle - one cluster spread across multiple clouds. Google Anthos vs. CAST AI - feature comparison Google Anthos: one cluster on a single cloud, one tool for 3 clouds Anthos is Google’s new managed service offering for multi-cloud environments launched in 2019. Apart from access to multiple clouds, it offers hybrid cloud infrastructures or even a combination of on-premises and cloud-based functionalities. Anthos is a meta-service that groups Kubernetes clusters by configuration and policy. Every cluster is independent - it has its own control plane, points of ingress, deployments, and configurations. This means that developers must continuously synchronize the clusters. However, Anthos comes with cloud environments that are configured and may behave in a drastically different manner. While using Google, it offers a mostly streamlined experience. But try it on AWS or on-premises setup, and you’ll see that management is hard to automate. It requires ample knowledge of the destination cloud. CAST AI: one cluster on many clouds, unified multi-cloud for 3 clouds When you create a multi-cloud cluster with CAST AI, you get a single logical Kubernetes cluster connected across multiple cloud services. The solution creates a virtual multi-cloud private network using your VPN of choice or direct fiber connectivity between clouds. CAST AI abstracts away the differences between compute, network, ingress and load balancing, and storage. As a result, you get an open and non-proprietary Kubernetes implementation for immediate production purposes. What about cost optimization? CAST AI has a heavy focus on optimization and cost of operation. Similar to GKE, CAST provides a set of out-of-the-box auto-scalers. Scaling a CAST AI cluster is controlled through customer defined policies. The CAST AI cost optimization engine focuses on ensuring that the best possible compute instances are used for currently deployed workloads. This includes the use of spot and preemptive instances when they are available to lower overall costs. CAST AI’s multi-cloud solution does not require a special configuration management / synchronization service because of the One Cluster approach. Configurations are synchronized across clouds utilizing the underlying etcd data store that naturally supports multiple distributed master nodes. One cluster spread across multiple clouds is a more evolved approach to multi-cloud. Dealing with a single multi-cloud cluster is easier than having to constantly synchronize your clusters and only taking advantage of common configuration or policies. Explore your options and try out CAST AI Multi-cloud doesn’t have to be hard. here . We’re offering free cloud credentials to help you build your first multi-cloud clusters and play around with our infrastructure for a limited time. Just join our community on Slack. Written by Leon Kuperman Co-founder and CTO, CAST AI Formerly Vice President of Security Products OCI at Oracle, Leon’s professional experience spans across tech companies such as IBM, Truition, and HostedPCI. He founded and served as the CTO of Zenedge, an enterprise security company protecting large enterprises with a cloud WAF. Leon has 20+ years of experience in product management, software design, and development, all the way through to production deployment. He is an authority on cloud computing, web application security and Payment Card Industry Data Security Standard (PCI DSS), e-commerce, and web application architecture. Previously published at https://resources.cast.ai/blog/google-anthos-vs-cast-ai-comparison

Google Anthos Vs. Cast.AI: A Review

Kubernetes requires extensive configuration, and keeping container security at the right level is always challenging. One of the best ways to tighten your clusters’ security is by implementing tactics that have become industry-standard.4 Here are 10 most important ones. 10 best practices for Kubernetes security 1. Don’t keep secrets in an environment variable It’s a good practice to have your objects use a secret in an environment variable since other parts of your system can access environment variables. That’s why it’s best to use secrets as files or take advantage of a secretKeyRef to minimize potential attacks. 2. Use Strong Kubernetes Authentication and Authorization Integrating Kubernetes with a third-party authentication provider is smart, as it delivers extra security features like multi-factor authentication. Avoid managing users at the API server level to make access to the control plane more secure. Use Identity and Access Management (IAM) solution from the cloud service provider if you’re running a managed Kubernetes service like EKS, GKE, AKS, or similar. Go for OpenID Connect (OIDC) alongside an SSO provider you got used to if CSP IAM isn’t an option. 3. Disable the NET_RAW capability If your K8s containers don’t drop the NET_RAW capability, you risk enabling a wide range of networking exploits from within the cluster. It’s best to use Policy Enforcement solutions like Open Policy Agents, Kyverno, or Kubernetes Pod Security admission controller to ensure you are following best practices. Use drop for ALL or NET_RAW capabilities in the pod’s securityContext definition to ensure that the NET_RAW capability is disabled. 4. Avoid sharing  the host’s IPC or network namespace Check if your pods are sharing the host’s IPC or network namespace. Sharing namespaces for pod and host interprocess communication may lead to opening access to shared information. Pods should not be allowed access to the host namespaces. And sharing pod and host network namespaces enables network access to the host network from the pod, which breaks network isolation. That’s why you better set the hostNetwork parameter to false in PodSecurityPolicy. 5. Use Kubernetes Role-Based Access Control (RBAC) RBAC helps to define who has access to the Kubernetes API and on what permissions. In Kubernetes 1.6 and higher, RBAC is usually enabled by default. Since Kubernetes combines authorization controllers, make sure to disable the legacy Attribute Based Access Control (ABAC) when enabling RBAC. Pick namespace-specific permissions over cluster-wide permissions. Even when debugging, don’t grant cluster administrator privileges. Otherwise, your container security may be compromised. 6. Avoid usage of root file system for container security Are your containers running without a read-only root file system? Using a read-only file system prevents malicious binaries from writing to a system or a system takeover by attackers. You can ensure that containers use only the read-only file system by setting readOnlyRootFilesystem to true in Pod securityContext definition. 7. Isolate your Kubernetes nodes Kubernetes nodes should never be exposed directly to public networks. Ideally, these nodes are located on a separate network and – if possible – without any direct connections to the general corporate network. You can achieve this by keeping Kubernetes control and data traffic isolated. Otherwise, they both flow through the same pipe. Open access to the data plane means that access is open to the control plane as well. Configure nodes with an ingress controller and set it to only allow connections from the master node via the specified port through the network access control list (ACL). 8. Avoid running containers in privileged mode Does your deployment have containers running in privileged (root) mode? This allows the container to gain access to critical resources of the host. Avoid running containers in privileged mode or enable the podSecurityPolicy with the privileged parameter set to false. This will ensure that containers can’t run processes that require root privileges on the host. 9. Check for unsafe /proc mount Deployments with unsafe /proc mount (procMount=Unmasked) will allow bypassing the default masking behavior of the container runtime. Setting a container with Unmasked /proc mount type can potentially expose the host information to the container and, as a result, cause a potential information leakage or container escape. Set procMount=Default to ensure the container doesn’t unsafely expose any parts of /proc. 10. Have a rolling update strategy Another smart move is developing a rolling update strategy. Rolling updates allow deployment updates to minimize your application downtime by incrementally updating pod instances with new ones. See for more details. this page in the Kubernetes docs Achieving Kubernetes security is challenging The Kubernetes ecosystem is constantly changing – and so are its security and configuration parts. Keeping up with these changes takes time and effort, stealing away engineers from other tasks and screaming for their attention. Another challenge teams experience around Kubernetes security is the prioritization of security issues – depending on the application’s size, prioritizing issues may become time-consuming. But automation is here to change that. CAST AI’s Cloud Security feature constantly checks clusters against industry best practices, Kubernetes recommendations, and container security system benchmarks – and prioritizes them automatically to get you started right away.

10 Best Practices from the Industry and Community in Kubernetes Security

Did you know that ?* Overprovisioning or cloud cost savings may not be a top-of-mind issue for your deployments yet. But it’s likely to become a growing concern as your cloud bill grows. 90% of applications have 5x more resources than they actually need And when the CFO and their financial team start asking you about how you allocate cloud expenses, better give them a good answer. You need to have a solid response to infrastructure utilization and cloud cost efficiency. Compute resources are the . Selecting the right VM instance types is a key aspect of cloud resource optimization and in some cases can . biggest line items on your cloud bill save you 50% of your overall compute bill If you’ve optimized your VM choices already, check our blog for more cost optimization tips. Perhaps your needs some optimizing? cloud bill Table of contents: 1. It’s so easy to overprovision, here’s why 2. How to choose the right VM for the job in 6 steps Define your minimum requirements Choose an instance type with cloud cost savings in mind Consider the pros and cons of different pricing models Take advantage of CPU bursting Check your storage transfer limitations Count in the network bandwidt 3. Saving on cloud costs – case study 4. Wrap up * Christina Delimitrou and Christos Kozyrakis, “Quasar: resource-efficient and QoS-aware cluster management”, . https://doi.org/10.1145/2644865.2541941 It’s easy to overprovision cloud resources Let’s say that you need a machine with 4 cores for your cluster. You can choose from some 40 different options, even in a single cloud scenario where you work with only one CSP. All these machine shapes are just too hard for human minds to understand and analyze. How can you compare all these instances? For example, take a look at the following from AWS: pricing table … and this is just a portion of the table, for one region, and one OS type. via GIPHY And we’re talking only about one cloud at this point. Things get exponentially more complicated when you go multi-cloud. So, let’s say that you pick an instance that costs $0.19 per hour. It seems reasonable on paper, so you modify your Infrastructure as Code (AoC) scripts and deploy. But when you deploy a compute-heavy application, . You’re using only half of the memory allocated per node, underutilizing the network, and not even utilizing the SSDs that may be attached to the instance. This doesn’t help to maximize cloud cost savings at all. you soon see that it ends up underutilizing other resources you’re paying for If you took a deeper look into the CSP offering, you might have found another VM shape that could support your application for a much lower cost. Sure, it gives you less memory and network, but that’s absolutely fine by your application. And it costs much less – only $0.10 an hour. This scenario isn’t made up – these are the results of a cost / performance test that we carried out on a demo e-commerce application. You’ll find the case study further down this article, so bear with us. How to choose the right VM for the job in 6 steps 1. Define your minimum requirements Your workload matters a lot when you’re choosing a VM instance type with cloud cost savings. You should be making a deliberate effort to order only what you need across all compute dimensions including CPU (and type x86 vs ARM), Memory, SSD, and network connectivity. On the flip side, while an affordable instance might look tempting, you might suffer performance issues when you start running memory-intensive applications. Identify the minimum size requirements of your application and make sure that the instance type you select can meet them across all dimensions: CPU Count CPU Architecture Memory SSD Storage Network For example, different VMs across different clouds have varying price vs. performance ratios. This means that it’s possible to get better performance for the cloud compute dollar, and save more on cloud costs overall. Once you have a targeted set of instance types that fit your application needs, buying a “good enough” instance type might not be the best possible choice. Next, you select between CPU and GPU dense instances. Consider this scenario: If you’re building a machine learning application, you’re probably looking for GPU dense instance types. They train models much faster than CPUs. Interestingly, the GPU wasn’t initially designed for machine learning – it was designed to display graphics. In 2007, Nvidia came up with to help developers with machine learning training for deep learning models. Today, CUDA is widely adopted across most popular machine learning frameworks. So, training your ML models will be much faster with GPUs. To find out exactly why that is the case, check out this that dives into the details. CUDA interesting post What about running predictions through your trained models? Can we achieve better price/performance with specialized instance types? CSPs are now introducing new instance types designed for inference – for example, . According to AWS, EC2 Inf1 instances deliver up to 30% higher throughput and up to 45% lower cost per inference than Amazon EC2 G4 instances. AWS EC2 Inf Side note: We haven’t tested this claim at CAST AI yet, as this instance type is fairly new. Unfortunately, the selection matrix gets more complicated. 2. Choose an instance type with cloud cost savings in mind CSPs provide a wide range of instance types optimized to match different use cases. They offer various combinations of CPU, memory, storage, and networking capacity. Each type includes one or more instance sizes, so you can scale your resources to fit your workload’s requirements. CSPs roll out different computers and the chips in those computers come with different performance characteristics. You might be getting an older-generation processor that’s slightly slower – or a new-generation processor that’s slightly faster. You might choose an instance type that has strong performance characteristics that you don’t actually need, without even knowing it. Reasoning about this on your own is hard. The only way to verify this is through benchmarking – dropping the same workload on each machine type and checking its performance characteristics. This is one of the first things that we did at CAST AI when we started over a year ago. Here are two examples. Example 1: Unpredictable CPUs within one provider This chart shows the CPU operations in AWS (Amazon t2-2x large: 8 virtual cores) at different times after several idle CPU periods. Source: CAST AIExample 2: Cloud endurance To understand VM performance better, we created a metric “Endurance Coefficient” and here’s how we calculate it: We measure how much work VM type can do in 12 hours and how variable CPU performance is. For a sustained base load, you’d want as much stability. For a bursty workload (website that experiences traffic once in a while or an occasional batch job), lower stability is fine. You should make an informed decision since sometimes with shared-core, hyperthreaded, overcommitted, between generations or burstable credit-based VM types. In our calculation, instances with stable performance edge close to 100, and ones with random performance are closer to 0 value. it’s not clear how much stability you’re getting for your money In this example, the DigitalOcean achieved the endurance coefficient of 0.97107 (97%), while AWS got only a weirdly shaped 0.43152 (43%) – not that it’s a burstable instance. s1_1 machine t3_medium_st Source: CAST AILet’s get back to choosing the instance type AWS, Azure, and Google all have these four instance types: – This instance type has a balanced ratio of CPU to memory, a good fit for general-purpose applications that use these resources in equal proportions such as web servers with low to medium traffic and small to medium databases. General purpose – Optimized for CPU-intensive workloads, this VM type comes with a high ratio of CPU to memory ratio. It’s a good fit for web servers experiencing medium traffic, batch preprocessing, network appliances, and application servers. Compute optimized – This type has a high memory-to-CPU ratio. That’s why it’s a good fit for production workloads like database servers, relational database services, analytics, and larger in-memory caches. Memory optimized – Workloads that require heavy read/write operations and low latency are a good fit for this instance type. Thanks to their high disk throughput and IO, such instances work well with Big Data, SQL and NoSQL databases, data warehousing, and large transactional databases. Storage optimized Each provider also offers instance types for GPU workloads under different names: AWS – These instances use hardware accelerators (co-processors) to carry out functions like data pattern matching, graphics processing, and floating point number calculations more efficiently than with CPUs. Great for machine learning (ML) and high-performance computing (HPC). Accelerated computing – for example, AWS EC2 Inf that promises up to 30% higher throughput and 45% lower cost per inference than AWS EC2 G4 instances. Inference type – Machines recommended for most deep learning purposes. GPU Azure – Powered by Nvidia GPUs, these instances were designed to handle heavy graphic rendering, video editing, compute-intensive workloads, and model training and inferencing (ND) in deep learning. GPU – This type offers the fastest and most powerful CPU VMs with underlying hardware optimized for compute- and network-intensive workloads (which includes high-performance computing cluster applications). A great combination with high-throughput network interfaces (RDMA). High-performance compute Google – Based on the NVIDIA Ampere A100 Tensor Core GPU, these VMs get you up to 16 GPUs in a single VM. Great for demanding workloads like HPC and CUDA-enabled machine learning (ML) training and inference. Accelerator-optimized Check these links for more info: AWS instance types Microsoft Azure instance types Google Cloud Platform instance types A note about ARM-powered VMs series, a new ARM-based System-on-Chip (SOC) the company announced in November 2020. According to Apple, the M1 will increase CPU performance by 3.5x and GPU performance by 6x. Recently, there has been a lot of attention around Apple’s M1 It’s as it’s based on ARM architecture. Cloud providers already offer ARM-powered VMs like the AWS EC2 A1 family powered by their ARM processor. incredibly fast Graviton2 it’s less power-hungry, so cheaper to run and cool – CSPs charge less for this type of processor. But if you’re toying with the idea of using it, Here’s what you need to know about ARM: you might need to re-architect your delivery pipeline to compile your application for ARM. If you’re running an interpreted stack like Python, Ruby or NodeJS, your apps will likely just run. 3. Consider the pros and cons of different pricing models Your next step is picking the right pricing model for your needs. Cloud providers offer the following models: — In this model, you pay for the resources that you use (for example, AWS charges compute capacity by the hour). There are no long-term binding contracts or upfront payments. You can increase or decrease your usage just-in-time. This flexibility makes on-demand instances great for workloads with fluctuating traffic spikes. On-demand (Committed use discounts in GCP) — Reserved Instances allow purchasing capacity upfront in a given availability zone for a much lower price than on-demand instances. In most cases, you commit to a specific instance or family, and can’t change it if your requirements change later. Generally, the larger the upfront payment, the larger the discount. Commitments range from 1 to 3 years. Reserved Instances — In this model, the same discounts as Reserved instances for committing to use a specific amount of compute power (measured in dollars per hour) over the period of one or three years. After committing to a specific amount of usage per hour, all usage up to it is covered by the Savings Plan, and anything extra is billed at the On-demand rate. Unlike Reserved Instances, you don’t commit to specific instance types and configurations (like OS or tenancy), but to consistent usage. Savings Plans (AWS) you’re running the risk of and securing resources that might not make sense for you in a year or two. In cloud terms, 3 years is an eternity. But didn’t you go to the public cloud to avoid CAPEX in the first place? By choosing reserved instances or the AWS savings plan, locking yourself in with the cloud vendor This is an evil practice on the part of CSPs. They lock customers in for a discount and cut them off from alternatives for years to come. (Preemptible in GCP) — In this pricing model, you bid on spare computing resources. This can bring cloud cost savings up to 90% off the on-demand pricing. The problem with spot instances is that they don’t guarantee availability, which varies depending on current market demand in a given region. Spot instances might make sense if your workload can handle interruptions (for example, if you’ve got stateless application components such as microservices). Spot instances (sole tenant nodes in GCP) — A Dedicated host is a physical server that has an instance capacity fully dedicated to you. It allows using your own licenses to slash costs but still benefit from the resiliency and flexibility of a cloud vendor. It’s also a good fit for companies that need to meet compliance requirements and not to share the same hardware with other tenants. Regulated markets that deal with lots of sensitive data often choose dedicated hosts for the perceived security benefits. Dedicated host AWS, Azure, and GCP all charge for things like egress traffic, load balancing, block storage, IP addresses, and premium support among other line items. Take them into account when comparing instance pricing and building your cloud budget. Note: Don’t forget about the extra charges. And each item deserves your attention. Take egress traffic as an example. In a mono-cloud scenario, you’ll have to pay egress costs between different availability zones, which is in most cases $0.01/GB. In a multi-cloud setup, you pay a slightly higher rate, like $0.02 when using direct fiber (for the US/EU). 4. Take advantage of CPU bursting Take a closer look at each CSP, and you’re bound to see “burstable performance instances.” These are instances designed to offer teams a baseline level of CPU performance, with the option to burst to a higher level when your workload needs it. They’re a good match for low-latency interactive applications, microservices, small and medium databases, and product prototypes, among others. Note: – larger instances collect more credits per hour. However, there’s also a cutoff to the number of credits you can collect, and larger instance sizes come with a higher cutoff. The amount of accumulated CPU credits depends on the instance type Where can you get burstable performance instances? AWS Instance families: T2, T3, T3a, and T4g.Restarting an instance in T2 family = losing all the accrued credits.Restarting an instance in T3 and T4 = credits persevere for seven days and then are lost. Learn more . here Azure B series VMs of CPU bursting. When you redeploy a VM and it moves to another node, you lose the credits.If you stop/start a VM while keeping it on the same node, it retains the accumulated credits.Learn more . here GCP Shared-core VMs offer bursting capabilities: e2-micro, e2-small, e2-medium.CPU bursts are charged by the on-demand price for f1-micro, g1-small, and e2 shared-core machine types. Learn more . here Our research into AWS showed that if you load your instance for 4 hours or more per day, on average, you’re better off with a non-burstable one. But if you run an online store that gets a stream of visitors once in a while, it’s a good fit. Note: CPU capacity has its limits We discovered that compute capacity tends to increase linearly during the first four hours. But after that, it becomes much more limited. The amount of available compute reduces almost by 90% until the end of the day. Source: CAST AI - 5. Check your storage transfer limitations Here’s another thing to consider when maximizing your cloud cost savings: data storage. AWS EC2 instances use Elastic Block Store (EBS) to store disk volumes, Azure VMs use data disks,And GCP has the Google Persistent Disk for block storage. You get local ephemeral storage in AWS, Azure, and GCP too. Every application has unique storage needs. When choosing a VM, make sure that it comes with a storage throughput your application requires. Also, – unless you expect to employ them to the fullest. don’t opt for pricy drive options like premium SSD 6. Count in the network bandwidth If you have a massive migration of data or a high volume of traffic, pay attention to the size of the network connection between your instance and the consumers assigned to it. You can find some instances that can bolster 10 or 20 Gbps of transfer speed. But here’s the caveat: only those instances will support this level of network bandwidth. Saving on cloud costs – case study We recently tested the CAST AI approach to cloud cost savings on an open-source from Google. e-commerce demo app adapted Here’s what we did: To get some meaningful costs, we load-tested our application with a high number of concurrent users (~1k). We then scaled the pods for each microservice accordingly. This was done on AWS EKS with a statically-scaled deployment. We ran the test for a 30-60 minute period to capture the metrics.Next, we extrapolated costs over a 30-day period, taking into account assumed traffic seasonality. We generated a likely 30-day usage pattern as part of our load testing scripts. The resulting usage experienced spikes every day at around the same time and during several days of the week traffic was somewhat heavier. This is how we calculated the monthly costs of running our app on the AWS test cluster and compared our solution to see how it helps to increase cloud cost savings. The original cluster doesn’t use auto scaling or spot instances. Note that our demo scenario assumes a fixed number of servers in the original EKS cluster and an instance type of m5.xlarge. Originally, we selected the m5.xlarge VM that comes with 4 CPUs, 16 GB of memory and a high throughput Network Interface (~10 GiB). The cost of this instance type on-demand is . $0.192/hour Then we launched CAST AI and let the magic happen. Through automated analysis, our optimizer selected an alternative shape: a1.xlarge. This instance type has 4 CPUs and only 8 GB of RAM. The pods deployed per VM could easily fit into 8 GB. The additional 8 GB on our former instance (m5.xlarge) was a pure waste of RAM. We were able to select the a1 (ARM) processor by re-compiling our apps and building new container images. Our new instance had a reduced network throughput. This wasn’t an issue for our app because we were not network bound. That is, the traffic generated with the available compute resources didn’t max out the bandwidth capacity of the selected instance. And now comes the best part: The a1.xlarge instance costs only $0.102/hour. This means that we instantly achieved cloud cost savings of 46% per compute-hour. Wrap up Compute is what you go to cloud providers for. It’s the biggest part of your cloud bill. So if you manage to optimize costs there, you’re on the right track to dramatically reducing your cloud bill. Try CAST AI for free . Previously published at https://cast.ai/blog/how-to-choose-the-best-vm-for-the-job/

6 Tips on How to Choose a Proper VM and Save on Your Cloud Costs

Bonus: Does your workload qualify for AWS spot instances? [CHEAT SHEET] You may already know what the catch is. The cloud provider can pull the plug at any time with as little as 30-second notice. We’re not saying that you should opt to reserve VM instances instead. Far from it. Reserved capacity is a path to vendor lock-in and paying more in the long term. There is a way how you can use spot instances effectively. Even for production workloads. Read this guide and learn and make financial team pleasantly surprised when they see the bill. how to handle spot instances This is part 3 of our cost optimization series. The rest of the series include: Surprised by your cloud bill? 5 common issues & how to deal with them How to choose the best VM type for the job and save on your cloud bill Why are spot instances so tricky? Interruptions are inevitable CSPs offer their unused capacity at prices that offer savings up to 90%. The only catch is that they can pull the plug with short notice, from 2 minutes to as little as 30 seconds. This is why spot instances are more difficult to manage for production workloads. Since you bid on spare computing resources, you have no guarantee on how long these capacities will stay available. Interruptions are bound to happen. That’s why you shouldn’t be using them for workloads that can’t tolerate them and are critical. Pulling the plug happens fast CSPs offer short interrupt notice. Amazon gives you 2 minutes, Azure and Google only 30 seconds. Is that enough time to drop everything and find a replacement for your instance? Not for a human. Let’s say that you already set your eyes on an on-demand instance. Creating a new VM takes around 5 minutes on AWS (and even longer if you use Kubernetes), so you’re looking at a few minutes of potential downtime. Another method is having some paused machines that can step in whenever you lose an instance. But then your savings aren’t going to be so spectacular. The best way to handle spot instance interruptions is through automation. Note: Rebalance recommendation In November 2020, AWS introduced a new feature that you can use to proactively rebalance workloads running on EC2 Spot instances without having to wait until your instance receives the interruption notice. It’s a signal that notifies you when the risk of interruption increases for a spot instance that you’re using. It arrives sooner than the interruption notice, giving you time to rebalance your workload to new or existing instances. Limited capacity The amount of available capacity sold as spot instances can vary a lot based on size, region, time of day, and many other factors. And all of them are subject to frequent changes. The availability of a spot instance is based on supply & demand. This might lead to unexpected behavior if you happen to pick the most popular instance types and a market surge like Black Friday occurs. So, why use spot instances at all? Some of your workloads probably don’t need on-demand machines at all times. Tech companies like Salesforce, Lyft, or AutoDesk use spot instances. If you’re still harboring some doubts, consider this scenario: Let’s say that you have 10 pods running for your application — a product catalog service. Half of the pods are running on a spot instance. At some point, the instance gives you a preemption notice — it’s about to be taken away from you. If that happens, you’ll lose half of your capacity. You’re not going to . Instead, the pods will be redistributed to other machines that are still available after the interruption. experience downtime immediately But what if you want to handle the interruption gracefully and replace that capacity before it becomes an issue? You can quickly order a new instance within the allotted time — for example, a different type of spot instance. Or go with an on-demand instance if there’s no spot instance capacity on the marketplace. And you can replace that on-demand instance with a spot instance a couple of hours later when the market pressures are alleviated. By not locking yourself up in a reserved plan, you get a lot of flexibility and avoid getting locked in with your vendor (or even a specific instance type). That’s why using spot instances is a good idea. When to use spot instances? If a service is stateless and can be scaled out — that is, have more than one replica — is a . good candidate for spot instances The good news is that most services are stateless in modern architectures. K8s was designed for stateless architectures. Here are some examples of workloads that work well in spot instances: — they’re fault-tolerant and instance-flexible. Batch processing jobs — they’re typically self-contained, highly available, fault-tolerant, and capable of handling interruptions. Containers and microservices — these apps usually need very high compute capabilities, massive amounts of memory, fast storage, and high network performance. Spot instances can support them via bursting or even serve as primary compute infrastructure. High Performance Computing (HPC) — it doesn’t matter what tools you use; these instances can come in handy in your deployment process. CI/CD operations — Elasticsearch or MongoDB can handle an interruption without losing any data or affecting the service. Distributed databases Any app on an orchestrated environment Which CSP to choose for spot instances? Do this before getting a spot instance 1. Know your workload How aggressive are you going to be about implementing spot instances? Before getting into the spot instance business, you need to know how much time your application needs to finish a job. Can it handle interruptions well? Will you have an automation tool in place to move your workload to another instance before your time runs out? 2. Cherry-pick your instances Next, it’s time to examine what the CSP has to offer. Take a look around and consider going for slightly less popular instances. They might come with a lower chance of interruptions and run stable for a longer time. When looking through the available instances, be sure to check the frequency of interruption. Frequency of interruption is the rate at which the instance reclaimed capacity during the trailing month. AWS displays it in the Spot Instance Advisor in ranges of <5%, 5–10%,10–15%,15–20% and >20%: 3. You can still use spot instances for more important workloads For example, AWS offers a type of spot instance where you get uninterrupted time guarantee for up to 6 hours (in hourly increments) and pay just a little more. A spot instance running for a predefined duration can achieve a discount of up to 30–50% compared to on-demand pricing. 4. Bid your price Now it’s time to set the maximum price you’re willing to pay for the spot instance. Your spot instance will run only when the marketplace price matches your bid or is lower. The rule of thumb here is using maximum price that equals on-demand price. If you set a custom amount and the price goes up, you risk getting interrupted. 5. Manage spot instances in groups When using groups of spot instances, you can request multiple instance types at the same time. As a result, you increase your chances of getting filled. Another perk is that you can set a maximum price per hour for the entire fleet rather than a given spot pool (a group of instances with the same type, OS, availability zone and network platform. — you can manage a large fleet of spot instances with different allocation strategies (for example, considering the lowest price or only capacity optimized types). AWS Spot Fleet — use this feature to create and manage a group of load-balanced VMs, increasing or decreasing their number automatically. Azure VM scale set — you can bring preemptible instances together in a group after specifying the preemptible option in the instance template. Google managed instance group But to make it all work, prepare for a massive number of manual configuration, setup, and maintenance tasks. 6. Turn to automation You can avoid downtime from lost instances by implementing automation tools for managing your cloud infrastructure via autoscaling methods. By using an automation tool, you can pick how much of your workload will be running on a spot instance, and then automatically fall back to on-demand instances in case of interruptions. Automation is here to make sure that your workload has a place to run. And thanks to features like AWS Rebalance events, you can mitigate the risk even before receiving the interrupt notice. You can get away with adding some basic levels of automation to how you manage these instances. But to achieve the best results, you need a solution that carries out automated actions based on predictive analytics. Here’s what automating spot instances can do for you Remember the case study we shared with you in the article about ? choosing the best VM for the job To test our cost savings approach at CAST AI, we opted for an open-source from Google. e-commerce demo app that we adapted We first prepared the app: We load-tested our application with ~1k concurrent users. We scaled the pods for every microservice accordingly (using AWS EKS with a statically-scaled deployment). To capture the metrics, we ran the test for 30–60 minutes. We then extrapolated the costs over a 30-day period (we took assumed traffic seasonality into account). To make it happen, we generated a likely 30-day usage pattern using a Python script. The example usage experienced spikes every day at around the same time and had several days of a week with heavier traffic. This is how we calculated on the AWS test instance. the monthly costs of running the demo app Our initial monthly cost of running the app was . $691.20 We applied ensuring that our application saved on costs relative to the on-demand pricing. the CAST AI spot instance policy We used the most aggressive policy settings where all the instances in use are spot instances. This brought the total compute costs down to $65.01 — saving 90% over the original costs of an unoptimized deployment. Wrap up Spot instances are a beast that can be tamed. But to reap the pricing benefits and use them safely in production, you will need to use an automation solution like CAST AI . Previously published at https://cast.ai/blog/how-to-reduce-cloud-costs-by-90-spot-instances-and-how-to-use-them/

All You Need to Know About Spot Instances to Cut Down Your Cloud Service Expenses

Amazon Web Services (AWS) had a second in a row-major outage a few days ago. Most likely Slack, Zoom and Notion were affected by this. On the 25th of November, AWS had another major outage. Widely utilized services like Amazon API Gateway, Lambda, DynamoDB, CloudWatch, CloudTrail, CloudFormation, Autoscaling, Athena and others were affected. Our beloved products from Adobe, Roku, Spotify, and Flickr went down together with AWS. Our hearts go out to these companies and their users. “So many services being impacted simultaneously may be indicative of a large software rollout, or potentially localized underlying hardware or network issues,” said Leon Kuperman, CTO and Co-Founder at . “If it’s a software issue, it’s unusual that AWS would have chosen US-East as a canary region. US-East (Virginia) represents the largest AWS region and largest customer footprint, which would put it near last, if not last on the rollout schedule.” CAST AI Public-clouds are a godsend for tech companies. Migration to cloud allows startups and enterprises to skip over unnecessary investments in on-premise data centers and focus on what matters the most: develop products that people love, scale fast, and always be accessible. Unfortunately, public cloud providers like AWS, Azure or Google Cloud Platform are eager to lock in companies to use their services exclusively. When you rely on one vendor, you go down when that vendor fails. Today, AWS showed the true cost of putting all your eggs in one basket. It amplified the need for companies to have a multi-cloud disaster recovery and fail-over strategy. This strategy would still allow for a possibility of some cloud providers to go down, but it would also ensure that a cloud going down would not mean downtime for your users. “A cloud failure should not become a disaster. Never again.” said Laurent Gil, CPO and Co-Founder at . “You can rest easy knowing that your infrastructure is never down.” CAST AI Imagine that you show up to your office one day and your office building is gone. That’s what happens when cloud providers like AWS go down. All you are left with is the question “Why?“. With CAST AI you don’t need to wonder. By running your application on multi cloud, CAST AI automatically shifts your resources to Azure or Google Cloud when AWS has troubles. Explore CAST AI here. Previously published at https://resources.cast.ai/blog/when-aws-sinks-why-sink-with-it

The AWS Outage Downed Slack, Zoom, and Notion, Among Others. A Multi-Cloud Approach Could've Helped

“Multi-cloud is the worst practice” – is it, really? Earlier in August, Last Week In AWS published an interesting and insightful written by Corey Quinn, the Chief Cloud Economist at The Duckbill Group. The Duckbill Group specializes in reducing the AWS bills for their customers, a much-needed service in today’s cloud environment. article In this article, we take a look at the arguments Corey made in his post and dive into the details. Before we begin, let’s just say that it’s great to be able to have this type of discussion. A year or two ago, we would wholeheartedly agree with Corey. But as times change and technology evolves, some problems that previously seemed insurmountable are longer an impediment to progress. What is multi-cloud? We agree that true multi-cloud means that the same workload is running across two or more clouds, in an active-active or active-passive (DR) configuration. Corey is spot on here, we’re not talking about various SaaS and IaaS offerings for disparate applications and use-cases and then calling that type of usage multi-cloud. If that was our definition, most organizations would already be there today. We also agree that multi-cloud is really hard to achieve, or at least it was before we built CAST AI. Corey outlines some of the toughest problems in multi-cloud and makes a great case for why the world needs a tool like ours. Lowest common denominator At CAST AI, we start with cloud primitives. These are Computers, Disks, and Networks, including some networking primitives such as load balancers, VPN services, and Direct/Express Connects (dedicated network interconnects). These are the fundamental components we need to build reliable clusters of computers, especially if we chose Kubernetes for orchestration, scaling and management of containerized applications. Corey is right, if you haven’t containerized your application(s), going multi-cloud is going to be very hard. As an organization, you will need to carry out a re-architecture and modernization of your apps before taking advantage of multiple cloud providers. As you move to containers and Kubernetes, the solution becomes much more attainable. Let’s talk about higher-order services Of course, you should have access to and use cloud-native services that your organization has come to rely on. If you’re using RDS, BigQuery, Azure Machine Learning, Elastic Cache, or any number of services provided by AWS, Google, or Azure, it would be very hard to cut those dependencies. We believe that you should have access to ALL of them in a secure way, over a private IP network. These services are definitely not closed to you. We call this a Multi-Cloud Architecture for Services. However, this is the beginning of the multi-cloud journey where you can take some of your components and distribute them across clouds. Just as all technologies evolve, it’s going to take time for the market to develop multi-cloud solutions. Vendors like and already offer a choice (A or B) of clouds for deployment, and others will follow. If one doesn’t take the first step, it’s impossible to ever complete a journey. Snowflake Yugabyte For completeness, all clouds have good Load Balancers these days, so there’s no need to run your own Nginx instances. At CAST AI we orchestrate these things for you, and LBs become part of the Kubernetes ingress story. We agree that you shouldn’t be spending your time on these low-level primitives and should be focusing on your code instead. What about vendor lock-in? In his article, Corey says, “ ” We agree, you probably are today! I do a lot of martial arts (Brazilian Jiu-Jitsu) and if someone puts me in a really bad position (I don’t like getting choked out), I try to get out of that bad spot, not just accept the inevitable and tap. I have some bad news for you: You’re already locked in. We need to pause here and talk about open source strategy for a moment. If your application stack is using proprietary protocols (DynamoDB, Spanner, Redshift to name a few examples) that don’t have an open-source equivalent, it’s going to be harder to move away from a single cloud for that component. However, if you’re using RDS MySQL or RDS PostgreSQL, things become much easier. New-generation distributed databases such as or all speak PostgreSQL, for example. Yugabyte DB CockroachDB Take a look at your open source strategy, are there parts of your stack that are completely tied to a provider with no open-source equivalent? If that’s the case, it’s not the end of the world. You can still take advantage of multi-cloud for the parts that are portable. You can still use cloud services as well. Developer/team buy-in Here Corey makes a compelling argument that your team knows one cloud (like AWS) and prefers to stick to what it knows well. He actually wrote another article that discusses this topic in detail ( ). The Lock-In You Don’t See We agree here that adding a second cloud in a traditional sense would require an entirely new team of cloud ninjas who speak and understand the new set of cloud APIs, Terraform, and other nuisances. And people who have spent a good chunk of their career on understanding AWS aren’t going to be super excited about learning a completely new cloud in a crunch. However, the buy-in we are seeing in the industry is what is driving the movement. People want to learn how Cloud Native Computing Foundation to deploy their apps on Kubernetes and other cloud-native projects. It’s not about cloud-specific Terraform, CLI, commands, or API specs. Just like very few people need to code in assembler these days, there will come a time when software development engineers no longer need to know about low-level cloud APIs to provision compute, network, storage, and other resources. As technology evolves, so do levels of abstraction… This is progress. If you offer your team the opportunity to learn Kubernetes and other CNCF projects, they’re not going to quit. They’re going to thank you and take those courses. And… they will have a higher market value as a result. Negotiating leverage Here is Corey’s core argument with respect to sticking to one cloud provider: “Every cloud provider of substance (and also Google Cloud, zing) negotiate discounting percentages based upon percentage of spend. Cutting your spend in half reduces your negotiating base.” He goes on to advise: “Every time we’ve seen this happen with our clients, the discounting achieved from that threat is less than the discount that the customer would get simply by committing to higher spend levels.” Committing to multi-year deals with reserved instances or savings plans sounds a lot like dealing with telcos and data centers of the past. We live in the age of just-in-time cloud computing, why are we negotiating deals with a used-car salesperson? Our position at CAST AI is that reserved instances and savings plans are evil. They lock our customers for years with a particular cloud, and in cloud-years that feels like decades. Technology develops so quickly that we can’t predict what will happen in 6 months, never mind what will happen over a 3-year term. Let’s take AWS as an example. With , if you pay upfront on a standard 3-year term, you can save around 60% depending on your instance family choice. AWS claims 72%, but we see that most options are far lower in terms of savings. If you spend $4,000,000 to $10,000,000 you can get an extra 10% discount! That is a lot of commitment at $4 million. reserved pricing Let’s now consider another approach using Spot or Preemptible instances. What if you had a platform that could automatically buy the spot instances you need, and move your applications seamlessly in and out of those instance types? What could you save? Using the , we see savings between 70% to 85% (US East) and even more in some cases. That’s all without paying $1 in advance and no commitment! AWS Spot Instance Advisor Our argument assumes that you have a platform that can move containers and workloads between regular and spot instances and that your application is tolerant to instance interruption. AWS gives you a 2-minute notice. GCP and Azure provide only less than 1-minute notice. This is where CAST AI helps Kubernetes is a great orchestration tool to move pods/containers between instances (nodes) and our optimization engine catches preemption notifications and automatically reschedules work to nodes that are available. There is a lot more technical detail under the hood here, and not all workloads are going to be super tolerant to preemption. The nice thing is, you can deploy this optimization strategy on a single cloud, without even considering other cloud providers. You can get even more benefits from preemptive/spot instances when you leverage multiple clouds. What if AWS is experiencing super high demand in your region and spot instances aren’t available? You have at least 3 options to move forward: You can just buy an on-demand instance for a short time while market conditions normalize, that may increase some instance costs for a few hours, You can buy a different instance type that will meet your requirements, Kubernetes will automatically reschedule your workloads based on the new cluster topology, or You can buy a preemptive/spot instance on an adjacent cloud for a short-term window or for a longer-term. Without an automation platform to do the work, the choices are difficult to consider and reason about. Luckily, CAST AI does the work for you here and allows setting policies to govern optimization and overall cluster behavior. In short, we would much rather see customers use the open market to leverage just-in-time best pricing and not see them locked into multi-year painful contracts where they need to fork out truckloads of cash to obtain some savings. As Corey suggests, we don’t want you to have an adversarial relationship with your cloud providers. But we prefer that decisions are made based on the best price, performance, or technology - rather than locked-in, multi-year, pre-negotiated contracts. Multi-cloud doesn’t protect you from price changes We agree with Corey that cloud costs tend to get lower over time. That’s why locking-in pricing over 3 years is potentially very harmful. If you know that things will be cheaper next year, your negotiated discount is going to be less meaningful. Overall long-term pricing trends aside, you want to protect yourself from market-driven price spikes and take advantage of market-driven price drops. This is where real-time purchasing of compute across more than one cloud provider can really help. Having a dynamic, self-adjusting platform that can look for optimizations within one or more cloud providers is going to help with small and large price changes. If a particular instance drops in price and now becomes much more affordable from a price/performance perspective, your application should start benefiting from that new lower price instance automatically. Again, this is not easy to achieve without automation. Multi-cloud doesn’t exist in reality I personally love this argument. It’s kind of like saying that “automobiles don’t exist in reality” in the year 1880. Sure, just because the first car was invented in 1885, it doesn’t mean that people weren’t already working towards realizing that vision. Corey says “… there are no articles or conference talks of companies talking about their successful multi-cloud strategies paying off.” Well, that’s not entirely true. Even prior to his article published on August 5th, that they’re moving a part of their platform to Oracle (OCI), and today they are running Zoom on both clouds in parallel. In fact, Corey writes about this in his April 28th and praises Zoom for making that choice. In this case, the price/performance of compute and egress costs sealed the deal, so Corey is arguing against his own position here. Zoom announced blog post I used to work for Oracle Cloud, Zoom is an amazing example that is well covered. Caveat: We’ve also seen other examples where a prominent US retailer wanted to take advantage of the Oracle Database technology while using Azure for other parts of their stack. In this case, between the two clouds to enable this type of multi-cloud collaboration. Oracle and Microsoft announced an interconnect Conclusion In this post, we haven’t touched some obvious discussion points. Egress costs and latency across clouds are two very important topics that we will cover next. Data gravity is another important and closely related topic to cover. While multi-cloud and cost optimization are in their early stages, we believe that they can potentially bring tremendous growth and savings opportunities for customers that adopt cloud-native and modern architecture approaches. It’s going to take a while for customers to realize the full benefits of these capabilities, and that’s why we need to get started now. We launched CAST AI to help developers realize the benefits of multi-cloud and cost-optimized Kubernetes. We hope that you follow our journey. You can try CAST AI now by following this and let us know what you think - we look forward to hearing from you. link Post written by Leon Kuperman , CTO at CAST AI Co-founder and CTO, CAST AI Formerly Vice President of Security Products OCI at Oracle, Leon’s professional experience spans across tech companies such as IBM, Truition, and HostedPCI. He founded and served as the CTO of Zenedge, an enterprise security company protecting large enterprises with a cloud WAF. Leon has 20+ years of experience in product management, software design, and development, all the way through to production deployment. He is an authority on cloud computing, web application security and Payment Card Industry Data Security Standard (PCI DSS), e-commerce, and web application architecture.

Stories, News & Mentions about CASTAI

Castai’s Stories on HackerNoon

Castai's News & Mentions