Things move fast in the world of infrastructure technology. It wasn’t too long ago when running a database on was considered too tricky to be worth it. But that was yesterday’s problem. Builders of cloud-native applications have gotten good at running stateful workloads because Kubernetes is a powerful way to create virtual data centers quickly and efficiently. Kubernetes The last time I wrote about this, I widened the aperture a bit to consider other parts of the application stack in the virtual data center – in particular, streaming workloads and analytics. With these two moving into the mainstream in Kubernetes, the discussion about use cases gets more interesting. What will we do with these foundational data tools if we have access to them? Thankfully we don’t have to investigate too deeply, because the industry has already picked the direction: AI/ML workloads. What’s driving this is the need for faster and more agile MLOps to support , otherwise known as real-time artificial intelligence (AI). Companies like and have been early adopters, but a host of great projects are available to get you going faster with Kubernetes. online prediction Uber Netflix Feature Serving with Feast Building and maintaining machine learning (ML) models are moving out of the back office and closer to users in production. A feature store acts as a bridge between data and machine learning models, providing a consistent way for models to access data in both offline and online phases. It manages data processing requirements during model training and provides low-latency real-time access to models during the online phase. This ensures data consistency for both phases and meets online and offline requirements. is an example of a feature store running in Kubernetes. It’s open source and enables organizations to store and serve features consistently for offline training and online inference. Feast goes beyond traditional databases by providing specialized features like point-in-time correctness. Feast Model Serving with KServe is an API endpoint for deploying machine learning models in Kubernetes, handling model fetching, loading, and determining whether  CPU or GPU is required. It integrates with eventing for scale-out and offers observability features like metrics and logging. The best part? It’s simple to use. Just point KServe at your model file and it will create an API and handle the rest. KServe KNative The explainer feature provides insight into why a decision was made for each prediction, offering feature importance and highlighting factors in the model that led to a particular outcome. This can be used to detect model drift and bias, which are some of the “important but hard” parts of machine learning. These features reduce the effort involved in MLOps and build trust in the application. KServe recently split away from the Google KubeFlow project and has been by Bloomberg as part of its efforts to build an ML inference platform. highlighted Vector Similarity Search Augmenting the traditional ways we find data, vector similarity search (VSS) is a machine learning tool that uses vector mathematics to find how "close" two things are to one another. This is done through the K-nearest neighbor ( ) algorithm, which expresses data as a vector. The data is then vectorized using a CPU-intensive KNN algorithm and indexed for less CPU-intensive searching. End users can provide a vector and find things that are close to it using the query mechanism provided by VSS servers. Open-source VSS servers that you can deploy in Kubernetes include and . Both provide everything you need to add a similarity search to your application stack. KNN Weaviate Milvus Assemble the Team Combine my with this one and you have a recipe for the full stack deployed in Kubernetes. The outcome every organization should try to realize is increased productivity and reduced costs. Recent show that leaders in the data space are finding both when deploying data infrastructure in Kubernetes. previous article surveys workloads may be something you’re just starting to explore, so now might be the best time to start on the right foot. The three areas mentioned — feature serving, model serving, and vector similarity search — are all covered in the book I co-authored with Jeff Carpenter, “ .” The bigger picture for AI/ML in the application stack: Real-time requirements will soon become prevalent in most AI applications. Using Kubernetes to go fast and build reliably is no longer an AI hallucination. AI/ML Managing Cloud-Native Data with Kubernetes Also published here.

DataStax Astra DB is the vector database for getting GenAI apps into production, fast. Try it for free today!

Too Long; Didn't Read

Making security fun one episode at a time

3 Key Tools for Deploying AI/ML Workloads on Kubernetes

Too Long; Didn't Read

People Mentioned

DataStax

About Author

TOPICS

Languages

THIS ARTICLE WAS FEATURED IN...

3 Key Tools for Deploying AI/ML Workloads on Kubernetes

Too Long; Didn't Read

People Mentioned

DataStax

About Author

TOPICS

Languages

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES