Algoscale

algoscale

Introduction to Real-time Analytics and Data Processing When building software or web applications, you can add analytics, but what does it mean to be real-time? Generally speaking, there are three types of analytics. The first one is dashboards and BI tools. These are normally used for internal purposes. The second one is user-facing analytics. These are analytics you provide to the end-users of your software or web applications. The third one is machine-learning, machine-powered, or machine-fed type of analytics. These are when you feed analytics or events directly into your systems and then have your systems do the processing automatically—like anomaly detection or fraud detection. An important part of a real-time analytics system is its ability to ingest new data as soon as it is pulled in by a streaming source, to process all of this raw data into machine-readable data. Real-time analytic systems use data processing frameworks, including Apache Kafka and Apache Spark. What is Kafka? Before we learn about Kafka, let's learn how companies start. In the beginning, there is a source system and a target system, and there needs to be a data exchange between the two - That's pretty simple, right. But what happens is that as the company grows, the source and target systems also increase, and there must be a data exchange between them, complicating the matter. For instance, if there are 4 sources and 6 target systems, there would be a need for 24 integrations. Each integration comes with its share of difficulties: What protocol to choose, i.e., how the data is transported (TCP, HTTP, REST, FTP, JDBC..) What would be the data format, i.e., how the data is parsed (Binary, CSV, JSON, Avra...) What will be the data scheme and evolution, i.e., how the data is shaped and may change in the future Moreover, each time a source system is integrated with a target system, there will be an increased load from the connections. So, how do we solve this? Well, this is where Kafka comes in. Kafka is an open-source, streaming medium that allows for the development of real-time event-driven applications. distributed Why use Kafka? Kafka allows you to decouple data streams and systems. The source systems will have their data in Kafka, and the target systems will source their data directly from Kafka, leaving out the hassle of manually integrating the data. Kafka is super quick. The reproduced records are replicated and partitioned to allow many users to use the application simultaneously without any detectable lag in performance. Kafka maintains a high level of accuracy. The data records ingested into Kafka are accurate a Kafka prevents data loss and maintains the sequence. Kafka is also resilient and fault-tolerant Because ingested data in Kafka is replicated, the margin for errors is greatly reduced. These characteristics all together add up to a potent platform. Some applications of Kafka in real-time data analytics and data processing include: Decoupling of data streams and systems Activity tracking Location tracking Data gathering What is Spark? The goal of Spark is to provide a fast general-purpose cluster framework for large-scale data processing designed to overcome the limitations of MapReduce, which was the most common data processing method in Hadoop at the time of Spark development. The foundation of Spark is based on the resilient distributed data set or RDD, a programming abstraction representing a collection of read-only objects split across a computing cluster. Why use Spark? Spark can create the RDD from text files. Spark can create the RDD from text files, SQL databases, NoSQL databases, HDFS, cloud storage, and the list. RDDs work for multiple functions. RDDs allow for standard MapReduce functions but also join datasets filtering and aggregation. The processing of RDDs is done entirely in memory. The RDD is designed to hide complexity from users who then don't have to worry about where specific files are sent or what resources to store and retrieve. Spark has fast processing. Among many, one of the most significant attributes of Spark is its swift processing. Thanks to the RDD design and in-memory processing, its fast y processing makes it run significantly faster than other big data options. Some applications of Spark in real-time data analytics and data processing include: Real-Time Online Recommendation Event Processing Solutions Fraud Detection Live Dashboards Kafka streams and Spark structured streaming! How are they different? Both Kafka streams and Spark structured streaming are used in real-time analytics systems and for data processing, but both of these frameworks differ from each other in the following modes: Kafka streaming is part of the Kafka ecosystem, while Spark streaming is a newer generation 2 streaming library built on spark SQL. Kafka streams API only interacts with the Kafka cluster but does not run directly on top of it. In contrast, Spark structured streaming belongs and runs as a part of the spark cluster. The core abstractions of Kafka stream are KStream, KTable, and GlobalKTable, and the core abstractions of spark structured streaming are dataset and data frame. While Kafka streaming is event-driven, spark-structured streaming works on micro-batch and event-driven models. There is no master-slave architecture in Kafka streaming, while Spark structured streaming operates on master-slave architecture. Kafka streams use data retention for handling late data, whereas spark structured streaming uses watermarking for handling late data. Happy Learning!

Real-time Analytics and Data Processing with Kafka & Spark

With the latest development and research in artificial intelligence and machine learning, there are many frameworks available online, both paid and free. With the help of these platforms, different algorithms can be created and implemented. While talking about different frameworks and platforms of machine learning one of the most prominent platforms is TensorFlow. TensorFlow is one of the best, free, and open-ended platforms for machine learning and artificial intelligence implementations. The main aim of using TensorFlow is to train different models and to work on deep neural network models and their interference. The language used by TensorFlow is mostly JavaScript or Python. With the help of these languages, one can train different models of machine learning and artificial intelligence. With the advancement in CNN as most of the machines, automobiles, industries have used CNN, machine learning, artificial intelligence, and deep learning. The researchers need a free and open-ended framework to test and train their models and for this purpose, TensorFlow is most widely used. There are different applications and different purposes for using TensorFlow over conventional frameworks and platforms. Most of them are discussed in this article and their comparison is enlisted along with the figurative comparison. This article will provide a gateway for the new researchers in this field who want to pursue their careers in the field of machine learning or AI. (Introduction to TensorFlow, 2019) In the previous paragraph, a brief discussion was made on TensorFlow. To continue further, there must have been a brief introduction to style transferring. Style transfer is basically a computer vision-based technique combined with image processing. In style transferring the composition of an image is changed in order to get different data sets for training purposes. It can be explained by a simple example. Imagine you have a digital image and want to convert that picture painted by a famous artist the style transfer will convert this for us using different algorithms but having a prior dataset already uploaded to the platform by the developers. In style transferring there are two images as data set one is the image that has to be transferred and the other one is the image on the basis of image, the first image has to be changed. (Style Transfer Guide | Fritz AI, 2019) Figure 1. A common example of Style Transferring one is Content Image, the other is Style Image, and the result is Generated Image. (| Notebook.Community, 2019) There are different methods to create such images and related algorithms. One of them is GAN (Neural Network - What’s the Relationship between Style Transfer and GAN? - Stack Overflow, 2019). It is a more generalized form but style transfer in TensorFlow keeps the image harmless and just paste the style image on the content image and provide the result as we have already observed in figure 1 shown above. There is another technique adopted by researchers as well. The technique is a feed-forward neural network. With the help of this technique, they have provided much better and more accurate results. The main issue that occurs in the feed-forward technique is due to a large number of input images this technique requires a large computation speed and a large time is required. (Ma et al., 2020) In style transferring in tensor flow, we can observe that the content image can adapt the colors obtained from the reference image. In this way, we can get a single image having the features of two images. In tensor flow, there are different layers in the content and style images. In the input image, there are some low features like edges and features. There are certain other layers which are final layers which are high-level features like wheels and eyes and in order to get extracted such features a lot of computation power is required but in with the help of TensorFlow such computation speed can be achieved and a lot of features can be extracted from the reference image as well. (Neural Style Transfer  |  TensorFlow Core, 2019) The technique is pretty important while talking about convolutional neural networks (How Do Neural Style Transfers Work? | by Blackburn | Towards Data Science, 2020). As they have combined the features of one image and the color of the other. This can be helpful to develop AI-based software which is mostly related to image processing, image editing, and video editing. By training proper data, setting up a proper database, and having the system with proper computational speed. Even at present, Facebook, Twitter, YouTube, and face unlocking software use this technique in order to get better results. The most common type of style transfer is Text style transfer or (TST) which is very much important in Natural Language Programming (NLP). In TST, the different expressions like humor, anger, and excitement can be translated into subtitles in movies. The autogenerated captions on Facebook and YouTube videos are based on the TST technique. As this technique is new that is the reason the results are not accurate or up to the mark. (Two Minutes NLP — Quick Intro to Text Style Transfer | by Fabio Chiusano | NLPlanet | Medium, 2019) There are many uses of style transferring. The most common of them are enlisted as follows. (Gatys et al., 2016) i. With the help of this technique, one can create artwork from the natural image with the help of artificial intelligence. ii. There are many notable mobile applications like Prisma, and DeepArt which have used this technique to develop such applications. iii. There are many techniques in computer science in which style transferring is used. The most common of them is object detection. iv. In supervised and unsupervised learning of object detection where TST and fast-feed style transferring has been used. v. This technique has been widely used in image processing techniques like image stylization and image analogies. References | notebook.community. (n.d.). Retrieved February 19, 2022, from https://notebook.community/diegocavalca/Studies/books/deep-learning-with-python/8.3-neural-style-transfer Gatys, L., Ecker, A., & Bethge, M. (2016). A Neural Algorithm of Artistic Style. Journal of Vision, 16(12), 326. https://doi.org/10.1167/16.12.326 How Do Neural Style Transfers Work? | by blackburn | Towards Data Science. (n.d.). Retrieved February 19, 2022, from https://towardsdatascience.com/how-do-neural-style-transfers-work-b76de101eb3 Introduction to TensorFlow. (n.d.). Retrieved February 19, 2022, from https://www.tensorflow.org/learn Ma, W., Chen, Z., & Ji, C. (2020). Block Shuffle: A Method for High-resolution Fast Style Transfer with Limited Memory. IEEE Access, 8, 158056–158066. https://doi.org/10.1109/ACCESS.2020.3020053 neural network - What’s the relationship between Style Transfer and GAN? - Stack Overflow. (n.d.). Retrieved February 19, 2022, from https://stackoverflow.com/questions/55099376/whats-the-relationship-between-style-transfer-and-gan Neural style transfer  |  TensorFlow Core. (n.d.). Retrieved February 19, 2022, from https://www.tensorflow.org/tutorials/generative/style_transfer Style Transfer Guide | Fritz AI. (n.d.). Retrieved February 19, 2022, from https://www.fritz.ai/style-transfer/ Two minutes NLP — Quick intro to Text Style Transfer | by Fabio Chiusano | NLPlanet | Medium. (n.d.). Retrieved February 19, 2022, from https://medium.com/nlplanet/two-minutes-nlp-quick-intro-to-text-style-transfer-61de9cbd4083

Style Transferring with TensorFlow

Before going further, you’ll need to understand the concept of game theory. Game theory is basically a branch of applied mathematics. In-game theories ( ), there are different available tools with the help of which different situations are analyzed. There are parties in-game theories mostly referred to as players and the decision they have taken are interdependent. This is a kind of playing chess in which the turn of one player is associated with the future strategy of the opponent player. How Game Theory Strategy Improves Decision Making So, a complete strategy, decision, possible movement, and formulation of strategy basically come under the banner of game theory. Game theory is basically a framework with the help of which one can understand the strategy of the opponent player. With the help of optimal game theory, the players come to a point where optimal decision-making is reached and the strategy of the player becomes strong enough so that it can play in any situation with any kind of player. This concept is very important from a technical point of view for those researchers who are willing to work in the field of machine learning, deep learning, AI, or NLP. In this article, the association of game theory with AI and NLP will be discussed. Game Theory in AI The main aim behind the game theory is to understand the outcome of the optimal solution of the opponent player. There is another field which is an inverse game theory with the help of which the strategy of the opponent player and strategy can be explained. With the help of game theory, the AI agents can design with the help of which the opponent environment can be created in a game (Game Theory in Artificial Intelligence | by Pier Paolo Ippolito | Towards Data Science, 2020). This can be explained by an example like when you are playing a game against the computer there are different AI agents who are pre-defined inside the game are playing against us. The caliber of the AI agent depends on the difficulty level selected initially and with the passage of time and with the training data set this AI agent gets better and the difficulty level is enhanced. The use of game theory in AI is a very vast field. Game theory is mostly used in the training of GANs, multi-agent system, and deep reinforcement learning. In artificial intelligence, there are different types of categories in which different games are placed. These categories are perfect information games, imperfect information games, deterministic games, and non-deterministic games or chance moves games (Artificial Intelligence | Adversarial Search - Javatpoint, 2019). A brief explanation of these categories is as follows: Perfect Information games, in board games such games are those in which the AI agent is responsible to have all the information of complete board and the agent can see all the moves of the opponent player and make decisions accordingly. Example of such games is chess, checkers, go, etc. Imperfect Information Games, are those games in which the agent does not have the necessary information and is not aware of what going on next. Example of such games is card games, bridge, tic-tac-toe, etc. Deterministic games are those games in which there is a strict pattern to be followed and a proper set of rules for the games. The rules are properly defined and there is no random algorithm associated with them. Examples are chess, Go, and tic tac toe. Non-deterministic Games: In such games, there is no proper set of rules in the games. The events that can be occurred are mostly unpredictable. Such games mostly include the chance and luck of the user as well. Such games required very high-quality AI agents and their training have a large data set so that a competitive game environment can be obtained. Examples are a monopoly, checkers, etc. Figure 1. Game Theory and Prediction explained (The Basics Of Game Theory, 2020) Game Theory and NLP In in-game theory, another approach is neuro-linguistic programming comes into play. It is basically a psychological approach with the help of which different strategies are being analyzed and apply accordingly depending on the nature of the opponent player. In NLP, there are basically thoughts, languages, and behavior of the opponent. All of these strategies are combined with each other in order to get an optimal and specific outcome of each step taken by the opponent player. (Craft, 2001). Neurolinguistics is basically how our brain works but there is no significant research carried out on whether such technique really works or not. It is basically a psychological technique used by magicians as well. The pro players of card games like bridge, rung, etc. also used such techniques to find the actual situation of the opponent player but such technique is yet to be implemented in computer or mobile phone games. Figure 2. A brief description of NLP ( ) The Universal Self-Help Keys of NLP - Neuro-Linguistic Programming | The Whole Parent, 2020 From the figure shown above a complete understanding of NLP is stated. As it is basically the combination of three techniques one is neuro, the other is programming, and in the last linguistic. In programming, there are the thoughts of a person which are stated in words. The ideas and all the organization of the thoughts of the individual who is doing programming for the AI agent so that it can be trained accordingly with situations and environment that can be happened later on in the game. The second thing is neuro, in which the thinking process of the user can be compiled. In this portion, all the senses of the person are combined and compiled, and later on, these thoughts are embedded in the programming section. This portion is yet to accomplish in the simulation environment as it required a large force and large computational power. The last part is linguistic which is basically the psychological part. In simple words, the linguistic portion contains all the behaviors, languages, eye contact, and such kind of things to get possible outcomes from human behavior.

Game Theory Meets AI and NLP

The number of horseless carriages on the roads is increasing every day, and with every vehicle comes a registration number, license, and a couple of drivers. So, for each automobile on the road, a lot of data is processed. It is hard to monitor and manage this amount of data. Global law enforcement required a solution to help them with this tedious task. came up with a solution, and that solution is ANPR. Police Scientific Development Branch in Britain What is ANPR? ANPR (Automated Number Plate Recognition) is a computer vision tool used to read vehicle registration plates. Law enforcement around the globe widely uses it to verify vehicles’ registration, license, identify stolen cars, and monitor traffic. ANPR uses video surveillance as well as traffic enforcement cameras to fulfill this purpose. ANPR was initiated in 1976 and had a working prototype by 1979. It can work on live video surveillance streams or images. Tools and Techniques behind ANPR ANPR mainly uses OCR to recognize registration plate characters. It captures high-res images of vehicles using specially-made speed cameras and recognizes the plates following a sequence of algorithms. ANPR takes in a video surveillance feed and monitors it frame by frame. Infra-red cameras with narrow wavelength retro-reflective techniques capture any frames with clear or processible images. The system uses localization to find and isolate registration plates from captured images. Once a plate is recognized, the tool takes the localized part of images and applies normalization. Normalization adjusts the cropped images into the required dimension, size, brightness, and contrast. ANPR uses the output of normalization and processes it using character segmentation to identify individual characters. Segmented images pass on to OCR for recognition. This step is crucial as OCR must be able to detect different font faces. High-speed processors aid character recognition and identify if a number plate exists in the system. Additionally, ANPR uses geo-analysis on recognized images depending on image specifications and the country that uses it. Each capture gives data about a new vehicle or information about an existing vehicle. How to Create ANPR with YOLO Let us understand how we can combine multiple technologies to create a small ANPR. We will combine three tools: , , and . YOLO OpenCV Google’s Tesseract First, we need to . YOLO and OpenCV can install with pip. To install Tesseract, we need to download and run the binary files available on its website. OpenCV is a tool that we use to capture and process live feed and help in detection. install these tools in the system Once we install the tools, we to detect moving vehicles. YOLO identifies vehicles in fast-moving traffic. Whenever a car passes in the video stream, YOLO identifies it and captures the frame. We apply the given image processing algorithms and extract characters from the capture. set up OpenCV on a live feed and apply on top of OpenCV YOLO that processes to recognize characters. Characters identify as an array which we concatenate into a string. The capture gets passed to the Tesseract Applications of ANPR Applications of ANPR mainly focus on law enforcement and traffic crime prevention but are not limited to these. The private sector uses ANPR for security and management. Government agencies globally use it for the automation of many traffic-related processes. ANPR uses include parking management systems, journey time analysis, weighbridges, tolling, bus-lane enforcement, and intelligent transport systems. Wrapping Up! We hope you find this article helpful. ANPR is an effective tool as well as a fusion of tons of great technologies. It carries the legacy of decades-old efforts of scientists and enlightens us on how quickly technology is evolving. It contains a mix of state-of-the-art hardware and top-of-the-line software. Happy Learning.

What is Automated Number Plate Recognition (ANPR)?

As the name suggests, object detection is a phenomenon used to spot different objects in image frames. Object detection is a product of Computer Vision and is a very effective technique to precisely locate items of different shapes and sizes and label them. What is Object Detection? Object detection is a way to provide computers with a set of eyes. Once applied, computers can quickly identify objects just like humans do and use different operations based on the presence of specific objects in the frame. For example, imagine you have an image of a road on a busy day, and you want your computer to be able to label the different kinds of vehicles in the photo. Once you input that photo to an object detector such as , it will instantaneously the vehicles present on the road, as well as any human or other animal present inside it. YOLO locate and label However, it's essential to realize that an object detector will only detect the objects it's trained to. For example, suppose you're using a custom object detector that's only trained to spot and label dogs. In that case, it won't magically label everything in the images provided and will only label dogs. Object Detection Vs. Image Recognition An essential difference worth mentioning here is between Object Detection and Image Recognition since most people confuse the two. Image recognition only predicts what object is present inside a frame and prints it along with the picture, while object detection labels each instance of an object inside the frame by using a . bounding box For example, if two cats are in a frame, the object detector will draw bounding boxes around both cats and label them using their exact coordinates. At the same time, the image recognition will only print 'Cat' along with the frame, no matter how many cats are in the photo. Source Object Detection Use Cases Now that we have a sound knowledge of object detection, let's move forward and see some actual use cases. We’ll go through each application one-by-one, discussing the implementations in detail. Self-Driving Cars Let's start with the most significant and most revolutionary use-case of object detection in the tech world – self-driving cars. Real-time object detection has always played a crucial part in the world of self-driving vehicles. Autonomous vehicle systems use high accuracy object detectors that work in real-time to spot any objects on the road to let the car drive safely. The object detectors used by self-driving cars are state-of-the-art and can even classify stationary or moving objects. While there are many other things as well that go into the making of self-driving vehicles, object detection sits at the core of it. In a nutshell, the process breaks down into three categories: localization, mapping, and tracking. Localization refers to figuring out the vehicle's position relative to the environment; mapping refers to taking the context of the environment into account. Finally, tracking means identifying moving objects to plan the path of the vehicle. 2. Mask Detection Mask detection has probably been the most sought-after use case of object detection in the past year. Ever since the pandemic has broken out, businesses have been implementing mask detectors in their buildings to spot people violating the regulation of wearing masks. Right now, many malls and shopping centers are using object detectors to implement mask detection systems to ensure the safety of customers and staff members. Once you learn how to train your custom object detectors, implementing a face mask detector isn't that hard! All you've got to do is find a suitable dataset, and then you're good to go. There are various available on the internet as well if you need a start. tutorials 3. Video Surveillance Since the top-notch object detectors are highly accurate and can label multiple instances of objects in a single frame in real-time, they're highly effective tools when it comes to video surveillance. Video frames are sequentially fed to the object detectors, and each frame can be used to spot any abnormal activity, given the high-quality video input. Many warehouses, banks, and retail stores are currently employing object detectors in their video surveillance cameras. Whenever something irregular or alarming pops up, the relevant authorities are automatically alarmed by the surveillance systems. 4. Crowd Counting Source This is yet another vital scenario when it comes to post-pandemic times. Due to the safety of crowds in places like malls, it's essential to avoid many people getting together in tight spots. Crowd counting systems use object detectors to measure how many persons are present in a given area and the density of people in a particular location. Consequently, highly crowded places can be automatically spotted by the object detector without anybody manually interfering. So, you see, object detectors are not only used to identify and localize objects in a given frame, but they can also calculate the proximity between particular objects and calculate the distances between them. 5. Anomaly Detection Anomaly detection is a simple yet compelling use case of object detection applied in a wide variety of industries. It works differently given the specific industry. Industries that produce goods or process raw material often have to go through materials that have expired or gone wrong, and not being able to spot those materials could prove to be disastrous. For example, in the agriculture industry, a object detector could identify plants that have been affected by a disease. While farmers could manually do this as well, they might not be very accurate and end up wasting a lot of time. custom-trained While this process used to be wholly manual and many human resources were wasted in this, object detection has completely revolutionized the process. The automated systems are not only fast, but they're also highly cost-effective once they're developed. 6. Media & Entertainment As surprised as you might be, the media & entertainment industry has also started to get around the use of object detection in its daily business. Sports matches such as football and rugby are amongst the first ones to use object detection; in fact, NFL uses real-time object detection to track the football in a match. Not only does this make the analysis easier, but it also makes tracking the ball easier when it’s surrounded by a lot of players and no single camera angle is enough to spot it fully. Moreover, the sports-betting industry also uses object detection heavily to obtain information efficiently, making the analysis much faster and accurate. 7. Medical Image Analysis Last but not least, the has also seen some huge benefits from object detection, such as medical image analysis. Medical imaging such as ultrasounds produces a heavy amount of visual data that needs to be analyzed in detail to diagnose issues with the patients. health industry Custom-trained object detectors are great in spotting minor details, saving considerable time for the surgeons. According to an estimate, radiologists go through around 200 medical images per day. Imagine the time they would save if a certain amount of their work were automated. Wrap Up Object detection is a crucial and highly developing technique that comes under the hood of computer vision. While the technique is directly linked to data availability and processing speed, it has been making fantastic progress in the past decade, given the abundance of data and highly increasing processing capabilities. Throughout the article, we have gone through some of the most important applications of object detection that are currently being applied in different sectors. As you can see, giving a computer ‘a set of eyes’ could prove to be very beneficial to humans in almost every field of life.

How Machines See the World: 7 Use Cases of Object Detection

Artificial Intelligence (AI) has had a huge impact on the healthcare industry. From improving patient outcomes to enhancing medical research and even streamlining healthcare operations, the potential of AI in healthcare is massive. AI-powered tools also help medical practitioners to detect diseases early and develop individualized treatment plans. Consequently, many healthcare organizations are heavily investing in AI. As per , as of 2021, around one-fifth of the surveyed healthcare organizations were in the early stage of AI adoption. It was also seen that the most common kind of AI software being used in the healthcare industry was related to natural language processing (NLP) and healthcare data integration. Statista That being said, many healthcare providers face massive challenges when it comes to seamless AI integration, such as data privacy and integration with legacy systems. To address these challenges, they must partner with AI developers who can help them successfully navigate real-world healthcare problems. It is predicted that the impending 5 to 10 years will be critical for the expansion of AI in the healthcare industry. Thus, progressive healthcare organizations need to collaborate with AI developers to make the most of this technology and deliver superlative patient outcomes. Understanding AI in Healthcare Artificial Intelligence is all set to become a transformational force in healthcare. Essentially, AI in healthcare is an umbrella term that is used to define the application of ML algorithms along with many other cognitive technologies. In other words, AI is when a machine mimics human cognition and becomes capable of learning, discerning, and making informed decisions. AI in healthcare helps to evaluate and predict patient outcomes based on past medical data gathered through electronic health records. Here are some of the most notable uses of AI in healthcare. Medical Imaging: AI algorithms can help medical providers analyze medical images, such as X-rays and MRI scans, to find signs of disease. This can help them make more accurate diagnoses and develop more effective treatment plans. Personalized Medicine: AI can help medical providers develop individualized treatment plans for patients based on their individual medical history, genetics, and lifestyle. This can lead to better outcomes for patients. Drug Discovery: AI can help researchers identify new drugs and treatments for diseases by analyzing vast amounts of data and predicting how different compounds will interact with the human body. Virtual Assistants: AI-powered virtual assistants, such as chatbots, can help patients schedule appointments, ask basic medical questions, and get information about their health without much delay. Predictive Analytics: AI can help medical providers predict which patients are at risk for certain diseases, such as diabetes or heart disease, and intervene early to prevent or treat the disease. Overall, the use of AI in healthcare can help medical providers provide better care to patients, improve medical research, and streamline healthcare operations. Importance of Collaboration between Healthcare Providers and AI Developers Collaboration between healthcare providers and AI developers is critical for improving patient outcomes, reducing healthcare costs, and driving innovation in healthcare. By collaborating with AI developers, healthcare providers can gain access to cutting-edge technology and data analysis tools that can help them make better-informed decisions. Additionally, AI developers can help healthcare providers identify gaps in data collection and develop tools to collect more accurate and complete data. This, in turn, will ensure quality data that helps in accurate diagnosis. Collaboration between healthcare providers and AI developers can also lead to the development of new and innovative technologies and approaches to healthcare. By working together, healthcare providers and AI developers can identify unmet needs and develop solutions to address them. At Algoscale, we can help you collaborate with expert AI developers who can build even highly complex yet sophisticated AI software for healthcare. As a leading , we have developed several specialized software to help healthcare providers streamline their medical processes as well as enhance patient outcomes. Artificial Intelligence Company Here is how Algoscale helped its clients in the healthcare industry. Case Study: ML-Based Analytics Solution for a Healthcare Supply Chain Platform In a highly regulated field like healthcare, transparency in the supply chain is immensely critical. Long lead periods and slow-moving products may cause a lack of visibility in the supply chain, resulting in stock shortages. It may also increase unnecessary expenditures associated with erroneous inventory counts. To resolve this, Algoscale developed a solution that generated business intelligence from data that is dispersed throughout the organization. We developed an ML-based integrated platform for advanced Spend Analytics, offered the client complete visibility on the spending process, and gathered insights to standardize their medical business processes. . Read to know more Case Study: Optimizing Costs for Healthcare Supply Chain with Predictive Analytics Business Intelligence is very important in the healthcare supply chain. Without it, healthcare organizations may face increased costs, substandard patient outcomes, and a loss of business opportunities.With this in mind, Algoscale helped its client, a provider of a cloud-based SaaS purchasing platform, to bring greater transparency to its supply chain. We built a global database to handle massive amounts of data and standardize it across all regions. We gathered info on product catalogs from all the leading healthcare equipment industries by crawling across sources. All this data was used to enrich the database.Our developers also built an end-to-end healthcare eCommerce platform for the Indian healthcare market. The platform included an AI-based recommendation engine that studied past history to provide recommendations and further simplify inventory replenishment for customers.Our solution helped the client discover new-fangled opportunities for savings and generate new revenue. It helped him check spending and supply chain analysis with a vast network of products and suppliers. . Read to know more Challenges of AI in Healthcare AI in healthcare indubitably has many benefits. However, there are also a few challenges that must be addressed. From the point of view of healthcare organizations, the most pressing challenges are as below: Data quality The quality of data is a leading challenge in integrating AI in healthcare. AI systems rely on massive amounts of data to train and enhance their accuracy. Therefore, the data provided by these systems must be complete, accurate, and reliable. Poor-quality data can result in faulty predictions and diagnoses. Privacy and security The use of personal health information raises concerns about privacy and security. AI systems must be designed to protect patient data, comply with privacy regulations, and prevent unauthorized access. Ethical considerations There are ethical considerations when developing and implementing AI in healthcare, including issues around bias, transparency, and accountability. AI models must be developed in a way that is fair and unbiased and that takes into account the potential impact on patients. Adoption and acceptance Healthcare providers may be hesitant to adopt new AI technologies, particularly if they are not familiar with them. It may take time to gain acceptance and trust in AI systems. Integration with existing systems Integrating AI systems with existing healthcare systems can be challenging, particularly if those systems are outdated or not designed for interoperability. AI must be able to work seamlessly with existing systems to be effective. To reach its full potential in the healthcare industry, AI developers can help to overcome the most daunting barriers to adoption. For instance, AI developers can improve and boost engagement with healthcare providers to support AI training which has the potential to increase the likelihood of AI adoption in the future. Similarly, close collaboration between AI developers and healthcare organizations can ensure the highest quality of data. AI developers can implement data governance to comply with the legal and ethical norms of conduct and ensure good practices for the collection, storage, and use of sensitive patient data. Finally, the developers can also streamline the process of AI integration with legacy systems, ensuring complete interoperability. How Algoscale can collaborate with Healthcare Providers to Adopt AI-powered Healthcare Solutions? At Algoscale, we can help you scale your IT team and work with certified developers who are well-versed in all AI technologies. Our specialized can meet the demands of your organization in unique ways. Whether it is a short-term project or a long-term one, our AI developers can help you run your IT operations smoothly, efficiently, and cost-effectively. staff augmentation services We follow a stringent screening process that carefully evaluates each professional in categories like mathematical reasoning, technical proficiencies, and soft skills. With our services, your organization can reduce costs and improve productivity while still maintaining quality standards. Get in touch with our reps today to schedule a free consultation.

The Importance Of Collaboration Between Healthcare Providers And AI Developers

La importancia de la colaboración entre proveedores de atención médica y desarrolladores de IA

L'importance de la collaboration entre les prestataires de soins de santé et les développeurs d'IA

स्वास्थ्य सेवा प्रदाताओं और एआई डेवलपर्स के बीच सहयोग का महत्व

医療提供者と AI 開発者のコラボレーションの重要性

A importância da colaboração entre profissionais de saúde e desenvolvedores de IA

Tầm quan trọng của sự hợp tác giữa các nhà cung cấp dịch vụ chăm sóc sức khỏe và nhà phát triển AI

医疗保健提供者和人工智能开发人员之间合作的重要性

Which is the best Chatbot development company in India & the United States? This is the most confusing question for modern developers and business owners.  But, not all chatbot companies are fully perfect in terms of quality.This is what creates an uproar among people to find the best when Chatbot is already a hot topic among us. Taking this into account, I have pre-selected some chatbot companies after analyzing several parameters namely: Diversified portfolio Previous work experience Technical experience Strictly NDA agreement Years of experience 100% Money back guarantee The list provided below of chatbot companies will surely help you find the right technology partner for all your Chatbot development needs. Additionally, several references have been taken from companies that evaluate companies that name Good companies, Clutch, etc. to provide you with the best collection of chatbot companies.  This will not only save time, but also your manual efforts. Here is a summarized list of chatbot development companies: Top in the list of chatbot companies. Develop custom chatbots that cater to business needs. 1. ValueCoders : Build ai chatbots that can communicate with customers on all channels. 2. CedexTechnologies : Stand on the third position in the list of best chatbot companies. 3. PixelCrayons : On of the Leading chatbot companies in India, improving position in the list. 4. E2logy : Offer companies to help develop chatbots from ideation to integration. 5. OnGraph Technologies Corporation : Reputed name in the list of chatbot companies by delivering top-notch ai chatbots messengers to the clients. 6. Affle Enterprise : Offers self-evolving, intuitive and engaging bot solutions. 7. Enterpi Software Solutions : Serves top-notch chatbot development services to clients across various industries. 8. IndiaNIC Infotech Ltd : One of the trusted chatbot companies in India. 9. Chapter247 Infotech : A considerable choice among the best chatbot companies in India. 10. ELEKS : Let's discuss these Chatbot development companies in detail: 1) ValueCoders (One of the best chatbot companies in India) Location: India/USA Pricing: Starting from $12/hr Employees: 700+ Established: 2004 Chatbots of the new era is creating a different way of working. ValueCoders is one of top which offers a full cycle of chatbot development services to revolutionize the way businesses interact with their clients and streamline processes. software development companies in India The company has above 14-years of experience and 450+ developers. It has worked for SMEs, startups, digital agencies, and the same type companies. It has 2500+ satisfied clients, and its score is 96 out of 100 for the client retention rate. Therefore, ValueCoders turns out to be one of the ISO and CMMI Level 3 certified Chatbot development firms. You can hire top chatbot developers from ValueCoders and can save up to 60% of development cost. They provide highly sophisticated and intelligent chatbot solutions that can be utilized in various domains such as healthcare sector, entertainment, customer support, e-commerce, and supply chain. 2) Cedex Technologies LLP (One of the best chatbot development agencies in India) Location: Cochin, India Pricing: < $25 / hr Employees: 10 - 49 Established: 2015 With so many chatbot companies, Cedex Technologies LLP is a young and dynamic company offers chatbot development solutions, with the office in Cochin, India. They are specialized in building web applications, chatbots and voice apps (Amazon Alexa and Google Assistants). They love challenges and thinks innovatively and disruptively to help our clients to turn their great ideas into reality. 3) PixelCrayons (Offer top chatbot development services in India) Location: India Pricing: $25-49 Employees: 400 Established: 2004 PixelCrayons is a well-known full-service best chatbot development company in India with its physical presence United Kingdom and the United States too. It offers a variety of web design/development and mobile full-service application development services including top chatbot development services in India. Its unmatched customized artificial intelligence and machine learning services to SMEs and startups are known all over the world and are beyond description. Its of the trusted chatbot companies help other  companies to deploy Messaging as a Platform (MAAP) model with tailored chatbot solutions. So, hire best chatbot developers from PixelCrayons and you will be able to save around 60% of development cost and get services of highly trained and skilled machine learning developers, quality infrastructure and all the modern technological developments. 4) E2logy (Best chatbot development company in India) Location: Ahmedabad, India Pricing: < $25 / hr Employees: 50 - 249 Established: 2005 E2logy is a full service top chatbot development company in India with expertise in any custom application development.Their focus area is Enterprise apps, Web applications & Websites, Mobile Apps, Remote Cloud Support and Digital Analytics Implementation. They make custom web applications based on client’s needs. They make mobile apps using Native SDKs of iOS, Android & Windows Phone.As one of the best chatbot companies, It offers services to transform the way businesses collaborate with customers and streamline interactions. 5) OnGraph Technologies Corporation (Best AI chatbot development service providers in India) Location: Noida, India Pricing: $25 - $49 / hr Employees: 50 - 249 Established: 2007 Founded in 2007, OnGraph is one of the best chatbot development firms in India. Since their establishment, they have brought custom software products to clients coming from small, mid-sized and large companies. Ongraph is in the list of top chatbot companies due to service offered by the them to transform the way business collaborate with clients and streamline interactions. Currently, they have rendered quality web and mobile development services to companies engaged in varied markets like health care, banking &amp; finance, eCommerce, education, telecom, media & entertainment, and more. 6) Affle Enterprise (One of the best chatbot development firms in India) Location: Gurgaon, India Pricing: $25 - $49 / hr Employees: 50 - 249 Established: 2005 Affle earned its position among the list of best chatbot companies in the industry due to proprietary consumer intelligence platform that delivers consumer engagement, acquisitions and transactions through relevant Mobile Advertising. The platform aims to enhance returns on marketing investment through contextual mobile ads and also by reducing digital ad fraud. 7) Enterpi Software Solutions (Top chatbot development company in India) Location: Hyderabad, India Pricing: < $25 / hrc Employees: 50 - 249 Established: 2008 EnterPi is a full spectrum software development company. Over the past 10 years, EnterPi has developed highly robust software systems for the worlds biggest companies. Applications built by us are used by multiple Fortune 500 companies with thousands of users, touching millions of customers. 8) IndiaNIC Infotech Ltd. (one of the best chatbot development firms in India) Pricing: < $25/hr Location: United States, India Employees: 250 - 999 Founded: 1998 Its main objective is to work with personalized web development, mobile application development, e-commerce business solutions, top chatbot development services with engineering practices guided by design and provide best chatbot development services in India. They have proficient engineers, artists, strategists, geeks and some rebellious personalities in their team, to make IndiaNIC the best place to be.They bring together intelligent and talented thinkers and help unleash their combined creativity in the world. 9) Chapter247 Infotech (One of the best chatbot development agencies in India) Location: Indore , India Pricing: < $25 / hr Employees: 10 - 49 Established: 2012 Chapter247 Infotech is a full service software partner specialized in architecting, designing & developing tech solutions by binding Web, IoT, Analytics, AI and Cloud technologies. Their global clients include organizations right from Enterprise, Midsize to startups and provide best chatbot development services in India. Chapter247 considered as the most reliable among other chatbot companies as they have talented chatbot developers who can make tedious job of creating bots for messaging platforms, website or a mobile a simple affair. In 6 years, they have launched over 100 products with ambitious clients in industries like Ecommerce, financial services, healthcare, education, shipping, energy and retail. 10) ELEKS (Best chatbot development company in India) Pricing: $25 - $49/hr Location: Ukraine, United States Employees: 1,000 - 9,999 Founded: 1991 They offer high-tech innovations to Fortune 500 companies, large companies, and technological challenges, improving the way they work and increasing the value they create for the modern world. They have more than 1,200 professionals located in delivery centers in Eastern Europe and sales offices in USA, Europe and Japan. They are helping their clients by digitally transforming their businesses through expert consulting, software engineering services and top chatbot development services.They provide their customers with a full range of software services. These include product development, dedicated equipment, R & D and technology consulting. 11) Ahex Technologies Pvt. Ltd. (Best among the list of chatbot compan ies in India) Location: India Pricing: < $25 / hr Employees: 50 - 249 Established: 2009 Founded in July 2009, Ahex Technologies is an ISO 9001:2015 certified software development Agency. They are provider of customized web application, mobile application, best chatbot development services in India and e-business solutions across the globe. 12) Quovantis Technologies (Best chatbot development company in India) Location: Noida, India Pricing: $25 - $49 / hr Employees: 50 - 249 Established: 2010 It is a full-lifecycle product development services company with a mission of creating beautiful products through design led product development. Quovantis Technologies has helped many leading product companies and innovative startups build delightful digital experiences. Some of the major clients include LOtus, Synaptics, LInklabs, Jason, Unity, Caarmo, Zoomvy etc. 13) TECHTIC SOLUTIONS INC (One of the top chatbot development agencies in India) Pricing: $25 - $49/hr Location: United States, India Employees: 50 - 249 Founded: 2009 Techtic Solutions Inc. is a leading company in the development of web and mobile applications known for offering innovative solutions especially top chatbot development services in India for any complex problem. It was incorporated in 2009 and based in the United States. It is an interactive creative agency, passionate about offering successful, efficient and successful solutions for your business. Whether it's a new company or a large corporation, they can offer innovative marketing strategies, exceptional brands, mobile application development and web solutions that exceed their expectations. 14) ArStudioz (One of the best chatbot development firms in India) Location: Jaipur, India Pricing: < $25 / hr Employees: 10 - 49 Established: 2017 ArStudioz is a solution-oriented team that develops user-friendly Mobile Apps and Software to help you increase engagement, transactions an overall satisfaction with their customers. They use the leading technology and systems to develop an app designed to your specifications that truly works and creates satisfaction among their customers. 15) ElkanIO Research Labs (One of the top chatbot development firms in India) Location: Cochin, India Pricing: < $25 / hr Employees: 2 - 9 Established: 2017 Founded in 2017 with a mindset to tackle the real world problems by using emerging digital technologies. A team with hands-on experience in developing Artificial Intelligence and Advanced Analytics solutions to cater to various industry needs. They have expertise in IoT-Software development support, Retail Analytics solutions, Predictive Maintenance Solutions and also provide top chatbot development services in India. 16) DevCom (Best chatbot development company in India) Pricing: $25 - $49/hr Location: Ukraine, India Employees: 50 - 249 Founded: 2000 DevCom proves that it was founded in the year 2000 and is a reliable provider of top chatbot development services in India for its customers worldwide. Devcom has carved a name for itself among the best chatbot companies by building AI chatbots that often outrun humans in virtual assistance. They understand that the value of technology lies in the benefits it provides to customers.This is the reason why they select among the best cutting-edge technologies to design solutions that meet customer needs. They comply with the latest technological standards and specialize in the development of software using all latest technologies. 17) XenonStack (Best chatbot development company in India) Location: Chandigarh, India Pricing: $25 - $49 / hr Employees: 50 - 249 Established: 2012 It is a Product Engineering and Technology Services company provides Digital enterprise services and solutions with DevOps , Big Data Engineering , Data Analytics , Cloud Migration ,Machine learning, Data Science and providing best chatbot development services in India. 18) Algoscale Technologies, Inc. (One of the best chatbot companies in India) Location: Noida, India Pricing: < $25 / hr Employees: 10 - 49 Established: 2014 Algoscale is a US incorporated, privately held boutique Big Data Analytics and Decision Science firm helping organizations derive valuable business insights from mountains of data through application of analytics and technology. It is helping organizations make strategic business decisions by deriving insights through application of analytical tools, techniques, and technology. 19) Webtunix AI (One of the top chatbot development companies in India) Location: Mohali, India Pricing: < $25 / hr Employees: 10 - 49 Established: 2015 Webtunix AI is a leading data science consulting company that is offering services in Chatbot development which are related to Industry needs. At Webtunix, they are offering services like data analytics services, machine learning services, data scraping, Data Mining, Object Detection, Video and Image Processing, Natural language processing and more. Hire best chatbot developers from this company. 20) XeliumTech Solutions (One of the best chatbot development services provider) Location: Gurgaon, India Pricing: < $25 / hr Employees: 50 - 249 Established: 2010 XeliumTech Solutions is a prominent Mobile App Development Company in Gurgaon, India, offering a range of Mobile App Design and Development, Android Application Development, IOS Application Development, Chatbot development, Cross-compatible, and Hybrid App Development Services, globally since 2010. They have a passionate team of skilled professionals who make it happen for them to deliver the niche solutions required by our valued customers. Conclusion: In this article, we have enlisted top 20 Chatbot development companies in India/USA. You can pick any company as per your business needs and budget constraint. Now, you will be definitely in a position to decide the best chatbot development firms in India after reading this article. I am ending by saying that picking the best chatbot development company in India seems easy, however, you must take all these points into account for your own good and successful business. Note: The list above is still in progress mode. If you like to include your Chatbot development company with us, then please write to us in the comment box.  Otherwise, you can message to us along with the details of the respective company.

Best Chatbot Development Companies For SME's and Startups in India, USA| Hire Best Chatbot Developer

AWS has just made a new open-source tool called Karpenter generally available. I will dig into how it works and will be unavoidably comparing it with what we have built at CAST AI. What exactly is Karpenter, how does its autoscaling mechanism work, and should you use it? Karpenter is a very simple autoscaler with two reactive policies. Its biggest selling point? No node pools. Let’s take a look at how Karpenter works and compare it to the autoscaling mechanisms of CAST AI. Quick feature comparison – CAST AI vs. Karpenter Detailed Feature Comparison of Karpenter and CAST AI Installation and Onboarding After trying Karpenter out, I’d say that onboarding isn’t trivial. Still, it’s not as hard as the Kubernetes Cluster Autoscaler. Karpenter onboarding requires you to have eksctl CLI installed and configured, even if you created an EKS cluster without the help of eksctl. It takes around to get started with Karpenter. 6 to 7 steps CAST AI doesn’t have an eksctl dependency, and you’re looking at a 2-step process your IT manager could complete on their own in a minute or two. Autoscaling Policies Karpenter has only two of the most basic autoscaling policies. At first glance, they are quite similar to two of the policies in CAST AI (Unscheduled Pod Policy and Node Deletion policy). Let’s compare the two. 1. Provisioner In its simplest form (without sub-features), CAST AI’s Unscheduled Pod Policy works in a similar way as Karpenter’s Provisioner. If there are any pods that can’t start because no suitable node has been found by the Kubernetes vanilla scheduler, a new node will be created based on those pending pod constraints like zone, subnet, PodAntiAffinity, capacity preference (spot or on-demand). Karpenter has these constraints available for NodeSelector, NodeAffinity, TopologySpreadConstraints: requirements:
    - key: node.k8s.aws/instance-type #If not included, all instance types are considered
      operator: In
      values: [ , ]
    - key: #If not included, all zones are considered
      operator: In
      values: [ , ]
    - key: #If not included, all architectures are considered
      values: [ , ]
    - key: #If not included, the webhook the AWS cloud provider will to on-demand
      operator: In
      values: [ , ] "m5.large" "m5.2xlarge" "topology.kubernetes.io/zone" "us-east-1a" "us-east-1b" "kubernetes.io/arch" "arm64" "amd64" " karpenter.sh/capacity-type" for default "spot" "on-demand" Karpenter supports GPU and ARM with the following tolerations: , , and . To add a GPU node, one needs to define a specific list of nodes (and we’re back to the node pool idea). nvidia.com/gpu amd.com/gpu aws.amazon.com/neuron CAST AI allows you to specify more pod constraints, like but doesn’t require you to specify the instance types on your own. In case you haven’t checked lately, there are 475 available types on AWS, which makes it easy to get back to implicit Node Pools by accident. get me instances with fast local storage for Spark jobs You can also specify the spot interruption rate per cluster and override per workload in CAST AI. For example, you may want your casual spot workload in a cluster to be on more cost-efficient spot instances – even if that means these instances could be interrupted once in a while. But those huge Jupyter notebook pods with a ton of resources? You want them to run spot instances that are so unpopular that they barely get ever interrupted. Back to the Node Pools against one’s will Running your Kubernetes workloads on burstable T3a instance types is a crime to your business and DevOps. Unfortunately, this is what you will get by default with Karpenter as they are the most affordable instances. Not excited about burstable instances? You’ll have to list all the acceptable instance types that you want your Kubernetes cluster to have in the Karpenter Provisioner CRD. And with that, you have just specified a cherry-picked node list and are implicitly back to Node Pools. And what if these cherry-picked node lists are not available during Black Friday or Christmas holidays? You will need to extend the list with even more alternatives. 2. Delete Empty Node policy As the name implies, Karpenter can and will delete empty nodes. This policy allows you to set the Time To Live for empty nodes ( with a default of 30). ttlSecondsAfterEmpty CAST AI’s Delete Node policy works in the same fashion. The CAST AI engine will not remove nodes if there are workloads running on them (excluding DaemonSets of course). The Kubernetes scheduler by default acts fairly and will always distribute workloads for maximum performance and availability. And if you’re running EKS, you can’t change the Kubernetes scheduler profile. Let’s look at this with an analogy: Imagine a train (Kubernetes cluster) with 10 train cars (nodes); each train car can seat 100 passengers (pods), the train has a capacity to transport 1000 passengers at peak. The train moves from the central station fully utilized with 1000 seats taken and with every stop more people get off the train than on it. Even a few stops before the end of the train journey, if the train is carrying only 15 passengers out of 1000, someone needs to make sure that there are empty train cars. It’s likely that there are 1 to 2 passengers in a train car enjoying their privacy, and the Delete Empty Node train car can’t find anything to remove. So how does Karpenter help with over-provisioned resources for off-peak times when the load is lower? Enter Node Expiry Karpenter offers a Node Expiry feature. If a node expiry time-to-live value ( ) is reached, that node is drained of pods, and the node is deleted. ttlSecondsUntilExpired Great! But wait a minute, what about your workloads that are running on these nodes? Sorry, end of shift. Nodes are going down. Alright, but how would you know what the expiration time should be? If you get it too short, your pods are going down, you then have to wait several minutes for the Karpenter provisioner to add new nodes with refreshed ttlSecondsUntilExpired. If you set the ttlSecondsUntilExpired too long, you’re wasting money. So what we see here is that Karpenter reacts in time to add new capacity but leaves you hanging with reducing any excess capacity. CAST AI, on the other hand, offers Evictor that shrinks the cluster to the minimum number of nodes by bin packing pods (simulating evictions, honoring Pod Disruption Budgets, etc). Once Evictor makes a node empty, it’s deleted automatically. Cloud Vendor Support At the moment, Karpenter only supports AWS. The code has a lot of AWS-specific hardcoded logic. For instance, using EC2 Fleet and Spot Fleet, without a way forward for other clouds. On the other hand, Karpenter has empty interfaces created in the code so the project looks pluggable and open enough if someone were to take the initiative of adding other cloud provider support. CAST AI supports AWS (EKS, kOps), Google Cloud Platform (GKE), and Microsoft Azure (AKS). No Node Pools The promise of being free of node pools sounds great. But the way it is now, Karpenter doesn’t offer users almost any control over the scheduling decisions when using a “naked” Provisioner. Provisioner is Karpenter’s Kubernetes CRD which defines the node creation constraints. For example, if you had workloads that needed a GPU and you wanted the GPU nodes to have a taint (so that only the pods that actually need it are placed on those nodes), you would create a Provisioner like that: apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: gpu
spec:
  requirements: 
  - key: node.kubernetes.io/instance-type
    operator: In
    values:
      - p3 xlarge
      - p3 xlarge
      - p3 xlarge
  taints:
  - key: nvidia.com/gpu
    value: effect: “NoSchedule” .2 .8 .16 true In Karpenter, the engineer still needs to have deep knowledge of the available instance types with all of their nuances. To use the SSDs in a K8s cluster, you would need to create a user-data script that makes a single logical disk out of those local SSDs and creates a symbolic link for the kubelet to use. That requires creating an EC2 Launch Template and referencing it in the Provisioner resource. And, most importantly, you need to keep it up to date if you want to achieve cost savings. In more complicated clusters, you’d have to create many different Provisioners, i.e., node pools: one for GPUs, another one for storage optimized, yet another one for infrequently interrupted Spot instances, etc. Conclusion: Provisioner Conflicts When there are multiple Provisioner resources defined and an unschedulable pod can tolerate/select any of them, the result will be non-deterministic. Here’s an example: apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: team-a
spec:
  labels:
    team: a
  requirements:
    - key: operator: In
      values: [ ]
---
apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: team-b
spec:
  labels:
    team: b
  requirements:
    - key: operator: In
      values: [ ]
---
apiVersion: v1
kind: Pod
metadata:
  name: pod
spec:
  containers:
  - image: nginx
    name: pod "node.kubernetes.io/instance-type" "m5.large" "node.kubernetes.io/instance-type" "m5.2xlarge" The nginx pod would be scheduled on either the m5.large or the m5.2xlarge node (chosen based on the sequence the API-Server returns for the records). This is another dimension that the DevOps engineer would need to think about. The Provisioners should clearly define constraints, and all pods should clearly define the selectors to reduce or eliminate randomness. In CAST AI, the selectors/tolerations are predefined, and engineers don’t have to come up with them on their own. Pod Grouping (a.k.a. Bin Packing on Provisioning) Karpenter’s GitHub states that the solution uses bin packing, but only in the nodePlacer sense. That is, it groups pods to be placed on nodes to avoid creating a separate node for each pod. In CAST AI, we use Permutations and Clique functions. Their purpose is to find out the most optimal pod groupings. Karpenter’s algorithm is basically creating a hash of the pod constraints. This means that otherwise compatible pods will end up on different nodes (Karpenter will create multiple nodes). Consider this example: apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  nodeSelector:
    topology.kubernetes.io/zone: containers:
    - image: nginx
      name: pod
---
apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                  - - containers:
    - image: nginx
      name: pod "eu-central-1a" "eu-central-1a" "eu-central-1b" Even though the pods could fit into a single node placed in the eu-central-1a zone, Karpenter will create two nodes (that might even end up in the same zone) because the hash of the pod constraints is different. Cost Optimization When Karpenter bin-packs pods, it produces a list of viable instance types per each pod group. For example, if there’s a pod group that needs 2 CPU and 8 GiB, then a list of all instance types that have at least 2 CPU and 8 GiB is retrieved. The list is sorted by resource weight. Karpenter sorts those instance types using a weight function, where 1 CPU = 1 GiB, favoring nodes with higher CPU and lower memory: func weightOf(instanceType cloudprovider.InstanceType) float64 { euclidean(
		float64(instanceType.CPU().Value()),
		float64(instanceType.Memory().ScaledValue(resource.Giga)), float64(instanceType.NvidiaGPUs().Value())* , float64(instanceType.AMDGPUs().Value())* , float64(instanceType.AWSNeurons().Value())* , )
} // weightOf uses a euclidean distance function to compare the instance types. // Units are normalized such that 1cpu = 1gb mem. Additionally, accelerators // carry an arbitrarily large weight such that they will dominate the priority, // but if equal, will still fall back to the weight of other dimensions. return // 1 gb = 1 cpu 1000 // Heavily weigh gpus x 1000 1000 // Heavily weigh gpus x 1000 1000 // Heavily weigh neurons x 1000 After sorting, only the first 20 nodes are picked. This list might not contain the cheapest instance types because of Karpenter’s particular weighing system. There is no inventory database that can tell exactly how much something costs. After the bin-packing and sorting, the list is passed to the EC2 Fleet API. The mode of choosing the instance type depends on whether it’s an on-demand node or spot instance. On-demand instances In this case, Karpenter passes the “lowest cost” allocation method that makes the EC2 Fleet API choose the cheapest instance type out of the 20 received viable instance types. There might be cheaper instance types, but the API works with what it was given. Spot instances Karpenter passes the “capacity optimized prioritized” allocation method that makes the EC2 Fleet API choose the instance types that have the highest availability, prioritizing the instance types at the top of the list. This has nothing to do with cost optimization and could be considered something similar to the CAST AI “Least interrupted” Spot reliability setting. CAST AI offers an entirely different approach to cost optimization. The platform selects the most cost-efficient instances that match the workload requirements and automatically rightsizes instances to avoid overallocation. CAST AI also comes with features such as cost reporting on any level, information about cluster cost, or potential savings. Karpenter doesn’t have any of those. Karpenter vs. CAST AI – Which Autoscaler is a Better Pick for You? As you can see, Karpenter offers a limited set of functionalities right now. AWS Karpenter is an option if you’re on AWS and considering the native Kubernetes Cluster Autoscaler. Today, , so node pools on which the Kubernetes Cluster Autoscaler works become unmanageable considering the exhaustible instance inventory. This is essentially what Karpenter kind of tries to solve. AWS offers 400+ instances If you’re looking for an autoscaler that can significantly reduce your cloud bill for production workloads without impacting their availability, you need to look elsewhere. CAST AI offers a substantially more robust set of features which makes it a great fit for reliable automated cost reduction on production and development workloads. It also works with all three major cloud providers: AWS, Google Cloud, and Azure. To see what results CAST AI could bring you, you can or book a quick demo by clicking the button below. register and start with a free cluster analysis

AWS' Karpenter Autoscaler and How it Stacks Up Against CAST AI

Introduction One of the major advantages of using Kubernetes for container orchestration is that it makes it really easy to scale our application horizontally and account for increased load. Natively, horizontal pod autoscaling can scale the deployment based on CPU and Memory usage but in more complex scenarios we would want to account for other metrics before making scaling decisions. Welcome . Prometheus is the standard tool for monitoring deployed workloads and the Kubernetes cluster itself. Prometheus adapter helps us to leverage the metrics collected by Prometheus and use them to make scaling decisions. These metrics are exposed by an API service and can be readily used by our object. Prometheus Adapter Horizontal Pod Autoscaling Deployment Architecture Overview We will be using Prometheus adapter to pull custom metrics from our Prometheus installation and then let the use it to scale the pods up or down. Horizontal Pod Autoscaler (HPA) Prerequisite Basic knowledge about horizontal pod autoscalingPrometheus deployed in-cluster or accessible using an endpoint. We will be using a Highly Available deployment. Prometheus-Thanos Deploying the Sample Application Let’s first deploy a sample app over which we will be testing our Prometheus metrics autoscaling. We can use the manifest below to do it apiVersion: v1
kind: Namespace
metadata:
  name: nginx
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: nginx
  name: nginx-deployment
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        prometheus.io/path: prometheus.io/scrape: prometheus.io/port: labels:
        app: nginx-server
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - nginx-server
              topologyKey: kubernetes.io/hostname
      containers:
      - name: nginx-demo
        image: vaibhavthakur/nginx-vts:1.0
        imagePullPolicy: Always
        resources:
          limits:
            cpu: 2500m
          requests:
            cpu: 2000m
        ports:
        - containerPort: 80
          name: http
---
apiVersion: v1
kind: Service
metadata:
  namespace: nginx
  name: nginx-service
spec:
  ports:
  - port: 80
    targetPort: 80
    name: http
  selector:
    app: nginx-server : LoadBalancer "/status/format/prometheus" "true" "80" type This will create a namespace named and deploy a sample Nginx application in it. The application can be accessed using the service and also exposes at the endpoint over port . For the sake of our setup we have created a DNS entry for the ExternalIP which maps to nginx nginx vts metrics /status/format/prometheus 80 nginx.gotham.com root$ kubectl get deploy 
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
nginx-deployment   1/1     1            1           43d

root$ kubectl get pods 
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-65d8df7488-c578v   1/1     Running   0          9h

root$ kubectl get svc
NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
nginx-service   ClusterIP   10.63.253.154   35.232.67.34      80/TCP    43d

root$ kubectl describe deploy nginx-deployment
Name:                   nginx-deployment
Namespace:              nginx
CreationTimestamp:      Tue, 08 Oct 2019 11:47:36 -0700
Labels:                 app=nginx-server
Annotations:            deployment.kubernetes.io/revision: 1
                        kubectl.kubernetes.io/last-applied-configuration:
                          { : , : , :{ :{}, : , : }, :...
Selector:               app=nginx-server
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  1 max unavailable, 1 max surge
Pod Template:
  Labels:       app=nginx-server
  Annotations:  prometheus.io/path: /status/format/prometheus
                prometheus.io/port: 80
                prometheus.io/scrape: Containers:
   nginx-demo:
    Image:      vaibhavthakur/nginx-vts:v1.0
    Port:       80/TCP
    Host Port:  0/TCP
    Limits:
      cpu:  250m
    Requests:
      cpu:        200m
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
OldReplicaSets:  <none>
NewReplicaSet:   nginx-deployment-65d8df7488 (1/1 replicas created)
Events:          <none>


root$ curl nginx.gotham.com
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href= >nginx.org</a>.<br/>
Commercial support is available at
<a href= >nginx.com</a>.</p>

<p><em>Thank you using nginx.</em></p>
</body>
</html> "apiVersion" "extensions/v1beta1" "kind" "Deployment" "metadata" "annotations" "name" "nginx-deployment" "namespace" "nginx" "spec" true "http://nginx.org/" "http://nginx.com/" for These are all the metrics currently exposed by the application $ curl nginx.gotham.com/status/format/prometheus nginx_vts_info{hostname= ,version= } 1 nginx_vts_start_time_seconds 1574283147.043 nginx_vts_main_connections{status= } 215
nginx_vts_main_connections{status= } 4
nginx_vts_main_connections{status= } 215
nginx_vts_main_connections{status= } 0
nginx_vts_main_connections{status= } 15577
nginx_vts_main_connections{status= } 3
nginx_vts_main_connections{status= } 1 nginx_vts_main_shm_usage_bytes{shared= } 1048575
nginx_vts_main_shm_usage_bytes{shared= } 3510
nginx_vts_main_shm_usage_bytes{shared= } 1 nginx_vts_server_bytes_total{host= ,direction= } 3303449
nginx_vts_server_bytes_total{host= ,direction= } 61641572
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 15574
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 2
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 15576
nginx_vts_server_request_seconds_total{host= } 0.000
nginx_vts_server_request_seconds{host= } 0.000
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_bytes_total{host= ,direction= } 3303449
nginx_vts_server_bytes_total{host= ,direction= } 61641572
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 15574
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 2
nginx_vts_server_requests_total{host= ,code= } 0
nginx_vts_server_requests_total{host= ,code= } 15576
nginx_vts_server_request_seconds_total{host= } 0.000
nginx_vts_server_request_seconds{host= } 0.000
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0
nginx_vts_server_cache_total{host= ,status= } 0 # HELP nginx_vts_info Nginx info # TYPE nginx_vts_info gauge "nginx-deployment-65d8df7488-c578v" "1.13.12" # HELP nginx_vts_start_time_seconds Nginx start time # TYPE nginx_vts_start_time_seconds gauge # HELP nginx_vts_main_connections Nginx connections # TYPE nginx_vts_main_connections gauge "accepted" "active" "handled" "reading" "requests" "waiting" "writing" # HELP nginx_vts_main_shm_usage_bytes Shared memory [ngx_http_vhost_traffic_status] info # TYPE nginx_vts_main_shm_usage_bytes gauge "max_size" "used_size" "used_node" # HELP nginx_vts_server_bytes_total The request/response bytes # TYPE nginx_vts_server_bytes_total counter # HELP nginx_vts_server_requests_total The requests counter # TYPE nginx_vts_server_requests_total counter # HELP nginx_vts_server_request_seconds_total The request processing time in seconds # TYPE nginx_vts_server_request_seconds_total counter # HELP nginx_vts_server_request_seconds The average of request processing times in seconds # TYPE nginx_vts_server_request_seconds gauge # HELP nginx_vts_server_request_duration_seconds The histogram of request processing time # TYPE nginx_vts_server_request_duration_seconds histogram # HELP nginx_vts_server_cache_total The requests cache counter # TYPE nginx_vts_server_cache_total counter "_" "in" "_" "out" "_" "1xx" "_" "2xx" "_" "3xx" "_" "4xx" "_" "5xx" "_" "total" "_" "_" "_" "miss" "_" "bypass" "_" "expired" "_" "stale" "_" "updating" "_" "revalidated" "_" "hit" "_" "scarce" "*" "in" "*" "out" "*" "1xx" "*" "2xx" "*" "3xx" "*" "4xx" "*" "5xx" "*" "total" "*" "*" "*" "miss" "*" "bypass" "*" "expired" "*" "stale" "*" "updating" "*" "revalidated" "*" "hit" "*" "scarce" Among these we are particularly interested in . We will be using the value of this metric to determine whether or not to scale our Nginx deployment. nginx_vts_server_requests_total Create Prometheus Adapter ConfigMap Use the manifest below to create the Prometheus Adapter Configmap apiVersion: v1
kind: ConfigMap
metadata:
  name: adapter-config
  namespace: monitoring
data:
  config.yaml: |
    rules:
    - seriesQuery: resources:
        overrides:
          kubernetes_namespace:
            resource: namespace
          kubernetes_pod_name:
            resource: pod
      name:
        matches: as: metricsQuery: (sum(rate(<<.Series>>{<<.LabelMatchers>>}[1m])) by (<<.GroupBy>>)) 'nginx_vts_server_requests_total' "^(.*)_total" " _per_second" ${1} This config map only specifies a single metric. However, we can always add more metrics. Create Prometheus Adapter Deployment Use the following manifest to deploy Prometheus Adapter apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: custom-metrics-apiserver
  name: custom-metrics-apiserver
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: custom-metrics-apiserver
  template:
    metadata:
      labels:
        app: custom-metrics-apiserver
      name: custom-metrics-apiserver
    spec:
      serviceAccountName: monitoring
      containers:
      - name: custom-metrics-apiserver
        image: quay.io/coreos/k8s-prometheus-adapter-amd64:v0.4.1
        args:
        - /adapter
        - --secure-port=6443
        - --tls-cert-file=/var/run/serving-cert/serving.crt
        - --tls-private-key-file=/var/run/serving-cert/serving.key
        - --logtostderr= - --prometheus-url=http://thanos-querier.monitoring:9090/
        - --metrics-relist-interval=30s
        - --v=10
        - --config=/etc/adapter/config.yaml
        ports:
        - containerPort: 6443
        volumeMounts:
        - mountPath: /var/run/serving-cert
          name: volume-serving-cert
          readOnly: - mountPath: /etc/adapter/
          name: config
          readOnly: volumes:
      - name: volume-serving-cert
        secret:
          secretName: cm-adapter-serving-certs
      - name: config
        configMap:
          name: adapter-config true true true This will create our deployment which will spawn the Prometheus adapter pod to pull metrics from Prometheus. It should be noted that we have set the argument . This is because we have deployed a Prometheus-Thanos cluster in the monitoring namespace in the same Kubernetes cluster as Prometheus adapter. You can change this argument to point to your Prometheus deployment. --prometheus-url= http://thanos-querier.monitoring:9090/ If you notice the logs of this container you can see that it is fetching the metric defined in the config file I1122 00:26:53.228394       1 api.go:74] GET http://thanos-querier.monitoring:9090/api/v1/series?match%5B%5D=nginx_vts_server_requests_total&start=1574381213.217 200 OK
I1122 00:26:53.234234       1 api.go:93] Response Body: { : , :[{ : , : , : , : , : , : , : , : , : , : },{ : , : , : , : , : , : , : , : , : , : } "status" "success" "data" "__name__" "nginx_vts_server_requests_total" "app" "nginx-server" "cluster" "prometheus-ha" "code" "1xx" "host" "*" "instance" "10.60.64.39:80" "job" "kubernetes-pods" "kubernetes_namespace" "nginx" "kubernetes_pod_name" "nginx-deployment-65d8df7488-sbp95" "pod_template_hash" "65d8df7488" "__name__" "nginx_vts_server_requests_total" "app" "nginx-server" "cluster" "prometheus-ha" "code" "1xx" "host" "*" "instance" "10.60.64.8:80" "job" "kubernetes-pods" "kubernetes_namespace" "nginx" "kubernetes_pod_name" "nginx-deployment-65d8df7488-mwzxg" "pod_template_hash" "65d8df7488" Creating Prometheus Adapter API Service The manifest below will create an API service so that our Prometheus adapter is accessible by Kubernetes API and thus metrics can be fetched by our Horizontal Pod Autoscaler. apiVersion: v1
kind: Service
metadata:
  name: custom-metrics-apiserver
  namespace: monitoring
spec:
  ports:
  - port: 443
    targetPort: 6443
  selector:
    app: custom-metrics-apiserver
---
apiVersion: apiregistration.k8s.io/v1beta1
kind: APIService
metadata:
  name: v1beta1.custom.metrics.k8s.io
spec:
  service:
    name: custom-metrics-apiserver
    namespace: monitoring
  group: custom.metrics.k8s.io
  version: v1beta1
  insecureSkipTLSVerify: groupPriorityMinimum: 100
  versionPriority: 100 true Testing the Set-Up Let’s check what all custom metrics are available root$ kubectl get --raw | jq .

{ : , : , : , : [
    { : , : , : , : , : [ ]
    },
    { : , : , : , : , : [ ]
    }
  ]
} "/apis/custom.metrics.k8s.io/v1beta1" "kind" "APIResourceList" "apiVersion" "v1" "groupVersion" "custom.metrics.k8s.io/v1beta1" "resources" "name" "pods/nginx_vts_server_requests_per_second" "singularName" "" "namespaced" true "kind" "MetricValueList" "verbs" "get" "name" "namespaces/nginx_vts_server_requests_per_second" "singularName" "" "namespaced" false "kind" "MetricValueList" "verbs" "get" We can see that metric is available. Now, let’s check the current value of this metric nginx_vts_server_requests_per_second root$ kubectl get --raw | jq .

{ : , : , : { : }, : [
    { : { : , : , : , : }, : , : , : }
  ]
} "/apis/custom.metrics.k8s.io/v1beta1/namespaces/nginx/pods/*/nginx_vts_server_requests_per_second" "kind" "MetricValueList" "apiVersion" "custom.metrics.k8s.io/v1beta1" "metadata" "selfLink" "/apis/custom.metrics.k8s.io/v1beta1/namespaces/nginx/pods/%2A/nginx_vts_server_requests_per_second" "items" "describedObject" "kind" "Pod" "namespace" "nginx" "name" "nginx-deployment-65d8df7488-v575j" "apiVersion" "/v1" "metricName" "nginx_vts_server_requests_per_second" "timestamp" "2019-11-19T18:38:21Z" "value" "1236m" Create an HPA which will utilize these metrics. We can use the manifest below to do it. apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-custom-hpa
  namespace: nginx
spec:
  scaleTargetRef:
    apiVersion: extensions/v1beta1
    kind: Deployment
    name: nginx-deployment
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - : Pods
    pods:
      metricName: nginx_vts_server_requests_per_second
      targetAverageValue: 4000m type Once you have applied this manifest, you can check the current status of HPA as follows: root$ kubectl describe hpa
Name:               nginx-custom-hpa
Namespace:          nginx
Labels:             <none>
Annotations:        autoscaling.alpha.kubernetes.io/metrics:
                      [{ : , :{ : , : }}]
                    kubectl.kubernetes.io/last-applied-configuration:
                      { : , : , :{ :{}, : , : "type" "Pods" "pods" "metricName" "nginx_vts_server_requests_per_second" "targetAverageValue" "4" "apiVersion" "autoscaling/v2beta1" "kind" "HorizontalPodAutoscaler" "metadata" "annotations" "name" "nginx-custom-hpa" "namespace" "n...
CreationTimestamp:  Thu, 21 Nov 2019 11:11:05 -0800
Reference:          Deployment/nginx-deployment
Min replicas:       2
Max replicas:       10
Deployment pods:    0 current / 0 desired
Events:             <none> Now, let's generate some load on our service. We will be using a utility called for this. Vegeta In a separate terminal run the following command | vegeta attack -rate=5 -duration=0 | vegeta report echo "GET http://nginx.gotham.com/" Meanwhile monitor the nginx pods and horizontal pod autoscaler and you should see something like this root$ kubectl get -w pods
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-65d8df7488-mwzxg   1/1     Running   0          9h
nginx-deployment-65d8df7488-sbp95   1/1     Running   0          4m9s
NAME                                AGE
nginx-deployment-65d8df7488-pwjzm   0s
nginx-deployment-65d8df7488-pwjzm   0s
nginx-deployment-65d8df7488-pwjzm   0s
nginx-deployment-65d8df7488-pwjzm   2s
nginx-deployment-65d8df7488-pwjzm   4s
nginx-deployment-65d8df7488-jvbvp   0s
nginx-deployment-65d8df7488-jvbvp   0s
nginx-deployment-65d8df7488-jvbvp   1s
nginx-deployment-65d8df7488-jvbvp   4s
nginx-deployment-65d8df7488-jvbvp   7s
nginx-deployment-65d8df7488-skjkm   0s
nginx-deployment-65d8df7488-skjkm   0s
nginx-deployment-65d8df7488-jh5vw   0s
nginx-deployment-65d8df7488-skjkm   0s
nginx-deployment-65d8df7488-jh5vw   0s
nginx-deployment-65d8df7488-jh5vw   1s
nginx-deployment-65d8df7488-skjkm   2s
nginx-deployment-65d8df7488-jh5vw   2s
nginx-deployment-65d8df7488-skjkm   3s
nginx-deployment-65d8df7488-jh5vw   4s

root$ kubectl get hpa
NAME               REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
nginx-custom-hpa   Deployment/nginx-deployment   5223m/4   2         10        3          5m5s It can be clearly seen that the HPA scaled up our pods as per the requirement, and when we interrupted the Vegeta command, we got the vegeta report. It clearly shows that all our requests were served by the application. root$ | vegeta attack -rate=5 -duration=0 | vegeta report
^CRequests      [total, rate, throughput]  224, 5.02, 5.02
Duration      [total, attack, ]      44.663806863s, 44.601823883s, 61.98298ms
Latencies     [mean, 50, 95, 99, max]    63.3879ms, 60.867241ms, 79.414139ms, 111.981619ms, 229.310088ms
Bytes In      [total, mean]              137088, 612.00
Bytes Out     [total, mean]              0, 0.00
Success       [ratio]                    100.00%
Status Codes  [code:count]               200:224  
Error Set: echo "GET http://nginx.gotham.com/" wait Conclusion This set-up demonstrates how we can use the Prometheus adapter to autoscale deployments based on some custom metrics. For the sake of simplicity we have only fetched one metric from our Prometheus server. However, the adapter Configmap can be extended to fetch some or all the available metrics and use them for autoscaling. If the Prometheus installation is outside of our Kubernetes cluster, we just need to make sure that the query end-point is accessible from the cluster and update it in the adapter deployment manifest. With more complex scenarios, multiple metrics can be fetched and used in-combination to make scaling decisions. Feel free to reach out should you have any questions around the set-up and we would be happy to assist you. . Previously published on https://appfleet.com/blog/prometheus-metrics-based-autoscaling-in-kubernetes/

How To Use Prometheus Adapter to Autoscale Custom Metrics Deployments

Cómo usar el adaptador Prometheus para implementar métricas personalizadas de escalado automático

Spawning an AWS EKS cluster has never been easier and options are many: CloudFormation, Terraform or CDK. For the lazy, you can even use the great CLI utility from Weavework. eksctl But once your cluster is set up, In this post, we will setup a new cluster from scratch using CDK and take a look at the component how to assure the number of nodes is scaling properly? Cluster Autoscaler ⚙️ AWS Setup First you will need an AWS account. You can create a new account on the or reuse an existing account. Be aware that: sign-in page EKS is not included in the AWS free tier, meaning you will be billed for the managed Kubernetes control plane.You can use t3.micro instances for free with limitations Choosing the node instance type For an EKS setup, you certainly don’t want to use these free t3.micro instances: t2/t3 instances are capped in term of CPU use the default AWS CNI plugin assign one IP from the subnet for each pods, meaning the number of pods running on a node is limited by the networking capacity of the node. For example, a t3.micro instance supports up to 4 pods which is simply too limited. You can find the number of supported pod by instance type . (2 ENI * (2 IP — 1) + 2) here To keep it simple and cheap, which supports up to 17 pods by node. They are not free but quite inexpensive. we will use t3.medium instances By the way, if you want to bypass the pod networking limitation, you need to opt out the AWS CNI plugin and use another plugin, for example . Calico Create an admin user Operating infrastructure with CDK requires extensive rights. To keep it simple, create a new user and grant it the managed AdministratorAccess policy. In real life you may want to create a custom IAM policy tailored to your needs. Install CDK Follow through the for getting started with CDK. official documentation 📝 Write the CDK stack Let’s begin by creating a dedicated VPC and a basic cluster: vpc = ec2.Vpc( , ); cluster = eks.Cluster( , , {
  vpc: vpc,
  defaultCapacity: , version: eks.KubernetesVersion.V1_17,
}); const new this "eks-vpc" const new this "Cluster" 0 // we want to manage capacity ourselves There is still no compute resources allocated. Let’s add a default managed node group for this cluster using between 1 and 3 instances: ng = cluster.addNodegroup( , {
  instanceType: ec2.InstanceType( ),
  minSize: ,
  maxSize: ,
}); const "nodegroup" new "t3.medium" 1 3 Managed node groups are automatically kept up to date by AWS and are created in an Autoscaling Group. 🎬 Run the stack First you will need to bootstrap the CDK tool, creating a bucket with CloudFormation manifests, then run the stack. Take a ☕️ as it will take more than 10 minutes. cdk bootstrap
cdk deploy 🔑 Get your Kubernetes credentials You should get back some outputs with the AWS CLI command to generate you kubectl configuration: CdkStack.testclusterClusterConfigCommand1851F735 = aws eks update-kubeconfig --name testclusterCluster00507BD3-639846f8ec5241a69f54eabd38c730a0 --region us-east-1 --role-arn arn:aws:iam::xxx:role/CdkStack-testclusterClusterMastersRoleAAD0ED84-DR14A5TYS195 Note that a master role has been created for you. Run this command to generate your kubectl configuration. Then validate you can access your new cluster with a simple command: kubectl get no

NAME                           STATUS   ROLES    AGE   VERSION
ip-10-0-165-141.ec2.internal   Ready    <none>   48m   v1.17.9-eks-4c6976 We have now a working cluster with one node like expected. ⚖️ Autoscale the cluster Our nodes are part of an Autoscaling Group which allow us to easily change the number of running nodes from 1 to 3. Of course, we want to automate this scaling depending on the load. One classical way of doing this is to add some automatic scaling rules based on CPU, RAM or other custom metrics. But let’s consider a basic use case: You want to run a pod with high memory or CPU requirements Your current nodes are underutilized so there’s no memory or CPU pressure to trigger your Autoscaling Group rules You submit the pod manifest The scheduler cannot find a node large enough to accommodate the pod even if there is plenty of capacity in global You pod is left unscheduled! To solve such a use case we need a . Let’s enter the . more Kubernetes specific approach to autoscaling that can detect unscheduled pods Cluster Autoscaler The Cluster Autoscaler is capable of detecting unscheduled pods due to resource constraints and increase the node count accordingly. On the opposite, when nodes are underutilized, pods are rescheduled on other nodes and the node count is decreased. 🛠 Installing the Cluster Autoscaler Because we want the Cluster Autoscaler be a a fundamental part of our cluster, we can add it to our CDK stack. Let’s add some code. Main steps are: To create a policy allowing the Cluster Autoscaler to do his job Attach this policy to the managed node group role Tag the nodes with the properly to allow the autoscaler to auto-discover them Install the Cluster Autoscaler manifest Another option would have been to use Helm to install the autoscaler manifests but it’s simpler this way as everything is in the same place. enableAutoscaling(cluster: eks.Cluster, ng: eks.Nodegroup, version: = ) { autoscalerStmt = iam.PolicyStatement();
    autoscalerStmt.addResources( );
    autoscalerStmt.addActions( , , , , , , ); autoscalerPolicy = iam.Policy( , , {
      policyName: ,
      statements: [autoscalerStmt],
    });
    autoscalerPolicy.attachToRole(ng.role); clusterName = CfnJson( , , {
      value: cluster.clusterName,
    });
    cdk.Tag.add(ng, , , { applyToLaunchedInstances: });
    cdk.Tag.add(ng, , , { applyToLaunchedInstances: }); eks.KubernetesManifest( , , {
      cluster,
      manifest: [  {
          apiVersion: ,
          kind: , 
          ... }], });
} string "v1.17.3" const new "*" "autoscaling:DescribeAutoScalingGroups" "autoscaling:DescribeAutoScalingInstances" "autoscaling:DescribeLaunchConfigurations" "autoscaling:DescribeTags" "autoscaling:SetDesiredCapacity" "autoscaling:TerminateInstanceInAutoScalingGroup" "ec2:DescribeLaunchTemplateVersions" const new this "cluster-autoscaler-policy" "ClusterAutoscalerPolicy" const new this "clusterName" `k8s.io/cluster-autoscaler/ ` ${clusterName} "owned" true "k8s.io/cluster-autoscaler/enabled" "true" true new this "cluster-autoscaler" "v1" "ServiceAccount" // full code is available here: https://gist.github.com/esys/bb7bbeb44565f85f48b3112a8d73a092 : we use the object to wrap the clusterName as a CDK token (a value that is still not resolved) cannot be used as key in tags. By doing this, the tag operation is delayed until the clusterName is resolved. Note CfnJson Update the CDK stack again with cdk deploy and check everything is installed correctly: kubectl -n kube-system get all -l app=cluster-autoscaler
NAME                                      READY   STATUS    RESTARTS   AGE
pod/cluster-autoscaler-57dfd566f9-27j29   1/1     Running   0          70m
NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster-autoscaler   1/1     1            1           103m
NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-autoscaler-57dfd566f9   1         1         1       103m 🏋🏼‍♂️ Testing the autoscaling capability First create a Deployment manifest for the nginx image. Note that we require from Kubernetes to guarantee a comfortable amounts of CPU and RAM (see the limit/requests section) for each pod. apiVersion: apps/v1 kind: Deployment metadata: name: nginx-scale spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: service: nginx app: nginx spec: containers: - image: nginx name: nginx-scale resources: limits: cpu: 500 m memory: 512 Mi requests: cpu: 500 m memory: 512 Mi And apply it on your cluster, you should have exactly one replica of nginx running: kubectl apply -f nginx-deploy.yaml

kubectl get po -l app=nginx
NAME                           READY   STATUS    RESTARTS   AGE
nginx-scale-69644568d9-t995l   1/1     Running   0          68s Let’s scale up the number of replicas and watch what happens: kubectl scale deploy nginx-scale --replicas=5
> kubectl get po -l app=nginx -w
NAME                           READY   STATUS    RESTARTS   AGE
nginx-scale-69644568d9-ng7nj   1/1     Running   0          27s
nginx-scale-69644568d9-pv62h   0/1     Pending   0          27s
nginx-scale-69644568d9-sng2v   0/1     Pending   0          27s
nginx-scale-69644568d9-t995l   1/1     Running   0          6m42s
nginx-scale-69644568d9-xfwkf   1/1     Running   0          27s Of the 5 requested replicas, 2 stay pending. If you describe the pending pods, you should see : there is not enough CPU kubectl describe po nginx-scale-69644568d9-pv62h
...
0/1 nodes are available: 1 Insufficient cpu. If you look in the Cluster Autoscaler logs, you can observe that : scaling up to 2 instances should be in progress kubectl -n kube-system logs deployment/cluster-autoscaler | grep Scale-up
I0904 13:43:55.584100       1 scale_up.go:700] Scale-up: setting group eks-4eba2c80-0b01-9821-207d-572d47bedd1a size to 2
... Pending pods should soon be running and the nodes count is appropriately scaled to 2 🎉 kubectl get no
NAME                           STATUS   ROLES    AGE     VERSION
ip-10-0-165-141.ec2.internal   Ready    <none>   107m    v1.17.9-eks-4c6976
ip-10-0-228-170.ec2.internal   Ready    <none>   7m19s   v1.17.9-eks-4c6976 The reverse should also be true: if you scale down the number of replicas, your cluster should shrink after a few minutes. 🧹 Cleaning up Simply destroy the CDK stack by running the cdk destroy command. Note that CDK bootstrapping resources have to be cleaned by hand! Thank you for reading! 🙇‍♂️ You can find the complete CDK code example . here Also published at https://medium.com/@emmanuel.sys/spawning-an-autoscaling-eks-cluster-52977aa8b467

Stories, News & Mentions about ALGOSCALE

Algoscale’s Stories on HackerNoon