SigmaOS

sigmaos

There’s more to web developments than just blockchain, chatbots, voice search and other stuff everybody is chattering about. I had a chance to talk to Uladzimir Yankovich, founder of , a productivity extension for Chrome. And here’s what else is going on on the web right now. Manganum What are the most important trends in web technology today? A few years ago, Google started to popularize the concept of (PWAs). Simply put, PWAs are websites built with the most advanced web technology to provide their visitors with a user experience comparable to using desktop or mobile applications. Progressive Web Apps In fact, this has been the main trend in web technology in recent years—getting as close to desktop software as possible. A great example is Adobe's to make Photoshop a web application. And I have to say, it looks amazing: the once cumbersome application that you had to first download, install, and run as a desktop application now just opens up like a website in your browser's tab. project But even more interesting is another trend that has emerged through the development of PWA: browser as an operating system (OS). After all, what is an OS? This is a platform for running applications, and if applications now run in a browser, why do I need an OS? My browser can take over its functions. It's no coincidence that in 2020, ChromeOS overtook macOS in terms of market share and became the world's second-largest OS: Do browser developers have a chance then to really compete with Chrome? In my view, only a corporation comparable with Google can do that. Browser development and promotion are very expensive. It is not for nothing that browser competition is called a war. So far, Chrome is the obvious winner on this battlefield. All the attempts of smaller teams to oppose it have been failing and at best these browsers are simply present with only few users. Edge from Microsoft and Safari from Apple feel much better. No doubt they are significant players in the browser market, but I can't imagine that one of them would suddenly swap shares with Chrome. However, local players also have their own success stories. The most recent is the corporate browser , which attracted $115 million one week after its launch. Another example is Yandex.Browser, which has a in its home region. But I don’t think we can talk about it as a global Chrome’s competitor. Island 30% market share Despite the incredible complexity of the task, every year several startups try to challenge Chrome. Here are just a few examples: , , , etc. And while they all do a tremendous amount of work on their products, they lack the resources to achieve massive adoption. Unfortunately for them, users are too inert and find migrating from one browser to another just too weary. Stack Browser Sidekick SigmaOS The Browser, I am sincerely glad that all these browsers are emerging, and I wish them good luck. I am also sure they all contribute a lot to Chromium and make Chrome evolve. That said, I don't really believe they will be as successful. Why do you think Chrome is so popular? Would you recommend using Chrome in 2022? In 2008, when Chrome was launched, it was by far the best browser on the market, fast and convenient. Add to that the Google's distribution powers and over the next few years, we have the #1 browser in the world. Today, Chrome is still the most popular browser in the world. It’s safe, fast, and familiar to millions of users. But is Chrome the best browser? Well, there is no objective answer. Browsers are now rapidly transforming from simple applications for surfing the Internet to full-fledged operating systems. With these new-generation browsers, we don't need a separate tool to work with documents, edit photos and videos, or engage in video conferencing. So it turns out that comparing browsers by their page-loading speed or memory usage, as we used to do, is no longer as meaningful. What matters more is how well the browser helps you manage your everyday tasks, and whether it makes you more productive. This, of course, is very individual. And while Chrome is still the most popular browser on the market, there are dozens of other options. Some of them are definitely worth looking at. For example, if privacy is important to you, you would probably choose Firefox or DuckDuckGo. If you wish to fully customize the browser, is the right choice. If you are a Web3 fan and use cryptocurrencies, then choose . For productivity geeks, I’d recommend considering which turns your browser into a multifunctional workspace. Besides, there are a few solutions being developed right now that might be of interest, such as , , and (a dedicated browser for research.) Vivaldi Brave Sidekick The browser MightyApp Synth So is Chrome going to yield the pas any time soon? No, I don’t think so. Chrome is still the favorite browser for millions of users. I personally adore Chrome. Besides, I feel comfortable entrusting all my data to Google. And since I also use Google Workspace products for work, the browser choice for me is obvious. But at the same time, I genuinely admire some features of Vivaldi or Sidekick. But do we really have to choose between the good old Chrome and modern multifunctional browsers from young and daring teams? I don’t like this dilemma, so I decided to try to solve it by developing my own solution. This is how was born, a browser extension that completely changes the Chrome new tab, and adds a sidebar with a set of productivity applications, such as bookmarks, calendar, tasks, notes, and even a translator. Manganum How do you view further development of the industry? What ideas, applications, or technologies should we be aware of? I believe we will continue to see the migration of desktop applications to the web, with the browser increasingly replacing our OS. Apart from that, I am watching with interest the development of the cloud access industry. So far, we have seen the rapid development of cloud gaming services, such as Nvidia’s . But it is easy to imagine how these technologies will change the world of browsers. Just look at this browser. GeForce Now Mighty

Web Technology Trends in 2022: PWAs, Browsers as OS, Cloud Access

How to Accurately Measure Binomial Proportions for Reliable Conversion Metrics

Diagramming Tools: When Should You Use No Code, DSL, or Code?

Using the Stratification Method for the Experiment Analysis

Gebruik die stratifikasiemetode vir die eksperimentanalise

ለሙከራ ትንተና የስትራቴሽን ዘዴን መጠቀም

استخدام أسلوب التقسيم الطبقي لتحليل التجربة

Método de Estratificación ukampi Análisis de Experimento ukataki

Təcrübə Təhlili üçün Stratifikasiya Metodundan istifadə

Выкарыстанне метаду стратыфікацыі для эксперыментальнага аналізу

Използване на метода на стратификация за анализ на експеримента

Korištenje metode stratifikacije za analizu eksperimenta

Ús del mètode d'estratificació per a l'anàlisi de l'experiment

Použití stratifikační metody pro experimentální analýzu

Brug af stratificeringsmetoden til eksperimentanalysen

Χρήση της Μεθόδου Στρωματοποίησης για την Πειραματική Ανάλυση

Estratifikazio-metodoa erabiltzea Esperimentuaren analisirako

استفاده از روش طبقه بندی برای تجزیه و تحلیل تجربی

Stratifikaatiomenetelmän käyttäminen kokeen analyysissä

Usando o método de estratificación para a análise do experimento

שימוש בשיטת הריבוד לניתוח הניסוי

Sèvi ak Metòd Estratifikasyon pou analiz Eksperyans lan

A rétegződési módszer használata a kísérleti elemzéshez

Փորձի վերլուծության համար շերտավորման մեթոդի օգտագործումը

Menggunakan Metode Stratifikasi untuk Analisis Eksperimen

Utilizzo del metodo di stratificazione per l'analisi degli esperimenti

სტრატიფიკაციის მეთოდის გამოყენება ექსპერიმენტის ანალიზისთვის

Экспериментті талдау үшін стратификация әдісін қолдану

ដោយប្រើវិធីសាស្រ្ត stratification សម្រាប់ការវិភាគពិសោធន៍

Эксперименттик талдоо үчүн стратификация ыкмасын колдонуу

Kosalela Méthode ya Stratification pona Analyse ya Expérience

ການນໍາໃຊ້ວິທີການ Stratification ສໍາລັບການວິເຄາະການທົດລອງ

Stratifikacijos metodo naudojimas eksperimento analizei

Stratifikācijas metodes izmantošana eksperimenta analīzei

Mampiasa ny fomba fisavana ho an'ny fanadihadiana andrana

Користење на методот на стратификација за анализа на експериментот

Туршилтын шинжилгээнд давхаргажуулах аргыг ашиглах

Menggunakan Kaedah Stratifikasi untuk Analisis Eksperimen

प्रयोग विश्लेषणको लागि स्तरीकरण विधि प्रयोग गर्दै

Het gebruik van de stratificatiemethode voor de experimentele analyse

Go šomiša Mokgwa wa Stratification bakeng sa Tshekatsheko ya Teko

Wykorzystanie metody stratyfikacji do analizy eksperymentu

د تجربې تحلیل لپاره د Stratification میتود کارول

Análisis de Experimento nisqapaq Método de Estratificación nisqawan yanapachikuspa

Utilizarea metodei de stratificare pentru analiza experimentală

Gukoresha Uburyo bwa Stratification Uburyo bwo Gusesengura Ubushakashatsi

අත්හදා බැලීම් විශ්ලේෂණය සඳහා ස්තරීකරණ ක්‍රමය භාවිතා කිරීම

Použitie stratifikačnej metódy na experimentálnu analýzu

Kushandisa iyo Stratification Method yeKuedza Ongororo

Isticmaalka Habka Stratification ee Falanqaynta Tijaabada

Përdorimi i metodës së shtresimit për analizën e eksperimentit

Коришћење методе стратификације за анализу експеримента

Användning av stratifieringsmetoden för experimentanalysen

Kutumia Mbinu ya Uwekaji Utabaka kwa Uchambuzi wa Majaribio

சோதனைப் பகுப்பாய்விற்கான அடுக்குமுறை முறையைப் பயன்படுத்துதல்

Истифодаи усули стратификация барои таҳлили таҷриба

การใช้การแบ่งชั้นวิธีสำหรับการวิเคราะห์การทดลอง

ንፈተነ ትንተና ዝኸውን ስትራቲፊኬሽን ሜላ ምጥቃም

Synag derňewi üçin stratifikasiýa usulyny ulanmak

Gamit ang Paraan ng Stratification para sa Pagsusuri ng Eksperimento

Ku tirhisa Ndlela ya Stratification eka Nxopaxopo wa Swikambelo

Використання методу стратифікації для аналізу експерименту

تجرباتی تجزیہ کے لیے Stratification طریقہ استعمال کرنا

Tajriba tahlili uchun tabaqalanish usulidan foydalanish

Ukusebenzisa iNdlela yoLungiselelo loVavanyo lokuVavanya

Ukusebenzisa Indlela Yokuhlanganisa Yokuhlola Ukuhlaziya

Technology advancements like generative AI, cloud computing, and robotics have been enhancing efficiency in various industries for a while now. As a result, global operations executives are actively deploying automation tools and strategies, alongside ongoing IT modernization, to elevate process efficiency in their organizations. Intelligent automation, a key focus of different organizations’ practice for years, has significantly contributed to them . In this article, I'll share practical insights on how to improve efficiency across different business contexts through intelligent automation and other solutions. achieving operational excellence How does automation boost process efficiency? Efficiency is crucial for a resilient enterprise, and achieving it requires a shift toward automated, insight-driven processes. Intelligent automation is one of the crucial activities that streamlines operations and also lets enterprises gain a competitive edge. Intelligent process automation alone is valued at $14.4 billion this year, with the potential to reach $29.5 billion by 2030. More companies are adopting automation solutions to boost business process efficiency due to the benefits automation offers. Consistency and accuracy. Automation ensures processes run consistently and precisely, eliminating errors from fatigue or distractions, and reducing quality inconsistencies. Increased speed and output. Automated systems operate much faster than humans, especially in repetitive tasks. Improved document processing enhances task speed and accuracy. Cost savings. Despite initial investment, automation leads to substantial long-term cost reductions. It cuts labor costs, minimizes overtime, and reduces costly errors. There are direct and indirect cost savings, like speeding up document filing in financial services. Scalability. Automated processes are easily scalable based on demand fluctuations. That is particularly beneficial for industries with variable workloads like retail or finance. Data insights and analytics. Automation generates extensive data for in-depth analysis and insights. It identifies bottlenecks, inefficiencies, and areas for improvement. It enables a proactive, data-driven approach for continuous process optimization. As you see, automation enhances process efficiency through consistency, speed, cost savings, scalability, and data-driven insights. Altogether these benefits make it a powerful tool for streamlining operations, reducing errors, and improving overall productivity. How to enable process efficiency through automation Critical decisions require clear reasoning. This means, first of all, businesses should carefully monitor their process efficiency to identify bottlenecks. Below are the 6 essential steps that will help you master this process. Step 1: Form a leadership team Establishing management and innovation leadership is crucial for enhancing process efficiency as this team will be guiding the company through optimization and fostering an innovative culture. Their tasks include: Setting vision. Clearly defining efficiency improvement objectives, specifying goals like cost reduction or cycle time enhancement. Also, creating a compelling vision that will be considered a roadmap for all employees. Change management. Leaders are responsible for effectively communicating the vision and goals to all stakeholders, from top management to employees. It involves transparency about the need for change and the benefits it will bring. Overcoming resistance. Resistance to changes, even positive ones, may occur due to a lack of awareness. The leadership must communicate the changes, the impact on employees, and the general direction of the team and the business. This awareness has to come before the change, ensuring successful adoption. Risk management. Changes always bring risks, and there has to be a plan for addressing them. Leadership assesses threats and potential impacts on the enterprise, developing contingency plans to minimize disruption. Forming and empowering teams. Leaders assemble cross-functional teams for specific process improvement projects. Apart from assembling such teams, leadership empowers them to make decisions and implement changes within the scope of their projects. Performance monitoring. The leadership team defines and monitors metrics enabling continuous evaluation of progress, identifying areas that need specific attention. Continuous improvement culture. Leaders set an example of commitment to continuous improvement by actively engaging in and supporting efficiency initiatives. They maintain the right culture by encouraging and rewarding employees for suggesting and implementing process improvements. Step 2: Assess current process efficiency In the discovery phase, focus on reviewing the current state to find processes suitable for automation. Look into the execution of existing processes Engage in discussions with stakeholders Analyze metrics the company already tracks Document the current process flow The exploratory step, known as process mining, helps define Process mining combines data science and process analytics to uncover, validate, and enhance workflows. It uses data from information systems like ERP or CRM tools to create a process model, providing a detailed view of processes and variations. the current process efficiency situation. Specialized algorithms analyze deviations from the standard, help management assess process performance, and identify opportunities for robotic process automation. This approach accelerates automation initiatives within the company. Step 3: Identify and address deficient processes Once you understand current processes, focus on identifying the most deficient ones. What's not working well? What's affecting the team's goals? Optimizing every process simultaneously is impossible, so Focus on processes that are slower or hinder critical aspects of the enterprise. For example, delays in procurement can impact resource availability and lead to significant consequences like supply chain breakdowns. prioritize those needing immediate improvements. Techniques like Lean and Six Sigma (or their combination, Lean Six Sigma) are effective in improving efficiency and enhancing process outcomes. Lean and Six Sigma both accelerate cost and process efficiency, but differ in approach: Lean targets waste elimination, while Six Sigma focuses on reducing variation. Lean uses less technical tools like workplace organization and continuous improvement. In contrast, Six Sigma relies on statistical data analysis, design of experiments, and hypothesis testing. Implementation often starts with Lean, optimizing workplace efficiency and reducing waste. If issues persist, Six Sigma's more advanced statistical tools come in handy. Step 4: Come up with an implementation schedule Introducing improvements in a company can be challenging amidst daily routines and employee resistance to established practices. An implementation schedule provides for change, ensures successful automation execution, and helps to manage expectations. a structured roadmap Following this schedule minimizes disruptions, encourages a smoother transition, and increases the probability of automation success. It establishes a systematic change management approach and lays the groundwork for more effective operations. Step 5: Train your teams Training is crucial for project success. The team managing new processes and tools needs to be well-versed in their operation. Technical training should and it's important to share change goals, encouraging team participation and commitment. cover system use, Motivate the team by showing how the new solution improves daily work. Engage them in the process to ensure active participation. Otherwise, the project might lack support from those directly involved in daily work. Step 6: Evaluate and effectuate modifications Implementing new processes isn't the end of your efforts. It's important to constantly oversee them, assess impact, and make adjustments as needed. Modifying processes is a natural part of improvement, and monitoring KPIs ensures effectiveness. Use KPIs before, during, and after automation to have a retrospective view of changes. Integrating to regular reviews of data allows for identifying areas needing adjustments. a systematic approach Swift action is crucial once adjustments are identified, as delays can lead to prolonged disruptions in realizing positive outcomes. Agility is key in this phase, allowing the organization to adapt and respond quickly to changing circumstances, minimizing downtime, and keeping progress toward goals. To sum up While the potential costs, time commitment, and planning may initially give organizations pause in embracing automation for process efficiency, the implementation of the practices discussed in this article is going to bring only positive outcomes. These strategies not only offer short-term benefits but also , ultimately improving overall process efficiency across the enterprise. contribute to long-term success In dynamic industries, where adaptability is crucial, proficiency and innovation become key drivers for continuous process enhancement. Embracing these practices not only positions organizations for success, but also ensures a sustained commitment to efficiency and innovation.

6 Ways to Optimizing Enterprise Processes

This is a practical guide for using Apache Cassandra as a real-time feature store. We explore real-time AI and the unique performance and cost attributes of Cassandra that make it an excellent database for a feature store and then dive into the basics of feature stores and their role in real-time applications. Cassandra is used as a feature store by large companies, including and Netflix; under real-world conditions, it can serve features for real-time inference with a tp99 < 23ms. Uber The guide is divided into several key sections. We start by introducing Cassandra and its features that make it an ideal choice for a feature store. Then, we explain the basics of feature stores, including what they are and how they can be used in real-time applications. After that, we explore the implementation details of creating a feature store using Cassandra. This includes data modeling, ingestion and retrieval of features, and handling data updates. Finally, we offer up best practices and tips for using Cassandra as a feature store to ensure optimal performance and scalability - from latency requirements to estimated performance metric requirements to reference architectures and ecosystem compatibility. This guide doesn’t discuss the or the in a . The best practices we’ll cover are based on technical conversations with practitioners at large technology firms such as Google, Facebook, , , and on how they deliver real-time AI experiences to their customers on their cloud-native infrastructures. Although we’ll specifically focus on how to implement real-time feature storage with Cassandra, the architecture guidelines really apply to any database technology, including Redis, MongoDB, and Postgres. data science aspects of real-time machine learning lifecycle management aspects of features feature store Uber AirBnB Netflix What is real-time AI? Real-time AI . Traditionally, training models and inferences (predictions) based on models have been done in batch – typically overnight or periodically through the day. Today, modern machine learning systems perform inferences of the most recent data in order to provide the most accurate prediction possible. A small set of companies like TikTok and Google has pushed the real-time paradigm further by including on-the-fly training of models as new data comes in. makes inferences or training models based on recent events Because of these changes in inference and changes that will likely happen to model training, the persistence of feature data – data that is used to train and perform inferences for a ML model – needs to also adapt. When you’re done reading this guide, you’ll have a clearer picture of how Cassandra and DataStax Astra DB, a managed service built on Cassandra, meets real-time AI needs, and how they can be used in conjunction with other database technologies for model inference and training. What’s a feature store? A feature store is a data system specific to machine learning (ML) that: Runs data pipelines that transform raw data into feature values Stores and manages the feature data itself, and Serves feature data consistently for training and inference purposes Real-time AI places specific demands on a feature store that Cassandra is uniquely qualified to fulfill, specifically when it comes to the and of features for model serving and model Training. storage serving Best Practices **Implement low-latency queries for feature serving For real-time inference, features need to be returned to applications with low latency at scale. Typical models involve ~200 features spread across ~10 entities. Real-time inferences require for collecting features, lightweight data transformations, and performing an inference. According to the (also confirmed by our conversations with practitioners), feature stores need to return the features to an application performing inference in under 50ms. time to be budgeted following survey Typically, models require “inner joins” across multiple logical entities – combining rows values from multiple tables that share a common value; this presents a significant challenge to low-latency feature serving. Take the case of Uber Eats, which predicts the time to deliver a meal. Data needs to be joined from order information, which is joined with restaurant information, which is further joined by traffic information in the region of the restaurant. In this case, two inner joins are necessary (see the illustration below). To achieve an inner join in Cassandra, one can either denormalize the data upon insertion or make two sequential queries to Cassandra + perform the join on the client side. Although it’s possible to perform all inner joins upon inserting data into the database through denormalization, having a 1:1 ratio between model and table is impractical because it means maintaining an inordinate number of denormalized tables. Best practices suggest that the feature store needs to allow for 1-2 sequential queries for inner joins, combined with denormalization. Here’s a summary of the performance metrics that can be used to estimate requirements for real-time ML pipelines: Testing Conditions: features = 200 number of tables (entities) = 3 number of inner joins = 2 Query TPS : 5000 queries / second Write TPS : 500 records / second Cluster Size : 3 nodes on AstraDB* Latency Performance summary (uncertainties here are standard deviations): tp95 = 13.2(+/-0.6) ms tp99 = 23.0(+/-3.5) ms tp99.9 = 63(+/- 5) ms Effect of compaction: tp95 = negligible tp99, tp999 = negligible, captured by the sigmas quoted above Effect of Change Data Capture (CDC): tp50, tp95 ~ 3-5 ms tp99 ~ 3 ms tp999 ~ negligible *The following tests were done on DataStax’s , which is a serverless environment for Cassandra. Users should expect similar latency performance when deployed on three notes using the . Astra DB’s free tier following recommended settings The most significant impact on latency is the number of inner joins. If only one table is queried instead of three, the tp99 drops by 58%; for two tables, it is 29% less. The tp95 drops by 56% and 21%, respectively. Because Cassandra is horizontally scalable, querying for more features does not significantly increase the average latency, either. Lastly, if the latency requirements cannot be met out of the box, Cassandra has two additional features: the ability to support denormalized data (and thus reduce inner joins) due to high write-throughput capabilities and the ability to selectively replicate data to in-memory caches (e.g. Redis) through Change Data Capture. You can find more tips to reduce latency here. Implement fault tolerant and low latency writes for feature transformations A key component of real-time AI is the ability to use the most recent data for doing a model inference, so it is important that new data is available for inference as soon as possible. At the same time, for enterprise use cases, it is important that the writes are durable because data loss can cause significant production challenges. *Object stores (e.g. S3 or HIVE) can be replaced with other types of batch-oriented systems, such as data warehouses. There is a trade-off between low-latency durable writes and low-latency feature serving. For example, it is possible to only store the data in a non-durable location (e.g., Redis), but production failures can make it difficult to recover the most up-to-date features because it would require a large recomputation from raw events. A common architecture suggests writing features to an offline store (e.g., Hive / S3) and replication of the features to an online store (e.g., in-memory cache). Even though this provides durability and low latency for feature serving, it comes at the cost of introducing latency for feature writes, which invariably causes poorer prediction performance. Cassandra provides a good trade-off between low-latency feature serving and low-latency “durable” feature writes. Data written to Cassandra has typically been replicated a minimum of three times, and it supports multi-region replication. The latency from writing to availability to read is typically sub-millisecond. As a result, by persisting features directly to the online store (Cassandra) and bypassing the offline store, the application has faster access to recent data to make more accurate predictions. At the same time, CDC, from the online store to the offline store, allows for batch training or data exploration with existing tools. Implement low latency and writes for prediction caching and performance monitoring In addition to storing feature transformation, there is also the need to store predictions and other tracking data for performance monitoring. There are several use cases for storing predictions: – In this scenario, a database used to cache predictions made by either a . The streaming architecture is particularly useful when the time it takes to inference is beyond what is acceptable in a request-response system. Prediction store batch system or a streaming system It is often necessary to monitor the prediction output of a real-time inference and compare to the final results. This means having a database to log the results of the prediction and the final result. Prediction performance monitoring Cassandra is a suitable store for both use cases because of its high-write throughput capabilities. Plan for elastic read and write workloads The level of query and write transactions per second usually depends on the number of users simultaneously using the system. As a result, workloads may change based on the time of day or time of year. Having the ability to quickly scale up and scale down the cluster to support increased workloads is important. Cassandra and Astra DB have features that enable dynamic cluster scaling. The second aspect that could affect write workloads is if there are changes in the feature transformation logic. With a large spike in write workloads, Cassandra automatically prioritizes maintaining low-latency queries and writing TPS over data consistency, which is typically acceptable for performing real-time inference. Implement low-latency, multi-region support As real-time AI becomes ubiquitous across all apps, it’s important to make sure that feature data is available as close as possible to where inference occurs. This means having the feature store in the same region as the application doing inference. Replicating the data in the feature store across regions helps ensure that feature. Furthermore, replicating just the features rather than the raw data used to generate the features significantly cuts down on cloud egress fees. Astra DB supports multi-region replication out of the box, with a replication latency in milliseconds. Our recommendation is to stream all the raw event data to a single region, perform the feature generation, and store and replicate the features to all other regions. Although theoretically, one can achieve some latency advantage by generating features in each region, event data often needs to be joined with raw event data from other regions;. from a correctness and efficiency standpoint, it is easier to ship all events to one region for processing for most use-cases. On the other hand, if the model usage makes the most sense in a regional context, and most events are associated with region-specific entities, then it makes sense to treat features as region specific. Any events that do need to be replicated across regions can be placed in keyspaces with global replication strategies, but ideally, this should be a small subset of events. At a certain point, replicating event tables globally will be less efficient than simply shipping all events to a single region for feature computations. Plan for cost-effective and low-latency multi-cloud support Multi-cloud support increases the resilience of applications and allows customers to negotiate lower prices. Single-cloud online stores such as DynamoDB result in both increased latency for retrieving features and significant data egress costs but also creates lock-in to a single cloud vendor. Open-source databases that support replication across clouds provide the best balance of performance cost. To minimize the cost of egress, events and feature generation should be consolidated into one cloud, and feature data should be replicated to open-source databases across the other clouds. This minimizes egress costs. Plan for both batch and real-time training of production models Batch processing infrastructure for building models is used for two use cases: building and testing new models, and building models for production. Therefore it was typically sufficient for feature data to be stored in slower object stores for the purpose of training. However, newer model training paradigms include updating the models in real-time or near real-time (real-time training); this is known as “online learning” (e.g. ). The access pattern for real-time training sits somewhere between inference and traditional batch training. The throughput data requirements are higher than inference (because it is not usually accessing a single-row lookup), but not as high as batch processing that would involve full table scans. TikTok’s Monolith Cassandra can support a TPS rating in the hundreds of thousands per second (with an appropriate data model), which can provide enough throughput for most real-time training use cases. However, in the case the user wants to keep real-time training from an object store, Cassandra achieves this through CDC to object storage. For batch training, CDC should replicate data to object storage. It’s worth noting that machine learning frameworks like and PyTorch are particularly optimized for parallel training of ML models from object storage. Tensorflow For a more detailed explanation of “online learning”, see Chip Huyuen’s explanation on , or this . Continual Learning technical paper from Gomes et. al Support for Kappa architecture is gradually replacing Lambda architecture due to costs and data quality issues due to online/offline skew. Although lots of articles discuss the advantages of moving from separate batch and real-time computation layers to a single real-time layer, articles don’t often describe how to architect the serving layer. Kappa architecture Using Kappa architecture for generating features brings up some new considerations: Updating features are being updated en masse and can result in a significant number of writes to the database. It’s important to ensure that query latency does not suffer during these large updates. The serving layer still needs to support different types of queries, including low-latency queries for inference, and highTPS queries for batch training of models. Cassandra supports Kappa architecture in the following ways: Cassandra is designed for writes; an increased influx of writes does not significantly reduce the latency of queries. Cassandra opts for processing the writes with eventual consistency instead of strong consistency, which is typically acceptable for making predictions. Using CDC, data can be replicated to object storage for training and in-memory storage for inference. CDC has little impact on the latency of queries to Cassandra. Support for Lambda architecture Most companies have a Lambda architecture, with a batch layer pipeline that’s separate from the real-time pipeline. There are several categories of features in this scenario: Features that are only computed in real-time, and replicated to the batch feature store for training Features that are only computed in batch, and are replicated to the real-time feature store Features are computed in real-time first, then recomputed in the batch. Discrepancies are then updated in both the real-time and object store. In this scenario, however, DataStax recommends the architecture as described in this illustration: The reasons are the following: Cassandra is designed to take batch uploads of data with little impact on read latency By having a single system of record, data management becomes significantly easier than if the data is split between the feature store and object store. This is especially important for features that are first computed realtime, then recomputed in batch. When exporting data from Cassandra via CDC to the object feature store, the data export can be optimized for batch training (a ), which significantly cuts down on training infrastructure costs. common pattern used at companies like Facebook If it is not possible to update existing pipelines, or there are specific reasons that the features need to be in the object store first, our recommendation is to go with a two-way CDC path between the Cassandra feature store and the object store, as illustrated below. Ensure compatibility with existing ML software ecosystem To use Cassandra as a feature store, it should be integrated with two portions of the ecosystem: machine learning libraries that perform inference and training, and data processing libraries that perform feature transformation. The two most popular frameworks for machine learning are TensorFlow and PyTorch. Cassandra has Python drivers that enable easy retrieval of features from the Cassandra database; in other words, multiple features can be fetched in parallel ( ). The two most popular frameworks for performing feature transformation are Flink and . Connectors for both and are available for Cassandra. Practitioners can use tutorials for and , and Cassandra. see this example code Spark Structured Streaming Flink Spark Flink Spark Structured Streaming Open Source feature stores such as FEAST also have a and for Cassandra as well. connector tutorial Understand query patterns and throughput to determine costs The number of read queries for Cassandra as a feature store is dependent on the number of incoming inference requests. Assuming the feature data is split across multiple tables, or if the data can be loaded in parallel, this should give an estimate of the fanout between real-time inference that can be made. For example, 200 features across 10 entities in 10 separate tables give you about a 1:10 ratio between real-time inference and queries to Cassandra. Calculating the number of inferences being performed will depend on the inference traffic pattern. For example, in the case of an inference will be performed whenever a relevant feature changes, so the total number of inferences is dependent on how often the feature data changes. When inference is performed in a “request-reply” setting, it is only being performed when a user requests it. “streaming inference,” Understand batch and real-time write patterns to determine costs The write throughput is primarily dominated by how frequently the features change. If denormalization occurs, this too could impact the number of features that are written. Other write throughput considerations include caching inferences for either batch or streaming inference scenarios. Conclusion When designing a real-time ML pipeline, special attention needs to be paid to the performance and scalability of the feature store. The requirements are particularly well satisfied by NoSQL databases such as Cassandra. Stand up your own feature store with Cassandra or and try out the with the . AstraDB Feast.dev Cassandra connector

A Guide to Using Apache Cassandra as a Real-time Feature Store

Una guía para usar Apache Cassandra como una tienda de funciones en tiempo real

Guide d'utilisation d'Apache Cassandra en tant que magasin de fonctionnalités en temps réel

रीयल-टाइम फ़ीचर स्टोर के रूप में अपाचे कैसेंड्रा का उपयोग करने के लिए एक गाइड

Apache Cassandra をリアルタイム機能ストアとして使用するためのガイド

Um guia para usar o Apache Cassandra como uma loja de recursos em tempo real

Hướng dẫn sử dụng Apache Cassandra làm kho lưu trữ tính năng thời gian thực

使用 Apache Cassandra 作为实时特征存储的指南

THE STARS OF WINTER

“This holiday season is all one more reason to cry…” - (the best-ever Christmas song) Table of Contents How holidays affect people and businesses. Is it possible to manage a high load and minimize its impact? How to get accurate predictions for high load, and do we need the exact numbers? What size of margins is sufficient? Anomalies: how to predict the unpredictable? Example: assemble the anomaly criteria and alerts system Conclusion How Holidays Affect People and Businesses Most people got stuck in traffic at least once in their life trying to fetch last Christmas gifts or were unable to make phone calls to friends on New Year's Eve due to a high load on the mobile network. The holiday season (a time of joy and celebration) usually brings an increased load on various aspects of life. This is especially evident in areas such as travel, postal services, and banking, as there is a higher demand for these services due to the increased number of people traveling and sending gifts. The high load can lead to delays, congestion, and other issues that can be a source of stress for individuals and businesses. Companies have to work harder to meet the increased demand for their products or services. Failure to tolerate the higher pressure can lead to customer disappointment and negatively impact a company's image. This can be particularly damaging for businesses that rely directly on positive customer experiences to drive sales and attract new customers, but it also affects B2B products. When a supplier may need to intensify product manufacturing and distribution, a service provider has to deal with an increased number of operations. Is It Possible to Manage a High Load and Minimize Its Impact? If it wasn't possible, I wouldn't be writing this article. For many years, businesses of all sizes - from small bakeries to large manufacturing companies – have been studying how demand changes over time (trends) and periodically (seasonality). For example, a bakery might produce more heart-shaped cakes around Valentine's Day. To prepare for a higher volume of production, businesses introduce scaling of all kinds: increased staffing levels (e.g., hiring seasonal workers), paid overtime, goods production beforehand, and moving critical software services to clouds that support auto-scaling out of the box. But to activate scaling, businesses need predictions, etc. How to Get Accurate Predictions for High Load, and Do We Need the Exact Numbers? Specialists analyze worldwide trends, perform social studies, and rely on historical. However, there is no known method that will give a perfect forecast. The real world is way too complicated, and it is impossible to take all the factors into account and include them in any sort of model. While manual processing of data is time-consuming and not very accurate, modern businesses tend to rely on machine learning (ML) methods to approach the task. They need to identify the target value (or several values) to predict: demand for products or services, traffic on a website or social media platform, or the number of customer service calls. Then they work with the time-series forecasting problem. I won't describe the approaches to time-series forecasting in this article (all the standard techniques are applicable), but I will highlight what is special about them when we are interested in high load as a narrower class of tasks. The main thing is that, in many cases, we don't need accurate numbers. The insight would be more of "if this day/week we'll see a high demand or not" rather than "what precise number of customers we expect" – rather flagging the fact of high load and its degree than producing the precise number. For example, if a company needs to prepare for an open-air festival in a small town that is expected to attract a large number of visitors, it's important to know if the demand for hotel rooms will be high and plan accordingly. However, it's hard to predict the exact number of visitors as many make last-minute decisions that are affected by factors such as the weather. When it comes to software applications, the load is often mapped to a number of servers (nodes) with large margins included, so that resources are not fully utilized at all times. This allows for more efficient use of resources, and when a new machine is needed, it is a granular decision. Whether it's a need for 0.05 or 0.7 machines, it still means allocating an additional node. The key principle of machine learning remains: use the simplest model that produces sufficient results for a particular task. What Size of Margins Is Sufficient? While businesses can implement backup plans to cover potential increases in demand, it also increases costs. If resources are not used, it also affects revenue. That's why it's important to get the forecast as accurate as possible and include margins that balance between being overly optimistic and overly pessimistic. For example, imagine a small newsstand that normally sells around 100 papers a day. A trivial prediction would be 100. But the owner knows that based on news and other sporadic factors, he can sometimes sell more items, and he doesn't want to lose his customers. So, in an optimistic approach (if the forecast is precise enough), he adds 5%, and in a pessimistic approach, he adds 10%. When looking at the calculation, we can see that the optimistic approach generates more revenue  – fewer papers are recycled, but there are 5 unhappy customers in total (on the 6th and on the 10th day). In the case of papers, it makes sense to maximize revenue – possibly the customers were late in the evening and didn't expect to buy a paper, so they weren't as upset. But if we're talking about a bank or a hospital instead of a newsstand, the damage from the inability to handle a higher number of transactions or patients is crucial, and management would likely accept higher costs. It is important to consider the nature of the business, the cost of overproduction, and the cost of not meeting the demand while defining the margin. The margin is a balance, and it needs to be tuned to the particular case. Anomalies: How to Predict the Unpredictable? Let's assume that we've built a model that produces fair predictions for most of the time, including general trends, seasonal, weekly, and daily changes, and the business has developed measures to deal with high load and even has contingency plans in place to deal with anomalies. But how do we detect such anomalies? If a business is able to notice it early enough, the company can minimize the negative impact of high load and maintain a high level of service for their customers. To achieve this, we can compare the predicted data with real-time data coming from the field. If there is a significant discrepancy, an alert can be triggered, and proactive measures can be activated, either automatically or through human intervention. Example: Assemble the Anomaly Criteria and Alerts System I once worked for a B2B company* that processed millions of requests per day, with one stage of the processing performed as a nightly job, which typically finished before users became active again in the morning. However, one night it failed, processing only around 60% of requests which resulted in serious issues for the morning and additional costs to mitigate the impact. Fortunately, an ad-hoc solution was quickly implemented, preventing any damage to the company's reputation. But the lesson was learned. The company wanted to investigate the situation and ensure that it had mechanisms in place to activate alerts as soon as a mild anomaly was detected, in order to mitigate risks in the future. To do this, we assembled a dataset by aggregating transaction numbers for the last 10 years since the product was launched in production. By performing Exploratory Data Analysis (EDA) on the data, we noticed weekly patterns, a trend, and a clear outlier on April 17th, 2022. The daily volume was than the maximal historical number. Such a deviation would have a big impact on the model's accuracy; therefore, it is essential to determine if it can be explained naturally and is possible to predict. ~2.5 times higher 65% of requests that day had a null amount of money associated. When we dug deeper, we found that the difference was caused by a single business user who had mistakenly requested past data for every request. So, the conclusion was made: this was a true anomaly caused by an unpredictable external factor – the perfect candidate for an early anomaly detection challenge. As the company is interested in knowing the total number of requests for a day to be handled tonight and knowing if the actual numbers are much higher than predicted, I suggested splitting the day into hourly buckets and comparing the predicted requests for each bucket. By doing this, we transformed the problem into multi-target regression and predicted a number for each hour of the day. This allowed us to compare numbers throughout the day and spot any mismatches by the end of the day with granularity. To check the hypothesis, we split the day into four parts: When we built the model, the standard deviation (std) of relative error did not exceed 3.4% for day parts and 5% for the entire day. The error distribution was close to normal, so the 3-sigma rule was natural to be applied to formulate the anomaly criteria: if the difference between predicted and actual counts is more than 3 standard deviations, an alert is triggered. When we tested this criterion for April 17th, we noticed that the alarm would have been triggered as early as 6 AM, when the requests for the first buckets were calculated. Monitoring systems can maintain events of different severity levels, such as a warning for a mismatch breaking 2 sigmas, and a critical alert for breaking 3 standard deviations. Since the system operates on a daily basis and buckets allow for scaling, this adds a layer of early detection and improves the company's ability to respond to unexpected events in real-time. The suggested criteria are just an example and shall be adjected for particular business scenarios based on the nature of the data. However, the described approach can also be applied to low load detection by looking at deviations in a negative direction. For example, in retail, many offline shops are closed on holidays, and delivery from online shops may not work on New Year's Eve, which can affect warehouse loads and the number of people needed to work those days. Conclusion In conclusion, while it is challenging to predict high load with 100% accuracy, businesses can use data and machine learning techniques to build models that can anticipate and prepare for high load periods, set up early detection of anomalies, and by these means, minimize the negative impact and maintain a high quality of service for their customers. *The name of the company is left out for privacy, and the real data was altered

Holiday Season: Predicting and Tolerating High Load in Business -  Dealing With Unexpected Anomalies

How to choose the right javascript charting library? Top javascript charting libraries Conclusion In today’s world, people are struggling more and more with the problem of scattered attention. So it is highly important to present information in a structured and well-designed way; especially if you have a . Presenting huge chunks of data in a standard spreadsheet to analyze or to study is as inconvenient as it can get. Creating different is a big issue that we will consider. complex business application charts Javascript charts are an essential part of a web application for presenting data. It means JavaScript charting libraries are inevitable. The way the human brain itself is programmed is that it understands visual data much better than anything else. Well visualized data creates much more influence than the data being presented as it is, no matter how well it is explained. Companies build strategies around the charts, numbers, and tables they’re presented with, and often use them to choose whether to pursue business opportunities. It is very important to choose the right tool to build a workaround. But how do you choose the tool that will fit your requirements in the javascript technology zoo? How to choose the right graphics for your data? Should it be paid or free? Build your own solution or take a ready-made tool? We will try to cover all these questions in this . First, we will talk about how to choose the right javascript charting library. Then we will define the criteria for evaluating the tools, and finally, compare the most popular javascript charts on the market. guide How to choose the right javascript charting library? When you approach the choice of the chart js library, everything is highly individual and depends on many criteria. Therefore, in this section, we will consider things you should pay attention to while choosing the tools for drawing charts. There are quite a few javascript charting libraries around now, but which are the best to use? This can depend on many factors like business needs, types of data, the purpose of the chart itself and many more. A nice interface and copy/paste code are all well and good if we need something generic to our needs. But frankly speaking, most of the time we actually need something tailored to our use case. Even more than that  –  we need a reliable charting library we could build upon. There are several factors to consider here: What kind of charts does the company want to build? Pie charts, maps, lines, bars? How large is the dataset? Is the app going to be used for Web, mobile, or both? SVG or Canvas base? Libraries based on SVG are usually better for smaller to medium datasets, as each element is a unique node and exists in the DOM tree. On the other side, Canvas is really fast. What is the browser support for a given library? Check your browser market share to figure this out. Which JavaScript framework do you use? What kind of customization of the look and feel do you need? Note that in some cases you may not need a data visualization library at all. Sometimes it’s better just to write one from scratch using vanilla JavaScript. Here is an example of a decision diagram that I have found on the . internet Top JavaScript chart libraries In this article, each JavaScript charting library will be compared with some key factors including chart types, commercial or free, and open-source status. These beautiful libraries have been analyzed thoroughly with hands-on experience to maximize the very best comparison. Highcharts Free for non-commercial, paid for commercial Free to $7060 No dependencies License: Price: Main dependencies: Web-site: https://www.highcharts.com/ HighCharts a modern javascript charting library based on SVG technology. It doesn’t require any plugin. The integration with all of the major web is very simple. frameworks In all of its simplicity, Highcharts is also very much compatible with old browsers, so you can pick it if you don’t need to represent data using advanced charting styles. Notable Features: Optimized for both responsive design and touch devices; Capable of working with Big Data; On-hover tooltips rendering is super-quick; Ability to annotate graphs; Data can be loaded to charts directly from a CSV file. Highcharts allows you to configure the theme separately from the data. This allows you to have a common theme for your brand to apply across all the charts on your website. Highcharts is extensively documented and has most use cases covered. HighCharts is used by some major companies across the world, including Facebook, IBM, MasterCard, and StackOverflow. It’s probably the most advanced library out there regarding available, but of course, comes at a cost for commercial use. If the pricing is not a stumble, Highcharts is an excellent choice. types of charts Chartist-js Open-source Free No dependencies License: Price: Main dependencies: Web-site: http://gionkunz.github.io/chartist-js Chartist is a very modern, SVG-based library. Its biggest feature is the SVG animations in the charts produced with this library. It has a solid technology base and is very easy to implement. Within minutes you can make an incredibly impressive chart that interacts easily with any backend data source. Chartist is really easy to configure, as well as easy to customize with Sass. This library has only 8 base chart types which can be used to improvise over a few more different types. Each is fully responsive but doesn’t have a great transitional effect as others. Features: Filtering by labels; Click on a legend to show and hide data on the chart; Non-numeric Y-Axis, have labels instead; Easy customization with interpolation of line charts. The Chart.js visualization library is completely open-sourced with the MIT License and available to modify, distribute, and use. Source files are available to ‘fork’ on GitHub too. Chart.js offers a vast documentation base including precise instructions on installing the library. The library can be quickly installed with Bower, NPM, jsDelivr, and can even be linked up from CDNJS. Besides, you can download the source files directly from the GitHub Repo. Chartist is a very powerful charting library, but it requires more work on the developer’s end to get things to look right. C3.js Open-source Free D3.js License: Price: Main dependencies: Web-site: https://c3js.org/ C3 is a very efficient D3 based chart visualization library. C3 library is fast to render, has good compatibility across browsers, and is very simple to integrate. If you’re looking for no-frills, C3 is a decent choice. It also includes good documentation for what is an inherently simple library. Features: Extensive tutorials and documentation; Responsive and mobile-ready; Stylish tooltips already integrated; Filterable data series. C3 provides a getting started guide that instructs on how to get the basic library setup with your project. Chartjs Open-source Free Moment.js License: Price: Main dependencies: Web-site: https://www.chartjs.org/ The Chart js is an HTML5 based JavaScript library for creating animated, interactive, and customizable charts and graphs. Chart.js is a much lighter product than HighCharts and doesn’t offer quite as much choice. The Chart.js is fairly simple and well-documented. Chart.js uses canvas instead of SVG. The library is actively maintained and has a few plugins to extend its functionality. API Chart.js offers 8 different chart types for data visualization with out-of-the-box animations. It is compatible with all modern browsers. Also, the responsive chart behavior of the charts can be enabled by some configuration. Plotly Open-source Free, paid for enterprise D3.js, Stack.gl License: Price: Main dependencies: Web-site: https://plot.ly/ Plotly is one of the most common libraries around. Plotly is a very rich library and has outstanding documentation, including a tutorial for each of the chart types. It has been open-source since 2015, meaning anyone can use it for free. Plotly.js supports 20 chart types, including SVG maps, 3D charts, and statistical graphs. It’s built on top of D3.js and stack.gl. The charts and graph types available have a professional look and feel. Creating a chart is just a matter of loading in your information and customizing the layout, axes, notes, and legend. NVD3 Open-source Free, paid for enterprise D3.js License: Price: Main dependencies: Web-site: http://nvd3.org/ NVD3 is also on the list of the most popular libraries. Built upon D3.js like the others above, it does have a solid technical base. The performance is relatively good, and it does have basic animations to inject some visual stimulation in an otherwise fairly plain interface. Data can be pumped directly from .json files, meaning NVD3 is very easy to integrate with existing data API solutions. When compared to other libraries on this list, it looks rather small with many charts not available, but most of the general graph-types are present. This visualization library is completely open-sourced with the Apache 2.0 License. Fusion Charts Paid From $497 No dependencies License: Price: Main dependencies: Web-site: https://www.fusioncharts.com/ FusionCharts probably has the most complete collection of charts and maps. With over 90+ chart types and 965 maps, you’ll find everything that you need right out of the box. FusionCharts supports both JSON and XML data formats, and you can export charts in PNG, JPEG, SVG, or PDF. They have a nice collection of business dashboards and live demos for inspiration. Their charts and maps work across all devices and platforms, are highly customizable, and have beautiful interactions. But with all of that, FusionCharts is slightly expensive. DyGraphs Open-source Free for all users – License: Price: Main dependencies: Web-site: http://dygraphs.com/ DyGraphs is a fast, flexible open-source JavaScript charting library. It is highly customizable, works in all major browsers (including IE8), and has an active community. Features: Linear regression; Synchronization across multiple graphs; Zoom capability; Highlighted regions. DyGraphs is well suited for large and complex data sets. D3.js Open-source Free for all users – License: Price: Main dependencies: Web-site: https://d3js.org/ D3 is an open-source JavaScript library released under the BSD license. It provides a tremendous amount of charts, graphs, and other methods for data visualization. D3 gives you almost everything that you need to visually represent your data of any kind. The website provides comprehensive documentation. There are examples provided to help with getting started and using the library. D3 supports all modern browsers. It has been tested on browsers including Firefox, Google Chrome, Safari, Opera, IE9+, Android, and iOS. There are two major concerns with D3.js: it has a steep learning curve and it is compatible only with modern browsers (IE 9+). Pick it up only if you have enough time to learn and adopt it. Sigma charts Open-source Free for all users No dependencies License: Price: Main dependencies: Web-site: http://sigmajs.org Sigma.js is built on Canvas and WebGL and has a public API, embracing a wide range of plugins contributed by the GitHub community. Sigma is fully responsive and touch interactive. It allows developers to directly add their own functions to the scripts and render nodes and edges exactly to spec. Sigma provides a lot of different settings to make it easy to customize drawing and interaction with networks. Sigma is a rendering engine, and it’s up to you to add all the interactivity you want. The public API makes it possible to modify the data, move the camera, refresh the rendering, listen to events, etc. It suits best for developers who need a powerful, dedicated graph drawing tool. Morris charts Open-source Free for all users jQuery License: Price: Main dependencies: Web-site: http://morrisjs.github.io/morris.js/ Morris.js charting library is quite popular as well. It is used in many – both free and premium. The charts used in Morris focus on simplicity and effectiveness. admin templates There are 4 types of charts in the library – line, area, bar, and donut charts. Morris charts provide a free license. The license details are provided on the website. There is also detailed documentation for the charts. Cytoscape Open-source Free for all users No dependencies License: Price: Main dependencies: Web-site: http://js.cytoscape.org/ Cytoscape.js is an open-source graph theory library written in JS. You can use Cytoscape.js for graph analysis and visualization. Cytoscape.js allows you to easily display and manipulate rich, interactive graphs. Cytoscape.js includes all the gestures out-of-the-box, including pinch-to-zoom, box selection, panning, et cetera. Cytoscape.js also has a graph analysis. The library contains many useful functions in graph theory. You can use Cytoscape.js headlessly on Node.js to do graph analysis in the terminal or on a web server. Rickshaw by Shutterstock Open-source Free for all users D3.js, jQuery, jsdom License: Price: Main dependencies: Web-site: https://tech.shutterstock.com/rickshaw/ Rickshaw is a JavaScript toolkit for creating interactive time-series graphs. Rickshaw provides the elements you need to create interactive graphs: renderers, legends, hovers, range selectors, etc. It’s based on d3 underneath, so graphs are drawn with standard SVG and styled with CSS. Customize everything you like with the techniques you already know. Rickshaw is free and open-source, available under the MIT license. Developed by Shutterstock company. CanvasJS charts Paid. Free licenses to students and for Non-Commercial use. from $149 to $4999 – License: Price: Main dependencies: Web-site: https://canvasjs.com/ CanvasJS is a responsive HTML5 charting library with high performance and a simple API. It supports 30 different chart types (including line, column, bar, area, spline, pie, doughnut, stacked charts, etc.), which are all well documented. All charts include interactive features like tooltips, zooming, panning, animation, etc. CanvasJS can be integrated with (Angular, , and jQuery) and server-side technologies (PHP, Ruby, Python, ASP.Net, Node.JS, Java). popular frameworks React Koolchart Paid for all users from $350 to $1280 – License: Price: Main dependencies: Web-site: https://www.koolchart.com/ KoolChart is an HTML5 canvas-based JavaScript charting library. The visuals are clean and modern. The use of canvas offers better performance at the expense of being raster-based. The API is well documented with example charts for each type. A two month trial period is available for evaluation. Licensing is required after the trial period expires. Rawgraphs Apache 2.0 Free d3.js License: Price: Main dependencies: Web-site: https://rawgraphs.io/ RAWGraphs is an open web tool to create custom vector-based visualizations on top of the d3.js library. It has been developed by DensityDesign Research Lab (Politecnico di Milano) and Calibro and sustained through corporate stewardship by ContactLab. It works with tabular data (spreadsheets and comma-separated values) as well as with copied-and-pasted texts from other applications. Based on the SVG format, visualizations can be edited with vector graphics applications for further refinements, or directly embedded into web pages. Here’s an example gallery to explore before diving in. Taucharts Apache 2.0 Free d3.js License: Price: Main dependencies: Web-site: https://taucharts.com/ TauCharts is one of the most flexible JavaScript charting libraries out there. It is also D3 based and is a data-focused JavaScript charting library that allows for improved data visualization. The library provides a declarative interface for fast mapping of data fields to visual properties. Its architecture allows you to build facets and extend chart behavior with reusable plugins. Talking about flexibility, TauCharts gives you easy access to their API, thus giving users the opportunity to seamlessly map and visualize data to get more amazing insights. Anychart Paid for commercial use From $49 to $799 to custom price. Free for non-commercial use – License: Price: Main dependencies: Web-site: https://www.anychart.com/ AnyChart is a robust, lightweight, and feature-rich JS chart library with rendering in SVG/VML. It actually gives web developers a great opportunity to create different charts that will help them conduct data analysis and make data-driven decisions. Key Features: More than 80 JS chart types, including basic charts, stock charts, maps, as well as Gantt and PERT charts; Many ways to set data: XML, JSON, CSV, JS API, Google Sheets, and HTML Table; Drill down into chart data; Stock technical analysis indicators and drawing tools (annotations) out-of-the-box; Integration with Angular, Qlik, Oracle APEX, React, Elasticsearch, , Android, iOS, etc. Vue.js The watermarked version is free. To get rid of the branding, as well as to use AnyChart for any commercial purpose, it’s necessary to buy a license (starts from $49). Recharts MIT Free d3.js License: Price: Main dependencies: Web-site: http://recharts.org/en-US/ Recharts provides a set of modular javascript charting components and does a great job by letting you mix those components together to easily build things like a composed line and bar chart. It is the most popular library to date. It has more than 11k stars on Github, but there is a huge number (600 to date) of open issues as well. The documentation is quite extensive but lacks details in some parts. You can find a bunch of examples at Recharts website, which could be a good starting point for building your own charts. What’s a bit troubling is the high number of unsolved issues on GitHub. These issues may not be critical, but creators don’t seem to drop by to answer them too often. So if you get stuck, be prepared to dig deep into the library. Conclusion With charting being something we all need to implement so often now, it’s no great surprise that there are many open-source charting libraries available for us to choose from. All of the javascript charting libraries we mentioned above have very good browser compatibility and fallbacks available for people using legacy browsers. The libraries built upon D3 have a solid base, but personally, I find them a little bit underwhelming from an aesthetic aspect. Javascript chart Libraries like FusionCharts, GoogleCharts, Dygraphs, or one of the D3 derivatives may work best for corporations with large data sets, or small businesses that rely heavily on data analysis. Both commercial offerings, Highcharts, and FusionCharts are mature libraries that can fit most use cases very well. Both of them have varying degrees of support for large datasets (>100k data points) and support a large variety of charts. If there is a specific type of chart you need to render, the choice might narrow down to only one of them. If you just need something small and quick, Morris.js or Chart.js might work better for you. For graphs and networks, Cytoscape or Sigma.js is probably the way to go. If you want to stick to a free and open-source library, use Chart.js. It is extremely simple to use for common use cases. If you need a little more control over rendering, then you could look into Chartist. P.S. We have prepared a table to show the popularity of libraries by downloading them through . We see that the most popular are d3, chart.js, and Highcharts. npm Conclusion To select the best JS chart solution for your unique needs, I recommend testing your own data against a couple of the libraries listed above to ensure a perfect fit for your current and future projects. Previously published at https://flatlogic.com/blog/best-19-javascript-charts-libraries/

19 JavaScript Data Chart Libraries

Creating an efficient user experience is not an easy task and does not have a “proven recipe”. Standard approaches and practices are not applicable to UX; each and every case is unique and unrepeatable. The nuances in business model, application work, or target audience that may seem insignificant at first glance can play a decisive role in success of UX, and thereby, of your product. The main criteria that really matters in UX is whether it fixes your users’ problem. Many entrepreneurs think that they may forget about UX once and for all after inviting a UX specialist to a project. In fact, creating a good UX requires the participation of at least three parties. Shall we take a closer look at them? UX From Users’ Point Of View When thinking about user experience, you should rely on . Although this is not very hard to understand, you'll be surprised to hear how often people forget about it. users’ needs Rigorous gathering and analysis of user expectations is the number one job to start working on UX. Moreover, remember well that you create UX for them, not for yourself. So, your comfort using the product does not count if it does not match the way your users feel. A bright example below shows how much you can go wrong if you do not bother to ask your audience what they need. A story about UX design while searching for airport names Once we were involved in upgrading a system used by operators of an airport call center to assist passengers in booking and buying airplane tickets. Searching the departure and arrival points was one of the main tools for operators to provide fast and quality service to end-users. When analyzing the future UX, we suggested applying one of the best practices that are widely used by hundreds of similar resources. That is to tie the search to airport names or names of cities, where airports are located. As regular users, we considered this approach as a proven and handy way to map a route. Imagine how astonished we were when we heard back: “ ”. We don’t need it that way After some negotiations, it turned out that searching by cities and airports names take more time than entering airport alphabetic codes, which every operator learns by heart on the very first day of the work. It happens, of course, that there is no possibility to ask users directly, and you are forced to step on a slippery path of guessing. A business owner and a UX specialist should do their best to walk-in users’ shoes and think through as many user scenarios as can be. UX From Business / Product Owners’ Point Of View Entrepreneurs can simplify the process of creating user experience and make it more efficient. Here are three things to begin with: dive deep into the process of UX creation and do not step aside from it; make an in-depth analysis of own business model; define your target audience and get a fix on why and how people are going to use your product or service. Treat your users with equal care regardless of whether they are your customers or your employees. It is a common delusion that only external users determine the success of a product. It is true that if an app or a service is inconvenient to use, eventually it will just fade out. When it comes to systems designed to simplify and optimize the inner work of a company, user experience is often not taken seriously. Therefore, instead of skimming the cream off investments made to optimize employees' work, businesses get unhappy and inefficient people forced to use a system that is neither handy nor easy to use. So, do not neglect your internal users’ opinions. Who but they know best how to streamline your corporate products and systems and make their daily work more productive. UX From UX Specialists’ Point Of View A UX specialist should act as an expert able to see and recognize the nuances that may affect the eventual success of a service or product. Here’s another example worth mentioning, since it demonstrates how a slight difference in the audience portrait may change the user experience. An application our team has been working on was intended as a tool for parents to motivate their children to learn, go for sports, help with household chores, etc. Using this solution, parents could set goals and reward kids for achieving those goals. Studying user scenarios, we discovered an interesting fact: two-parent and solo-parent families . In the first case, a kid could inform parents about the work done in person, while in the latter they needed a chat to communicate. Different conditions dictate different approaches even when a user portrait remains almost the same. required different sets of features Many people believe that only a UX specialist with relevant experience in a specific industry can create an effective UX. Surely, a UX expert should dive into the domain and understand the customer’s business. At the same time, the lack of experience in a particular industry should not become a stop factor. Extensive experience in different areas endows a UX specialist with a certain open-mindedness that can bring fresh ideas to the user experience of your application. To Sum Up What makes a good UX specialist? An ability to see certain common factors in users’ behavior and understand how to adapt a product to these regularities. Technically, any person, who can do that already is a UX specialist, even if he or she doesn`t have it written down in a resume. Who is in charge of UX? A good UX is teamwork of business owners, product users, and a UX specialist, where every part of the process should be able to hear the other party and understand their needs. What does it take to create an effective UX? Take attention to every little detail, since it may count for a great deal later. Remember that best practices and solutions that are considered convenient in most cases may not be applicable for the specific needs of your audience. Be able to compromise in order to give your users a handy tool for solving their problems. Previously published at https://sigma.software/about/media/three-sides-efficient-ux

What Does It Take To Create An Effective UX?

Stories, News & Mentions about SIGMAOS

Categories

Trending Topics

Stories, News & Mentions about SIGMAOS

Sigmaos’s Stories on HackerNoon

Categories

Trending Topics