Moesif

moesif

For any API-first company, implementing the right API analytics platform is critical for tracking the utilization of your APIs and to discover any performance or functional issues that may be impacting customers. There are a variety of open-source projects you can leverage to build a complete API analytics platform. Before jumping into building an API analytics solution yourself, you should first list out your requirements and use cases. Not all tools support all use cases directly, and may require heavy investment in development and integration. API analytics build requirements Need to answer engineering or business questions Engineers commonly turn to API logs and metrics to understand what’s happening over their APIs, inspect payloads, and root cause issues with their services that come up. capabilities is a must for engineers that want to depend on their API analytics solution to put out the (hopefully far and few between) fires due to an API outage or reliability issue. Because real-time logging can raise the compute and storage cost, not all analytics infrastructure maintains a real-time pipeline. On the flip side, only the most recent data is needed to answer hair on fire engineering problems, so data can be after a short time period such as after 24 hours. Real-time API logging retired Engineering and product leaders make strategic decisions based on lines, not dots. Business questions are answered from historical trends in data, which may be over months or even years. This means your API analytics build should be capable of storing data for long retention periods such as for multiple years. This also requires infrastructure that can roll up and compress your data as storing and running aggregations on raw event logs will cripple your analytics infrastructure Real-time alerting and monitoring? While building dashboards and throwing them up on a monitor can be a great way to monitor your metrics, it still requires manually checking them periodically. To become more proactive, many API analytics builds also have some sort of monitoring and alerting features. These can vary from simple threshold-based alerts that sends an email when a metric reaches a certain value, to complex monitoring rules and workflows that can perform complex aggregations in real-time and route alerts to incident response platforms like PagerDuty and BigPanda. Some customer-facing teams like developer relations and customer success may want to build automated workflows to notify customers or internal specific internal stakeholders based on complex user behavioral flows. Access data anywhere vs controlled access The post-COVID world has accelerated the shift towards work from home. This means internal users of your API analytics build may need to access metrics on home networks or on the go without a VPN. Placing unnecessary security restrictions to access the API data may limit the value your API analytics provides to your company or can even backfire creating bad habits such as password sharing or exporting large amounts of data to a personal device rather than using the API analytics build the way it was designed. On the flip side, providing self-service API analytics from anywhere also means having strong security and access control. If you think typical enterprise access control like single sign on and role-based access control may be needed in the future, you should plan your API analytics build accordingly even if not needed immediately. Ripping out and changing authentication and authorization design is not an easy task and could force a complete rewrite. You don’t want to be caught off guard and be the engineer who failed at foreshadowing typical future enhancements. Flexibility of visualizations While most analytics platforms can display event data or plot basic single-value metrics over time, you platform may also be used for more advanced and specialized analysis like or cohort retention analysis. These are common queries used by business functions like marketing and growth teams but rarely used by developers themselves. Yet, building a funnel can be quote challenging if the right data model was not used. funnel analysis Many times, you don’t know the types of queries that need to be displayed but should choose a project that enables flexibility in both the data model and the visualization layer. Comparison There are a variety of open-source analytics and monitoring projects out there. Some are focused on monitoring infrastructure metrics like Kibana and Grafana. While others are more focused on web analytics like Matomo (also known as Piwik). While none of these are designed for API products, you could develop custom code to piece a few components together to build an open-source API analytics platform Kibana Kibana is one of the de facto open-source log visualization tools out there for engineers. It’s part of the official ELK stack (Elasticsearch Logstash Kibana) which makes it one of the quickest visualization tools to get up and running due to it’s tight integration with Elasticsearch relative to tools like Grafana which are far more complex to set up. Elasticsearch itself is well suited for high-cardinality, high-dimension log data which is a must for API logs. The downside is that Kibana is compatible with Elasticsearch. If you want to visualize data stored in a SQL database or other data store, you’ll need to look elsewhere. only While quick to set up, Kibana is also quite limited in the types of visualizations and flexibility supported. Kibana’s primary use case is to provide log search and light analysis on raw event data rather than offering a true API monitoring tool. For debugging use cases, this may be sufficient, but popular business metrics like funnels and retention analysis cannot be performed by Kibana limiting it’s application outside of engineering teams. Kibana and Elasticsearch is designed to be paired with a Logstash instance which enables you to design a logging pipeline to process and enrich API logs such as normalized any HTTP headers or add geo IP information to each API call. Keep in mind Logstash cannot perform aggregations across multiple events at a time. Such processing requires a separate cluster-computing framework for map-reduce operations like Spark or Hadoop. Kibana is designed for use cases where you want to explore your data in an ad hoc fashion rather than create a daily dashboard. You’re able to leverage Elasticsearch query DSL or Lucene query syntax providing great flexibility, but these do have a steep learning curve. By default, Kibana is purely a visualization tool which means things like alerting, anomaly detection, and authentication are separate. This means anyone with access to your Kibana endpoint can access your data so you shouldn’t have it public. However, you can purchase and install the Elasticsearch X-Pack to gain some monitoring functionality along with access control. Grafana Unlike Kibana which focuses on log search, Grafana focuses on time-series based metrics. You can visualize data in a variety of databases including Elasticsearch, InfluxDb, OpenTSDB, Graphite, and Prometheus. Grafana does one thing, and one thing really well, which is visualizing time series metrics stored in a database with beautiful dashboards. This does leave everything else up to you including configuring your data source and processing your data into a time series metric that can be displayed by Grafana. Compared to Kibana, Grafana only works on time-series data already stored in a database and does not have any real-time log search nor a way to browse or explore your raw data in an ad hoc way. The primary use case for Grafana is to design a dashboard to monitor time-series metrics regularly such as on a TV in the office. For example, you may want to display disk utilization, system CPI, and requests per minute for your servers. Grafana has a lot of options to display your metrics in the way you want, like showing storage capacity using base 2 units and percentile based metrics. Due to it’s time-series based architecture, Grafana’s application for high-cardinality, high-dimension analytics on API calls is limited. Instead, you need to spend time deciding exactly what specific time-series metrics you want to track ahead of time and model your data accordingly. This also limits Grafna’s use case for the self-service data exploration that business users may be looking for where you’re looking to segment with multiple or correlate multiple dimensions in your data. group by’s Compared to Kibana, Grafana is known for built-in support for authentication and access control. You can also hook up your Grafana instance to an external Lightweight Directory Access Protocol (LDAP) server or SQL server to better control access in an enterprise setting. You can also connect Grafana to an incident response platform like PagerDuty to create and trigger alerts from your Grafana instance. Keep in mind these alerts are limited to the same time series metrics you are already monitoring in Grafana. Grafana does have a separate product called Loiki, which provides some of the log exploration features that Kibana has. Jaeger Like Grafana which specializes in time-series metrics, Jaeger does one thing and one thing well, which is visualizing distributed traces. This makes Jaeger quite a bit of a different tool than Grafana and Kibana in that each trace is created and viewed in isolation vs monitoring metrics or logs over time. A trace is a snapshot of all context and timing info as a request propagates through a service mesh or hits various services in a microservice architecture. Because trace generation is expensive, sampling is usually employed to snapshot every X requests or specific criteria. Unlike Grafana which focuses on monitoring time-series metrics and Kibana which focuses on log search, Jaeger focuses on root causing specific issues with a service mesh or dependency issues. Jaeger supports multiple data sources like Grafana including Cassandra and Elasticsearch. Since traces are created in isolation, the only view is a trace view as expected. There is no way to create trends over time. Jaeger also doesn’t have any alerting or monitoring features so you’ll want to still have a Grafana or similar instance handy. Moesif Many API teams find needing a multitude of tools for their monitoring needs. Grafana has time-series metrics whereas Kibana enables log search. A faster option is an end to end solution designed specifically for API products like . Unlike Grafana where time-series metrics need to be preplanned, Moesif is designed for ad hoc data exploration using high-cardinality, high-dimension API analytics which is a must for product and engineering leaders looking to make strategic decisions from their API usage data. Moesif Compared to infrastructure monitoring tools like Kibana and Grafana, Moesif leverages a which enables you to align your API metrics to business goals by tying API metrics to individual customers rather than tied to infrastructure. This is also known as which enables understanding complex user flows holistically across multiple user actions and API calls rather than looking at each time-series in isolation. user-centric data model user behavioral analytics A classic example of user behavioral analytics is monitoring a and then breaking it down by user acquisition channel to decide where to invest marketing dollars or viewing for different features or endpoints to make your API products more sticky. This also enables tracking higher level account health for security research and customer success. conversion funnel user retention Compared to Grafana and Kibana, Moesif also provides elaborate real-time alerting and reporting features. This enables you to see which endpoints are causing the most performance issues broken down by each customer email. With behavioral emails and workflows, you’re able to scale customer outreach and support efforts with automated emails with a set of steps.

Open Source API Analytics and Monitoring Tools - A Comparison

9 Vital API Security Threats Every Team Should Watch For

API Analytics are key for any platform company that wants to gain insights into their API and platform usage. These insights can be leveraged by product owners, growth teams, developer relations, and more to make more strategic decisions based on the raw health of your platform business rather than just gut feelings. However, many API platforms have a very high volume of API calls per day. This volume creates a set of unique challenges when designing an analytics system that scales without crippling their platform or having sticker shock when looking at their cloud vendor’s bill. This post goes into some of the inner workings of how we designs Moesif’s API analytics platform to handle companies with billions of API calls a day. Agents/SDKs Designing for scale starts with a well-designed agent to collect metrics from various applications and send to your analytics platform. Since the agent or SDK is embedded in your application, a poorly designed one could cripple your core services if your analytics service gets overloaded. In order to prevent this, your agent should have automatic fail safes and kill switches. For starters, the agent should collect any metrics asynchronous from your core application code and should sit out-of-band, This way, even if your analytics service is down for extended periods of time, your application availability is not affected. Ensure any errors in processing metrics data are handled and not propagated back to the application itself. Since you’ll be collecting metrics asynchronously, you’ll want to leverage local queuing. This can be done in memory or to disc depending on the data consistency required. Keep in mind though, any resources that need to be allocated locally such as memory or disc required for a queue needs to be constrained. Otherwise, a queue could continue to grow without bounds if your analytics service goes down for an extended period of time eventually causing the process to run out of memory and crash. One way to solve this via fixed queue size and drop older events. Dropping older events during a network disconnect means you sacrifice consistency for availability, a key pillar of which is usually referenced in when discussing distributed data stores, but is applicable here also. Queueing also enables you to batch events which reduces network overhead. Brewer’s CAP theorem For more info, read Best practices for building SDKs for APIs Sampling If your API is processing billions of API calls a day, even with queueing and batching, you may quickly exceed your compute and and storage requirements. The compute cost to run such a system could easily be in the hundreds of thousands or millions of dollars a year. One way to reduce this pressure is via a smart sampling algorithm. While a rudimentary algorithm that consistently chooses a random percentage of events to log vs drop may work for the short term, this can heavily reduce the usefulness of your analytics system. This is because most products exhibit a power law curve when it comes to customer usage. Your top 20% of customers will create 80% of your traffic. This also means you have a of customers that send very little traffic. Heavily sampling customers who send very little data in the first place can create distorted views of your customer journey. long tail One way to reduce this issue is by implementing a more intelligent sampling scheme. One way is sample on a per user or per API key basis. For example, any user who is a top 20% customer will have a sample rate of 10% whereas the rest of your customers will have a sample rate of 100% (i.e. capture all activity). This means you need a mechanism to propagate the sample rates to your agent. Collection Network Regardless if your analytics service is accessible only from your own infrastructure only or is accessible from the intranet, you should have a system that can handle fail over in case one of the collection nodes goes down. A well designed network will leverage DNS-based load balancing across multiple data centers around the world such as AWS Route 53 or Azure Traffic Manager with a relatively short TTL. Within each data center, you can then leverage a proxy server like HAProxy, NGINX, AWS ELB to distribute traffic among your collector nodes. We at Moesif are heavy users of HAProxy to load balance within each data center and Azure’s Traffic Manager to load balance across different data centers. Your collector logic should be very light wight. It’s job should be only authentication, some lightweight input validation, and persisting to disc as quick as possible. This is where solutions like Kafka can be extremely useful. Roll Ups Even if you sample, you may find that your storage costs are ever increasing and queries take exceedingly long time. Modern databases like Elasticsearch and Cassandra can hold hundreds of terabytes of data with their distributed architecture, yet you’ll run into increasingly long queries (or worse: out of memory errors) when you attempt to perform aggregations on these massive data sets. In addition, the storage requirements continue to increase. When it comes to analytics, you can choose the resolution you want to store the data. This can be the raw events as they come in, or rolled up into for all activity that happened in a particular period such as every minute or every hour. Rolling up a metric can drastically reduce storage requirements, but it sacrifices metric resolution. stories Many analytics systems have a tiered roll up strategy. For example, all data in last 7 days is stored down to the event resolution or on 5 second intervals. Older data greater than 7 days ago can be rolled up into hourly intervals. While even data that’s older than a year ago may be rolled up in daily intervals. It’s unlikely you’ll be looking at events down to the exact timestamp. colder Metric Estimation Sometimes you don’t need an exact count and can live within a certain level of error. Let’s take the example calculating weekly active API keys. Let’s say you have an ordered API log that contains the request time stamp and the API key used to access your API. A naive approach would be to create a map indexed by the weeks and then for each , create a set that contains the unique keys for each bucket. bucket An issue with this approach is you can quickly run out of memory if you have a large number of unique API keys (also known as high-cardinality). A different way to approach this would be leveraging a probabilistic data structure to infer or estimate the uniqueness of the set rather than directly counting. Example probabilistic data structures include and . HyperLogLog Bloom Filters After all, you probably are interested in if there was 100 or 10,000 API keys active last week, but if your analytics system reported 101 instead of 100, your decision making wouldn’t differ. For this case, we can leverage HyperLogLog which approximates the number of unique values by estimating the number of leading zeros when the API keys are hashed together via a hashing algorithm. Accuracy is dependent on how many bits are allocated for this bit vector required to store the hashed values. Many modern databases for analytics like Druid and Elasticsearch have HyperLogLog built in, but it’s up to you to ensure your data can leverage them. Previously published at https://www.moesif.com/blog/technical/scaling-analytics/How-to-Design-API-Analytics-Data-Collection-for-High-Volume-APIs/

How to Design API Analytics Data Collection for High Volume APIs

In order to capture API calls from arbitrary environments, we had to create middleware for many of the common web API frameworks. Here's what we learnt. Introduction to HTTP Middleware What Is Middleware? I use the term middleware, but each language/framework refers to the concept differently. Node.JS and Rails call it middleware. In the Java enterprise world (i.e. Java Servlet), it’s called filters. C# calls it delegate handlers. Essentially, middleware performs some specific function on the HTTP request or response at a specific stage in the HTTP pipeline before or after the user defined controller. Middleware is a design pattern to eloquently add cross-cutting concerns like logging, handling authentication, or gzip compression without having many code contact points. Since these cross-cutting concerns are handled in middleware, the controller/user defined handlers can focus on the core business logic. What Can Be Done With Middleware Middleware is generally pretty flexible. Some middleware is passive, such as logging middleware. Other middleware, such as gzip compression, can perform transforms on the request or response body. Middleware can add HTTP headers, or add internal flags for use by your business logic, etc. It’s an implementation of the . pipeline's design pattern In fact, even basic framework features like body parsing can be considered middleware. You can add custom body parsing for various binary protocols if the ones included with a framework don’t fit your use case. You can even call other services, such as looking up or performing a GeoIP lookup from a MaxMind dataset. Key Things to Consider One of the first things to consider if you want to perform a task using middleware is that middleware is best for cross-cutting concerns. If it is a specific business logic, and only applies to very few cases. If your app has many common tasks, such as logging, authentication, parsing JSON, or adding some common share data to every request or response, then refactoring out that logic into middleware makes sense. Performance Due to the cross-cutting nature of middleware, ensuring it’s performant is particularly important, as any added latency could impact the entire application. Latency could come from the I/O, such as the disk or network access. For example, if you create authentication middleware for your API that needs to lookup user information in a SQL database, that I/O read will stall the HTTP request pipeline until a response is received from the SQL database. That added latency will still be seen by clients even with non-blocking frameworks like Node.js. Primer On Latency vs Bandwidth and are two orthogonal, yet related metrics. refers to metrics such as Request Per Second or the maximum number of active clients or connections at a time. on the other hand, refers to metrics like response time to first byte, or the average time needed to perform a particular query. But, since an increase in latency can also reduce the bandwidth of a system, they are related. For example, an increase in latency may also increase the resident time of queues. Latency bandwidth Bandwidth Latency, Generally, non-blocking frameworks such as Node.js focus mostly on increasing the of an application by not tying up resources (i.e. threads). This is similar to adding an additional VM or load balancer to handle additional traffic. However, each individual user or client may still experience high latency, regardless of the bandwidth of the system, or if they're using a non-blocking or blocking architecture. bandwidth For our example, a better solution may be to use an in-memory hash table, like Redis, for looking up data via session tokens. An even lower latency solution would be to use JSON Web Tokens (JWT) where secure authentication and authorization requires only CPU cycles. Some frameworks like Node are designed from the bottom up to be a non-blocking I/O. However, in other frameworks, such as PHP or Ruby on Rails, it's important to do I/O or process heavy tasks on background threads. Passive Middleware Even with the limitations of NIO described above, it could still be beneficial in reducing latency. Especially for passive middleware with write only I/O, architecting your middleware to be non-blocking or asynchronous is ideal. For example, the middleware for does not modify the response, so there is no reason for the HTTP pipeline to wait for any writes to Moesif. Thus, we designed the Moesif middleware to be asynchronous. Moesif For blocking frameworks such as PHP Laravel, you may have to implement your own methodology to be asynchronous. For the , we leveraged a Unix fork to handle the sending of data to Moesif in a separate process. Moesif Laravel middleware Forking a process in Linux is a surprisingly lightweight task. For more info on why, please read . What every web developer should know about CPU Arch Ordering Each middleware is instantiated as one step in a pipeline, thus ordering may be important for functional correctness. However, even if two pieces of middleware can be re-ordered, you should think about the performance or security implications. Lightweight middleware that has a short circuited return path should go before the heavier middleware. As an example, middleware that redirects non-HTTPS to HTTP domain via a 301 Permanent Redirect should be placed before middleware that decompresses and parses the request body. Otherwise, you’re wasting CPU cycles decompressing a body that will be thrown away. Think about security and DDoS protection also. If only a specific whitelist of IP addresses are allowed to access your API, and all others are denied, you probably want to check the IP address before performing expensive operations, such as querying account information in a database. Underlying Frameworks Often times, the framework you are using is actually built on top of another framework. For example, Ruby on Rails is built on top of Rack. So when you are building a rails middleware, you are actually building one for Rack. This can be used to your advantage. At Moesif, we didn’t need to create a separate middleware for every Java framework. Instead, we released a single Java Servlet SDK. Many modern Java frameworks are built on a Java Servlet such as Spring, Struts, and Jersey. Sometimes, you have the option to build on a base framework like Java Servlet or a higher level web framework such as Spring. Closing Thoughts Build middleware can force you to dig into the more underlying technology of each framework. Many common services (such as authentication and body parsing) of a framework are implemented as middleware. Previously published at https://www.moesif.com/blog/engineering/middleware/What-Is-HTTP-Middleware/

How To Define HTTP Middleware and Best Practices

Enterprise software companies sell very differently today than just a decade ago. Previously, most software was shrink-wrapped, required a lot of effort to distribute and implement, and was sold to an executive who would have to deploy it throughout their department or organization. The buying process involved a long sales cycle and often included pilots, cost-benefit analyses, procurement, and legal reviews. Today, enterprise companies are building developer platforms where an individual developer can sign up for a SaaS service, integrate the SDK, and start paying via a credit card in a matter of minutes. Everything from payments and telecommunications like Stripe and Twilio to e-commerce search like Algolia can now be delivered via an API and SDK. With the rise of SaaS, enterprise companies are disrupting new industries by taking a page out of the go-to-market playbook for latest generation of mobile focused and web 2.0 companies like Facebook, AirBnB and Uber. This means ads, onboarding experience, and self-service is taking a higher priority over field or inside sales. The transition is not easy though. This article lays out some of the key elements required. Self-Service Onboarding How to transition from enterprise-only sales and add a developer first offering that's self-service. Any developer platform or self-service SaaS must start with their onboarding flow. Whereas the previous generation of enterprise companies could simply rely on a "Schedule Demo" or "Contact Us" form, developer platforms must now enable everything from sign up, to paying for the implementation via a no-touch or low-touch model. Because many enterprise software products require engineering expertise to set up and are more complicated than signing up for Netflix, a large emphasis should be placed on your onboarding flow. You'll want to gather some information such as the company name and the role of the person. This enables the experience to be personalized - is the person is a developer or non-developer. You'll want to clearly define the next steps for implementation, whether that's installing an SDK or authenticating with a third party tool. Measuring Onboarding Success Time To First Hello World (TTFHW0, or sometimes called Time to First API Call, is a key metric to track the effectiveness of your onboarding. Specifically, TTFHW is the time it takes for a developer to sign up for your service and make their first API call. Some products may have a short TTFHW in the range of minutes, but others may be weeks or even months if serious investigation is needed. Keep an eye out for any user segments that exhibit a better TTFHW than others. For example, which marketing channel is the most effective, what integrations are they using, etc. Their first API call could be as simple as a test from Postman or from their production code. For more info, read Best practices for Developer Relations Programs to measure success of an API platform Reducing Onboarding Friction To reduce friction, keep the onboarding flow to around three steps or less. Even if your platform has a large surface area or large integration component, identify what is the minimum integration required to demonstrate value to the developer as quick as possible. This means less critical steps such as setting up custom rules or configuring app settings should not be shown during onboarding.  For example, at Moesif we only prompt our new customers to integrate the middleware SDK to get instant visibility into their APIs. Moesif has additional SDKs to enrich the user profiles with demographic info, like plan quota or company name, but we don't prompt users to install those during our onboarding. Instead, we nudge our users via progress check lists and drip emails. When a user redirects to an area of the product not set up, then we display the relevant secondary onboarding page. This practice is sometimes called . gradual onboarding Self-Service Pricing Self-service should imply no- or low-touch sales and almost always include a free tier or a limited-duration trial. If someone from your team has to be on the phone to demo the product or provision a new account, and involve engineering for implementation, then self-service will be extremely hard to pull off. Instead, simply share your enterprise pricing publicly, as the cost of acquisition will far exceed the lifetime value of the customer. Similarly, self-service is not ideal for high contract values. initial While a customer who was initially self-service can increase their usage over time and mature into more expensive plans, they may not have been willing to initially pay for higher plan. Don't launch a developer platform that's self-service just to reduce cost of sales and support effort. Instead, view self-service as a way to open new markets that are too expensive for traditional sales (such as smaller businesses), or reduce the effort required for demand generation (such as in markets with a large user base that can be driven by word-of-mouth). Ways To Price While some companies can garner higher rates, most self-service models do best when the price point is less than $1,000/month, or even less than $500/month. Ideally, you want multiple price points that target a variety of customer profiles and sizes. This can be done through usage based, number of seats, multiple or bundles, add-ons, or a combination of the above. Pick only one pricing model to prevent decision fatigue for customers. Mixing tiers with usage based and seats can confuse the customer. tiers What To Price On In order to scale with customers, many API first companies have some sort of usage-based pricing connected to a value metric. This can be transaction volumes (such as SMS sent), monthly active users, dollar volume, etc. If different customers care about different value metrics, you may need more than one in order to capture the full market. Auth0, a login API, is a clear example of this. Besides company size, they have two types of customers: B2C and B2B. B2C has very high monthly active user counts. Auth0 may appear extremely expensive if they only offered 100 or 1,000 MAU in their lower tiers; We've seen 1,000 MAU for invite-only beta apps, that aren't even launched yet; 7,000+ MAU might be more appropriate for their free plan. Many B2B companies can be in the $10's to $100's of millions of annual run rate, without ever breaching 7,000 MAU. In order to capture value at this end of the spectrum, Auth0 decided to package certain high-value features, such as enterprise sign-in support and advanced role based access control, in their B2B plan. More information on pricing can be found in . The Anatomy of SaaS Pricing Strategy Data-Driven Pricing Once you decide on the packaging and positioning of your self-service platform, you still have to determine actual limits. Does the free tier have 10,000 API transactions per month or does the free tier include 1 million API transactions per month? The best way to determine these amounts is first decide on the price points based on your customer profiles such as free, $100/month, and $500/month. Then, use your tool of choice to cluster users based on their usage data. A simple quick visual glance even scripted k-means will do. Can you find a local maxima in certain groups? You may find trends such as majority of companies with less than 10 employees have around X usage. In order to do this, you need to track usage data per customer and be able to visualize that easily through an API analytics tool like or alternative analytics for API platforms. Moesif Self-Service Support Third-party developers hate emailing or getting on the phone with someone just to resolve a support issue. In addition, solutions engineers, consultants, and engineers are very expensive as first line support. They should be available if a developer cannot find their answer online. This means you need to put effort in documentation, tutorials, GitHub examples, and other self-service resources. Empower the developer to research and resolve their own problem, rather than relying on expensive first-tier support staff. Read for an overview on how to structure your various resources. Designing good Static REST Documentation

How To Launch A New Developer Platform That's Self-Service Software

A list of the most important API metrics every API product manager and engineer should know, especially when you are looking into API analytics and reporting. Identifying key API metrics Each team needs to track different KPIs when it comes to APIs. The API metrics important to infrastructure teams will be different than what API metrics are important to API product or API platform teams. Metrics can also be dependent on where the API is in the product lifecycle. A API recently launched will focus more on improving design and usage while sacrificing reliability and backwards compatibility. An team that maintains an API that’s been widely adopted by enterprise teams may focus more on driving additional feature adoption per account and give precedence to reliability and backwards compatibility over design. In general, there are three to four teams that care about API metrics. Infrastructure/DevOps Ensure the servers are running and limited resources are correctly allocated, potentially for multiple engineering teams. Application Engineering/Platform API developers are responsible for adding new features to APIs while debugging application specific issues in the API business logic. These products could be API as a Service, plugins and integrations for partners, APIs incorporated in a larger product, or something else. Product Management API product managers are in charge of roadmapping API features, ensuring the right API endpoints are being build, and balancing the needs ot customers (whether internal or external) with engineering time and personal constraints. Business/Growth Business facing teams such as marketing and sales are not thinking in terms of API endpoints, rather they are mostly interested in customer adoption, ensuring they’re successfully using the APIs, where they come from, and seeing which users could be new sales opportunities. Infrastructure API metrics Many of these metrics are the focus of Application Performance Monitoring (APM) tools and infrastructure monitoring companies like Datadog. 1. Uptime While one of the most basic metrics, uptime or availability is the gold standard for measuring the availability of a service. Many enterprise agreements include a SLA (Service Level Agreement), and uptime is usually rolled up into that. Many times, you’ll here terms like triple 9’s or four 9’s which is a measure of how much uptime vs downtime there is per year. Of course, going from four to five nines is far harder than going from two to three nines which is why you won’t see five nines except for the most mission critical (and expensive) of services. With that said, certain services can actually have lower uptime while ensuring graceful handling of outages without impacting your service. For example, Moesif is designed such that it can continue to collect data from our SDKs even during a full outage of the website and dashboard. Even in absolutely worst case where our collection network is down, the SDKs will queue locally and not cause disruption of the application. Uptime is most commonly measured via a ping service or synthetic testing such as via or . You can configure probes to run on a fixed interval such as every minute to probe a specific endpoint such as /health or /status. This endpoint should have basic connectivity tests such as to any backing data stores or other services. Pingdom UptimeRobot These metrics can be published on your website using tools like . Moesif in fact uses an built on Lambda. Statuspage.io open source status page There are also more sophisticated ping services called Synthetic testing that can perform more elaborate test set ups such as running a specific sequence and asserting the response payload has a certain value. Keep in mind though synthetic teesting may not be representative of real world traffic from your customers. You can have a buggy API while maintaining high uptime. What is ? As name implies, it is a predefined set of API calls that a server (usually by one of the providers of Monitoring services) triggers to call your service. While it doesn’t reflect real world experience of your users, it is useful to see the sequence of these APIs are as expected. Synthetic Monitoring 2. CPU Usage CPU usage is one of the most classic performance metrics that can be a proxy to application responsiveness. High Server CPU usage can mean the server or virtual machine is oversubscribed and overloaded or it can mean a performance bug in your application such as too many spinlocks. Infrastructure engineers use CPU usage (along with its sister metric, memory percentage) for resource planning and measuring overall health. Certain types of applications like high bandwidth proxy services and API gateways naturally have higher CPU usage than other metrics along with workloads that involve heavy floating point math such as video encoding and machine learning workloads. When your debugging APIs locally, you can easily see system and process CPU usage via (or ) to see CPU usage. On a server though, you probably don’t want to be SSH’ing and running the top command. This is where various APM providers can be useful. APMs include a agent that you can embed in your application or on the server that capture metrics such as CPU and memory usage. It can also perform other application specific monitoring like thread profiling. Task manager on Windows Activity Monitor on Mac When looking at CPU usage, it's important to look at usage per virtual CPU (i.e. physical thread). Unbalanced usage can imply applications not correctly threaded or and incorrectly sized thread pool. Many APM providers enable you to tag an application with multiple names so you can perform rollups. For example, you may want to have a breakout of each VM metrics like , , , etc while having these rolled up in a single app called . my-api-westus-vm0 my-api-westus-vm1 my-api-eastus-vm0 my-api 3. Memory Usage Like CPU usage, memory usage is also a good proxy for measuring resource utilization as CPU and memory capacity are physical resources unlike a metric which may be more configuration dependent. A VM with extremely low memory usage can be either be downsized or have additional services allocated to that VM to consume additional memory. On the flip side, high memory usage can be an indicator of servers overloaded. Traditionally, big data queries/stream processing and production databases consume much more memory than CPU. In fact, the size of memory per VM is a good indicator for how long your batch query can take as more memory available can reduce check pointing, network synchronization, and paging to disk. When looking at memory usage, you should also look at number of page faults and I/O ops. An easy mistake to make is an application that’s configured to allocate at a maximum only a small fraction of available physical memory which can cause artificially high page virtual memory thrashing. Application API metrics 4. Request Per Minute (RPM) RPM (Requests per Minute) is a performance metric often used when comparing HTTP or database servers. Usually, you’re end to end RPM will be much lower than an advertised RPM, which serves more as an upper bound for a simple “Hello World” API. Since a server will not consider latency incurred for I/O operations to databases, 3rd party services, etc. While some like to brag about their high RPM, an engineering team’s goal should be efficiency and attempt to drive this down. Certain business functions that require many API calls can be combined into fewer number of API calls to reduce this number. Common patterns like batching multiple requests in a single request can be very useful along with ensuring you have a flexible pagination scheme. Your RPM may vary depending on day of week or even hour per days especially if your API is geared for other businesses which exhibit lower usage during nights and weekends. There are other related terms to RPM such as RPS (Requests per Second) and QPS (Queries per Second). 5. Average and Max Latency One of the most important metrics to track customer experience is API latency or elapsed time. While an increase in infrastructure level metrics like CPU usage may not actually correspond to a drop in user perceived responsiveness, API latency definitely will. Tracking latency by itself may not give you full understanding to why an increase occurred. It’s important to track any change on your API such as new API versions being released, new endpoints added, schema changes, and more to get to a root cause why latency increased. Because problematic slow endpoints may be hidden when looking only at aggregate latency, it’s critical to look at breakdowns of latency by route, by geography, and other fields to segment on. For example, you may have a POST /checkout endpoint that’s slowly been increasing in latency over time which could be due to an ever-increasing SQL table size that’s not correctly indexed. However, due to a low volume of calls to POST /checkout, this issue is masked by your GET /items endpoint which is called far more than the checkout endpoint. Similarly, if you have a GraphQL API, you’ll want to look at the average latency per GraphQL operation. We put latency under application/engineering even though many DevOps/Infrastructure teams will also look at latency. Usually an infrastructure person looks at aggregate latency over a set of VMs to make sure the VMs are not overloaded, but they don’t drill down into application specific metrics like per route. 6. Errors Per Minute Similar to RPM, Errors per Minute (or error rate) is the number of API calls with non 200 family of status codes per minute and is critical for measuring how buggy and error-prone your API is. In order to track errors per minute, it’s important to understand what of errors are happening. 500 errors could imply bad things are happening with your code where as many 400 errors could imply user errors from a poorly designed or documented API. This means when designing your API, it’s important to use the appropriate type HTTP status code. You can further drill down into seeing where these errors come from. Many errors from one specific geo region could imply bots are attempting to hack your API. 401 Unauthorized API product metrics APIs are no longer just an engineering term associated with microservices and SOA. APIs as a product is becoming far more common especially among B2B companies who want to one up their competition with new partners and revenue channels. API-driven companies need to look at more than just engineering metrics like errors and latency to understand how their APIs are used (or why they are not being adopted as fast as planned). The role of ensuring the right features are built lies on the , a new role that many B2B companies are rushing to fill. API product manager What is Moesif? is the most advanced API analytics platform used by Thousands of platforms to understand what your most loyal customers are doing with your APIs, how they’re accessing them, and from where. Moesif focuses on analyzing real customer data vs just synthetic tests to ensure you’re building the best possible API platform for your customers. Moesif 7. API usage growth For many product managers, API usage (along with unique consumers) is the gold standard to measure API adoption. An API should not be just error free, but growing month over month. Unlike requests per minute, API usage should be measured in longer intervals like days or months to understand real trends. If measuring month-over-month API growth, we recommend choosing 28-days instead as it removes any bias due to weekend vs weekday usage and also differences in number of days per month. For example, February may have only 28 days whereas the month before has a full 31 days causing February to to have lower usage. appear 8. Unique API consumers Because a month’s increase in API usage may be attributed to just a single customer account, it’s important to measure (Monthly Active Users) or . This metric can give you an overall health of new customer acquisition and growth. Many API platform teams correlate API MAU to their web MAU, to get a full product health. If web MAU is growing far faster than API MAU, then this could imply a leaky funnel during integration or implementation of a new solution. This is especially true when the core product of the company is an API such as for many B2B/SaaS companies. On the other hand, API MAU can be correlated to API usage to understand where that increased API usage came from (New vs. existing customers). API DAU unique consumers of an API Tools like Moesif can track both individual calling and API and also link them to or . users companies organizations 9. Top customers by API usage For any company with a focus on B2B, tracking the top API consumers can give you a huge advantage when it comes to understanding how your API is used and where upsell opportunities exist. Many experienced product leaders know that many products exhibit power law dynamics with a handful of power users having a disproportionate amount of usage compared to everyone else. Not surprisingly, these are the same power users that generally bring your company the most revenue and organic referrals. This means its critical to track what your top 10 customers are actually doing with your API. You can further break this down by what endpoints they are calling and how they’re calling them. Do they use a specific endpoint much more than your non-power users? Maybe they found their ah ha moment with your API. 10. API retention Should you spend more money on your product and engineering or put more money into growth? Retention and churn (the opposite of retention) can tell you which path to take. A product with high product retention is closer to product market fit than a product with a churn issue. Unlike subscription retention, product retention tracks the actual usage of a product such as an API. While the two are correlated, they are not the same. In general, product churn is a leading indicator of subscription churn since customers who don’t find value in an API may be stuck with a yearly contract while not actively using the API. API retention should be higher than web retention as web retention will include customers who logged in, but didn’t necessarily integrate the platform yet. Whereas API retention looks at post-integrated customers. 11. Time to First Hello World (TTFHW) TTFHW is an important KPI for not just tracking your API product health, but your overall . Especially if your API is an open platform attracting 3rd party developers and partners, you want to ensure they are able to get up and running as soon as possible to their first moment. TTFHW measures how long it takes from first visit to your landing page to an MVP integration that makes the first transaction through your API platform. This is a cross-functional metric tracking marketing, documentation and tutorials, to the API itself. developer experience aka DX ah ha 12. API Calls per business transaction While more equals better for many product and business metrics, its important to keep the number of calls per business transactions as low as possible. This metric directly reflects the of the API. If a new customer has to make 3 different calls and piece the data together, this can mean the API does not have the correct endpoints available. When designing an API, it’s important to think in terms of a business transaction or what the customer is trying to achieve rather than just features and endpoints. It may also mean your API is not flexible enough when it comes to . design filtering and pagination. 13. SDK and version adoption Many API platform teams may also have a bunch of SDKs and integrations they maintain. Unlike mobile where you just have iOS and Android as the core mobile operating systems, you may have 10’s or even hundreds of SDKs. This can become a maintenance nightmare when rolling out new features. You may selectively roll out critical features to your most popular SDKs whereas less critical features may be rolled out to less popular SDKs. Measuring API or SDK version is also important when it comes to deprecating certain endpoints and features. You wouldn’t want to deprecate the endpoint that your highest paying customer is using without some consultation on why they are using it. Business/Growth Business/growth metrics can be similar to product metrics but focused on revenue, adoption, and customer success. For example, instead of looking at the top 10 customers by API usage, you may want to look at top 10 customers by revenue, then by their endpoint usage. For tracking business growth, analytics tools like Moesif, support enriching user profiles with customer data from your CRM or other analytics services to have a better idea who your API users are. Conclusion For anyone building and working with APIs, its critical to track the correct API metrics. Most companies would not launch a new web or mobile product without having the correct instrumentation for engineering and product. Similarly, you wouldn’t want to launch a new API without a way to instrument and track the correct API metrics. Sometimes the KPIs for one team can blend into another team as we saw with the API usage metrics. There can be different ways of looking at the same underlying metric. However, teams should stay focused on looking at the right metrics for their team. For example, product managers shouldn’t worry about CPU usage just like infrastructure teams shouldn’t worry about API retention. Tools like can help you get started measuring these metrics with just a quick SDK installation. Moesif API Analytics

13 API Metrics That Every Platform Team Should be Tracking

Meltdown has definitly taken the internet by storm. The attack seems quite simple and elegant, yet the leaves out critical details on the specific vulnerability. It relies mostly on a combination of cache timing side-channels and speculative execution that accesses globally mapped kernel pages. whitepaper This deep dive assumes some familiarity with CPU architecture and OS kernel behavior. Read the first for a primer on paging and memory protection. background section Simplified version of the attack: Speculative memory reads to kernel mapped (supervisor) pages and then performing a calculation on value. Conditionally issuing a load to some other non-cached location from memory based on the result of the calculation. While the 2nd load will be nuked from the pipeline when the faulting exception retires, it already issued a load request out to L2$ and beyond ensuring the outstanding memory request still brings the line into the cache hierarchy like a prefetch. Finally, a separate process can issue loads to those same memory locations and measure the time for those loads. A cache hit will be much quicker than a cache miss which can be used to represent binary 1’s (i.e. hits) and binary 0’s (i.e. misses). Parts 1 and 2. has to do with speculative execution of instructions while parts 3 and 4 enable the microarchitecture state (i.e. in cache or not) to be committed to an architectural state. Is the attack believable? What is not specified in the Meltdown whitepaper is what specific x86 instruction sequence or CPU state can enable the memory access to be speculatively executed AND allow the vector or integer unit to consume that value. or L2$. In modern Intel CPUs, when a fault happens such as a page fault, the pipeline is not squashed/nuked until the retirement of the offending instruction. However, memory permission checks for page protection, segmentation limits, and canonical checks are done in what is called the address generation (AGU) stage and TLB lookup stage before the load even look up in the L1D$ or go out to memory. More on this below. Performing memory permission checks Intel CPUs implement L1D$ and L1I$ which requires translating the linear (virtual) address to a physical address before the L1D$ can determine if it hits or misses in the cache via a . This means the CPU will attempt to find the translation in the (Translation Lookaside Buffer) TLB cache. The TLB caches these translations along with the page table or page directory permissions (privileges required to access a page are also stored along with the physical address translation in the page tables). physically tagged tag match A TLB entry may contain the following: Valid Physical Address minus page offset. Read/Write User/Supervisor Accessed Dirty Memtype Thus, even a speculative load already knows the permissions required to access the page is compared against the Current Privilege Level (CPL) and required op privilege and thus can be blocked from any arithmetic unit from ever consuming the speculative load. Such permission checks include: Segment limit checks Write faults User/Supervisor faults Page not present faults This is what many x86 CPUs in fact are to do. The load would be rejected until the fault is later handled by software/uCode when the op is at retirement. The load would be zeroed out on the way to the integer/vector units. In other words, a fault on User/Supervisor protection fault would be similar to a page not present fault or other page translation issue meaning the line read out of the L1D$ should be thrown away immediately and the uOp simply put in a state. designed waiting Preventing integer/floating units from consuming faulting loads is beneficial not just to prevent such leaks, but can actually boost performance. i.e. Loads that fault won’t train the prefetchers with bad data, allocate buffers to track memory ordering or allocate a fill buffer to fetch data from L2$ if it missed the L1D$. These are limited resources in modern CPUs and shouldn’t be consumed by loads that are not good anyway. In fact, if the load missed the TLB’s and had to perform a page walk, some Intel CPUs will even kill the page walk in the PMH (Page Miss Handler) if a fault happens during the walk. Page walks perform a lot of pointer chasing and have to consume precious load cycles, so it makes sense to cancel if it’ll be thrown away later anyway. In addition, the PMH Finite State Machine can usually handle only a few page walks simultaneously. In other words, aborting the L1D Load uOp can actually a good thing from a performance stand point. The press articles saying Intel slipped because they were trying to extract as much performance as possible with the tradeoff of being less secure isn’t true unless they want to claim the basic concepts of speculation and caching are considered tradeoffs. The fix While this doesn’t mean the Meltdown vulnerability doesn’t exist. There is more to the story than what the whitepaper and most news posts discuss. Most posts claim that only the of having speculative memory accesses and cache timing attacks can create the attack, and now Intel has to completely redesign it’s CPUs or eliminate speculative execution. mere act In fact, the bug fix would probably be a few gate changes to add the correct rejection logic in the L1D$ pipelines to mask the load hit. Intel CPUs certainly have the information already as the address generation and TLB lookup stages have to be complete before a L1D$ cache hit can be determined anyway. It is unknown what are all the scenarios that cause the vulnerability. Is it certain CPU designs that missed validation on this architectural behavior? Is it a special x86 instruction sequence that bypasses these checks, or some additional steps to set up the state of the CPU to ensure the load is actually executed? Project Zero believes the attack can only occur if the faulting load hits in L1D$. Maybe Intel had the logic on the miss path but had a logic bug for the hit path? I wouldn’t be surprised if certain Intel OoO designs are immune to Meltdown as it’s a specific CPU design and validation problem, rather than a general CPU architecture problem. Unfortunately, x86 has many different flows through the Memory Execution Unit. For example, certain instructions like MOVNTDQA have different memory ordering and flows in the L1D$ than a standard cacheable load. Haswell Transactional Synchronization Extensions and locks add even additional complexity to validate correctness. Instruction fetches go through a different path than D-side loads. The validation state space is very large. Throw in all the bypass networks and then you can see how many different places where fault checks need to be validated. One thing is for certain, caching and speculation are not going away anytime soon. If it is a logic bug, it may be a simple fix for future Intel CPUs. Why is this attack easier today? Let’s say there is an instruction that enable loads to hit and consumed even in presence of faults, or I am wrong on the above catch, why is it happening now rather than discovered decades ago. Today’s CPUs have much deeper pipelines Prescott which provides a wider window between the speculative memory access and the actual nuke/squash of those faulting accesses. Faulting instructions are not handled until the instruction is up for retirement/committal to processor architectural state. Only at retirement is the pipeline nuked. Long pipelines allow for a large window between the execution of the faulting instruction and the retirement of it allowing other speculative instructions to race ahead. cough cough Larger cache hierarchy and slower memory fabric speed relative to fast CPU only ops such as cache hit/integer ops which provides a much larger time difference in cycles between a cache hit and a cache miss/memory to enable more robust cache timing attacks. Today’s large multi-core server CPUs with elaborate mesh fabrics to connect tens or hundreds of cores exaggerates this. Addition of performance enhancing features for fine granularity cache control such as the x86 CLFLUSH and PREFETx give more control for cache timing attacks. Wider issue processors that enable parallel integer, floating, and memory ops simultaneously. One could place long floating point operations such as divide or sqrt right before the faulting instruction to keep the core busy, but still keeping the integer and memory pipelines free for the attack. Since the faulting instruction will not nuke the pipeline until retirement, it has to wait until any earlier instructions in the instruction sequence to be committed including long running floating point ops. Virtualization and PaaS. Many web scale companies are now running workloads cloud providers like AWS and Azure. Before cloud, Fortune 500 companies would run their own trusted applications on their own hardware. Thus, applications from different companies were physically separated, unlike today. While it is unknown if Meltdown can allow a guest OS to break into the hypervisor or host OS, what is known is that many virtualization techniques are more lightweight than full blown VT-x. For example, multiple apps in Heroku, AWS Beanstalk, or Azure Web apps along with Docker containers are running within the same VM. Companies no longer spin up a separate VM for each application. This could allow a rogue application to read kernel memory of the specific VM. Shares resources were not a thing in the 90’s when OoO execution became mainstream with the Pentium Pro/Pentium III. The use of the Global and User/Supervisor bits in x86 paging entries which enables the kernel memory space to be mapped into every user process (but protected from Ring3 code execution) to reduce pressure on TLBs and slow context switching to a separate kernel process. This performance optimization has been done since the 1990’s. Is this x86 specific? First of all, cache timing attacks and speculative execution is not specific to Intel or x86 CPUs. Most modern CPUs implement multi-level caches and heavy speculation outside of a few embedded microprocessors for your watch or microwave. This isn’t an Intel specific problem or a x86 problem but rather a fundamental problem in general CPU architecture. There are now claims that specific OoO ARM CPUs such as those in iPhones and smartphones exhibit this flaw also. Out of order execution has been done since being introduced by Tomasulo algorithm. At the same time, cache timing attacks have been known for decades as it’s been known stuff may be loaded into caches when it shouldn’t have. However, cache timing attacks have traditionally been used to find the location of kernel memory rather than the ability to actually read it. It’s more of a race condition and window that is enabled depending on the microarchitecture. Some CPUs have shallower pipelines than others causing the nuke to happen sooner. Modern desktop/server CPUs like x86 have more elaborate features from CLFLUSH to PREFETCHTx that can be additional tools to make the attach more robust. Background on Memory Paging Since the introduction of paging to the 386 and Windows 3.0, operating systems have used this feature to isolate the memory space of one process from another. A process will be mapped to its own independent virtual address space which is independent from another running processes’s address space. These virtual address spaces are backed by physical memory (pages can be swapped out to disk, but that’s beyond the scope of this post). For example, let’s say needs 4KB of memory thus the OS allocates a virtual memory space of 4KB which has a byte-addressable range from 0x0 to 0xFFF. This range is backed by physical memory starting at the location 0x1000. This means processes 1’s is “mounted” at the physical location . If there is another process running, it also needs 4KB, thus the OS will map a second virtual address space for this _Process 2_with the range 0x0 to 0xFFF. This virtual memory space also needs to be backed by physical memory. Since process 1 is already using 0x1000-0x1FFF, the OS will decide to allocate the next block of physical memory [0x2000-0x2FFF] for Process 1 [0x0-0xFFF] [0x1000-0x1FFF] Process 2. Given this setup, if Process 1 issues a load from memory to linear address 0x0, the OS will translate this to physical location 0x1000. Whereas if Process 2 issues a load from memory to linear address 0x0, the OS will translate this to physical location 0x2000. Notice how there needs to be a translation. That is the job of the page tables. A range of mapped memory is referred to as a . CPU architectures have a defined page size such as 4KB. Paging allows the memory to be fragmented across the physical memory space. In our above example, we assumed a page size of 4KB, thus each process only mapped one page. Now, let’s say _Process 1_performs a malloc() and forces the kernel to map a second 4KB region to be used. Since the next page of physical memory [0x2000–0x2FFF] is already utilized by , the OS needs to allocate a free block of physical memory [0x3000–0x3FFF] to (Note: Modern OS’s use deferred/lazy memory allocation which means virtual memory may be created before being backed by any physical memory until the page is actually accessed, but that’s beyond the scope of this post. See x86 Page Accessed/Dirty Bits for more). page Process 2 Process 1 The address space contiguous to the process but in reality is fragmented across the physical memory space: appears Process 1 Process 2 There is an additional translation step before this to translate logical address to linear address using x86 Segmentation. However, most Operating Systems today do not use segmentation in the classical sense so we’ll ignore it for now. Memory protection Besides creating virtual address spaces, paging is also used as a form of protection. The above translations are stored in a structure called a . Each 4KB page can have specific attributes and access rights stored along with the the translation data itself. For example, pages can be defined as read-only. If a memory store is executed against a read-only page of memory, a fault is triggered by the CPU. page table Straight from the x86 Reference Manual, the following list of attribute bits (which behave like boolean true/false) are stored with each page table entry: non-exhaustive Minimizing context switching cost We showed how each process has it’s own virtual address mapping. The kernel process is a process just like any other and also has a virtual memory mapping. When the CPU switches context from one process to another process, there is high switching cost as much of the architectural state needs to be saved to memory so that the old process that was suspended can resume executing with the saved state when it starts executing again. However, many system calls need to be performed by the kernel such as I/O, interrupts, etc. This means a CPU would constantly be switching between a user process and the kernel process to handle those system calls. To minimize this cost, kernel engineers and computer architects map the kernel pages right in the user virtual memory space to avoid the context switching. This is done via the access rights bits. The OS maps the kernel space but designates it as (a.k.a. Ring0) access only so that any user code cannot access those pages. Thus, those pages appear invisible to any code running at user privilege level (a.k.a. Ring3) User/Supervisor supervisor while running in , if a CPU sees a instruction access a page that requires supervisor rights, a page fault is triggered. In x86, page access rights are one of the paging related reasons that can trigger a . user-mode #PF (Page Fault) The Global bit We showed how each process has it’s own virtual address mapping. The kernel process is a process just like any other and also has a virtual memory mapping. Most translations are private to the process. This ensures _Process 1_cannot access ’s data since there won’t be any mapping to [0x2000–0x2FFF] physical memory from . However, many system calls are shared across many processes to handle a process making I/O calls, interrupts, etc. Process 2 Process 1 Normally, this means each process would replicate the kernel mapping putting pressure on caching these translations and higher cost of context switching between processes. The Global bit enables these certain translations (i.e. the kernel memory space) to be visible across all processes. Closing thoughts It’s always interesting to dig into security issues. Systems are now expected to be secure unlike in the 90’s, AND is only becoming much more critical with the growth of crypto, biometric verification, mobile payments, and digital health. A large breech is much more scary for consumers and businesses today that in the 90’s. At the same time, we also need to keep discussion going on new reports. The steps taken to trigger the Meltdown vulnerability have been proven by various parties. However, it probably isn’t the of having speculation and cache timing attacks that caused Meltdown nor is a fundamental breakdown in CPU architecture, rather seems like a logical bug that slipped validation. Meaning, speculation and caches are not going away anytime soon, nor will Intel require an entirely new architecture to fix Meltdown. Instead, only change needed in future x86 CPUs is a few small gate changes to the combinational logic that affects determining if a hit in L1D$ (or any temporary buffers) is good. mere act

A technical deep dive on Meltdown and does it work?

188 Stories To Learn About Analytics

Let's learn about via these 53 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Apis /Learn Repo 1. Public ASX100 APIs: The Essential List We’ve conducted some initial research into the public APIs of the ASX100 because we regularly have conversations about what others are doing with their APIs and what best practices look like. Being able to point to good local examples and explain what is happening in Australia is a key part of this conversation. 2. How To Monitor Your API Server 'Have you developed or are you in the process of creating an API Server that will be used on a production or cloud environment? In this 4th instalment of my Node JS Performance Optimizations series, I show you how to test the availability of your API Server, so that you can understand how many requests per second it can handle whilst performing heavy duty tasks. 3. Get Your Postman Workspace Organized Learn tricks for Postman that ease your Workspace organization! 4. A Guide to Delivering a Low-code API This guide will take you through the steps to build and host an API using a low-code platform. 5. How To Build An API Without Coding [Python] I’ve spent the last 10 years of my life as a System Engineer, which means when I first had to build an API, I had no idea how to do it. I knew python quite a bit since as a SysAdmin I often had to write some smaller or bigger scripts, but building the whole API was a challenge for me. Sure — I could just google the phrase and find dozens of tutorials on how to build an API, but all of them were somehow complicated or created for raw developers (which means — with assumptions that you know already all the tools and frameworks that developers use). 6. My Experience with GraphQL 7. Working with Salesforce APIs? Of CORS! In this post, we’ll take a closer look at how Cross-Origin Resource Sharing,/CORS, operates. By working with a small Node.js app that interacts with salesforce. 8. Why API Designs are Stuck in the Past The industry has embraced statically typed languages, but API design remains twenty years in the past. Schema driven development presents an opportunity to pull API design into the present. 9. 4 Standard Business Models for API Monetization API has been rapidly driving new opportunities for business in last few years. Read in details how top API monetisation models helps in business growth. 10. How to Introduce Collaboration into your API Development Lifecycle We all know that APIs have become foundational to how organizations streamline integration and build new applications. In essence, they are the connective tissue between different systems, data, and applicationlayers. 11. 51% of Business Leaders Believe They Will Be Left Behind If They Don't Innovate Learn what IT leaders are prioritizing one year into the COVID-19 pandemic, per new research on the adoption of microservices, open-source tech, and more. 12. Creating Developer-Friendly Data Services with Apache Cassandra Using data APIs and advanced data modeling will make it far easier for JSON-oriented developers to connect to Apache Cassandra. 13. How To Define HTTP Middleware and Best Practices In order to capture API calls from arbitrary environments, we had to create middleware for many of the common web API frameworks. Here's what we learnt. 14.  How Do APIs Provide a Foundation for Software Innovation Get a comprehensive Guide about Application Programming Technology or API  in a layman's language. 15. Exploring the API-First Design Pattern Learn how the API-first design pattern is a carbon copy of the successful writing approach that John Vester has leveraged for several years. 16. How To Launch A New Developer Platform That's Self-Service Software Enterprise software companies sell very differently today than just a decade ago. Previously, most software was shrink-wrapped, required a lot of effort to distribute and implement, and was sold to an executive who would have to deploy it throughout their department or organization. The buying process involved a long sales cycle and often included pilots, cost-benefit analyses, procurement, and legal reviews. 17. Here is the Fastest GraphQL Playground It's been 2 months of Firecamp's GraphQL playground launching, throughout this time I have been discussing with so many folks about their GraphQL usage and experience with Playground. 18. Introduction to Containers, Microservices on Kubernetes Containers and Microservices have become the most popular way for new software deployments, and for application re-factoring, such as twelve factor apps. 19. Low-code: Solving the Dark Arts of API Deployment Creating APIs may be easy, but maintenance and deployment is more often than not, time-consuming and difficult. But it doesn't have to be. 20. How We Delivered New Services Without Building New Services Our API serves more than 300 million API calls per year. We used an open source integration platform to deliver new services without building from scratch. 21. API Integration in 2021 and a Look at Insights from 2020 What's next for API integration? Share your thoughts in Cloud Elements' industry survey to win prizes and contribute to the industry-renowned report. 22. How to Design API Analytics Data Collection for High Volume APIs API Analytics are key for any platform company that wants to gain insights into their API and platform usage. These insights can be leveraged by product owners, growth teams, developer relations, and more to make more strategic decisions based on the raw health of your platform business rather than just gut feelings. However, many API platforms have a very high volume of API calls per day. This volume creates a set of unique challenges when designing an analytics system that scales without crippling their platform or having sticker shock when looking at their cloud vendor’s bill. This post goes into some of the inner workings of how we designs Moesif’s API analytics platform to handle companies with billions of API calls a day. 23. An Intro to APIs for Beginners In layman's terms, API is a messenger. API sets the rules for communication. 24. 54% of Developers Cite Lack of Documentation as the Top Obstacle to Consuming APIs Recently, I worked with my team at Postman to field the 2020 State of the API survey and report. We’re insanely grateful to the folks who participated—more than 13,500 developers and other professionals took the survey, helping make this the largest and most comprehensive survey in the industry. (Seriously folks, thank you!) Curious what we learned? Here are a few insights in areas that you might find interesting: 25. 7 Companies with Recipe APIs in 2022 An overview of 7 Companies with recipe APIs in 2022 26. Solving for Real ‘Real-Time’ API Monetization The API Monetization Platform under consideration had an option of choosing between Pay-Per-Use (PPU), Pay-Fixed-Rate (PFR), lease-based, subscription-based and 27. Stoplight's Open-Source October: Hacktoberfest Involvement & Pull-Requests for Trees Complete an open-source pull request from one of our open-source tools, & we will plan 400 trees in your honor to celebrate Open-Source October & Hacktoberfest. 28. How to Choose the Best API Integration Platform Discover the most important things to consider when choosing an API integration platform for your B2B SaaS company. 29. 5 Major Factors Impacting the Evolution of APIs In today’s world of ever-increasing digital interconnectivity, APIs (Application Programming Interfaces) have emerged as essential tools for  integrating data efficiently and cost-effectively. 30. Building Maintenance Mode for your API Gateway Using Terraform In my previous blog post, I went through how to create a maintenance mode page for your application, and how to implement it using Terraform and github pages. But the website is just one part of an Application, and often there’s also a public (or private) API that also needs to have a maintenance mode. Let’s see how we can do that using Terraform on API Gateway. 31. Another approach to design API for improved performance! GraphQL is an amazing specification, no doubt about that. I loved it when it first came into my knowledge but as I started learning it, well, I started to hate it and I still do at some extent because I am still learning it. So much boilerplate code to even starting the graphQL server. 32. Open banking and APIs Can Work Wonders for Crypto Startups Before the pandemic, the crypto startup scene had a nemesis: lacking demand. In 2021, this changed radically—the number of people leveraging cryptocurrencies reached almost 300 million, and venture capitalists invested a record $25.2 billion in blockchain companies, an eight-fold increase from the previous year, according to a new CB Insights report. 33. API Choice Overload With so many approaches to proper API design, development and management, it’s pretty easy to get lost just in the planning process. 34. How to Build Email Lookup Extension for Chrome HazelBase is a powerful identity network that collects data from the internet and provides users with a wealth of information. 35. Learn How to Build an HTTP API with Rust In this article, I'll write about the developer experience of developing the above application in Rust, compared to what I'm used to with Spring Boot. 36. My Best Travel APIs APIs are useful tools in general but can be made exceptionally useful when combining a number of them to accumulate information to help you plan trips. 37. Never Make Assumptions: 5 Golden Rules for Web App Development No matter if you are a startup company or a big enterprise, your app needs to be intuitive and customer-facing to win many hearts. However, designing a user-friendly web app for your business is not so easy. You need to be aware of the complete rules. 38. What's New in WSO2 API Manager 3.2.0? WSO2 API Manager 3.2.0 the third release of the API Manager 3.x series, simplifies integrating with other ecosystems while reducing complexity of large scale deployments. 39. 8 Most Popular Weather APIs for 2020 The climate is a hot issue and weather forecasting technology is suddenly cool. The urgent need for innovative weather and environmental conditions forecasting solutions is obvious whether you believe in climate change or not. 40. What Is The True Cost of Using Public APIs How ubiquitous are APIs in today’s development processes? Try asking an engineer how many APIs their project integrates. Most teams won’t know the answer. From analytics tools to maps and cloud hosting, modern applications use a hefty collection of internal and public APIs. Developers use these to quickly assemble applications that would otherwise take much more effort to build. However, there’s a forgotten expense not typically calculated early in a project. 41. Top Tools for Rapid API Development From design to deploy, these tools will have your API and microservices up and running in no time. 42. How & Why We Built a Git Workflow to Accelerate API Testing With the release of ReadyAPI v3.3.0, our all-in-one collaborative API quality platform, the team here at SmartBear completed the first round of work on a rethought Git workflow – all available directly within the tool. 43. Great Tools To Help You Build an Awesome API Learn more about API-FIRST and how it can deliver practical solutions to today's modern IT environment. 44. A Simple Guide to API Development Tools APIs can be as simple as 1 endpoint for use by 100s of users or as complex as the AWS APIs with 1000s of endpoints and 100s of thousands of users. Building them can mean spending a couple of hours using a low-code platform or months of work using a multitude of tools. Hosting them can be as simple as using one platform that does everything we need or as complex as setting up and managing ingress control, security, caching, failover, metrics, scaling etc. 45. What’s the Difference Between APIs and Webhooks? An API (Application Programming Interface) enables two-way communication between software applications driven by requests. A webhook is a lightweight API that powers one-way data sharing triggered by events. 46. GraphQL, Streaming, and RESTful API development Come Together in this Open Source Software Swell is a cross-platform compatible (Mac, Linux, & Windows) desktop application and the leading API dev tool for testing and visualizing steaming TCP connections (ie WebSockets, SSEs, GraphQL subscriptions). Swell functions similarly to Postman, but with the added perks of full streaming and GraphQL support. Being an Open Source project, Swell is 100% free and open to outside contributions. 47. How to Deploy Java GraphQL APIs Using Apifi — A Quick Start Guide A Quick Start Guide to Getting a Java GraphQL API up and Running in no time Using Apifi. 48. API Development in the Time of COVID-19 Let’s face it. 49. REST API Tools Smackdown Introduction 50. A Brief Guide to Everything You Need to Know About APIs In this article, we will be discussing in-depth APIs and web services. Also learn the basics of making an API call, its vulnerabilities, and the future. 51. 13 API Metrics That Every Platform Team Should be Tracking A list of the most important API metrics every API product manager and engineer should know, especially when you are looking into API analytics and reporting. 52. What is REST APIs? In this blog, I am going to discuss REST API, what is Rest API, what is the use of REST API and how it works. So I hope you will get some idea about the REST API after reading this blog. 53. An API-First Approach For Designing Restful APIs I’ve been working with Restful APIs for some time now and one thing that I love to do is to talk about APIs. Thank you for checking out the 53 most read stories about Apis on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

53 Stories To Learn About Apis

And this is according to research from 2020. So, it’s probably even more today. It would be cool if someone did research on how many API-as-a-service companies got founded in the last decade. I’m betting that number is increasing exponentially. 83% of the internet is API! With the of API-as-a-service companies, API-driven platforms and applications, it is essential for organizations to have a simple-to-use, centralized platform/tool in place to monitor and manage these services. rise API-centric and General Monitoring Tools I’ve been looking extensively at API monitoring and tools for the past 2 years and I’ve noticed that many tools are basically, something else first 😅 and then they also have some API-centric metric or (usually) a way to test your APIs through checkpoints from all over the world. Not many differentiators there I must admit. I’ve decided I shall call these ones observability “General Monitoring Tools”. For some of you this is maybe the way to go, depending on your need. However, as I am writing this piece of content my colleague just sent me the news of how 😲😬. Data from 37 million accounts stolen! API increased by a staggering 681% last year alone. T-Mobile got hacked through API data breach attacks I am not trying to bash any of these tools, however, threats to your APIs are real and maybe (just maybe) metrics like uptime and you running your own test isn’t good enough anymore. To actually know that something like this is happening you’ll need a more API-centric tool that can tell you what is actually happening so that you can react faster and on time. So, the first part of my list will have some of the tools that can help you with that since some of them can provide you with real-time API metrics generated by your users. Here are my picks (there are more, so don’t hesitate to ask me for more if need be). 9 API-centric and API Observability Tools/Platforms Some of these are a bit older, but you know what they say: an oldie but a goodie. However, recently some tools emerged answering the call for more observability and transparency. In these “hAPI Hacker” times, it is a good time to consider using one of those. The first 3 tools on this list are more focused on answering this requirement, the rest is still API-centric, but offer more of the traditional things engineers are used to following, like uptime, speed and such. 1. Treblle Treblle is an all-in-one platform for the entire API lifecycle. A very intuitive UI gives value to almost everyone who needs to work with APIs. Advanced API metrics, enriched data at a glance and even a helpful hand to grade your REST API. Auto-detection of endpoints, and real-time API traffic with enriched data points that give you useful metadata about who, how and from where is your API being accessed, auto-generated docs and more. It’s a premium-looking product with accessible pricing for everyone and it’s basically brand new (founded in July 2021). Integration is simple and if you are versed in the language of your choice it shouldn’t take you more than 4-5 minutes to have this puppy up and running with virtually no impact on your API. It has great apps! Treblle is also the only one on this list with super-optimized native apps for iOS, Android and Windows, so you can monitor your APIs and get notified about potential problems on the go. : Treblle has a free forever (no credit card required) account that gives 250K requests that renew every month and they even award you with extra requests if you integrate within 24 hours of signing up. Pricing If you need more, there are also team and enterprise options. A great bonus with apps is that you can run local tests without them being counted as a request from your plan. So, test away as much as you love! 2. Akita Software This is an interesting tool in beta. Similarly to Treblle, it detects your endpoints and proceeds to paint a picture about your API performance from there. You can have your eye on everything that is happening in the dashboard, however, it doesn’t provide you with as many data points and analytics as Treblle does. However, it is in beta, so if you just want to dip your toes in the water of this kind of observability, then this might be a way. If you are serious about API your endpoints, then this is a tool that should enter your consideration. observability and monitoring 3. Moesif Moesif is another API analytics platform that can help you analyze how your API is being used. Features that can help you do this are funnel analysis, web analytics, behavioral emails and behavioral cohorts. Similar to RapidAPI, Moesif also offers a solution for monetizing your APIs. So, analytics come in handy if you are looking to grow and cash in. : Moesif offers a 14-day free trial. As with Treblle, there is no credit card required. You get 30K events for maximum of 1 user on the free options. Pricing 4. API Science API Science is another modern API monitoring tool that understands REST, JSON, OAuth and XML. It works by setting up monitors where you set up your tests. It’s a good tool for trying out different scenarios where the tool lets you test up to every 1 minute from around the world. The user experience is great and the interface is intuitive and easy to understand. What is also great is that API Science is a great tool if you rely on 3rd party APIs. You can easily keep track of any API in your “API supply chain” by setting up custom monitors. : Pricing starts from $20 for a Starter package where you get 30K API calls to work with. The largest plan offers 750K API call for $500. Pricing 5. Assertible Assertible seems to be very straightforward in what it offers. It’s (again) all about testing and making sure your API is delivering. The tool provides you with simple but powerful assertions to test and monitor your websites and APIs. : Similar to Treblle, Assertible also offers a free forever plan. It seems to be going for smaller teams and is currently not aiming at enterprises. Pricing 6. Postman Postman is probably the most recognizable name on this list with over 20M developers actually using the product in one way or the other. The most popular way of using Postman is for API testing, which you can do with their desktop app. The app itself is fine. However, it could be a little bit more optimized for my taste. Other than the most popular way to use Postman the platform itself comes with a set of tools that can cover your API design and API governance needs. There is a free option that comes with core tooling and can be used by up to 3 users, all the way up to enterprise level where the pricing becomes $99 per user. Pricing: 7. RapidAPI RapidAPI is probably the best place to learn a lot about how to use, build and ship APIs at the moment. They own the world’s largest API Hub where you can find, connect and manage APIs. RapidAPI offers a variety of products depending on your needs, however in my opinion they are more suited for larger teams if you are looking for a tool to help you manage your APIs. : RapidAPI does have a free trial. It is not super straightforward on the rest of the pricing model. However, this is the platform where you can actually monetize your API product and I suspect this is where RapidAPI also gets some of its revenue from. Pricing 8. Checkly Checkly is definitely more on the modern side of things when it comes to API monitoring. It still requires some setup to show its full potential, however, it might be worth the time. With Checkly you can automatically collect error traces, screenshots and performance metrics with every browser check you run. : There is a free forever option that gives you a 14-day free trial of team option. Pricing 9. Runescope Runescope is another tool that emphasizes testing and uptime. However, they are.. a bit more API-centric than some of the other tools on this list. Similar to some features on Treblle it can tell you information about the response size, speed and if the return payload was correct. Runescope offers free trial for its plans that start from $79 for 250K requests. Pricing: 11 General Monitoring Tools you can use for more traditional API monitoring These are mostly tools that are not new, with most of them being used by enterprises with big that prefer all of their monitoring requirements in one place. That is of course convenient. DevOps and engineering teams It becomes inconvenient when the same tool simply can’t or doesn’t have the power to show you advanced API statistics that are becoming more and more important. Especially if we are talking . security Nevertheless, these are still note-worthy and quite big and famous tools and platforms. 10. Saucelabs If all you do is test everything, every day, as much as you can, then this might be the sauce you need with your meal. Saucelabs covers much of what you might need like mobile app testing, cross-browser testing,  UI/Visual Testing and of course API testing. There are also options depending on your needs, so you can do continuous testing, automated or live. : For startups and medium teams the pricing starts from $39 up to almost $5K (depending on how many parallel tests you need.) Pricing 11. Sematext Synthetics Sematext offers synthetic monitoring for your APIs and Websites. The key word here is uptime, which is sometimes really the only information you are looking for. So if you need to know info about the availability of your website, APIs and critical web transactions, then this might be a solution to consider. Similarly to API Science, it requires you to set up designated checkpoints that will run the tests you want learnings from. : Pricing page looks a bit complicated, but it seems there is a free option for monitoring, although it is mainly for infrastructure monitoring, not just APIs. Pricing 12. Dotcom-Monitor Dotcom-Monitor is an overall great platform with a little bit of something for everybody. If you are mainly looking to see what is happening with your API, then you’ll look at Dotcom’s Load/Stress Testing solution. This is a real browser-based load testing for websites, web apps, APIs and streaming media. : Dotcom offers a free trial period of 30 days. It’s not super straight forward on what the pricing actually is after that. Pricing 13. DataDog Alongside Postman, probably the most recognizable brand on this list. Both have been around for a while. As a platform, it offers a lot and in enterprise-level DevOps teams is where it probably brings the most value. Can it show deep and enriched data about API requests automatically like say Treblle or can you learn all about APIs like with RapidAPI? Probably not! However, the main strong argument for DataDog is the centralization of everything that needs to be monitored, including APIs. It just depends on how much data is enough for your team to be efficient. : I am not even going to try to get into this. There are however free trials for whatever you might want to try out. Pricing 14. Better Uptime This is mainly an monitoring tool. It does check your API among other things (hence the name). It has an interesting approach to how it logs errors on an API. It records the response message and basically screenshots the webpage. This could be helpful to someone, however, there are tools that can tell you more actionable details. infrastructure : There is a free forever option that gives you 10 monitors and 5 status pages and it also has enterprise solutions ready. Pricing 15. Prometheus Prometheus is an open-sourced monitoring solution, usually used alongside Grafana (a little bit further down the list). It’s a very specific tool with a lot of integration options, so depending on your needs it might be worth your time to take a look. Integration time might be an issue if you need some information fast. Integrating this kind of solution usually takes up time, and on top of that, there will most probably be a on how to actually use it. learning curve 16. Graphite This is a very interesting solution that on top of getting valuable information for you, also allows you to get it in graph visuals for your website. I wouldn’t say this is the way to go if you are serious about API monitoring, but if you just need some simple and fast data that you can show, then Graphite might be a fun tool to use. 17. Uptrends Uptrends is another tool that gives you a solution for uptime, , web performance and API monitoring. When it comes to API monitoring, with Uptrends you can build out multi-step monitoring and run tests from 231 global checkpoints. web app : There is a 30-day free trial with no limitations, after which you can choose the plan that suites you the most, starting from $17.5/month Pricing 18. App Dynamics This Cisco product is advertised as full-stack observability for apps. Although it does bring some interesting solutions, and it does visualize infrastructure performance, I’d say in terms of monitoring APIs, it is similar to most tools. However, if you are a part of an app development team, this tool might be an interesting one to consider. The website gives you the option to get a quote, so I don’t have a disclosed price range, unfortunately. Pricing: 19. ReadMe ReadMe is primarily a tool that helps you with your API docs by essentially turning the into interactive developer hubs. However, among other features, they also have API metrics that can help your developers understand what is happening with the API. : On a free plan you get 1M API logs, which sounds great, however it is restricted to one project. If you want more, you’ll need to refer to one of the paid plans, starting from $99. Pricing 20. Grafana Grafana is too big not to be mentioned. Although Grafana offers a lot of possibilities and integrations, to me it also looks like a trap. It takes forever to integrate and requires continuous setup and learning, figuring out how to get what you want and so on. Big companies stick to the software even if it doesn’t cover all of their needs because of all of the work that went into setting it up in the first place. : This will depend on how and what you’ll use so it stands to reason that there isn’t any straight forward pricing for Grafana. Pricing 21. Open Telemetry OT is essentially a of APIs, SDKs and tools that you can use to get data you are interested in (like API metrics). It is free and , so if you are up to the task of finding what works best for you and you have a way to visualize the data, then maybe Open Telemetry is a way to go. collection open source Wrap Up So there you have it folks. Like I said at the start, there are more tools out there, and if you think it’s worth your time and resources you can always try to build something instead of taking something off the shelf. However, most of the things on the list have a free trial, and some come with free forever plans, so I would recommend trying some of those before you decide to commit to building. If you are already using something, 2023 might be a good year to make a switch to tools that will ensure your security and provide you with advanced observability which can position you ahead of your competitors. Whatever you decide, stay hAPI!

Stories, News & Mentions about MOESIF

Moesif’s Stories on HackerNoon