OpenRent is the way people rent property online

OpenRent

Lua-Land CPU Flame Graphs in OpenResty XRay

Selenium has been the de facto automation standard/tool for browser/UI automation for quite sometime. Creators of selenium also created selenium grid for scaling the UI testing. The grid consists of a selenium hub and several selenium nodes which connects/hosts the browsers and thus providing a way to run several UI tests simultaneously. Selenium also provides some easy way to customize/extend existing capabilities. For most of the use cases the selenium hub /node based architecture scales pretty well and doesn’t need much customization. Need for higher availability and scale The architecture, although supports scaling the selenium nodes, it doesn’t make it easy to scale the hub itself. On a typical deployment there would be a single hub and several nodes attached to it. The mapping of sessions/browser instances is maintained in memory inside the hub. So, if the hub fails, the whole grid will become unusable. A simple way to enable some level of HA is to run at least 2 hubs. Challenge with multiple hubs Requirement of sticky session Since selenium hubs can’t talk to each other, the distribution of load to different hubs need some kind of stickiness. Stickiness here needs to make sure that once the session is created on one hub, all subsequent request for that specific session comes to the same hub. Session stickiness is not a new thing. A typical stickiness for browser based clients is done via cookies. The LB looks at the cookie information being passed to decide which upstream server the request would be forwarded. Cookie based stickiness ruled out Passing this extra bit of information using cookie is not possible in selenium automation scenarios, so this was not an option for our use case. Client ip based stickiness One option that we have been using so far is to create stickiness based on client ip. So all calls originating from one client ip will always land to a given hub. While this works for most of the scenarios, the load on hubs are not distributed evenly. This particular technique solves the HA problem as such, but this may not be a great solution for scaling if the load is not even. Uneven load and uneven test cases in test suites A large test suite with huge number of UI tests can completely chock the one off hub where all tests are landing. For making our test infrastructure more reliable and robust, we may have to explore some different avenue. Openresty to the rescue Openresty is a software bundle of nginx with some high quality Lua libraries included. It allows you to customize load balancing behavior and add custom behaviors. Design We can add openresty as a proxy layer before the hub. We can deploy couple of instances of openresty in front of hub and the LB can forwards the request to one of these openresty instances in a round robin fashion ( earlier the LB was forwarding the request to hub directly using the client ip hashing). Openresty instances works as a reverse proxy and all instances of hub are upstream server for each openresty instances. The Magic The way the request forwarding works is: Each new session request will be forwarded to one of the hub in a round robin fashion. Once the session is created at hub and response comes back to the openresty proxy instance, it inspects the response body, looks for session id and appends the upstream host ip(host ip in number format) to the existing session id and responds back to the client. The client receives a new session id which has the upstream hub node embedded in it. Any subsequent operation on that session from client uses this session id in the url path, which now has the upstream hub’s ip embedded in the session id. The proxy inspects the request url and extracts the session id from the request. It rewrites the url path by removing the embedded host ip part. It also sets the upstream host to the specific host found in the ip section of the session id. Then it forwards the request to the desired hub. The hub sees the session id in the exact same format as it had generated and continues to process as usual. Sample nginx conf with embedded lua code: _Contribute to Mitendra/code_samples development by creating an account on GitHub._github.com Mitendra/code_samples Taking a step further While solving the scale and availability with openresty proxy, we can use this proxy for some more features built on top of Selenium hub. Rate limiting: Even though this would help in scaling the hub and nodes, there are scenarios where the Spike in number of new sessions may exceed the number of available resources. Currently all requests are put in the queue and processed when the resources are available which results into delayed session creation. Different test cases/frameworks have different timeout settings and while some of them proceed, many of them would get stuck. Many times, while the sessions get times out at client/test case layer, the hub still will go on and create a session. This unused sessions later gets cleaned up but it makes resource crunch even worse. To solve this we can add a rate limiting feature, where based on the capacity and traffic pattern only certain number of sessions in the queue would be allowed and beyond that we can return a http 429 status with some details like current active sessions and an estimated time after which it can be retried. Monitoring data: We can add a lot of monitoring data about sessions bring requested, time to create a new session, active session time etc from proxy layer, which can give further insight on how to optimize the infrastructure usage. References Image source: https://pixabay.com/en/skyscraper-highrise-building-tall-1209407/ Selenium: https://www.seleniumhq.org/ Selenium grid: https://www.seleniumhq.org/docs/07_selenium_grid.jsp Openresty: https://openresty.org/en/ _Embed the Power of Lua into NGINX HTTP servers. Contribute to openresty/lua-nginx-module development by creating an…_github.com openresty/lua-nginx-module _Embed the Power of Lua into NGINX HTTP servers. Contribute to openresty/lua-nginx-module development by creating an…_github.com openresty/lua-nginx-module

Scaling selenium with Openresty

How to Implement the Idempotency-Key Specification on Apache APISIX

Apache APISIX-এ আইডেমপোটেন্সি-কী স্পেসিফিকেশন কীভাবে প্রয়োগ করবেন

So implementieren Sie die Idempotency-Key-Spezifikation auf Apache APISIX

Cómo implementar la especificación de clave de idempotencia en Apache APISIX

Comment implémenter la spécification Idempotency-Key sur Apache APISIX

Apache APISIX पर Idempotency-Key विनिर्देश को कैसे लागू करें

Apache APISIX で Idempotency-Key 仕様を実装する方法

Apache APISIX에서 멱등성 키 사양을 구현하는 방법

Como implementar a especificação de chave de idempotência no Apache APISIX

Как реализовать спецификацию идемпотентного ключа в Apache APISIX

Idempotency-Key Belirtimi Apache APISIX'te Nasıl Uygulanır?

Cách triển khai Đặc tả khóa Idempotency trên Apache APISIX

如何在 Apache APISIX 上实现幂等密钥规范

In early 2021, I started to work on the project. I have to admit that I had never heard about it before. In this post, I’d like to introduce some Apache projects that are less well-known than HTTPD or Kafka. Apache APISIX Apache APISIX is an . It builds upon , a Lua layer built on top of the famous reverse-proxy. APISIX adds abstractions to the mix, , , , , and offers a plugin-based architecture. APISIX API Gateway OpenResty nginx e.g. Route Service Upstream Lots of plugins are provided out of the box: Transformation: , , gRPC, , etc. response-rewrite proxy-rewrite body-transformer Authentication: JWT, OPA, Keycloak, OpenID Connect, etc. Observability: metrics, logging, and traces Traffic: rate limiting, request validation, canary release, etc. Serverless: Azure functions, AWS Lambdas, OpenWhisk, etc. Messaging: Kafka, Dubbo, and MQTT Pre- and post-processing If no plugin fits your requirements, writing your own is possible. You can leverage APISIX on Kubernetes as an Ingress Controller. APISIX provides a Helm Chart for this. Apache ShardingSphere claims to offer an ecosystem able to transform any database into a distributed database system. It acts as a proxy between your code and your database(s). It comes in two flavors: ShardingSphere ShardingSphere-JDBC: a JDBC driver that acts as a proxy to your database(s). It’s only available for JVM-based applications. ShardingSphere-Proxy: a technology-independent deployable component. ShardingSphere offers several core features: Data Sharding is the core feature, as the project’s name implies. Most use cases focus on scaling purposes, but there are others, , requirements. e.g. data residency XA transactions for distributed transactions Read/write splitting Data encryption etc. Apache SeaTunnel is a data integration platform that offers the three pillars of data pipelines: sources, transforms, and sinks. It offers an abstract API over three possible engines: the Zeta engine from SeaTunnel or a wrapper around or . Be careful, as each engine comes with its own set of features. Apache SeaTunnel Apache Spark Apache Flink The power of SeaTunnel comes from its rich connector ecosystem. It does provide traditional SQL connectors, , Oracle, PostgreSQL, and MySQL, and NoSQL ones, , MongoDB, Cassandra, and Elasticsearch. However, it also comes bundled with some original ones, including Jira, Google Sheets, and Notion. I have a particular fondness for the CDC connector sources over MongoDB, MySQL, and Microsoft SQL Server. e.g. e.g. SeaTunnel comes with a web UI, which provides visual management of jobs, scheduling, running, and monitoring capabilities. Apache SkyWalking is an APM tool, focusing on microservices, Cloud Native apps, and Kuernetes architectures. It builds its architecture on four kinds of components: Apache SkyWalking collect telemetry data (metrics, logs, traces, and events). They support multiple output formats, including OpenTelemetry. Probes The aggregates and processes data platform The offers an interface over a supported backend. Supported backends include ElasticSearch, H2, MySQL, TiDB, and BanyanDB, a custom storage engine developed for SkyWalking storage Finally, a web allows visualizing SkyWalking’s data UI Skywalking supports a couple of formats, including OpenTelemetry. Given the industry’s current focus on OpenTelemetry, I recommend seriously considering this option. Apache Doris is a real-time data warehouse. Apache Doris Doris promotes four primary scenarios: Reporting analysis Ad-Hoc query Unified data warehouse construction Data lake query Doris is mostly MySQL compliant so that you can use a regular MySQL client. Discussion The Apache Foundation hosts the projects above, but they have another thing in common: they were all incepted in China. Have a look at the . You’ll probably be amazed at the sheer number; it’s close to 300! Apache project list In recent years, the number of projects incepted at the Apache Foundation has increased drastically. Look again at the list; I’m sure you only know a few of them - lots come from China. The trend is only growing; it’s a great move to integrate China with the OpenSource world! Just as I finish this post, my friend Stefano Fago has posted on another relevant project, , a streaming data lake platform. Apache Paimon To go further: Apache APISIX Apache ShardingSphere Apache SeaTunnel Apache SkyWalking Apache Doris Originally at A Java Geek on December 17th, 2023 published

Five Apache Projects You Don't (But Should) Know About

is an API Gateway, which builds upon the reverse-proxy to offer a plugin-based architecture. The main benefit of such an architecture is that it brings structure to the configuration of routes. It’s a help at scale, when managing hundreds or thousands of routes. Apache APISIX OpenResty In this post, I’d like to describe how plugins, priority, and phases play together and what pitfalls you must be aware of. APISIX plugin’s priority When you configure a route with multiple plugins, Apache APISIX needs to execute them in a order so that the results are the same over time. For this reason, every APISIX plugin has a . You can check a plugin priority directly in the code. For example, here’s the relevant code fragment for the plugin: consistent hardcoded priority basic-auth local _M = {
    version = 0.1,
    priority = 2520,                                 #1
    type = 'auth',
    name = plugin_name,
    schema = schema,
    consumer_schema = consumer_schema
} Priority is 2520 For documentation purposes, the file lists the priority of all out-of-the-box plugins. Here’s an excerpt: default-config.yaml plugins:
  - ldap-auth                      # priority: 2540
  - hmac-auth                      # priority: 2530
  - basic-auth                     # priority: 2520
  - jwt-auth                       # priority: 2510
  - key-auth                       # priority: 2500
  - consumer-restriction           # priority: 2400
  - forward-auth                   # priority: 2002
  - opa                            # priority: 2001
  - authz-keycloak                 # priority: 2000
  #- error-log-logger              # priority: 1091
  - proxy-cache                    # priority: 1085
  - body-transformer               # priority: 1080
  - proxy-mirror                   # priority: 1010
  - proxy-rewrite                  # priority: 1008
  - workflow                        # priority: 1006
  - api-breaker                    # priority: 1005
  - limit-conn                     # priority: 1003
  - limit-count                    # priority: 1002
  - limit-req                      # priority: 1001 Imagine a route configured with and . Because of their respective priority, would run before . proxy-mirror proxy-rewrite proxy-mirror proxy-rewrite upstream:
  - id: 1
    nodes:
      "oldapi:8081": 1

routes:
  - id: 1
    uri: "/v1/hello*"
    upstream_id: 1
    plugins:
      proxy-rewrite:                                 #1-3
        regex_uri: ["/v1(.*)", "$1"]
      proxy-mirror:                                  #2
        host: http://new.api:8082 The plugin ordering in this file is not relevant has priority 1010 and will execute first proxy-mirror has priority 1008 and will run second proxy-rewrite The above setup has an issue. For example, if we call , APISIX will first mirror the request, and then remove the prefix. Hence, the new API receives instead of . It’s possible to override the default priority to fix it: localhost:9080/v1/hello /v1 /v1/hello /hello routes:
  - id: 1
    uri: "/v1/hello*"
    upstream_id: 1
    plugins:
      proxy-rewrite:
        _meta:
          priority: 1020                             #1
        regex_uri: ["/v1(.*)", "$1"]
      proxy-mirror:
        host: http://new.api:8082 Override the default priority Now, has higher priority than : the former runs before the latter. proxy-rewrite proxy-mirror In this case, it works flawlessly; in others, it might not. Let’s dive further. APISIX phases APISIX runs plugins not only according to their priorities but also through dedicated phases. Because APISIX builds upon OpenResty, which builds upon ngxinx, the phases are very similar to the phases of these two components. nginx defines several an HTTP request goes through: phases NGX_HTTP_POST_READ_PHASE NGX_HTTP_SERVER_REWRITE_PHASE NGX_HTTP_FIND_CONFIG_PHASE NGX_HTTP_REWRITE_PHASE NGX_HTTP_POST_REWRITE_PHASE NGX_HTTP_PREACCESS_PHASE NGX_HTTP_ACCESS_PHASE NGX_HTTP_POST_ACCESS_PHASE NGX_HTTP_PRECONTENT_PHASE NGX_HTTP_CONTENT_PHASE NGX_HTTP_LOG_PHASE Each phase focuses on a task, , verifies that the client is authorized to make the request. i.e. NGX_HTTP_ACCESS_PHASE In turn, OpenResty offers similarly named phases. From nginx documentation: Finally, here are the phases of Apache APISIX: rewrite access before_proxy header_filter body_filter log We have seen two ways to order plugins: and Now comes the most important rule: ! by priority by phase. order by priority only works inside a phase For example, the plugin has a priority of 900 and runs in phase ; the plugin has a priority of 995 and runs in phase . Regardless of their respective priorities, will happen before , because happens before . Likewise, changing a priority won’t "move" a plugin out of its phase. redirect rewrite gzip body_filter redirect gzip rewrite body_filter The example above with and worked because both run in the phase. proxy-mirror proxy-rewrite rewrite The main issue is that priority is documented in the file, while the phase is buried in the code. Worse, some plugins run across different phases. For example, let’s check the proxy plugin and, more precisely, the functions defined there: config-default.yaml proxy-rewrite local function is_new_headers_conf(headers) local function check_set_headers(headers) function _M.check_schema(conf) function _M.rewrite(conf, ctx) The name of the function is suspiciously similar to one of the phases above. Looking at other plugins, we see the same pattern repeat. Apache APISIX runs plugin functions with the same name as the phase. rewrite() I took the liberty of summarizing all plugins and their respective phases in a table. +----------------------+---------+--------+-------------+--------------+------------+-----+
| Plugin               | Phase                                                            |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| General              | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| redirect             | X       |        |             |              |            |     |
| echo                 |         |        |             | X            | X          |     |
| gzip                 |         |        |             | X            |            |     |
| real-ip              | X       |        |             |              |            |     |
| ext-plugin-pre-req   | X       |        |             |              |            |     |
| ext-plugin-post-req  |         | X      |             |              |            |     |
| ext-plugin-post-resp |         |        | X           |              |            |     |
| workflow             |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Transformation       | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| response-rewrite     |         |        |             | X            | X          |     |
| proxy-rewrite        | X       |        |             |              |            |     |
| grpc-transcode       |         | X      |             | X            | X          |     |
| grpc-web             |         | X      |             | X            | X          |     |
| fault-injection      | X       |        |             |              |            |     |
| mocking              |         | X      |             |              |            |     |
| degraphql            |         | X      |             |              |            |     |
| body-transformer     | X       |        |             | X            | X          |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Authentication       | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| key-auth             | X       |        |             |              |            |     |
| jwt-auth             | X       |        |             |              |            |     |
| basic-auth           | X       |        |             |              |            |     |
| authz-keycloak       |         | X      |             |              |            |     |
| authz-casdoor        |         | X      |             |              |            |     |
| wolf-rbac            | X       |        |             |              |            |     |
| openid-connect       | X       |        |             |              |            |     |
| cas-auth             |         | X      |             |              |            |     |
| hmac-auth            | X       |        |             |              |            |     |
| authz-casbin         | X       |        |             |              |            |     |
| ldap-auth            | X       |        |             |              |            |     |
| opa                  |         | X      |             |              |            |     |
| forward-auth         |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Security             | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| cors                 | X       |        |             | X            |            |     |
| uri-blocker          | X       |        |             |              |            |     |
| ua-restriction       |         | X      |             |              |            |     |
| referer-restriction  |         | X      |             |              |            |     |
| consumer-restriction |         | X      |             |              |            |     |
| csrf                 |         | X      |             | X            |            |     |
| public-api           |         | X      |             |              |            |     |
| chaitin-waf          |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Traffic              | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| limit-req            |         | X      |             |              |            |     |
| limit-conn           |         | X      |             |              |            | X   |
| limit-count          |         | X      |             |              |            |     |
| proxy-cache (init)   |         | X      |             | X            | X          |     |
| proxy-cache (disk)   |         | X      |             | X            |            |     |
| proxy-cache (memory) |         | X      |             | X            | X          |     |
| request-validation   | X       |        |             |              |            |     |
| proxy-mirror         | X       |        |             |              |            |     |
| api-breaker          |         | X      |             |              |            | X   |
| traffic-split        |         | X      |             |              |            |     |
| request-id           | X       |        |             | X            |            |     |
| proxy-control        | X       |        |             |              |            |     |
| client-control       | X       |        |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Observability        | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| zipkin               | X       | X      |             | X            |            | X   |
| skywalking           | X       |        |             |              |            | X   |
| opentelemetry        | X       |        |             |              |            | X   |
| prometheus           |         |        |             |              |            |     |
| node-status          |         |        |             |              |            |     |
| datadog              |         |        |             |              |            | X   |
| http-logger          |         |        |             |              | X          | X   |
| skywalking-logger    |         |        |             |              |            | X   |
| tcp-logger           |         |        |             |              |            | X   |
| kafka-logger         |         |        |             |              | X          | X   |
| rocketmq-logger      |         |        |             |              | X          | X   |
| udp-logger           |         |        |             |              |            | X   |
| clickhouse-logger    |         |        |             |              | X          | X   |
| syslog               |         |        |             |              |            | X   |
| log-rotate           |         |        |             |              |            |     |
| error-log-logger     |         |        |             |              |            |     |
| sls-logger           |         |        |             |              |            | X   |
| google-cloud-logging |         |        |             |              |            | X   |
| splunk-hec-logging   |         |        |             |              |            | X   |
| file-logger          |         |        |             |              | X          | X   |
| loggly               |         |        |             |              | X          | X   |
| elasticsearch-logger |         |        |             |              |            | X   |
| tencent-cloud-cls    |         | X      |             |              | X          | X   |
| loki-logger          |         |        |             |              | X          | X   |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| Others               | rewrite | access | beforeproxy | headerfilter | bodyfilter | log |
+----------------------+---------+--------+-------------+--------------+------------+-----+
| serverless           |         | X      |             |              |            |     |
| openwhisk            |         | X      |             |              |            |     |
| dubbo-proxy          |         | X      |             |              |            |     |
| kafka-proxy          |         | X      |             |              |            |     |
+----------------------+---------+--------+-------------+--------------+------------+-----+ Conclusion I’ve detailed Apache APISIX plugin phases and priorities in this post. I’ve explained their relationship with one another. Icing on the cake, I documented each out-of-the-box plugin’s phase(s). I hope it will prove helpful. To go further: Default plugin priorities Apache APISIX plugin execution lifecycle Plugin execution code Originally published at , on December 10th, 2023 A Java Geek

Navigating Apache APISIX: A Strategic Approach to Plugin Priorities and Execution Phases

Let's learn about via these 74 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Observability /Learn Repo 1. So, How are Observability and Monitoring Different, Actually? The similarities and differences between monitoring and observability and how to pair the two strategies 2. Observability Tips and Tricks For Using Grafana and Prometheus Rapid prototyping tips when using Grafana and Prometheus observability and monitoring stack. 3. Observability and Monitoring in a nutshell A guide to understanding the concepts and differences between monitoring and observability. 4. Debugging AWS Lambda Runtime Errors Debugging AWS Lambda can get tricky in Amazon CloudWatch. KloudMate lets you get to Lambda errors quickly and effectively using Lambda error logs. 5. An Intro to Dynamic CDK Dashboards How to create a construct based on other constructs in the same stack without forcing the developer to call a method for each? That's exactly what aspects do! 6. N/A please delete this 7. Developer Trends from Thoughtworks’ Technology Radar 27 Thoughtworks’ Technology Radar is a regular time to take a look at what trends are changing our industry. 8. How Observability and Monitoring Produces Better Software An article focused on deep diving into observability and its significance in software. Its history, goals, the importance of observability, and the issues that 9. How to Build Your Own Observability Platform on Kubernettes End-to-end full observability using Odigos and other back-end observability tools. 10. Maintaining Quality When Transitioning from Monolith to Microservices Piece by piece, legacy monolith applications are being broken down and replaced by microservices. 11. How to Monitor Apache Flink with OpenTelemetry Apache Flink monitoring support is now available in the open source OpenTelemetry collector. 12. Observability vs Monitoring: What's the Difference? Monitoring has been a basic system to track the health of servers for years. Now it is not enough. 13. Observability and Monitoring Have a Symbiotic Relationship, but They Are Different Observability vs. monitoring, what is the difference? Monitoring is the what to observability’s why. Here we dig into the differences. 14. Microservice.add(observability) != Microservice.add(monitoring) You are reading this content, which means that you are not novice to the microservices field. So let me just  scratch the surface of it before moving to Observable Microservices.  Once upon a time Monolith Application was now transformed into Microservices based application. 15. How Do You Make Cloud Migration Easier? Engage in intelligent change. 16. Application Monitoring: Closing Observability Gaps with Custom Metrics Which application metrics should you collect for your microservices? 17. The Challenges of Modern Observability It’s tough, impossible, to get a holistic view of your entire application when it’s running on 200 pods deployed to 12 nodes distributed across the world. 18. Why OpenTelemetry Should Matter to Network and Systems Admins OpenTelemetry is young, even by internet standards. Born out of the merger of OpenTracing and OpenCensus projects at the Cloud Native Computing Foundation CNCF 19. What is the ideal memory size to lower costs of running a task on Lambda? Should you increase Lambda memory? It might sound crazy, but increasing your AWS Lambda memory could actually lower your bills. Find out how. 20. What "Shifting Left" in Software Really Means for Blameless DevOps A truly "blameless culture" in software must to evolve from incident reporting, to telemetry aimed at Proactive Observability in DevOps 21. CI/CD/CF — The "Missing-Link" in the DevOps Toolchain? Everyone is familiar with CI and CD processes, but whatever happened to Continuous Feedback? Leveraging observability in dev creates a new type of dev process 22. Why Your Monitoring Dashboard May Be Feeding You Phantom Metrics We trust our metrics to show us the status of our system and where it misbehaves. But do our metrics show us what really happened? 23. Cloud Native Applications: 4 Building Blocks You Need in Your Platform To create amazing cloud native applications your platform needs these components: Kubernetes, Continuous Delivery Pipelines, Edge Stack, and Observability. This article will explain the importance of each component. 24. Comparing Different Serverless Monitoring Platforms Technology touches almost every corner of the world economy. Even when it’s an indirect relation, in many cases tech is an essential, vital part of our societies. It just can’t fail without causing too much distress and losses. Not only financially, but especially to the human aspect. 25. Get Started With Sidekick Open Source Live Debugger in 5 Mins Sidekick is a live application debugger that lets you troubleshoot your applications while they keep on running. Here is how you can start using it in 5 minutes 26. Logging in Observability - Part 1 rsyslog vs journald vs filebeat 27. Managing Collectors on K8 – Why We Chose the OpenTelemetry Collector for Odigos Without proper implementation and continuous configuration of your collectors, observability tools will be limited at best at best and many times ineffectual. 28. On Dynamic Observability and Team Culture with Liran Haimovitch, Rookout CTO Rookout Co-Founder and CTO, Liran Haimovitch, shares the origin story of their debugging tool, what excites him about the startup life, PLG, and more. 29. 9 Data Trends You’ll See in 2023 2022 saw the data space grow by leaps and bounds. Here are the top 9 things our team of data experts expects to see in 2023. 30. Observability and AI Partner Well to Tackle All of That Data Gone are the days when operations and development were a back-office affair. 31. Logging in Observability - Part 2 In this article, I want to describe how to work with logs, analyze them in the command line, and we will consider new modern tools to visualize logs. 32. Tracing: An Essential Component of Observability In this article, I am going to shed light on one important aspect of observability - tracing. 33. Microservice Observability Patterns [Part 1] Logging is one of the most important parts of software systems. Whether you have just started working on a new piece of software, or your system is running in a large scale production environment, you’ll always find yourself seeking help from log files. Logs are the first thing people look for when something goes wrong, or something doesn’t work as expected. 34. What is Observability? Is it Cultural? In order to leverage observability we need a significant shift in our corporate culture that encapsulates the entire company & goes beyond the tools. 35. Murre - the lightweight K8s metrics monitoring tool 36. Introductory Guide to Debugging Lambda Configuration Errors The “Lambda configuration error” is as generic as it gets but at the end of the day, it's a pathing issue. Let's go over how to resolve this. 37. Contextual Intelligence and Observability: Without the Former, You Really Don’t Have the Latter Observability is a hot term in the industry, but don’t let it fool you: having visibility into your organization's apps and services only gives you a piece. 38. The Everything Guide to Data Collection in DevSecOps Collecting wide swaths of observability and security data is key to a high-quality digital experience. Find out what you need to know to get started. 39. Better Performance and Security by Monitoring Logs, Metrics, and More Monitoring is a crucial part of observability. Learn how monitoring can specifically improve security, performance, and reliability. 40. Goodbye, Cost-depth Trade Off: Cloud-Native Observability, Redefined It turns out that, by rethinking the fundamentals of your approach to monitoring, you can have it all 41. Prometheus — Easy In, Difficult Out Setting up a Prometheus server can be easy, but getting actionable information out is a very long and never-ending journey. 42. Choosing a Computing Method: a Serverless SWOT Analysis If you ever find yourself deciding for or against serverless the following tries to make the decision easier for you. 43. Setting Up AWS CloudWatch Alerts (vs Dashbird Alerts) To Monitor Your Applications Learn about the best practices for AWS Cloudwatch Alerts and Dashbird Alarms, to not miss out on critical info about your serverless app. 44. Exploring Differences Between Monitoring And Observability Monitoring vs Observability: in this article, we're explaining what is observability exactly and how does it differ from monitoring. 45. Kuma and Prometheus for Observability in Kubernetes Microservices Clusters A year ago, Harry Bagdi wrote an amazingly helpful blog post (link at bottom of article) on observability for microservices. And by comparing titles, it becomes obvious that my blog post draws inspiration from his work. 46. To Log Or Not To Log, Is The (Eternal) Question An alternative logging strategy to make loggers your friends, not enemies 47. 2023 Observability Predictions: Open Source is the Key to Explosive Growth The observability industry is projected to grow about 8% in 2023, a continuation of a strong growth trend since the 90’s. 48. How Goji Investments Enhances Developer Experience via Observability This post was written by Dean Record, Engineer at Goji Investments. 49. What is API Observability? APIs are the blood vessels of digital business. 50. A Guide to Deploying Jaeger on Kubernetes in Production Logs, metrics, and traces are the three pillars of the Observability world. The distributed tracing world, in particular, has seen a lot of innovation in recent months, with OpenTelemetry standardization and with Jaeger open source project graduating from the CNCF incubation. 51. How To Use a FastAPI Abstraction To Query AWS Data Lake Our guide in creating FastAPI abstraction that allows us to query the AWS Glue metadata catalog - a Data Discovery API for AWS Data Lake. 52. Sizing Your Cloud Resources: 10 Common Mistakes to Avoid Learn the common pitfalls of moving to the cloud and how you can avoid them to truly benefit from the cloud’s elasticity. 53. CNCF Tools Overview: Are You Cloud-Native? Cloud computing is becoming more and more of a household name, with even the most conservative fields of business figuring out how to make the best use of it. Cloud computing usually starts with running a private cloud solution on premises before venturing onto the public cloud. Of course, the cloud is not a single uniform being. It may come from different providers, Amazon Web Services, Google Cloud Platform, and Microsoft Azure being the biggest players here. Or it may come with different visibility and hosting, that is, public (resides with the provider), private (self-hosted), or hybrid (which uses a bit of both). And the cloud can use different tools and APIs for management as well. 54. Microservice Observability Patterns [Part 2] In my previous article, I talked about the importance of logs and the differences between structured and unstructured logging. Logs are easy to integrate into your application and provide the ability to represent any type of data in the form of strings. 55. Improving Observability Through Service Level Metrics 56. Observability on Heroku: How to Monitor Apps on a Managed Infrastructure Cloud platforms like Heroku make it easier than ever to host applications: just upload your code, and they’ll deploy it for you. But a common misconception is that, because you don’t own the infrastructure, you can’t really monitor your applications or see under the hood. 57. Setting Up Notification Systems to Observe and Analyze Your Application By building observability and analytics into your notification system, you can identify and quickly resolve issues by monitoring how your product is performing. 58. Better Failure Detection in Serverless Apps Having reliable failure detection in place is a must for any application in production. Here's how to best achieve it for AWS Lambda. 59. 4 Key Observability Metrics for Distributed Applications A common architectural design pattern these days is to break up an application monolith into smaller microservices. Each microservice is then responsible for a specific aspect or feature of your app. For example, one microservice might be responsible for serving external API requests, while another might handle data fetching for your frontend. 60. Lua-Land CPU Flame Graphs in OpenResty XRay This post will introduce the idea of Lua-land CPU flame graphs and use OpenResty XRay to produce real flame graphs for several small and standalone Lua examples 61. 5 Key Takeaways from New Relic’s 2022 State of Logs Report The 2022 State of Logs Report provides a detailed view of how this practice is shaping engineering and the technologies of the future. 62. Introduction to Observability in ITOM and AIOps Observability is a best practice implemented by AIOps, enabling automation and expanding visibility into the entire organizational ecosystem. 63. Internal AWS Monitoring is Hard to Grasp Learm how a serverless monitoring solution can catch problems for you without the painful learning curve connected to serverless failure detection. 64. Jaeger Persistent Storage with Elasticsearch, Cassandra and Kafka Running systems in production involve requirements for high availability, resilience and recovery from failure. When running cloud-native applications this becomes even more critical, as the base assumption in such environments is that compute nodes will suffer outages, Kubernetes nodes will go down and microservices instances are likely to fail, yet the service is expected to remain up and running. 65. How to Monitor Serverless Applications With AWS CloudWatch Alarms Running any application in production assumes reliable monitoring to be in place and serverless applications are no exception. 66. How to Optimize Large Scale Serverless Applications for Operational Excellence Managing applications at scale often comes up as one of the biggest concerns for businesses; How can it work smoothly? How do we monitor so many resources? How do we maintain best practices with constantly evolving infrastructure? In this article, we run through the best approach for operational excellence looking at serverless monitoring strategy, serverless alerting strategy, and security and compliance best practices. 67. Fixing The ClickHouse Node Failure On Distributed Systems - A How-To Guide Part One: ClickHouse Failures, by Marcel Birkner 68. Introduction To Distributed Tracing Pattern A distributed architecture brings in several challenges when it comes to operability and monitoring. Here, one may be dealing with tens if not hundreds of microservices, each of which may or may not have been built by the same team. 69. 10 Threats to an Open API Ecosystem Despite tight economic situations worldwide, the API economy continues to grow. 70. Data Access for Microservices If you want to access data in a distributed environment such as in a microservice architecture, then data services are the way to go. The idea is to create a data abstraction layer (DAL) that the rest of the system’s applications and services can share. Thus, a data service gives you a generalized interface to the data you’re exposing and provides access to it in a standard manner. This would be in a well-understood protocol and a known data format. For example, a popular approach is to use JSON via HTTP/S. 71. Rethinking Programming: Automated Observability Introduction 72. Watching the Requests Go By: Reconstructing an API Spec with APIClarity In this post, we’re going to look at what API reconstruction is and how APIClarity solves the API observability problem. 73. Navigate Your Way to Production Bliss with Caretta The easiest way to map our cluster: Caretta - a standalone OSS tool, leveraging eBPF technology to be lightweight and frictionless 74. The Importance of Monitoring Big Data Analytics Pipelines In this article, we first explain the requirements for monitoring your big data analytics pipeline and then we go into the key aspects that you need to consider to build a system that provides holistic observability. Thank you for checking out the 74 most read stories about Observability on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

74 Stories To Learn About Observability

171 Stories To Learn About Monitoring

2022 is over, and not a moment too soon. I'll never forget it: some of my friends had to flee their own country; others are fighting for their freedom as I write this post. I hope they will be safe and that their wishes will come true in 2023. On the personal and technical side, here's a summary of the past year from my perspective. Job Change First and foremost, I changed jobs. I worked for for 3½ years. However, I started to become dissatisfied with the company's direction and my position within it. Hazelcast I looked for a couple of months for my next gig. In February, I started to work on : Apache APISIX Full Lifecycle API Management API Gateway, Ingress Controller, etc. Apache APISIX provides rich traffic management features like Load Balancing, Dynamic Upstream, Canary Release, Circuit Breaking, Authentication, Observability, etc. When I joined Hazelcast, I had to distributed systems and concepts related to them: clocks, order, split brain, leader, etc. In comparison, an API Gateway such as APISIX is pretty simple. Hence, my focus is more on underlying technologies: understand learning NGINX OpenResty Lua Kubernetes Ingress etc. It means I had to shift the subjects I write about: you won't find any Java and hardly any Kotlin. My Blog In 2022, I published 50 blog posts on : one each week on Sunday, but on Christmas and the New Year - for obvious reasons. Here are the top 5 most viewed pages: https://blog.frankel.ch/ Rank Post Views % Avg. time on page 1 Hard things in Computer Science 26,123 5.29 00:05:35 2 A poor man's API 21,297 4.32 00:05:04 3 Discussing Backend For Front-end 18,626 3.78 00:04:57 4 Homepage 18,062 3.66 00:00:59 5 A list of cache providers 14,646 2.97 00:03:32 Two things are of interest: The 4th most viewed page is the homepage. The 5th, about cache providers, was published in October 2021. It's a long tail, indeed! Views are broken down as the following during the year: You can notice a couple of huge spikes. They are front page occurrences. I advertise on HN when I believe the post has potential; sometimes, others do. Most viewed posts correlate somewhat with referrals from Hacker News. Hacker News Rank Post Views % 1 A poor man's API 10,924 38.57 2 Discussing Backend For Front-end 8,128 28.69 3 Hard things in Computer Science 4,072 14.38 4 What I miss in Java, the perspective of a Kotlin developer 2,180 7.70 5 Toying with Kotlin's context receivers 1,023 3.61 Cross-Posting 2022 also saw consistent cross-posting on third-party content aggregator sites, which allow linking to the original post. As a reminder, Google (and other search sites) flag similar content as duplicate. If the aggregator site has more "page rank" than your site, then the content on the latter will be marked as such. The attribute is the way to tell Google: here's the original content, don't flag either as duplicate. I only post on sites with such a feature. rel canonical That being said, here are my followers depending on the site: : 564 Medium : 1,838 Dev.to : 80 Hashnode : no follow feature DZone : no follow feature Hacker Noon : no follow feature either foojay.io It's interesting to see that though I thought focused on web and front-end, I got many followers though I rarely write on such subjects. Conversely, I still need to understand why my follower base on Hashnode doesn't take off. dev.to Public Speaking Indeed, 2022 was not my best year in terms of conference attendance. Here's a comparison of the previous years: Raw numbers are a bit misleading: because of Covid, I spoke at many events online in 2020 and 2021. It's easier to do so than traveling to another country (or continent!); on the other hand, engagement is much lower, not to mention the lack of social interactions. much Anyway, 2022 numbers are below 2019's, the last "regular" year. Despite this, I'm ok with it: 2019 was hard on me, and conference attendance is more of a marathon than a sprint. I think I've found the sweet spot between public speaking and other activities. Regarding content, I mainly did two talks: Evolving your APIs Chopping the monolith It's below the number of talks I usually try to achieve, but they were pretty popular, especially the first one. I'm working on more diversity for 2023. GitHub Contributions As a Developer Advocate, my are different from the ones of a regular software developer, especially one working on Open Source. GitHub contributions Most of my contributions are commits because I'm the only one working on my demos. Furthermore, I regularly update them with the latest version of whatever dependency I use. I'm trying to open issues on Apache APISIX since I'm a huge user. Eating one's dog food is a great way to uncover either bugs or usability improvements. Finally, pull requests are for blog posts and reviews for my colleagues' blog posts. Conclusion Years when one changes jobs are always challenging but very interesting; 2022 was no different. My resolution for 2023 are: To deepen my understanding of the Apache APISIX ecosystem. Write as many blog posts as in 2022 - it's hard to do better, anyway. Design at least three new talks What are yours? Originally published at on January 8th, 2023 A Java Geek Photo by on Michael Skok Unsplash

OPENRENT

openrent.co.uk

-- employees

Since n.d.

COMPANY RANKING

OPENRENT

EVERGREEN INDEX #8311

Representing this company? Click here to claim and customize this page!

OpenRent WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

OPENRENT

openrent.co.uk

-- employees

Since n.d.

COMPANY RANKING

OPENRENT

EVERGREEN INDEX #8311

RELATED COMPANIESRANK

STORIES ABOUT OPENRENT

Representing this company? Click here to claim and customize this page!

OpenRent WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!