i3Systems

i3systems

Uni-OVSeg: A Step Towards Efficient and Bias-Resilient Vision Systems

How Logical Clocks Keep Distributed Systems in Sync

Why Smart Design Decisions in Payment Systems Can Save You Millions

The Paradox of Scaling Through Inaction: Smart People Choose to Build Systems - Here's Why

مفارقة التوسع من خلال التقاعس: الأشخاص الأذكياء يختارون بناء الأنظمة - وإليك السبب

Paradox škálování prostřednictvím nečinnosti: Chytří lidé se rozhodli stavět systémy – zde je důvod

Το παράδοξο της κλιμάκωσης μέσω της αδράνειας: Οι έξυπνοι άνθρωποι επιλέγουν να κατασκευάσουν συστήματα - Να γιατί

La paradoja de escalar mediante la inacción: las personas inteligentes eligen construir sistemas. He aquí el porqué

निष्क्रियता के माध्यम से स्केलिंग का विरोधाभास: स्मार्ट लोग सिस्टम बनाने का विकल्प चुनते हैं - यहाँ कारण बताया गया है

Paradoks nan Eskalad Atravè Inaksyon: Moun Entelijan Chwazi Bati Sistèm - Men Poukisa

何もしないことで拡大するというパラドックス: 賢い人はシステム構築を選択する - その理由はここにある

Әрекетсіздік арқылы масштабтау парадоксы: ақылды адамдар жүйе құруды таңдайды - міне, неге

Paradoks Penskalaan Melalui Tidak Bertindak: Orang Pintar Memilih untuk Membina Sistem - Inilah Sebabnya

O paradoxo da escalabilidade por meio da inação: pessoas inteligentes escolhem construir sistemas - eis o porquê

ፓራዶክስ ናይ ምዕባይ ብዘይተግባር: በላሕቲ ሰባት ስርዓታት ክሃንጹ ይመርጹ - ስለምንታይ ከምኡ ከም ዝኾነ እንሆ

Nghịch lý của việc mở rộng quy mô thông qua sự không hành động: Những người thông minh chọn xây dựng hệ thống - Đây là lý do

The Noonification: The Dark Systems Manual (10/10/2024)

The Dark Systems Manual

ডার্ক সিস্টেম ম্যানুয়াল

Priručnik za mračne sisteme

Manual de sistemas oscuros

کتابچه راهنمای سیستم های تاریک

डार्क सिस्टम्स मैनुअल

ダークシステムマニュアル

Харанхуй системийн гарын авлага

De handleiding voor donkere systemen

Mwongozo wa Mifumo ya Giza

தி டார்க் சிஸ்டம்ஸ் கையேடு

Buku ya Tisisiteme ta Munyama

Imanuwali Yezinhlelo Ezimnyama

The thundering herd problem is a critical challenge in distributed systems that can bring even robust architectures to their knees. This article explores the nature of this issue, its recurring variant, and how jitter serves as a crucial defense mechanism. We'll also examine practical solutions for Java developers, including standard libraries and customization options for REST-based clients. Understanding the Thundering Herd The thundering herd problem occurs when a large number of processes or clients simultaneously attempt to access a shared resource, overwhelming the system. This can happen in various scenarios: After a service outage, when all clients try to reconnect at once
When a popular cache item expires, causing multiple requests to hit the backend
During scheduled events or cron jobs that trigger at the same time across many servers The impact can be severe, leading to: Increased latency
Service unavailability
Cascading failures across dependent systems Recurring Thundering Herd: A Persistent Threat While a single thundering herd event can be disruptive, recurring instances pose an even greater danger. This phenomenon happens when: Clients use fixed retry intervals, causing repeated traffic spikes
Periodic tasks across multiple servers align over time
IoT devices or smart home appliances check for updates on a fixed schedule Jitter: The Unsung Hero of Distributed Systems Jitter introduces controlled randomness into timing mechanisms, effectively dispersing potential traffic spikes. Here's why it's crucial: Prevents synchronization: By adding small random delays, jitter keeps processes from aligning their actions.
Smooths traffic: Instead of sharp spikes, jitter creates a more even distribution of requests over time.
Improves resilience: Systems with jitter can better handle load variations and recover from failures. Implementing Jitter in Java Java developers have several options for implementing jitter: Standard Libraries java.util.concurrent.ThreadLocalRandom:
javalong jitter = ThreadLocalRandom.current().nextLong(0, maxJitterMs);




java.util.Random:
javaRandom random = new Random();
long jitter = random.nextLong(maxJitterMs); Third-Party Libraries Guava's ExponentialBackOff:
javaExponentialBackOff backoff = ExponentialBackOff.builder()
    .setInitialIntervalMillis(500)
    .setMaxIntervalMillis(1000 * 60 * 5)
    .setMultiplier(1.5)
    .setRandomizationFactor(0.5)
    .build();




Resilience4j's Retry:
javaRetryConfig config = RetryConfig.custom()
    .waitDuration(Duration.ofMillis(1000))
    .maxAttempts(3)
    .build();
Retry retry = Retry.of("myRetry", config); Customizing REST Clients with Jitter When working with REST clients, you can incorporate jitter in several ways: Custom Interceptors: Implement an interceptor that adds a random delay before each request.
Retry Policies: Use libraries like OkHttp or Apache HttpClient that allow custom retry policies with jitter.
Circuit Breakers: Implement circuit breakers with jittered retry mechanisms using libraries like Hystrix or Resilience4j. IoT and Smart Home Devices: A Special Case The thundering herd problem is particularly relevant for IoT and smart home devices. These devices often use a common pattern of periodically checking for updates or sending data to a central server. To mitigate potential issues: Implement device-side jitter for update checks and data transmissions.
Use push notifications instead of frequent polling when possible.
Stagger initial boot times and update schedules across device fleets. Conclusion The thundering herd problem remains a significant challenge in distributed systems, but with proper understanding and implementation of jitter, developers can create more resilient and scalable applications. By leveraging Java's built-in libraries and third-party solutions, along with custom REST client configurations, you can effectively tame the herd and ensure your systems remain stable under heavy load. Remember, in the world of distributed systems, a little randomness goes a long way in maintaining order and preventing chaos. References: [1] Distributed Systems Horror Stories: The Thundering Herd Problem https://encore.dev/blog/thundering-herd-problem
[2] Retry policy to avoid Thundering Herd Problem - Temporal Community https://community.temporal.io/t/retry-policy-to-avoid-thundering-herd-problem/790
[3] This is known generally as the "Thundering Herd" problem https://news.ycombinator.com/item?id=1722213
[4] Using the REST Client - Quarkus https://quarkus.io/guides/rest-client
[5] Thundering Herd Problem and How not to do API retries - YouTube https://www.youtube.com/watch?v=8sTuCPh3s0s
[6] YouTube Strategy: Adding Jitter isn't a Bug - High Scalability - https://highscalability.com/youtube-strategy-adding-jitter-isnt-a-bug/
[7] Timeouts, retries and backoff with jitter - AWS https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
[8] Connect to a REST API - Jitterbit Documentation https://success.jitterbit.com/design-studio/design-studio-reference/sources-and-targets/http/rest-api-tutorial/connect-to-a-rest-api/ [9] Figure 1: Figure 1: The thundering herd problem : Image generated using DALL-E 3 from the prompt "The Thundering Herd Problem: Taming the Stampede in Distributed Systems" (OpenAI, 2023) The thundering herd problem is a critical challenge in distributed systems that can bring even robust architectures to their knees. This article explores the nature of this issue, its recurring variant, and how jitter serves as a crucial defense mechanism. We'll also examine practical solutions for Java developers, including standard libraries and customization options for REST-based clients. Understanding the Thundering Herd The thundering herd problem occurs when a large number of processes or clients simultaneously attempt to access a shared resource, overwhelming the system. This can happen in various scenarios: After a service outage, when all clients try to reconnect at once When a popular cache item expires, causing multiple requests to hit the backend During scheduled events or cron jobs that trigger at the same time across many servers After a service outage, when all clients try to reconnect at once When a popular cache item expires, causing multiple requests to hit the backend During scheduled events or cron jobs that trigger at the same time across many servers The impact can be severe, leading to: Increased latency Service unavailability Cascading failures across dependent systems Increased latency Service unavailability Cascading failures across dependent systems Recurring Thundering Herd: A Persistent Threat While a single thundering herd event can be disruptive, recurring instances pose an even greater danger. This phenomenon happens when: Clients use fixed retry intervals, causing repeated traffic spikes Periodic tasks across multiple servers align over time IoT devices or smart home appliances check for updates on a fixed schedule Clients use fixed retry intervals, causing repeated traffic spikes Periodic tasks across multiple servers align over time IoT devices or smart home appliances check for updates on a fixed schedule Jitter: The Unsung Hero of Distributed Systems Jitter introduces controlled randomness into timing mechanisms, effectively dispersing potential traffic spikes. Here's why it's crucial: Prevents synchronization: By adding small random delays, jitter keeps processes from aligning their actions. Smooths traffic: Instead of sharp spikes, jitter creates a more even distribution of requests over time. Improves resilience: Systems with jitter can better handle load variations and recover from failures. Prevents synchronization: By adding small random delays, jitter keeps processes from aligning their actions. Prevents synchronization: Smooths traffic: Instead of sharp spikes, jitter creates a more even distribution of requests over time. Smooths traffic: Improves resilience: Systems with jitter can better handle load variations and recover from failures. Improves resilience: Implementing Jitter in Java Java developers have several options for implementing jitter: Standard Libraries java.util.concurrent.ThreadLocalRandom:
javalong jitter = ThreadLocalRandom.current().nextLong(0, maxJitterMs); java.util.Random:
javaRandom random = new Random();
long jitter = random.nextLong(maxJitterMs); java.util.concurrent.ThreadLocalRandom: javalong jitter = ThreadLocalRandom.current().nextLong(0, maxJitterMs); java.util.concurrent.ThreadLocalRandom: java.util.concurrent.ThreadLocalRandom: javalong jitter = ThreadLocalRandom.current().nextLong(0, maxJitterMs); javalong jitter = ThreadLocalRandom.current().nextLong(0, maxJitterMs); java.util.Random: javaRandom random = new Random();
long jitter = random.nextLong(maxJitterMs); java.util.Random: java.util.Random: javaRandom random = new Random();
long jitter = random.nextLong(maxJitterMs); javaRandom random = new Random();
long jitter = random.nextLong(maxJitterMs); Third-Party Libraries Guava's ExponentialBackOff:
javaExponentialBackOff backoff = ExponentialBackOff.builder()
    .setInitialIntervalMillis(500)
    .setMaxIntervalMillis(1000 * 60 * 5)
    .setMultiplier(1.5)
    .setRandomizationFactor(0.5)
    .build(); Resilience4j's Retry:
javaRetryConfig config = RetryConfig.custom()
    .waitDuration(Duration.ofMillis(1000))
    .maxAttempts(3)
    .build();
Retry retry = Retry.of("myRetry", config); Guava's ExponentialBackOff: javaExponentialBackOff backoff = ExponentialBackOff.builder()
    .setInitialIntervalMillis(500)
    .setMaxIntervalMillis(1000 * 60 * 5)
    .setMultiplier(1.5)
    .setRandomizationFactor(0.5)
    .build(); Guava's ExponentialBackOff: Guava's ExponentialBackOff: javaExponentialBackOff backoff = ExponentialBackOff.builder()
    .setInitialIntervalMillis(500)
    .setMaxIntervalMillis(1000 * 60 * 5)
    .setMultiplier(1.5)
    .setRandomizationFactor(0.5)
    .build(); javaExponentialBackOff backoff = ExponentialBackOff.builder()
    .setInitialIntervalMillis(500)
    .setMaxIntervalMillis(1000 * 60 * 5)
    .setMultiplier(1.5)
    .setRandomizationFactor(0.5)
    .build(); Resilience4j's Retry: javaRetryConfig config = RetryConfig.custom()
    .waitDuration(Duration.ofMillis(1000))
    .maxAttempts(3)
    .build();
Retry retry = Retry.of("myRetry", config); Resilience4j's Retry: Resilience4j's Retry: javaRetryConfig config = RetryConfig.custom()
    .waitDuration(Duration.ofMillis(1000))
    .maxAttempts(3)
    .build();
Retry retry = Retry.of("myRetry", config); javaRetryConfig config = RetryConfig.custom()
    .waitDuration(Duration.ofMillis(1000))
    .maxAttempts(3)
    .build();
Retry retry = Retry.of("myRetry", config); Customizing REST Clients with Jitter When working with REST clients, you can incorporate jitter in several ways: Custom Interceptors: Implement an interceptor that adds a random delay before each request. Retry Policies: Use libraries like OkHttp or Apache HttpClient that allow custom retry policies with jitter. Circuit Breakers: Implement circuit breakers with jittered retry mechanisms using libraries like Hystrix or Resilience4j. Custom Interceptors: Implement an interceptor that adds a random delay before each request. Custom Interceptors: Retry Policies: Use libraries like OkHttp or Apache HttpClient that allow custom retry policies with jitter. Retry Policies: Circuit Breakers: Implement circuit breakers with jittered retry mechanisms using libraries like Hystrix or Resilience4j. Circuit Breakers: IoT and Smart Home Devices: A Special Case The thundering herd problem is particularly relevant for IoT and smart home devices. These devices often use a common pattern of periodically checking for updates or sending data to a central server. To mitigate potential issues: Implement device-side jitter for update checks and data transmissions. Use push notifications instead of frequent polling when possible. Stagger initial boot times and update schedules across device fleets. Implement device-side jitter for update checks and data transmissions. Use push notifications instead of frequent polling when possible. Stagger initial boot times and update schedules across device fleets. Conclusion The thundering herd problem remains a significant challenge in distributed systems, but with proper understanding and implementation of jitter, developers can create more resilient and scalable applications. By leveraging Java's built-in libraries and third-party solutions, along with custom REST client configurations, you can effectively tame the herd and ensure your systems remain stable under heavy load. Remember, in the world of distributed systems, a little randomness goes a long way in maintaining order and preventing chaos. References: [1] Distributed Systems Horror Stories: The Thundering Herd Problem https://encore.dev/blog/thundering-herd-problem [2] Retry policy to avoid Thundering Herd Problem - Temporal Community https://community.temporal.io/t/retry-policy-to-avoid-thundering-herd-problem/790 [3] This is known generally as the "Thundering Herd" problem https://news.ycombinator.com/item?id=1722213 [4] Using the REST Client - Quarkus https://quarkus.io/guides/rest-client [5] Thundering Herd Problem and How not to do API retries - YouTube https://www.youtube.com/watch?v=8sTuCPh3s0s [6] YouTube Strategy: Adding Jitter isn't a Bug - High Scalability - https://highscalability.com/youtube-strategy-adding-jitter-isnt-a-bug/ [7] Timeouts, retries and backoff with jitter - AWS https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ [8] Connect to a REST API - Jitterbit Documentation https://success.jitterbit.com/design-studio/design-studio-reference/sources-and-targets/http/rest-api-tutorial/connect-to-a-rest-api/ https://encore.dev/blog/thundering-herd-problem https://community.temporal.io/t/retry-policy-to-avoid-thundering-herd-problem/790 https://news.ycombinator.com/item?id=1722213 https://quarkus.io/guides/rest-client https://www.youtube.com/watch?v=8sTuCPh3s0s https://highscalability.com/youtube-strategy-adding-jitter-isnt-a-bug/ https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/ https://success.jitterbit.com/design-studio/design-studio-reference/sources-and-targets/http/rest-api-tutorial/connect-to-a-rest-api/ [9] Figure 1: Figure 1: The thundering herd problem : Image generated using DALL-E 3 from the prompt "The Thundering Herd Problem: Taming the Stampede in Distributed Systems" (OpenAI, 2023)

The Thundering Herd Problem: Taming the Stampede in Distributed Systems

Governance in Decentralized Systems

Architecting Scalable Databases for Large-Scale Systems

Diagrams are fantastic tools for explaining complex systems. They act as a visual map, allowing viewers to grasp how various components interact and connect in a single glance. However, static diagrams sometimes lack the ability to convey intricate processes fully. Imagine presenting a complex system diagram – you can narrate, draw focus to specific elements, and guide your audience through the visual representation. However one of the primary goals of diagrams is to provide a clear explanation without needing a constant narration. Animating diagrams, particularly the flow between different components, bridges the gap between static images and detailed narration. Animations act as a visual guide, directing viewers’ attention and highlighting the order of operations within the system. Imagine seeing a building’s piping system statically on paper (which is still an amazing way to give information about, well the piping in a building). But imagine the drawing animates and shows the direction of the flow of water or sewer system. Increasing the information signal using animations, amplifies the impact of the system diagram. I recently ended up using Drawing.io that has easy way to do this. Most of the time the connections between components of the diagrams are the pieces we want to animate. I looked for this in a lot of other tools I used in the past. Both proper diagramming tools like lucid charts, mermaid, excalidraw, and miro. figma/figjam and general presentation apps like Keynote, and PowerPoint. None have easy animation options. The way I started doing easy animations, is to use my existing drawings, import them as static jpeg/png or SVG into draw.io then draw a connector line over my existing lines, give it a white color and make it thick, then check it’s “animation” option to animate white dashes lines that animate over my existing connector lines, making them animated. Beyond Animation, I like draw.io because it’s a fantastic drawing tool. It’s open-source and free to use and contribute to. Works cross-platform. It has a lot of drawing elements, and you can import your own png, svg icons to your library. You can install draw.io on your desktop (on Mac, using homebrew, as simple as brew install –cask drawio), use the online version for free, or signup for cloud plans to collaborate with your team. Diagrams are fantastic tools for explaining complex systems. They act as a visual map, allowing viewers to grasp how various components interact and connect in a single glance. However, static diagrams sometimes lack the ability to convey intricate processes fully. Imagine presenting a complex system diagram – you can narrate, draw focus to specific elements, and guide your audience through the visual representation. However one of the primary goals of diagrams is to provide a clear explanation without needing a constant narration. Animating diagrams, particularly the flow between different components, bridges the gap between static images and detailed narration. Animations act as a visual guide, directing viewers’ attention and highlighting the order of operations within the system. Imagine seeing a building’s piping system statically on paper (which is still an amazing way to give information about, well the piping in a building). But imagine the drawing animates and shows the direction of the flow of water or sewer system. Increasing the information signal using animations, amplifies the impact of the system diagram. I recently ended up using Drawing.io that has easy way to do this. Most of the time the connections between components of the diagrams are the pieces we want to animate. I looked for this in a lot of other tools I used in the past. Both proper diagramming tools like lucid charts, mermaid, excalidraw, and miro. figma/figjam and general presentation apps like Keynote, and PowerPoint. None have easy animation options. The way I started doing easy animations, is to use my existing drawings, import them as static jpeg/png or SVG into draw.io then draw a connector line over my existing lines, give it a white color and make it thick, then check it’s “animation” option to animate white dashes lines that animate over my existing connector lines, making them animated. Beyond Animation, I like draw.io because i t’s a fantastic drawing tool. It’s open-source and free to use and contribute to. Works cross-platform. It has a lot of drawing elements, and you can import your own png, svg icons to your library. i You can install draw.io on your desktop (on Mac, using homebrew, as simple as brew install –cask drawio), use the online version for free, or signup for cloud plans to collaborate with your team.

Using Draw.io to Animate Systems Diagrams

Stories, News & Mentions about I3SYSTEMS

I3systems’s Stories on HackerNoon