"It works on my machine."
We've all said it. But does your bash script work when the third-party API takes 15 seconds to respond? Does your backend service gracefully handle a sudden spike of 503 errors from a payment provider?
Writing a full mock server just to test a simple retry logic in a script is often overkill. In this tutorial, I'll show you a faster way: Chaos Engineering directly in the terminal.
We will use curl and a cloud-based Chaos Proxy to inject failures into real network requests without changing a single line of your application code.
The Problem: Localhost is Too Perfect
When developing locally, network latency is near zero. APIs either work (200 OK) or they don't (Connection Refused). But in production, you face:
High Latency: The server is busy.
Intermittent Failures: 5% of requests drop.
Throttling: You hit the rate limit.
Simulating this in a terminal usually involves complex iptables rules or local tools like tc (Traffic Control). There is an easier way.
The Solution: Cloud Chaos Proxy
Instead of configuring your OS, we will route specific requests through a proxy that "breaks" the traffic according to rules we define.
I'll be using chaos-proxy.debuggo.app for this, but the concept applies to any programmable proxy.
Video Guide (1:27)
Prefer watching? Here is the 90-second workflow:
https://www.youtube.com/watch?v=x_S-guPwPEk&embedable=true
Step-by-Step Tutorial
1. Define the Failure
First, we need to tell the proxy what to break.
- Target:
httpbin.org(or your API domain). - Delay:
7 seconds(Simulate lag). - Failure Rate: 1 (100% of requests will fail).
- Error Code: 503 Service Unavailable.
2. Trust the Certificate (The "One-Time" Setup)
Since we are intercepting HTTPS traffic, we need to trust the proxy's CA certificate.
Download the mitmproxy-ca-cert.pem from the dashboard.
MacOS: Add it to Keychain Access -> System and set "Always Trust."
Linux: Copy to /usr/local/share/ca-certificates/ and update.
3. The Magic Command
Now, we use curl with the -x(proxy) flag.
curl -v -x http://user:[email protected]:13979 https://httpbin.org/get
Understanding the Output
When you run this command multiple times:
Scenario A (The Chaos): You'll notice the terminal "hangs" for 7 seconds (our delay). Then:
< HTTP/1.1 500 Internal Server Error
< content-length: 56
< content-type: text/plain
...
Debuggo Chaos Injection: 500 Error
Scenario B (Success): The other 50% of the time, the request passes through to the real server:
< HTTP/1.1 200 OK
...
{
"args": {},
"headers": { ... }
}
Real World Use Cases
- Testing CI Pipelines: Verify that your deployment scripts don't crash if a dependency is slow.
- Cron Jobs: Ensure your nightly data sync retries correctly on failure.
- Quick Sanity Checks: Before pushing code, verify how the API client handles a 503 error.
Conclusion
You don't need heavy infrastructure to test network resilience. A simple proxy setup allows you to inject chaos into any HTTP client—be it curl, wget, or your Python/Node.js scripts.
Happy Breaking! 🔨