Go Context Timeouts and External Calls: Cook Them Properly

The context package effectively manages and propagates timeouts and deadlines through a chain of calls in your product architecture. However, be mindful when using it. It's essential to be aware that it can also introduce performance degradation to your service in some cases with external calls. In this article, I tell you about some of them.

TLDR;

ctx.WithTimeout or ctx.WithDeadline have a chance to destroy the performance of your service. Be careful. Use a good cook recipe.

For this article, I use Redis as an example of an external service we use. But the cases I’m talking about are generic and can be applied to any other service or database.

Let’s start with a basic example of code to make a call to Redis:

import "github.com/go-redis/redis/v9"

reddisClient := redis.NewClient(...)

ctx := context.Background() 
ctx, cancel := context.WithTimeout(ctx, 200*time.Millisecond) 
defer cancel()

result, err := reddisClient.Get(ctx, "redis-key").Result()

This code tells us that we expect a response for the Get request to come within 200 milliseconds. And everything looks logical. But what if the Get request exceeds the timeout of 200 milliseconds?

When the context deadline is exceeded, all operations associated with this context should be terminated.

Let's take a look under the hood at what happens when the context deadline is exceeded. The Redis client:

1. Closed the current connection as a connection is not safe to re-use it

2. Established new connection:

3. Performed handshakes using a new connection.

   
   

These three steps basically mean that we don’t use the connection pool anymore if we have a good amount of requests with exceeded deadlines. Every time for requests that exceed the deadline, we close the current connection and open a new one. It makes each operation slower and adds risks to exceeding the deadline one more time. As a result, it can be an infinite loop, It affects the performance SLA of your service and bottleneck performance for the whole product.

What can we do to avoid this problem?

Context is used to propagate timeouts through a chain of calls, and each call has a random timeout that depends on the timing of previous calls. To prevent it for some calls in a chain, we can use a separate timeout for every external call:

ctx := context.Background()

result1, err := reddisClient.Get(ctx.WithTimeout(ctx, time.Second), "redis-key-1").Result() 

result2, err := reddisClient.Get(ctx.WithTimeout(ctx, time.Second), "redis-key-2").Result()

Don’t set up timeout too small because it can cause a problem we discuss here.

In a case where we need to respect incoming context timeout and build a response within that timeout, we can also use a go-routine to perform a call to an external service:

ctx, cancel := context.WithCancel(ctx) 

// Process asynchronously in a goroutine. 
go func() { 
  defer cancel() 

  result, err := reddisClient.Get(context.Background(), "redis-key").Result() 
  ch <- result 
  close(ch)
}() 

select { 
case <-ctx.Done(): 
  // error happend 
case res := <-ch: 
  // success 
}

In the code above, we build a response in time, but we also give redisClient to complete the Get request in the background.