How Costly is AWS Redshift Serverless?

AWS Redshift is a popular data warehouse service that allows users to analyze large amounts of data quickly and efficiently. One of the newer offerings from AWS in this space is Redshift Serverless, which aims to provide the scalability and performance of Redshift with the cost-saving benefits of a serverless architecture.

However, it is not that simple.

One of the main selling points of Redshift Serverless is its ability to scale up and down automatically based on workload. This means that users only pay for the computing resources they consume rather than having to provision and maintain a fixed number of nodes. In theory, this should result in significant cost savings compared to traditional Redshift clusters, which require users to pay for a fixed amount of computing and storage regardless of usage.

Sounds amazing. Especially when you look at performance results and declared cost of 0,375 USD for RPU-hour.

After calculating the potential cost for one of my clients, I was so happy. It seemed like Redshift Serverless is really fast, and the average time for the query would be less than a second for the small data warehouse.

Here are my initial calculations:

1/6 (I assumed I will use it only for no more than 10 minutes a day)
* 30 days
* 32 RPU (DWH is small so our choice is the smallest cluster size)
* 0,375 USD
= 60 USD

I’ll explain about 10 minutes average time (spoiler: I was even pessimistic about my expectation). The total amount of the data warehouse shouldn't exceed 2–3 GB at the beginning, which is nothing for the Redshift. I plan to manually run 10–20 daily queries with an ETL tool and 100–200 short queries on the days I need to set up something.

However, after the first day of usage, I saw a ~25 USD decrease in the trial money account. I was surprised and started to investigate what was wrong.

There is the table SYS_SERVERLESS_USAGE. You can check what you were charged for. I was charged 1920 seconds for almost every query, even if it was calculated faster than a second.

After thorough research, I found the line, “You pay for the workloads you run in RPU-hours on a per-second basis (with a 60-second minimum charge)”. So that is where 1920 seconds come from (60 seconds * 32 RPU).

That gives the minimal cost of query 0.2 USD (60 seconds * 32 RPU * 0.375 USD / 3600 seconds in an hour). It may not seem like too much but let’s apply it to my initial calculations assuming that months will pass while I first hit 60 seconds threshold:

10–20 daily ETL queries * 30 days * 0.2 USD = 60–120 USD

100–200 manual queries * 10 days (assuming I will set up something half of my work days) * 0.2 USD = 200–400 USD

Total: 260–520 USD a month for storing and processing 2–3 GB of data

And I didn’t add the cost of connecting it to a BI Viz tool.

In summary, while Redshift Serverless may offer some benefits in certain situations, it is important to consider the costs and trade-offs carefully before choosing it as a data warehouse solution. Until now, BigQuery has been the top 1 for small data warehouses. When AWS removes a minimal 60-second period, it can change.

Will see…