Is Kafka the Key? The Evolution of Highlight's Ingestby@highlight
539 reads

Is Kafka the Key? The Evolution of Highlight's Ingest

tldt arrow
Read on Terminal Reader🖨️

Too Long; Didn't Read

At Highlight, we face a unique challenge when it comes to adapting to the load of our customers' web apps. Because we record web apps, we need to be able to handle bursts of traffic as they come from their users. We had to run some database migrations that ran an ALTER command with an aggressive table lock. The initial approach worked fine for a while. But as our customer base grew, we ran into key issues with this implementation. We needed to find a way to buffer data, we settled on a producer-consumer message strategy using Apache Kafka.

Companies Mentioned

Mention Thumbnail
Mention Thumbnail
featured image - Is Kafka the Key? The Evolution of Highlight's Ingest
Highlight HackerNoon profile picture

@highlight

Highlight

Credibility

react to story with heart
Highlight HackerNoon profile picture
by Highlight @highlight.See and act on your application's errors, interactions, and performance metrics. Get started at http://highlight.io!
Read my stories

RELATED STORIES

L O A D I N G
. . . comments & more!
Hackernoon hq - po box 2206, edwards, colorado 81632, usa