Kafka Storage Design  -  Making File Systems Cool Again!by@sahilmalhotra
753 reads
753 reads

Kafka Storage Design  -  Making File Systems Cool Again!

by Sahil Malhotra5mSeptember 27th, 2022
Read on Terminal Reader
Read this story w/o Javascript
tldt arrow

Too Long; Didn't Read

A deep dive into the internals of a distributed system like Kafka. My aim with this series of blogs is to be able to comprehensively explain the fundamental details of how Kafka was designed. Kafka is mostly written in Java, and only based on JVM. The developers of Kafka find that using a PageCache over an in-process cache would be much more efficient when amortised over time and under high load. It doesn’t wait for the data to actually persist to disk, leaving it to the OS to take care for, and just sends out ack once it writes to the PageCache.

Company Mentioned

Mention Thumbnail
featured image - Kafka Storage Design  -  Making File Systems Cool Again!
Sahil Malhotra HackerNoon profile picture
Sahil Malhotra

Sahil Malhotra

@sahilmalhotra

Building ML & Data Platforms at MongoDB 🍃

Learn More
LEARN MORE ABOUT @SAHILMALHOTRA'S
EXPERTISE AND PLACE ON THE INTERNET.

STORY’S CREDIBILITY

Original Reporting

Original Reporting

This story contains new, firsthand information uncovered by the writer.

Share Your Thoughts

About Author

Sahil Malhotra HackerNoon profile picture
Sahil Malhotra@sahilmalhotra
Building ML & Data Platforms at MongoDB 🍃

TOPICS

THIS ARTICLE WAS FEATURED IN...

Permanent on Arweave
Read on Terminal Reader
Read this story in a terminal
 Terminal
Read this story w/o Javascript
Read this story w/o Javascript
 Lite
L O A D I N G
. . . comments & more!