The Missing Introduction to Kafka

I have been fascinated by the open-source Apache Kafka service platform since I first heard about it some years ago. In part for the goal it has set out to achieve — to make microservices talk to each other, but also because of its name.

Why is it named Kafka? This is indeed something I find curious. There is a certain sense of intrigue when a software library or platform goes a bit further than the norm and picks a more unusual name. In this case the name Kafka goes well beyond the trivialities of daily existence and into a surreal literary realm.

I wholeheartedly welcome any sort of connection to anything foreign to software and technology, such as the literary world of Franz Kafka in the case of Apache Kafka. Working as a software engineer for more than a decade, I sometimes begin to look at software and technology as a sort of static, dehumanized space of data and algorithms where nothing much happens and where the banal and the mundane thrives. And I don’t mean dehumanizing in a dystopian technological singularity sort of way with artificial super-intelligence controlling the world. I mean it in a purely modern way, as someone who earns a living in the software space which is often void of human interactions.

According to the Wikipedia entry, the Kafka platform was named after writer Franz Kafka and that was because, being a system optimized for writing, the creators thought it was fitting to use the name of a writer. In addition, Kafka is well resonating name for a software library — short, interestingly sounding, easy to remember. But Kafka also happens to be a well fitting name for a system that operates with message queues and events for more reasons than just Franz Kafka being a writer.

As it happens, he had a peculiar affinity with messages and communication. In 1917, Kafka wrote a short story entitled Message from the Emperor, in which a subject is entrusted with an urgent message at the bedside of a dying emperor. In the piece, which is barely one page long and presented as a parable, the messenger, who is an “indefatigable man” travels tirelessly through crowds, then stairways and courtyards again and again for thousands of years, never reaching its destination.

Messages are also themes in some of Kafka’s other works, such as “In the Penal Colony”, where the message is inscribed on the back of a prisoner by a brutal apparatus or Amerika, which is set in an ultra-modern New York City and where communication is central to the story.

By now you have probably concluded that this particular article will not be diving deep into the workings of the Apache Kafka software platform. If you’re a software engineer and you happen to be reading this while you’re trying to get more familiar with the system, you might see this entire story as some kind of literary poppycock which is completely irrelevant to what you were looking for. By now you might feel a bit annoyed and potentially even revolted. Even more so if I tell you that I have never used Apache Kafka. I had used RabbitMQ for some time, which I believe is a competitor platform, but I am not an expert on it. Nor am I an expert in the literary works of Franz Kafka for that matter.

The Essay - In the Shadow of Kafka - Waving or Drowning: Kafka and Meaning - BBC Sounds

On the other hand, Karen Leeder — translator and professor of Modern German Literature at the University of Oxford — is an expert by all accounts and she notes in a 2015 essay for BBC Radio 3 that Kafka did write a lot of letters and he was paranoid with whether the letters will ever reach their destination. Somewhat in contrast with the software platform Apache Kafka which does everything possible in order to send those messages and has elaborate mechanisms to guarantee not only their arrival but also their acknowledgement.

But what would Franz Kafka think about his name being used by a software platform? No doubt, he would find it terribly funny, as he did find his works later in his life. Although not particularly comical, some of them do bear a surreal or absurd aspect that could be seen as amusing. Or maybe he would think the whole matter is kafkaesque. And if you’re still not convinced, consider this last sentence from Message from the Emperor which is suddenly directed at the reader: “You, however, sit at your window and dream of the message when evening comes”.