Event Bus Implementation(s)

Event driven architecture pattern is a distributed asynchronous architecture pattern to create highly scalable reactive applications. The pattern suits for on every level application stack from small to complex ones. The main idea is delivering and processing events asynchronously. There are couple of ways to implement event driven pattern to your stack. One of them is using “ ”. In broker topology, you send events to a central broker and all subscribers of these broker receive and process events asynchronously. Broker Topology In this story, I shared of event bus and with their and . In all implementations, all event generators sends events to the event bus. Event processors(subscribers) subscribe to the event bus. And subscribers can also create new events and pass it to the event bus. four implementations goods drawbacks i1) Notify the Event to All Subscribers Event Bus sends all incoming events(red arrows) to all subscribers(blue boxes) without any touch Actors Event Bus, Subscribers (Event Processor), Event Creators Implementation Whenever the event bus receive any event from event creators, it passes the event to the all subscribers. Subscribers process the event data with their own logic. Subscribers can also create new events and pass it to the event bus too. Event bus does not care about if receiver fails to receive the event message. Functions Needed , , , are enough to implement such event bus. Notify subscribe unsubscribe subscribers Goods It is very simple in terms of development, not too much responsibility. Drawbacks In this approach you are replicating the event data to the all subscribers, even if the subscribers do not have power to consume this event data. This means that your memory demand may increase with the multiplication of the subscribers in time. It also does not care about the assurance of the delivery. It tries to send the event to the subscriber only once but it does not care about the broken pipes, connection errors and other possible lacks. This implementation notify all subscriber without filtering. So, filtering should be done in subscribers. On JS front-end world ` ` library may be given as a sample of this implementation. When an event dispatched by `actions`, all `reducers` receive the same event type and event data and decides what to change. reduxjs Such implementation in Elixir lang: https://github.com/mustafaturan/event_bus/commit/08a955294c1d1c5bec2c7fef28ed255400e5f322 i2) Notify the Event Shadow to All Subscribers Event Bus with Event Store and Event Watcher Actors Event Bus, Event Creators, Subscribers, Event Store, Event Watcher What is Event Shadow? Probably, you never heard this term before, do not worry I made it up. Event shadow is basically a reference data to your original event data. Implementation Whenever the event bus receives any event, it save it to an . Then passes the event shadow to the all subscribers. Subscribers process the event shadow and fetch the event data from Event Store when they start processing. Event Store When an event comes to the event bus, it creates a watcher with the list of subscribers. is responsible for deletion of event data from the Event Store when all subscriber work done with the event data. Event Watcher Event bus does not care about if receiver fails to receive the event message. But it automatically mark the Event Watcher as ‘ ’ when any subscriber fails to receive event shadow. skipped Functions Needed , , , , , are enough to implement such an event bus. Notify subscribe unsubscribe subscribers save/delete/fetch_event_data mark_as_completed/skipped Goods This approach comes with a less memory consumption. It is possible to implement one more function to watch current status of a specific event or all events. Drawbacks In this approach each subscriber need to call and functions. You need to implement a . And you need to implement also an ‘event watcher’ to delete the event data when all subscribers processed the data. And sadly, event store write persistence should be blocking. fetch_event_data, mark_as_completed, mark_as_skipped read heavy event store Like in the implementation , this implementation notify all subscriber without filtering. So, filtering should be done in subscribers. Which is not too bad, but if you are using the event bus over networks, this will cause unnecessary network traffic. i1 Such implementation in Elixir lang: https://github.com/mustafaturan/event_bus/releases/tag/v0.2.1 i3) Notify the Event Shadow to Filtered Subscribers This implementation is same as the i2 but it is filtering the events before notifying the subscribers. In this approach, you need to register subscribers with their interests of topics. Note: It is good to have a regex filter on topics(event types) like in RabbitMQ. It has same goods and drawbacks as in the implementation . i2, except i3 supports filtering on events Such implementation in Elixir lang: https://github.com/mustafaturan/event_bus i4) Ordered Delivery to Subscribers To guarantee the ordered event delivery, the idea is partition your data before passing to interested in subscribers. The is a great way to start such an implementation. In basic, the producer decides for the event partition with a custom function like `CRC32`. And only one consumer is responsible only for that partition. Kafka paper Drawbacks Increasing, decreasing the number of partition might be pain. You need to implement your own partition function on producer. The consumer implementation will be as hard as broker implementation. Notes on Subscriber Implementations In and implementations, all subscribers should match the event type(topic) first in manner to . i1 i2 blocking prevent unnecessary stacks in memory In and implementations, you should match event type(topic) in asynchronous way because only the subscribed topic data comes to the event processors. i3 i4 non-blocking In general, all subscribers should process the event asynchronously without any blocking manner. Also subscribers can also create new events and pass those events to the event bus. In conclusion, promises , , systems. There are many implementation of event buses like AMQP (RabbitMQ is an implementation of AMQP), ZeroMQ and Kafka. There are all coming with different promises. For example, Kafka promises on the ‘guaranteed and ordered delivery’, AMQP promises on the ‘guaranteed delivery’ and ZeroMQ comes with patterns to develop your own custom broker. event broker topology reactive event driven asynchronous If you are using Elixir or Erlang for the development, try the library which is an implementation of the . event_bus i3 mustafaturan/event_bus