The Architecture of an Enterprise Chat Application That Can Serve 1 Million+ Users Simultaneously

Well, not every business works in the same way we wanted but most, especially a social chat platform, chat between doctor and patient and such work in this fashion.

Let’s simply put it, we need to create a chat platform to connect a minimum of 1k users and letting them chat whenever and wherever.    

So, we develop a chat application where the end-users types the text, sends the messages, it gets validated in the database and reaches the client-server.

It sounds pretty easy? Well. Not at all.

This is actually the first version. I.e., the basic version of the chat application architecture.

It’s not an enterprise chat application architecture.

So, What’s the problem with this basic version?

1. It lacks end-to-end encryption
2. It lacks in DBA (Database Administrator)
3. Lacks in the top-notch Database to scale concurrent users. 

Now, it’s time for the enterprise chat architecture version.

The Architecture of An Enterprise Chat Solution

It’s the architecture of MirrorFly - An enterprise chat application built on Erlang, transmitting information on XMPP protocol, ejabberd server and storing on Mnesia database. 

Here comes the entire chat api & sdk interface of the enterprise chat solution.

Ejabberd can inherently be broken up into three layers:

1. The Interface Layer. This is the layer we have just talked about. It handles all the incoming data from and the outgoing data to the client.

2. The Logic Layer. This layer is the most complicated of all. It handles all the XMPP logic, as well as many other features, e.g.: HTTP bindings, access controls, extensible modules and hooks, etc.

3. The Data Layer. This layer handles how data get stored in databases, and ensures the integrity and constraints of data.

The Chat API & SDK Interface Layer Set Up

This layer handles incoming data and outgoing data from the client. Its main functionality is to listen on a local port waiting for client connections, establish a connection with any clients or servers when necessary, and exchange data. It supports TCP/TLS connections as well as UDP transport, as is described in our last discussions. It serves as the interface between the outer world and the ejabberd server.

Business Layer

The logic layer is the main and most important part of the ejabberd system. Some of its functionalities include:

1. Jabber Logics:

The client to server connection (typically on tcp/5222) is handled by the module ejabberd_c2s. The server to server connection (typically on tcp/5269) is handled by the modules ejabberd_s2s, ejabberd_s2s_in, ejabber_s2s_out. The HTTP bindings are handled by the ejabberd_http module.

2. Router:

The router handles the routing of the messages on mirrorFly chat servers
, i.e., when a Jabber client sends a to another entity, how is the message routed to the correct destination? The first ejabberd determines whether the message is a local or a remote one. It does so by looking at the "to" attribute to see if the host implied by the "to" attribute is hosted in itself. If so, the message is local; and is handled by ejabberd_local; otherwise, it is treated as an s2s message.

3. Modules:

In addition to the core Jabber and Router logics, there is a large part of the ejabberd which can be plugged in only when necessary, and they are called modules. Modules can be started/stopped dynamically at any time, thus making the ejabberd server highly extensible even at runtime. Modules are widely used for various extensions (the so-called 'XEP's).

4. Hooks:

Do you want to change the core logic in the ejabberd server? No problem. Hooks are everywhere to help you. A hook is a way by which you can change the behavior of ejabberd by injecting your new code into the system, without changing any existing code. For example, if you want to roll up your message filter to filter out messages you don’t want, you can add a module hooking to the “filter_packet/3” hook. If you want to keep track of all the messages clients sent, you can write a function hooking to ‘user_send_packet/3’.

5. Access Control:

All users (including real jabber client users as well as administrators) are stored the same way in ejabberd. Their privileges are determined by the groups they belong to. Hence, the access control modules in ejabberd allow us to distinguish users with their groups, thus providing different services for different users.

6. Utils and Libraries:

Some other libraries and utils exist in ejabberd for common purposes, e.g.: XML processing, SASL authentication, Encodings, Logger, etc. It is worth noting the ejabberd_logger is a very good logger module perfectly usable by any other project.

Data Layer

Ejabberd primarily uses mnesia as its “database”. Mnesia is, in fact, a high-performance key-value pair storage system built into the erlang library. It provides many features such as:

1. Replication. Tables may be replicated at several nodes.
2. Atomic transactions. A series of table manipulation operations can be grouped into a single atomic transaction.
3. Location transparency. Programs can be written without knowledge of the actual location of data.
4. Extremely fast real-time data searches.

Some of its features, such as replication and real-time searches, making it eligible for an extremely extensible database system. However, ejabberd does not enforce its use; it provides.

ODBC interfaces to allow utilizing of other databases whenever possible. For example, sometimes it might be more convenient to allow the user to authenticate using data stored in an existing database server, or there may be some relational data that need to be used for some purpose. 

Generally, mnesia is suitable for real fast key-value searches but performs pretty badly when you use it for “relational” lookups (using the mnesia: select the wrong way). Avoid relational data whenever possible (use key lookup) if you want to make good use of it.

Conclusion

Finally, the competition has got us here, right from Erlang, AWS, agile methodology, microservices, and Kubernetes. In today’s world, Chat Channels and AutoScaling are the biggest things that are in demand in every enterprise chat api & sdk solution. So, most of the enterprise chat API providers are optimizing the architecture to handle a minimum of 1 million users with just a single server.