If you’ve ever visited a reasonably large airport, you must have noticed multiple flights taking off or landing at any given hour of the day. Now imagine how things would be if an airport could handle only one flight on any given day. Global travel would come to a screeching halt. Handling multiple flights at the same time makes air travel possible. This ability is also known as and it is the . concurrency key to doing more work with limited time and resources If you apply the concept to software development, . an application is concurrent if it can handle multiple tasks simultaneously How would you react if you visit the Amazon home page and get a message saying that a maximum number of users are already accessing the website and you need to wait for your turn? Chances are that you’ll never visit the website again and would lose a potential customer. Amazon As developers, when we use a platform (such as a web server) for running an application, we expect the platform to support concurrent behavior. Often, the concurrency model of platforms or frameworks is quite easy to understand. But I have seen that developers often get confused about . They know about the Event Loop but don’t know how it actually works. concurrency in Node.js In this post, my goal is to clear all the confusion about concurrency in Node.js. The biggest hurdle to Concurrency Why is it difficult to achieve concurrency? Is there any fundamental hurdle to concurrency in software systems? Yes. The main hurdle to concurrency is s . low and blocking input-output operations I/O operations are one of the fundamental operations of a computer system. But they are also the slowest. Accessing the RAM is in the order of nanoseconds. Accessing a disk is even slower (in the order of milliseconds). The bandwidth shares the same story. RAM has a transfer rate in the order of GB/second while disk and network can vary from MB/s to GB/s. While I/O is not computation heavy, it introduces a delay between the moment of request and the moment the operation completes. Also, I/O is not just a software concern. The human factor is probably even more important. The biggest delay in an application comes from waiting for user input (for example, mouse clicks and keystrokes). These delays are many orders of magnitude greater than those caused by disk access or network latency. Ultimately, slow and blocking I/O pushes you towards synchronous code that is not concurrent. How can platforms overcome blocking I/O? There’s no doubt that slow and blocking I/O is the biggest hurdle to concurrency. So, how do some platforms get around this issue? Let us look at a couple of popular methods to enable concurrency. Multi-Threading Multi-threading is a common approach to support concurrency. It is used by popular languages such as Java. In traditional blocking I/O programming, the function call that makes an I/O request will block the execution until the operation completes. The block can go from a few milliseconds for disk access to even minutes if we are waiting on the user to press a particular key. . Each I/O operation will block the processing of any other connection. A web server that uses blocking I/O will not be able to handle multiple connections within the same thread Therefore, to achieve concurrency, web servers start off a new thread for each concurrent connection. When a thread ‘blocks’ for an I/O operation, other requests will not be impacted as those requests have their own separate threads. Hence, the name multi-threading. See below illustration Unfortunately, a thread is not cheap in terms of system resources. Threads consume memory and cause context switches. Many times, a thread may sit in idle mode waiting for some I/O operation to complete. This leads to the wastage of resources. Suffice it to say, having a long-running thread for each connection is the best approach to increase efficiency. not Busy-Waiting Most modern operating systems support a mechanism known as non-blocking I/O. This type of I/O does not block the execution. The system’s call to access the resource returns immediately. If no results are available, the function returns a predefined constant. There is no waiting for data to be read or written. So, how is the data actually processed? The answer is . busy-waiting Busy-waiting is nothing but actively in a loop till it returns some actual data. polling a resource With busy-waiting, you can use the same thread to handle multiple I/O requests. But polling algorithms are also inefficient. A large portion of the precious CPU time is wasted in iterating over resources that are mostly unavailable. Why Node.js cannot use Multi-Threading When compared to busy-waiting or polling, multi-threading is a better approach to achieve concurrency. Many programming languages and frameworks use multi-threading successfully. However, Node.js cannot use multi-threading. This is because Node.js uses JavaScript and JavaScript is single-threaded. So is there no hope? Actually, there is another mechanism to enable non-blocking resources. This mechanism is known as . You can also call it the . Event Demultiplexing Event Notification Service The below illustration shows the concept. Event Demultiplexing There are three important steps in the Event Demultiplexing process: First, the resources are added to a watchlist. Each resource has an associated operation such as or . read write This watchlist of resources along with the callbacks is assigned to the . The demultiplexer makes a and call for any events generated from the watched resources. When the demultiplexer eventually returns from the blocking call, it has a new set of events available for processing. Event Demultiplexer synchronous blocking Each event returned by the is processed by the application callback methods. At this point, the resource is to be ready to read and it will not block during the operation. When all the events are processed, the demultiplexer will again make a blocking request for the next set of events. Event Demultiplexer guaranteed This process of checking for events, processing them, and checking again is known as the and forms the basis of concurrency in Node.js. Moreover, the makes it possible to handle the entire process in a single thread. Event Loop Event Demultiplexer There is no need for multi-threading. No need for busy waiting and polling. A single thread with very little idle time is enough. Tasks are spread over time instead of being divided between multiple threads. The Node.js Event Loop Having gone through all the explanations, we can now construct a model of how the Node.js event loop works. Check out the below diagram that shows all the steps in the event loop. Let us go through all the steps one by one: - The application generates a new I/O request and submits it to the . Along with the request, the application also provides a handler or a callback. This is like telling the demultiplexer whom to contact once the I/O request is complete. The call to the demultiplexer is non-blocking and the control returns back to the application immediately. STEP A Event Demultiplexer - The waits for events from I/O resources in its watchlist. When a bunch of I/O operations is completed, the pushes the corresponding events into the . STEP B Event Demultiplexer Event Demultiplexer Event Queue - The iterates over the items in the Event Queue. Technically, the is not just one queue but multiple queues that are processed in different stages. If interested, you can more about the different Event Loop queues. STEP C Event Loop Event Queue - For each event, the triggers the handler. The information about the handler was given to by the demultiplexer in STEP B. STEP D Event Loop Event Loop - The handlers are part of the application code. We also know them as callbacks. Once a callback runs, it may give the control back to the . This is denoted by path E1. However, the handler might also request for new async operations i.e. path E2. The E2 path results in new operations being sent to the . STEP E Event Loop Event Demultiplexer - When the is empty, the loop will block again on the and wait for another cycle. STEP 6 Event Queue Event Demultiplexer To describe things briefly, the Event Loop handles I/O by until new events are emitted from a set of watched resources. When events become available, it dispatches each event to its corresponding handler using the Event Queue. blocking When the Event Demultiplexer will run out of pending operations and there are no events left in the Event Queue, the Node.js application automatically exits. Conclusion Node.js is concurrent because of the . And the is made possible by the demultiplexer. Event Loop Event Loop However, very few developers know about the foundation of the . To give credit where it’s due, the complexities of the Event Demultiplexer are so nicely abstracted that developers can build useful applications with Node.js even if they haven’t heard about the Event Loop. Event Loop Nonetheless, it is good to know about the role of in the long run. It helps you make better choices about your application. Event Demultiplexer If you found this post to be useful, consider sharing it with friends and colleagues. You can also connect with me on other platforms: Twitter LinkedIn

This story contains new, firsthand information uncovered by the writer.

How Does Node.js Achieve Concurrency?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

The Essential Guide to Relearning Java Thread Primitives

A Guide to Improving Your Python Performance Speed

A simple introduction to Python’s asyncio

Assume It Worked and Fix Later

Async/Await in Golang: An Introductory Guide

Build a MapReduce flow in Elixir

The Essential Guide to Relearning Java Thread Primitives

A Guide to Improving Your Python Performance Speed

A simple introduction to Python’s asyncio

Assume It Worked and Fix Later

Async/Await in Golang: An Introductory Guide

Build a MapReduce flow in Elixir

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps