Concurrency Control in Promises With Bluebird

Bluebird is an interesting promise library, one of the best I’ve found, so far. Superior to Q (which I and my team had been using before bluebird) in terms of both speed and the convenience wrappers for performing various tasks.

Today I discovered Promise.map(), a convenient way to run operations on an array of promises (or operations on an array of values that will yield promises). Before this I used to run things as so:

<a href="https://medium.com/media/993d89840c8da296e741429e124a86a6/href">https://medium.com/media/993d89840c8da296e741429e124a86a6/href</a>

I used to iterate over my tasks using lodash and map them to the promises they’d return. These were then passed to Promise.all() to be executed in parallel.

Now this is all fine and good and something I’ve been using for a long time now. But what when I need to run a script that makes 10,000 connections in parallel? I could batch the requests, wait for a batch to finish before executing the next one and so on, but why bother with all that when I can just as easily use bluebird’s Promise.map() method with a concurrency option?

All I have to do:

<a href="https://medium.com/media/7da83043490821922809271c3b3b3f5f/href">https://medium.com/media/7da83043490821922809271c3b3b3f5f/href</a>

Now all these tasks will be handled 1000 at a time!

From the bluebird API Reference:

The concurrency limit applies to Promises returned by the mapper function and it basically limits the number of Promises created. For example, if concurrency is 3 and the mapper callback has been called enough so that there are three returned Promises currently pending, no further callbacks are called until one of the pending Promises resolves. So the mapper function will be called three times and it will be called again only after at least one of the Promises resolves.

No more handling batches with my own crappy code. This beautiful little library does the heavy lifting for me.

Huzzah!