Thanks! We'll be in touch in the next 12 hours
Oops! Something went wrong while submitting the form.

Understanding Node.js Async Flows: Parallel, Serial, Waterfall and Queues

Nikita Sinha

Full-stack Development

Promises in Javascript has been around for a long time now. It helped solve the problem of callback hell. But as soon as the requirements get complicated with control flows, promises start getting unmanageable and harder to work with. This is where async flows come to the rescue. In this blog, let’s talk about the various async flows which are used frequently rather than raw promises and callbacks.

Async Utility Module

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. Although it is built on top of promises, it makes asynchronous code look and behave a little more like synchronous code, making it easier to read and maintain.

Async utility has a number of control flows. Let’s discuss the most popular ones and their use cases:

1. Parallel

When we have to run multiple tasks independent of each other without waiting until the previous task has completed, parallel comes into the picture.

CODE: https://gist.github.com/velotiotech/08a07aaaee82f4af64816ffb7e9d27cc.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

In case an error is passed to a function's callback, the main callback is immediately called with the error. Although parallel is about starting I/O tasks in parallel, it’s not about parallel execution since Javascript is single-threaded.

An example of Parallel is shared below:

CODE: https://gist.github.com/velotiotech/1f7ade9f3581a6202269151d056eaad7.js

2. Series

When we have to run multiple tasks which depend on the output of the previous task, series comes to our rescue.

CODE: https://gist.github.com/velotiotech/b11fb4a5df97d7d55b33fd2c67c97d42.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

Callback function receives an array of result objects when all the tasks have been completed. If an error is encountered in any of the task, no more functions are run but the final callback is called with the error value.

An example of Series is shared below:

CODE: https://gist.github.com/velotiotech/cb07ef5321ee22ba49ee49a0cec7f9fc.js

3. Waterfall

When we have to run multiple tasks which depend on the output of previous task, Waterfall can be helpful.

CODE: https://gist.github.com/velotiotech/f35c7af9a4a9f1352f8a2dbe6e38f2ac.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable structure.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

It will run one function at a time and pass the result of the previous function to the next one.

An example of Waterfall is shared below:

CODE: https://gist.github.com/velotiotech/81290b3e768820ac03a7366c5f7feb3b.js

4. Queue

When we need to run a set of tasks asynchronously, queue can be used. A queue object based on an asynchronous function can be created which is passed as worker.

CODE: https://gist.github.com/velotiotech/e6dd3fd07ac0a45788ccb95f2bd1df45.js

Task: Here, it takes two parameters, first - the task to be performed and second - the callback function.

Concurrency: It is the number of functions to be run in parallel.

async.queue returns a queue object that supports few properties:

  • push: Adds tasks to the queue to be processed.
  • drain: The drain function is called after the last task of the queue.
  • unshift: Adds tasks in front of the queue.

An example of Queue is shared below:

CODE: https://gist.github.com/velotiotech/d92e399e1752782ec9387e434afaafef.js

5. Priority Queue

It is the same as queue, the only difference being that a priority can be assigned to the tasks which is considered in ascending order.

CODE: https://gist.github.com/velotiotech/9176d91adbd3d3f6538121b9ca9bd368.js

Task: Here, it takes three parameters:

  • First - task to be performed.
  • Second - priority, it is a number that determines the sequence of execution. For array of tasks, the priority remains same for all of them.
  • Third - Callback function.


The async.priorityQueue does not support ‘unshift property of the queue.

An example of Priority Queue is shared below:

CODE: https://gist.github.com/velotiotech/fed1ba482e5f17e816e345c9fb2d5fab.js

6. Race

It runs all the tasks in parallel, but as soon as any of the function completes its execution or passes error to its callback, the main callback is immediately called.

CODE: https://gist.github.com/velotiotech/8ed974dbda0eb336977193695bde5058.js

Task: Here, it is a collection of functions to run. It is an array or any iterable.

Callback: The result of the first complete execution is passed. It may be the result or error.

An example of Race is shared below:

CODE: https://gist.github.com/velotiotech/8da7d51c55209f1b8f56b808cbd1b19d.js

Combining Async Flows

In complex scenarios, the async flows like parallel and series can be combined and nested. This helps in achieving the expected output with the benefits of async utilities.

However, the only difference between Waterfall and Series async utility is that the final callback in series receives an array of results of all the task whereas in Waterfall, the result object of the final task is received by the final callback.

Conclusion

Async Utilities has an upper hand over promises due to its concise and clean code, better error handling and easier debugging. It makes us realize how simple and easy asynchronous code can be without the syntactical mess of promises and callback hell.

Get the latest engineering blogs delivered straight to your inbox.
No spam. Only expert insights.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.

We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.

Explore current openings

Understanding Node.js Async Flows: Parallel, Serial, Waterfall and Queues

Promises in Javascript has been around for a long time now. It helped solve the problem of callback hell. But as soon as the requirements get complicated with control flows, promises start getting unmanageable and harder to work with. This is where async flows come to the rescue. In this blog, let’s talk about the various async flows which are used frequently rather than raw promises and callbacks.

Async Utility Module

Async is a utility module which provides straight-forward, powerful functions for working with asynchronous JavaScript. Although it is built on top of promises, it makes asynchronous code look and behave a little more like synchronous code, making it easier to read and maintain.

Async utility has a number of control flows. Let’s discuss the most popular ones and their use cases:

1. Parallel

When we have to run multiple tasks independent of each other without waiting until the previous task has completed, parallel comes into the picture.

CODE: https://gist.github.com/velotiotech/08a07aaaee82f4af64816ffb7e9d27cc.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

In case an error is passed to a function's callback, the main callback is immediately called with the error. Although parallel is about starting I/O tasks in parallel, it’s not about parallel execution since Javascript is single-threaded.

An example of Parallel is shared below:

CODE: https://gist.github.com/velotiotech/1f7ade9f3581a6202269151d056eaad7.js

2. Series

When we have to run multiple tasks which depend on the output of the previous task, series comes to our rescue.

CODE: https://gist.github.com/velotiotech/b11fb4a5df97d7d55b33fd2c67c97d42.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

Callback function receives an array of result objects when all the tasks have been completed. If an error is encountered in any of the task, no more functions are run but the final callback is called with the error value.

An example of Series is shared below:

CODE: https://gist.github.com/velotiotech/cb07ef5321ee22ba49ee49a0cec7f9fc.js

3. Waterfall

When we have to run multiple tasks which depend on the output of previous task, Waterfall can be helpful.

CODE: https://gist.github.com/velotiotech/f35c7af9a4a9f1352f8a2dbe6e38f2ac.js

Tasks: A collection of functions to run. It can be an array, an object or any iterable structure.

Callback: This is the callback where all the task results are passed and is executed once all the task execution has completed.

It will run one function at a time and pass the result of the previous function to the next one.

An example of Waterfall is shared below:

CODE: https://gist.github.com/velotiotech/81290b3e768820ac03a7366c5f7feb3b.js

4. Queue

When we need to run a set of tasks asynchronously, queue can be used. A queue object based on an asynchronous function can be created which is passed as worker.

CODE: https://gist.github.com/velotiotech/e6dd3fd07ac0a45788ccb95f2bd1df45.js

Task: Here, it takes two parameters, first - the task to be performed and second - the callback function.

Concurrency: It is the number of functions to be run in parallel.

async.queue returns a queue object that supports few properties:

  • push: Adds tasks to the queue to be processed.
  • drain: The drain function is called after the last task of the queue.
  • unshift: Adds tasks in front of the queue.

An example of Queue is shared below:

CODE: https://gist.github.com/velotiotech/d92e399e1752782ec9387e434afaafef.js

5. Priority Queue

It is the same as queue, the only difference being that a priority can be assigned to the tasks which is considered in ascending order.

CODE: https://gist.github.com/velotiotech/9176d91adbd3d3f6538121b9ca9bd368.js

Task: Here, it takes three parameters:

  • First - task to be performed.
  • Second - priority, it is a number that determines the sequence of execution. For array of tasks, the priority remains same for all of them.
  • Third - Callback function.


The async.priorityQueue does not support ‘unshift property of the queue.

An example of Priority Queue is shared below:

CODE: https://gist.github.com/velotiotech/fed1ba482e5f17e816e345c9fb2d5fab.js

6. Race

It runs all the tasks in parallel, but as soon as any of the function completes its execution or passes error to its callback, the main callback is immediately called.

CODE: https://gist.github.com/velotiotech/8ed974dbda0eb336977193695bde5058.js

Task: Here, it is a collection of functions to run. It is an array or any iterable.

Callback: The result of the first complete execution is passed. It may be the result or error.

An example of Race is shared below:

CODE: https://gist.github.com/velotiotech/8da7d51c55209f1b8f56b808cbd1b19d.js

Combining Async Flows

In complex scenarios, the async flows like parallel and series can be combined and nested. This helps in achieving the expected output with the benefits of async utilities.

However, the only difference between Waterfall and Series async utility is that the final callback in series receives an array of results of all the task whereas in Waterfall, the result object of the final task is received by the final callback.

Conclusion

Async Utilities has an upper hand over promises due to its concise and clean code, better error handling and easier debugging. It makes us realize how simple and easy asynchronous code can be without the syntactical mess of promises and callback hell.

Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.

We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.

Explore current openings