Scaling to Infinity: Innovating the Worker Queue

Written by paragon | Published 2020/06/08
Tech Story Tags: architecture | engineering | programming | software-development | api | startups | hackernoon-top-story | good-company

TLDR Paragon provides a visual builder for creating APIs and workflows that connect to databases, 3rd party APIs and services, and logic or custom code for routing requests and transforming data. Traditionally, queues and APIs are kept separate; queues handle long-running or CPU intensive tasks and APIs serve quick responses to keep applications snappy. With that context, we had to build a worker queue that could support the following use cases: users can run arbitrary code (steps) on the server; the system should be able to scale to execute zero, thousands or millions of steps in parallel.via the TL;DR App

no story

Written by paragon | Visually build and run powerful workflows to automate tasks, integrate APIs, and connect your data.
Published by HackerNoon on 2020/06/08