Promise Concurrency Control

While Promise.all is a powerful tool for executing multiple asynchronous operations in parallel, it has a significant ar

While Promise.all is a powerful tool for executing multiple asynchronous operations in parallel, it has a significant architectural limitation: it triggers all tasks simultaneously.

When dealing with a small number of requests, this is rarely an issue. However, when you scale up to hundreds or thousands of tasks—such as batch-processing API requests, reading large volumes of files, or handling massive image uploads—uncontrolled concurrency can lead to several critical failures:

  • Resource Exhaustion: You may hit the maximum number of concurrent connections allowed by the browser or the Node.js runtime.
  • Rate Limiting: Target servers or third-party APIs may interpret the sudden burst of traffic as a Denial of Service (DoS) attack, triggering rate limits or outright blocking your IP.
  • Memory Pressure: Creating thousands of Promise objects at once significantly increases memory consumption and puts heavy pressure on the event loop.
  • Error Management Complexity: In a massive batch, a single failure can make it difficult to track which tasks succeeded and which were still in flight.

To mitigate these risks, we implement Promise Concurrency Control. The goal is to ensure that at any given moment, no more than $N$ tasks are executing. As soon as one task completes, the next one in the queue begins.
Effective concurrency control relies on three fundamental components:

  1. Task Queue: A collection of pending tasks that are waiting to be executed.
  2. Running Counter: A state variable that tracks how many tasks are currently "in flight."
  3. The Scheduler: A logic loop that monitors the counter. When the counter is below the limit, it pulls a task from the queue and executes it.

A Critical Implementation Detail:
To control concurrency, you must pass functions that return Promises (often called "thunks"), rather than the Promises themselves. If you pass an array of already-instantiated Promises, the execution has already started, and you lose the ability to throttle them.

Implementation 1: The Manual Pool Approach

The following asyncPool implementation uses a classic approach: managing a counter and a pointer to the next task. This method is highly efficient and ensures that the results are returned in the original order of the input array.

/**
 * Executes tasks with a limit on concurrent operations.
 * @param {number} limit - Maximum number of concurrent tasks.
 * @param {Array<() => Promise<any>>} tasks - Array of functions returning promises.
 * @returns {Promise<Array<any>>} - A promise that resolves to the ordered results.
 */
async function asyncPool(limit, tasks) {
  return new Promise((resolve, reject) => {
    const results = new Array(tasks.length);
    let nextIndex = 0;
    let activeCount = 0;
    let completedCount = 0;
    let isRejected = false;

    function runNext() {
      if (isRejected) return;

      // If all tasks are finished, resolve the main promise
      if (completedCount === tasks.length) {
        resolve(results);
        return;
      }

      // Fill the "slots" available up to the limit
      while (activeCount < limit && nextIndex < tasks.length) {
        const currentIndex = nextIndex++;
        activeCount++;

        // Execute the task
        Promise.resolve()
          .then(() => tasks[currentIndex]())
          .then((value) => {
            results[currentIndex] = value;
          })
          .catch((err) => {
            isRejected = true;
            reject(err);
          })
          .finally(() => {
            activeCount--;
            completedCount++;
            // Once a task finishes, try to trigger the next one
            if (!isRejected) runNext();
          });
      }
    }

    runNext();
  });
}

// Usage Example:
const urls = ['url1', 'url2', 'url3', 'url4', 'url5'];
const tasks = urls.map(url => () => fetch(url).then(res => res.json()));

asyncPool(2, tasks)
  .then(data => console.log('All tasks completed:', data))
  .catch(err => console.error('A task failed:', err));

In this implementation, even if the third task finishes before the first, its result is placed in results[2], preserving the integrity of the data mapping.

Implementation 2: The Modern Promise.race Approach

For a more concise and readable version, we can leverage async/await combined with Promise.race. This approach uses a Set to keep track of currently executing promises.

async function asyncPool(limit, tasks) {
  const results = [];
  const executing = new Set();

  for (const [index, task] of tasks.entries()) {
    // Create the promise and wrap it to ensure it's handled
    const p = Promise.resolve().then(() => task());
    results[index] = p;
    executing.add(p);

    // Remove the promise from the set once it settles
    const clean = () => executing.delete(p);
    p.then(clean, clean);

    // If we hit the limit, wait for the fastest task to finish
    if (executing.size >= limit) {
      await Promise.race(executing);
    }
  }

  // Wait for all remaining tasks to settle
  return Promise.all(results);
}

This version is much more "idiomatic" in modern JavaScript. By using Promise.race(executing), the loop effectively pauses when the concurrency limit is reached, resuming only when at least one task in the executing set has completed.

Error Handling Strategies

When managing batches of tasks, "failure" is not a one-size-fits-all concept. You must choose a strategy based on your business logic:

  • Fail-Fast: This is the default behavior of Promise.all. If any single task fails, the entire operation rejects immediately. This is ideal when tasks are interdependent.
  • Resilient (All-Settled): If you need to ensure that one failure doesn't stop the entire batch, use a pattern similar to Promise.allSettled. You can wrap each task in its own .catch() block to return a specialized error object instead of throwing.
  • Retry Logic: For transient errors (like network timeouts), it is often better to implement an exponential backoff retry mechanism inside the task function itself before it ever reaches the concurrency pool.

Example of a Resilient Pattern:

// Wrap tasks so they never reject the main pool
const safeTasks = tasks.map(task => () => 
  task().catch(err => ({ error: true, message: err.message }))
);

const results = await asyncPool(3, safeTasks);

Production-Ready Solutions

While writing your own utility is great for learning or lightweight needs, production environments often benefit from battle-tested libraries.

  • p-limit: The industry standard. It is incredibly lightweight and provides a simple API to wrap functions.
  • p-queue: A more robust alternative if you need advanced features like task prioritization, timeouts, or the ability to pause/resume the queue.

Using p-limit looks like this:

import pLimit from 'p-limit';

const limit = pLimit(3);
const input = urls.map(url => limit(() => fetch(url).then(res => res.json())));

const results = await Promise.all(input);

Summary of Best Practices

  1. Don't Over-concur: A higher limit isn't always better. Always balance your concurrency against the target service's limits and your local hardware capabilities.
  2. Pass Functions, Not Promises: Always pass () => promise to ensure the scheduler actually controls the start time.
  3. Handle Cancellation: Standard Promises cannot be cancelled once started. If you need to abort a batch, integrate AbortController into your task functions.
  4. Mind the Memory: If you are processing millions of items, do not create a massive array of task functions upfront. Instead, consider using a Stream or a Generator to feed the queue incrementally.