Asynchronous Programming Methods and Practices in JavaScript

1. Why Asynchronous Programming

JavaScript is a single-threaded programming language that can only execute one task at a time. In order to handle different task scheduling logic, asynchronous programming is unavoidable in JavaScript development.

The following scenarios necessarily involve asynchronous programming methods.

IO operations: external device access
- File access
- TCP / UDP network access
Asynchronous APIs
- setTimeout / setInterval
- setImmediate
- process.nextTick
- queueMicrotask
- Event

2. Several ways to implement asynchronous programming

Callback functions
Event Listening / Observer Pattern
Promise
Async / Await

Functions can be passed as arguments to functions, which is the most basic form of Javascript support for asynchronous programming. Therefore, the callback function pattern is the most widespread and almost the only available means of asynchronous programming in the early days (ES5-). With the introduction of the Promise concept, JavaScript’s modern syntax (ES6+) for asynchronous programming became more and more concise and friendly. Since ES7 introduced Async / Await based on Promise into the syntax standard, asynchronous programming can become as concise and clear as writing synchronous code.

2.1 Callback functions

The ability to pass functions as arguments to functions is the most basic form of Javascript support for asynchronous programming, and this feature makes JavaScript asynchronous programming very flexible and simple. The callback function pattern is the most widespread and almost the only option available for asynchronous programming in the early days (ES5-). However, it is most criticized for having to use a large number of asynchronous callbacks in complex business logic, resulting in a deeply nested callback hell pattern that makes the logic exceptionally complex to read and difficult to maintain.

const fs = require('fs');
 
fs.readFile('./a.txt', (err, data) => {
    if(err) {
        console.log('readFile.error:', err);
    } else {
        fs.readFile('./b.txt', (err1, data1) => {
            if (err1) {
                console.log('readFile.error:', err1);
            } else {
                return data + data1;
            }
        });
    }
});

2.2 Event Listening and Handling

Event listening and handling is a subscriber pattern, and is essentially based on callback functions. It is very common in JavaScript-based development. The subscriber pattern makes it easy to separate less relevant logic implementations and simplify inter-module dependency calls. The event-based subscriber pattern was used extensively in the jQuery era.

However, in large projects with heavy front-end architecture, data dependencies are often extremely complex, and in complex scenarios involving multiple variable factors, the data state of the subscriber pattern is highly uncontrollable and can easily lead to confusion in the subscription dependency logic due to logical fine-tuning of different variable factors. Therefore, there are many libraries of data flow management tools for medium and large front-end projects. Data flow management is still essentially based on the implementation of callback functions.

const uid = uuid.v4();
const payload = {...};
workerChannel.postMessage({ uid, payload });
workerChannel.once(uid, (result) => console.log(result));

2.3 Promise

With the introduction of the Promise concept, JavaScript’s modern syntax (ES6+) for asynchronous programming has become more and more concise and friendly. Promise is much better than callback functions and avoids the callback hell paradigm, but its then callback paradigm is not concise. However, Promise provides the foundation for almost all asynchronous-related APIs in the subsequent ES6+ standard.

fs.promises.readFile('./a.txt', 'utf8')
  .then(a => fs.promises.readFile('./b.txt', 'utf8').then(b => a + b))
  .then(result => console.log(result));
  .catch (err => {console.log(err));

function readFile(path) {
  return new Promise((resolve, reject) => {
    fs.readFile(path, (err, data) => err ? reject(err) : resolve(data));
  });
}
 
readFile('./a.txt')
  .then(a => readFile('./b.txt').then(b => a + b))
  .then(result => console.log(result))
  .catch(err => console.error(err));

2.4 `Async / Await`

Since ES7(ES2016) introduced Async / Await based on Promise into the syntax standard, asynchronous programming can become as concise and clear as writing synchronous code. ES13(ES2022) also includes Top-level Await in the specification, so that Await can be used anywhere in the ESM module outside the confines of the Async function. Its main features can be summarized as follows.

Syntactic sugar: based on Promise base support
Asynchronous programming as if you were writing synchronous code

try {
  const a = await fs.promises.readFile('./a.txt', 'utf8');
  const b = await fs.promises.readFile('./b.txt', 'utf8');
  const result = a + b;
  console.log(result);
} catch (err) {
  console.log(err);
}

3. JavaScript Asynchronous Programming in Practice

Promiseization
Using Async / Await
Concurrent performance: avoiding IO blocking
Anti-jitter and throttling

3.1 Delay handling

// delay 封装：callback 模式
function delay(timeout = 0, callback: () => void) {
  setTimeout(() => callback(), timeout);
}

// delay 封装：Promise 模式
function delay(timeout = 0) {
  return new Promise(resolve => setTimeou(() => resolve(), timeout));
}
 
await delay(3_00).then(() => callback());
callback();

// sleep 封装：TS 类型、回调值支持
export const sleep = <T>(timeout = 0, value?: T | (() => T | Promise<T>)): Promise<T> =>
    new Promise(resolve => setTimeout(() => resolve(), timeout))
      .then(() => typeof value === 'function' ? value() : value);
 
await sleep(3_000, 1);

3.2 Promiseing of event listeners and subscriptions

function request(payload) {
  return new Promise(resolve => {
    const uid = uuid.v4();
    workerChannel.postMessage({ uid, payload });
    workerChannel.once(uid, (result) => resolve(result));
  });
}
 
request({...}).then(body => console.log(body));

Q: How to implement timeout processing?

3.2.1 `setTimeout` and timeout handling

function request(payload, timeout = 5_000) {
  return new Promise(resolve => {
    const uid = uuid.v4();
    const timer = setTimeout(() => resolve({ errmsg: 'timeout' }), timeout);
    workerChannel.once(uid, (result) => {
      clearTimeout(timer);
      resolve(result);
    });
    workerChannel.postMessage({ uid, payload });
  });
}
 
request({...}).then(body => console.log(body));

Q: How to implement a generic package for timeout processing?

3.2.2 Generic encapsulation of timeout handling: `raceTimeout`

export function raceTimeout<T>(promise: Promise<T>, timeout: number, onTimeout?: () => T | undefined): Promise<T | undefined> {
    let promiseResolve: ((value: T | undefined) => void) | undefined = undefined;
    const timer = setTimeout(() => promiseResolve?.(onTimeout?.()), timeout);
    return Promise.race([promise.finally(() => clearTimeout(timer)), new Promise<T | undefined>(resolve => (promiseResolve = resolve))]);
}

Call request with raceTimeout to handle the timeout.

1
2

raceTimeout(request({...}), 3_000, () => ({ errmsg: 'timeout' }))
  .then(body => console.log(body));

Use raceTimeout to encapsulate generic timeout handling.

function request(payload, timeout = 5_000) {
  const p = new Promise(resolve => {
    const uid = uuid.v4();
    workerChannel.postMessage({ uid, payload });
    workerChannel.once(uid, (result) => resolve(result));
  });
  return raceTimeout(p, timeout, () => ({ errmsg: 'timeout' }));
}
 
request({...}).then(body => console.log(body));

3.2.3 Generic wrapper for timeout handling: `timeoutDeferred`

interface IScheduledLater extends IDisposable {
    isTriggered(): boolean;
}
function timeoutDeferred(timeout: number, fn: () => void): IScheduledLater {
    let scheduled = true;
    const handle = setTimeout(() => {
        scheduled = false;
        fn();
    }, timeout);
    return {
        isTriggered: () => scheduled,
        dispose: () => {
            clearTimeout(handle);
            scheduled = false;
        },
    };
}

Example of encapsulating generic timeout handling using timeoutDeferred.

function request(payload, timeout = 5_000) {
  const p = new Promise(resolve => {
    const uid = uuid.v4();
    const deferred = timeoutDeferred(timeout, resolve({ errmsg: 'timeout' }));
    workerChannel.postMessage({ uid, payload });
    workerChannel.once(uid, (result) => {
      deferred.dispose();
      resolve(result);
    });
  });
}
 
request({...}).then(body => console.log(body));

A simple use, no simpler than using setTimeout directly
A more convenient use of timeoutDeferred is to decide how to execute deferred.dispose() in complex logic processes based on different variable factors

3.2.4 Generic wrapper for timeout handling: `microtaskDeferred`

function microtaskDeferred(fn: () => void): IScheduledLater {
    let scheduled = true;
    queueMicrotask(() => {
        if (scheduled) {
            scheduled = false;
            fn();
        }
    });
 
    return {
        isTriggered: () => scheduled,
        dispose: () => { scheduled = false; },
    };
};

3.3 Debounce and throttle functions

Take the example of a postal worker delivering a letter.

Post office receiving mail - letters = [] ;
The postman delivers the letter - function deliver(){}

const letters = [];
/** 邮局接收信件 */
function onLetterReceived(l) {
  letters.push(l);
  deliver(); // 派送策略？
}
/** 邮政员派送信件 */
function deliver() {
  const lettersToDeliver = letters;
  letters = [];
  return makeTheTrip(lettersToDeliver);
}

Execute makeTheTrip as soon as the letter is received. Requirement.
- Receive mail less frequently?
- Very large number of postal workers?
- Very fast delivery?
- More…

3.3.1 `lodash` : function anti-shake and throttling

import { throttle, debounce } from 'lodash';
 
// 节流：100ms 内最多执行一次
const throttler = throttle(deliver, 100);
 
// 防抖：间隔 100ms 以上才触发
const debounced = debounce(deliver, 100);
 
// 防抖：高频调用 - 每隔 1s 至少会触发一次
const debounced = debounce(deliver, 100, { maxWait: 1000 });

Problems.

Optimal value: How to determine 100ms and 1000ms?
Unable to maximize CPU utilization
Can’t handle uncertain task calls with large time consumption better

Why?

Main reason: Can’t tell when the message is finished
Solution: callback => use back callback hell mode?
Solution: try Promise ?

3.3.2 `Promise`-style antijitter and throttling

Throttler : Execute async callback tasks in throttled fashion

export class Throttler {
  /** 正在执行的任务句柄 */
  private activePromise: Promise<unknown> | null;
  /** 等待执行的任务句柄 */
  private queuedPromise: Promise<unknown> | null;
  /** 等待执行的任务 */
  private queuedPromiseFactory: ITask<Promise<unknown>> | null;
  public queue<T>(promiseFactory: ITask<Promise<T>>): Promise<T>;
}

Application examples.

const throttler = new Throttler();
/** 邮局接收信件 */
function onLetterReceived(l) {
  letters.push(l);
  throttler.queue(deliver);
}

Delivery strategy
- Incoming mail initiates delivery tasks
- Take all emails for each delivery
- Waiting queue always caches only the latest one - queuedPromiseFactory

3.3.3 `Sequencer` : sequential execution of async callback tasks

export class Sequencer {
    private current: Promise<unknown> = Promise.resolve(null);
    queue<T>(promiseTask: ITask<Promise<T>>): Promise<T> {
        return (this.current = this.current.then(
            () => promiseTask(),
            () => promiseTask()
        ));
    }
}

Features.

One By One
Difference with Throttler: no limit on waiting queues ( queue)
Simple encapsulation, easy to call

3.3.4 Distinguishing between multiple types of sequential execution of async tasks

Cache different types of Sequencer by key.

export class SequencerByKey<TKey> {
    private promiseMap = new Map<TKey, Promise<unknown>>();
    queue<T>(key: TKey, promiseTask: ITask<Promise<T>>): Promise<T> {
        const runningPromise = this.promiseMap.get(key) ?? Promise.resolve();
        const newPromise = runningPromise
            .catch(() => {})
            .then(promiseTask)
            .finally(() => {
                if (this.promiseMap.get(key) === newPromise) {
                    this.promiseMap.delete(key);
                }
            });
        this.promiseMap.set(key, newPromise);
        return newPromise;
    }
}

3.3.5 Jitter-proof execution of asynchronous tasks

export class Delayer<T> implements IDisposable {
    private deferred: IScheduledLater | null;
    private completionPromise: Promise<unknown> | null;
    private doResolve: ((value?: unknown | Promise<unknown>) => void) | null;
    private doReject: ((err: unknown) => void) | null;
    private task: ITask<T | Promise<T>> | null;
 
    constructor(public defaultDelay: number) {}
    trigger(task: ITask<T | Promise<T>>, delay = this.defaultDelay): Promise<T>;
    isTriggered(): boolean;
    cancel(): void;
}

const delayer = new Delayer(10_000);
const letters = [];
function letterReceived(l) {
  letters.push(l);
  delayer.trigger(() => makeTheTrip());
}

Features.

Delayed execution
Stateful and cancelable
Simple calling method, clear business logic
Disadvantages: high-frequency calls keep getting canceled, can’t be called in time

3.3.6 `ThrottledDelayer` : Anti-jitter + throttling

The postman is smart enough to wait a certain amount of time before going out to deliver the mail (it doesn’t wait all the time).

export class ThrottledDelayer<T> {
    private delayer: Delayer<Promise<T>>;
    private throttler: Throttler;
 
    constructor(defaultDelay: number);
    trigger(promiseFactory: ITask<Promise<T>>, delay?: number): Promise<T> {
        return this.delayer.trigger(() => this.throttler.queue(promiseFactory), delay) as unknown as Promise<T>;
    }
    isTriggered(): boolean { return this.delayer.isTriggered(); }
    cancel(): void { this.delayer.cancel(); }
    dispose(): void { this.delayer.dispose(); }
}

Delayed execution of high-frequency tasks: wait for a certain amount of time before sending a message
A new delayed task is called in a shake-proof way when the message is delivered: delayer.trigger, delayer.completionPromise
When the delivery is complete, the next delivery journey is immediately started: throttler.queuedPromise

3.3.7 `Barrier` : call barrier before initialization

Creates a barrier with an initial state of closed and a final state of permanently open.

export class Barrier {
    private _isOpen: boolean = false;
    private _promise: Promise<boolean>;
    private _completePromise!: (v: boolean) => void;
    constructor() {
        this._promise = new Promise<boolean>((c, _e) => {
            this._completePromise = c;
        });
    }
    isOpen(): boolean { return this._isOpen }
    open(): void {
        this._isOpen = true;
        this._completePromise(true);
    }
    wait(): Promise<boolean> { return this._promise }
}

Q: How does a postal worker handle a triggered mail delivery task when he or she is not yet on duty?

Example.

const barrier = new Barrier();
async function letterReceived(l) {
  letters.push(l);
  await barrier.wait(); // 等待就绪后调用
  makeTheTrip();
}
 
// ...
barrier.open(); // 邮政员上班了

Advantages.

No cumbersome way to create buffers, wait for callbacks, etc.
Simplified call chain and clear and concise logic

3.3.8 Timeout auto-open barrier: `AutoOpenBarrier`

How to automatically enable an alternative (e.g. robotic delivery mode) when the postal worker keeps not coming to work?

export class AutoOpenBarrier extends Barrier {
    private readonly _timeout: NodeJS.Timer;
    constructor(autoOpenTimeMs: number) {
        super();
        this._timeout = setTimeout(() => this.open(), autoOpenTimeMs);
    }
    override open(): void {
        clearTimeout(this._timeout);
        super.open();
    }
}

3.3.9 `retry` Failure to retry

async function retry<T>(task: ITask<Promise<T>>, delay: number, retries: number, validator?: (r: T) => boolean): Promise<T> {
    let lastError: Error | undefined;
 
    for (let i = 0; i < retries; i++) {
        try {
            const result = await task();
            if (!validator || validator(result)) return result;
        } catch (error) {
            lastError = error;
            await sleep(delay);
        }
    }
 
    throw lastError;
}

retry Application example.

1
2

async function doLogin(): { success: boolean } {}
const result = await retry(doLogin, 1_000, 3, r => r.success);

Table of Contents

1. Why Asynchronous Programming

2. Several ways to implement asynchronous programming

2.1 Callback functions

2.2 Event Listening and Handling

2.3 Promise

2.4 Async / Await

3. JavaScript Asynchronous Programming in Practice

3.1 Delay handling

3.2 Promiseing of event listeners and subscriptions

3.2.1 setTimeout and timeout handling

3.2.2 Generic encapsulation of timeout handling: raceTimeout

3.2.3 Generic wrapper for timeout handling: timeoutDeferred

3.2.4 Generic wrapper for timeout handling: microtaskDeferred

3.3 Debounce and throttle functions

3.3.1 lodash : function anti-shake and throttling

3.3.2 Promise-style antijitter and throttling

3.3.3 Sequencer : sequential execution of async callback tasks