Faster NestJS REST APIs with Automated Parallelization of Business Logic

The use case / problem statement

While inspecting the performance of our internally developed REST APIs for an e-commerce project, my team noticed that the performance of certain endpoints was not as good as we wanted, with an average of 6 seconds per call. We felt that an average of 1 second per call would be good to strive for.

The code behind these endpoints was about 3 years old, and it involves calling a bunch of different services to get the job done. Some of those services maintained internally, others maintained by third parties. We started an investigation into the causes of the slowness.

We found 2 major causes of slowness. The first one was that not all asynchronous logic that could be parallelized, was actually parallelized. It was all relying on manual effort to parallelize bits of code where needed (e.g. using something like Promise.all), and developers sometimes just forgot to do that while working on their user stories.

Another cause of slowness was that some logic was executed multiple times per request, when it could have only been executed once per request. It was not always easy to oversee the impact of individual pieces of code, so developers sometimes just didn't realize that certain asynchronous functions would get called multiple times in certain scenarios.

We came up with the idea to develop a proof-of-concept with all code organized into single-responsibility 'procedures', plus a 'procedure manager' that would parallelize whatever can be parallelized, and de-duplicate whatever can be de-duplicated. A way to reduce manual effort by automating as much as possible, and making it less likely for developers to forget parallelization and de-duplication.

With this proof-of-concept, we wanted to validate how much faster our slowest endpoint would get and if it would be close enough to our ambition of 1 second per call on average.

The concept in short: 'procedures' and a 'procedure manager'

Let's start with what we see as 'procedures'. The endpoint that we wanted to build the proof-of-concept for, is the endpoint to add a product to your shopping cart. It involves a bunch of asynchronous procedures, such as:

Getting customer information
Determining customer discounts
Getting product configuration
Determining product prices
Determining product tax
Determining shipping prices
Determining startup costs
Etc.

Some of these procedures have prerequisites. For example, determining customer discounts first needs customer information. A lot can be parallelized, but whenever one procedure depends on another it will need to wait for the procedure it depends on.

We came up with the idea to write each procedure as a standalone NestJS service, and let it declare which other procedures it depends on by leveraging NestJS' dependency injection container. With this consistent procedure separation and consistent dependency declaration between procedures, we get a lot of transparency on how all procedures are related to each other.

A 'procedure manager' can then leverage this to figure out which procedures it can run in parallel, and which procedures need to wait for another procedure to finish. With a procedure manager to do this decision making, it will be easier for developers to write code. You can now focus on a small piece of logic at a time, without having to put manual work to parallelize things and you get the maximum performance gain automatically.

Functional requirements of the procedure manager

Procedures should be able to access NestJS services using dependency injection (those NestJS services can be other procedures, or just regular services such as a logger).
Procedures should be able to access request parameters (e.g. from NestJS route params or the request body as JSON).
Procedures should be able to declare their prerequisites (other procedures), and then access the data that is returned by those prerequisites.
Procedures should be able to base their prerequisites on the request parameters (e.g. have a certain prerequisite only if certain data is present in the request parameters).
Procedures should be able to expose their logic with a 'run' method on itself, which the procedure manager can run as soon as it is ready to do so.
Each procedure should be run only once per request, even if it is a prerequisite of multiple other procedures.

Example implementations

Procedures

The simplest version of a procedure is one without prerequisites. It looks like this:

import { Injectable } from '@nestjs/common';

// Each procedure implements the Procedure interface from
// the procedure manager, which dictates that the procedure
// declares what type of request params it expects,
// what type of output it returns, and that it should
// have a 'run' method.
import { Procedure } from 'path/to/procedure-manager.types';

// Each procedure can indicate what type of request params
// it expects, by importing the corresponding DTO and then
// passing it as a generic to the Procedure interface.
import ExampleRequestParamsDto from 'path/to/example-request-params-dto.dto';

// Each procedure can indicate what kind of data it returns.
// by passing an interface as a generic to the Procedure
// interface.
export interface ExampleOutput {
}

@Injectable()
export default class ExampleProcedure implements Procedure<ExampleRequestParamsDto, ExampleOutput> {
    async run(
        // When the procedure is run by the procedure manager,
        // the request params will be injected so that the
        // procedure can use it for conditional logic where needed.
        requestParams: ExampleRequestParamsDto,
    ) {
        // Some async logic here ...

        return {
            // The data that this procedure returns ...
        };
    }
}

The slightly more complicated version of a procedure is one with prerequisites. It looks like this:

import { Injectable } from '@nestjs/common';
import { Procedure } from 'path/to/procedure-manager.types';
import ExampleRequestParamsDto from 'path/to/example-request-params-dto.dto';

// Each procedure can import references to other procedures
// (NestJS services) and their output type, to then
// set these as prerequisites of itself.
import AnotherProcedure, { AnotherProcedureOutput } from 'path/to/another-procedure';
import YetAnotherProcedure, { YetAnotherProcedureOutput } from 'path/to/yet-another-procedure';

export interface ExampleOutput {
}

@Injectable()
export default class ExampleProcedure implements Procedure<ExampleRequestParamsDto, ExampleOutput> {
    // By implementing a 'getPrerequisites' method, the procedure
    // can indicate that it depends on data from other procedures.
    getPrerequisites(
        requestParams: ExampleRequestParamsDto,
    ) {
        // The procedure can return references to other procedures
        // (NestJS services) here as an array.
        // The procedure manager will take care of taking instances
        // of these references out of NestJS' dependency injection
        // container.

        // If this procedure only needs certain prerequisites
        // conditionally (e.g. based on certain data in the
        // request params), you can use if statements here to
        // return different arrays of prerequisites per scenario.
        if (requestParams.something) {
            return [AnotherProcedure, YetAnotherProcedure];
        }

        return [AnotherProcedure];
    }

    async run(
        requestParams: ExampleRequestParamsDto,
        // The output of each prerequisite is injected into the
        // 'run' method, in the same order as the prerequisites
        // were declared in the 'getPrerequisites' method.
        anotherProcedureOutput: AnotherProcedureOutput,
        yetAnotherProcedureOutput?: YetAnotherProcedureOutput,
    ) {
        // Some async logic here based on `anotherProcedureOutput` ...

        if (yetAnotherProcedureOutput) {
            // Some conditional logic here ...
        }

        return {};
    }
}

The procedure manager

Now that we have shown some examples of how procedures are written in the previous section, let's look into the implementation of the procedure manager.

First, let's look at the 'Procedure' interface it exports which is to be implemented by all procedures:

// With the 'Type' interface from NestJS, the 'Procedure'
// interface will declare that whenever a procedure declares
// any prerequisites, those prerequisites are expected to be
// injectable NestJS types/services (e.g. classes with
// `@Injectable()` on it). The procedure manager will use these
// to take procedure instances out of NestJS' dependency
// injection container.
import { Type } from '@nestjs/common';

// The Procedure interface dictates with generics that each
// procedure needs to specify the type of request params it
// expects, and its output type.
export interface Procedure<RequestParamsType, OutputType> {
    // The typings on the 'getPrerequisite' method dictate that
    // injected request params will match the type the procedure
    // expects, and that all returned prerequisites must also
    // be able to handle the same request params type.
    getPrerequisites?(requestParams: RequestParamsType): Array<Type<Procedure<RequestParamsType, unknown>>> | undefined;

    // The typings on the 'run' method dictate that injected
    // request params will match the type the procedure expects,
    // that prerequites are injected here as additional params,
    // and that the returned output type must match the one
    // that the procedure declared above.
    run(requestParams: RequestParamsType, ...prerequisites: unknown[]): Promise<OutputType>;
}

Then, we of course have the most important part: the implementation of the procedure manager that actually runs the code:

import { Injectable, Scope, Type } from '@nestjs/common';

// We will use NestJS' ModuleRef to take procedure instances
// out of the dependency injection container.
import { ModuleRef } from '@nestjs/core';

import { Procedure } from 'path/to/procedure-manager.types';

// We add `Scope.REQUEST` to the procedure manager,
// so that we can cache procedures for the duration of a request.
// For more information about injection scopes, see:
// https://docs.nestjs.com/fundamentals/injection-scopes
@Injectable({ scope: Scope.REQUEST })
export default class ProcedureManager {
    // We store the promise of each pending procedure in cache,
    // for the duration of a request. This makes sure that even
    // if a procedure is a prerequisite of multiple other procedures,
    // it is only executed once per request.
    private readonly procedureCache: {
        [procedureName: string]: Promise<any>,
    };

    constructor(
        private readonly moduleRef: ModuleRef,
    ) {
        this.procedureCache = {};
    }

    // The 'run' method of the procedure manager can be used
    // by NestJS controllers to initiate a procedure and its
    // tree of prerequisites.
    // With generics, we dictate that the controller initiating
    // a procedure is aware of which type of request params and
    // output will apply.
    async run<RequestParamsType, ProcedureOutputType>(
        // The controller will be responsible for injecting any
        // request params.
        requestParams: RequestParamsType,
        // The controller will pass a reference to the procedure
        // it wants to initiate.
        procedure: Type<Procedure<RequestParamsType, ProcedureOutputType>>,
    ): Promise<ProcedureOutputType> {
        const procedureName = procedure.name;

        // Using NestJS' ModuleRef, we take the procedure instance
        // from the dependency injection container.
        // Depending on the injection scope of a procedure, we will
        // either need to resolve asynchronously (in case of a
        // request-scoped or transient-scoped procedure) or just get
        // the instance synchronously (in case of a default-scoped
        // procedure).
        // For more information about ModuleRef, see:
        // https://docs.nestjs.com/fundamentals/module-ref
        const procedureInstance = this.moduleRef.introspect(procedure).scope !== Scope.DEFAULT
            ? await this.moduleRef.resolve(procedure, undefined, { strict: false })
            : this.moduleRef.get(procedure, { strict: false });

        // Resolve procedure from cache if available
        if (this.procedureCache[procedureName]) {
            return this.procedureCache[procedureName];
        }

        // Run prerequisites if needed
        if ('getPrerequisites' in procedureInstance) {
            const prerequisites = procedureInstance.getPrerequisites(requestParams);

            if (prerequisites?.length) {
                // Run each prerequisite. This will happen recursively,
                // as long as the procedure manager keeps finding
                // procedures with prerequisites.
                const prerequisitePromises = prerequisites.map(
                    (prerequisite) => this.run<RequestParamsType, unknown>(requestParams, prerequisite)
                );

                // Resolve with injected output of each prerequisite.
                // With Promise.all, we make sure that all
                // prerequisites are initiated in parallel.
                this.procedureCache[procedureName] = Promise.all(prerequisitePromises)
                    .then((prerequisiteOutputs) => (
                        procedureInstance.run(requestParams, ...prerequisiteOutputs)
                    ));

                return this.procedureCache[procedureName];
            }
        }

        // Resolve without prerequisites
        this.procedureCache[procedureName] = procedureInstance.run(requestParams);
        return this.procedureCache[procedureName];
    }
}

Example controller

In NestJS, each REST API endpoint is a method in a controller. So, controllers are where you would initiate a procedure and its tree of prerequisites:

import ExampleRequestParamsDto from 'path/to/example-request-params-dto.dto';

import ExampleProcedure, { ExampleProcedureOutput } from 'path/to/example-procedure';

import ProcedureManager from 'path/to/procedure-manager';

@Controller()
export default class ExampleController {
    constructor(
        private procedureManager: ProcedureManager,
    ) {
    }

    @Get('/')
    async exampleMethod() {
        // Request params can be based on for example the
        // endpoint's route params or request body.
        // For more information on how to access request related
        // data in controller methods, see:
        // https://docs.nestjs.com/controllers
        const requestParams = {};

        return this.procedureManager.run<ExampleRequestParamsDto, ExampleProcedureOutput>(
            requestParams,
            ExampleProcedure,
        );
    }
}

Bonus: additional caching and de-duplication in individual procedures

Additional caching and de-duplication in individual procedures will often not be needed, but we found a use case where we wanted to do this:

In our case the customer can add a product to their cart which is a combination of several things. For example, one product which contains the following variations of a t-shirt:

50 mens t-shirts in size M
50 mens t-shirts in size L
50 womens t-shirts in size M
50 womens t-shirts in size L

In this case, there are 2 different product configurations we need to fetch in one of our procedures, not 4. The mens t-shirts and womens t-shirts are in our case different product configurations, but each size within those are the same product configuration.

So, we would like to fetch product configuration twice instead of 4 times, by caching those product configurations for the duration of a request. We can cache them based on the 'product key' of each.

We can apply the exact same mechanism here that we use in the procedure manager itself: by making the procedure request-scoped and storing the pending product configuration requests in a private property of the procedure. This is how it looks:

// We will use the Axios HTTP client in this example to fetch
// the product configurations.
import axios from 'axios';

import { Injectable, Scope } from '@nestjs/common';
import { Procedure } from 'path/to/procedure-manager.types';
import ExampleRequestParamsDto from 'path/to/example-request-params-dto.dto';

interface ProductConfiguration {
}

export interface GetProductConfigurationOutput {
    [productKey: string]: ProductConfiguration,
}

@Injectable({ scope: Scope.REQUEST })
export default class GetProductConfigurationProcedure implements Procedure<ExampleRequestParamsDto, GetProductConfigurationOutput> {
    private productConfigurationCache: {
        [productKey: string]: Promise<ProductConfiguration>,
    };

    constructor() {
        this.productConfigurationCache = {};
    }

    async run(
        requestParams: ExampleRequestParamsDto,
    ) {
        return Promise.all(
            requestParams.productVariations.map(
                ({ productKey }) => this.fetchProductConfiguration(productKey)
            ),
        );
    }

    private async fetchProductConfiguration(productKey: string): Promise<ProductConfiguration> {
        // Resolve from cache if available.
        if (this.productConfigurationCache[productKey]) {
            return this.productConfigurationCache[productKey];
        }

        // No cached item was available, so here we can fetch the
        // data we want and then store it in the cache object.
        // Note that we cache the promises and not awaited values,
        // so that we can grab requests from cache even if they
        // are still pending.
        this.productConfigurationCache[productKey] = axios.get<ProductConfiguration>(
            'path/to/product-configuration-endpoint',
            // Additional Axios params left out for this example.
            // If you would like more information about the
            // Axios HTTP client, see: https://axios-http.com
        ).then((response) => response.data);

        return this.productConfigurationCache[productKey];
    }
}

Conclusion

By rewriting our slowest endpoint using this architecture, our performance improved drastically: from ~6 seconds per call on average to ~1 second per call on average. This was without any other performance improvements to the logic itself or to any of the external services we were calling.

We felt that the modular architecture also helps with focussing on smaller chunks of the logic, with clearer relations between all procedures involved.

With the exception of when you want to implement additional caching/de-duplication in individual procedures, the procedure manager does all the work. This way we can write the majority of code without the risk of forgetting to parallelize asynchronous procedures. The risk of forgetting to de-duplicate is also reduced, as there is now a pretty standard and straightforward coding convention to do that per procedure if needed.

Further improvements to the performance of this endpoint is mostly about improving the external services this endpoint relies on. Some of those are also maintained by us, others are maintained by third parties that we will reach out to with our findings where needed.

We are currently working on migrating more endpoints and services to this architecture. Not all endpoints and services are expected to profit as much, for example if there is not much to parallelize. We still consider writing those in the same architecture, since we feel it might still make it easier to maintain the code over time. The modular way of working and the possibility to share code as 'procedures' across endpoints and services can benefit us on the long term.

The real-world version of our procedure manager implementation has thorough logging and unit test coverage, and a script that renders a diagram of the procedure tree that was executed for a certain request (useful during debugging).

The question we had ourselves while we were building this, was: surely we can't be the first ones to think of this, surely there is a libary or well-known design pattern that already does this? So far we haven't found anything that resembles this approach. Maybe we will turn it into a library ourselves one day, if we feel that there is a demand for it in the NestJS community.

If you enjoyed this blog post and feel like talking to us about ideas for the future, I encourage you to reach out to us!