Customizing Angular builds with TypeScript AST transformers

Daan Scheerens - Feb 6, 2022

One of the things I generally dislike as a developer is having to repeat myself (in code). Mainly because it challenges my patience, but also (and more importantly) as this often hurts the maintainability of a code base. Most developers share this vision, and we refer to this as the DRY principle.

We often avoid repeating code fragments by generalizing code into utility functions and libraries. However, not all repeating patterns can simply be substituted with the usage of generalized utility code. Instead, automatic code generation is required to get rid of this form of repetition.

As a concrete example of this, consider how you write type guard functions in TypeScript. Every implementation is different, yet we can define a set of rules to systematically construct the correct implementation for such functions. If we want to do just that, then we need a tool that analyses the code and then generates the implementation of these functions. Wouldn't it be cool if I could just write is<SomeType>(someValue) and the type guard function for SomeType would automatically be generated?

One approach is to first scan the code for such usages of the is<T> function, generate the corresponding implementation and perform some substitution as a pre-build step. The disadvantage of this approach is that it doesn't update the generated code if you make changes to the application while it is running. You will also end up with a bunch of intermediate files that should be excluded from your version control system.

It would be far cleaner if the code generation doesn't require a separate build step and instead happens on the fly. So, can that be done?

Yes, this is possible with TypeScript AST transformers... and those can even be used in Angular applications!

AST-what?

AST is an abbreviation for Abstract Syntax Tree.

The AST is a data structure which captures the essential structure of source code. Compilers use this as an intermediate model for the representation of applications. They do so, because the AST offers a convenient model to perform further analysis (e.g. type checking) and transformation. TypeScript uses the AST as input when it actually transpiles your code to JavaScript.

What if, before passing the AST to the transpiler, we could apply some modifications? That would open up a range of interesting possibilities. For example, we could reduce the code size and make it slightly more performant if we simplify expressions involving literal integer values. Consider the expression 4 + 38: we can simplify that to just the literal value 42. To make this work we need to write something that detects such expressions and rewrites them. And that is exactly what AST transformers do!

How to get AST transformers working with Angular

While the TypeScript compiler API has support for AST transformers, getting them to work with an Angular application is not that trivial. This assumes that you're making use of the Angular CLI. If you have built your own compilation pipeline, then integrating AST transformers should be easier.

The problem with getting the AST transformers in place is twofold:

  • There is no way to configure them for the Angular CLI via the angular.json config file. (GitHub issue)

  • TypeScript also doesn't support them in the tsconfig.json configuration files. (GitHub issue)

So, with no way of specifying them through a configuration file, the only option left is to programmatically hook into the compilation pipeline. Fortunately, the custom builders from ngx-build-plus and @angular-builders/custom-webpack make this possible without having to ditch the Angular CLI.

These custom builders provide access to the webpack configuration, from which we can obtain the Angular plugin. After having obtained the Angular plugin, we can start adding our own TypeScript AST transformers into the mix. In fact, this has already been demonstrated by David Kingma before.

Because AST transformers are not officially supported in Angular, this means we have to rely on private APIs in the Angular plugin. But as we all know: you shouldn't use private APIs because they are subject to change! And that is exactly why David's method no longer works since Angular 12: the private APIs of the @ngtools/webpack package have changed!

Do we give up?

Of course not!

I strongly believe AST transformers have real value (regardless of the framework). So, I will advocate their use and hope they one day get the official support that they deserve.

Setting up the custom webpack configuration

Ok, enough chit chat. How to actually make use of AST transformers in combination with the Angular CLI? As mentioned before: we need programmic access to Angular's plugin for webpack. A good option for that is @angular-builders/custom-webpack, since it makes it easy to write the full solution in TypeScript. Note that same is possible with ngx-build-plus, but it takes a little more work.

The first step is to configure the custom builders in the angular.json file. Details can be found the README file of the repository. After that we can create the custom webpack configuration:

1
2
3
4
5
6
7
8
// webpack-config.ts
import { Configuration } from 'webpack';

export default function(webpackConfig: Configuration): Configuration {
  // ...

  return webpackConfig;
}

The custom webpack configuration needs to be specified in the angular.json file within the options of the build-architect

1
2
3
4
5
6
7
8
// angular.json: projects.{project-name}.architect.build
{
  "options": {
    "customWebpackConfig": {
      "path": "./webpack-config.ts"
    }
  }
}

Accessing the Angular webpack plugin

This setup now allows us to access and modify the webpack configuration. With that, the next step is to get a reference to the AngularWebpackPlugin:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// webpack-config.ts
import { AngularWebpackPlugin } from '@ngtools/webpack';
import { Configuration } from 'webpack';

export default function(webpackConfig: Configuration): Configuration {
  const angularWebpackPlugin = (webpackConfig.plugins ?? []).find(
    (plugin): plugin is AngularWebpackPlugin => plugin instanceof AngularWebpackPlugin)
  );

  if (!angularWebpackPlugin) {
    return webpackConfig;
  }

  // TODO: Add transformers to the `angularWebpackPlugin`.

  return webpackConfig;
}

Lines 6 through 8 contain the code that attempts to find the Angular webpack plugin. This plugin might not be found, so that is why a bailout is present on lines 10 through 12. In practice, however, this should not happen unless webpack-config.ts is used without the Angular CLI.

Adding transformers to the Angular webpack plugin

As a final step we need to add our own TypeScript transformers into the mix via the AngularWebpackPlugin reference. Unfortunately, this is not as simple as it once was. There no longer is a (private) _transformers property holding an array of the TypeScript transformers (see David's method). So, although it seems we've reached a dead end, we still have options. By inspecting the source code of the AngularWebpackPlugin we find that it eventually calls the createFileEmitter function. This function requires a transformers object of type CustomTransformers as second parameter:

1
2
3
4
5
6
7
8
9
10
11
12
class AngularWebpackPlugin {
  // ...

  private createFileEmitter(
    program: ts.BuilderProgram,
    transformers: ts.CustomTransformers = {},
    getExtraDependencies: (sourceFile: ts.SourceFile) => Iterable<string>,
    onAfterEmit?: (sourceFile: ts.SourceFile) => void,
  ): FileEmitter {

  // ...
}

If only we could modify the transformers argument somehow...

Well, since this is JavaScript, we can monkey patch it! We just overwrite the createFileEmitter function with our own to intercept the calls:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// webpack-config.ts
import { AngularWebpackPlugin } from '@ngtools/webpack';
import { Configuration } from 'webpack';

export default function(webpackConfig: Configuration): Configuration {
  const angularWebpackPlugin = (webpackConfig.plugins ?? []).find(
    (plugin): plugin is AngularWebpackPluginWithPrivateApi => plugin instanceof AngularWebpackPlugin)
  );

  if (!angularWebpackPlugin) {
    return webpackConfig;
  }

  const originalCreateFileEmitterFunction = angularWebpackPlugin.createFileEmitter;

  angularWebpackPlugin.createFileEmitter = function(program, transformers, getExtraDependencies, onAfterEmit) {
    // Here we can add our own transformers:
    transformers?.before.push(yourOwnTypeScriptAstTransformer);

    return originalCreateFileEmitterFunction.call(this, program, transformers, getExtraDependencies, onAfterEmit);
  };

  return webpackConfig;
}

The interception mechanism is coded in lines 14 through 21. First, we obtain a reference to the original function, so it can be invoked later. Next, a new version is installed with the same signature. It simply calls the original function, but not before modifying the transformers object to insert our own transformer.

Note that we also needed to introduce a new type called AngularWebpackPluginWithPrivateApi. This is because the createFileEmitter is private function of the AngularWebpackPlugin. So, the new type is simply an extension of AngularWebpackPlugin that exposes this function as a public member instead.

1
2
3
4
5
6
7
8
9
10
import { BuilderProgram, CustomTransformers, Program, SourceFile } from 'typescript';

type AngularWebpackPluginWithPrivateApi = Omit<AngularWebpackPlugin, 'createFileEmitter'> & {
  createFileEmitter(
    program: BuilderProgram,
    transformers: CustomTransformers,
    getExtraDependencies: (sourceFile: SourceFile) => Iterable<string>,
    onAfterEmit?: (sourceFile: SourceFile) => void,
  ): unknown;
};

Gift wrapping it

The whole solution can be cleaned up a bit, by creating a utility function that takes care of adding the TypeScript AST transformers to the webpack configuration. Another refinement we can make is to expose the Program, to the AST transformers. This will allow them to access additional information, such as the type checker, which enables a richer set of transformations.

After having made the necessary modifications, we end up with an addTransformer function. Using this function makes the custom webpack configuration a lot cleaner. The code below shows an example of how this looks like in a demo application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
import { addTransformer } from 'ngx-ast-transform';
import { Configuration } from 'webpack';

import * as packageInfo from '../package.json';

import { stringSubstitute } from './string-substitute.transformer';
import { typeGuardGenerator } from './type-guard-generator';

export default function(webpackConfig: Configuration): Configuration {
  addTransformer(webpackConfig, stringSubstitute({
    package: packageInfo,
    build: {
      timestamp: new Date().toISOString(),
    },
  }));

  addTransformer(webpackConfig, typeGuardGenerator());

  return webpackConfig;
}

Limitations

Having seen that it is in fact possible to make use of TypeScript AST transformers in Angular applications it is wise to also look at the limitations.

First and foremost: the solution outlined previously does make use of private APIs. The Angular CLI has no official support for AST transformers. This means that with any new Angular release, there is a chance something breaks.

To mitigate this problem, I wrapped the addTransformer function that was introduced previously into small NPM package: ngx-ast-transform. Breaking changes can be caught by this library. Should they occur, then you can just upgrade to the new version of the library and won't need to change the project setup.

Another issue is that this approach only works for applications. But things are different when you're building an Angular library. In that case webpack is not involved, meaning that another solution is required.

Most libraries are built using ng-packgr. Unfortunately, based on the ng-packgr programmtic API there seems to be no way to hook into the compilation pipeline and add AST transformers. So, unless you have setup your own compilation pipeline for Angular libraries, AST transformers currently cannot be used for those.

Summary

TypeScript AST transformers can be used as powerful utilities to boost productivity and improve code base maintainability. Despite their apparent advantages, there is no official support for them in the Angular CLI. You could ditch the Angular CLI and switch to your own compilation pipeline, but from personal experience I know that isn't the most pleasant experience.

Fortunately, there is way to keep the CLI and have AST transformers. Using custom builders like ngx-build-plus or @angular-builders/custom-webpack allow us to access the webpack configuration used by the Angular CLI. This configuration contains an Angular plugin which we can hook into to add our own AST transformers. An outline of how this can be done has been shown in this article.

Since there is no official support in the Angular CLI we are forced to make use of private APIs. This means that we are at risk that a future release might break the way we add AST transformers. To accomodate for these possible breaking changes, the solution has been wrapped in a small library: ngx-ast-transform. When a new release requires a change in the way AST transformers are added, those can be caught in the library, thereby only requiring you to upgrade to a newer version of the library.