Incremental AOT compilation is 5 times slower in Angular 11.1 comparing to 10.1

🐞 bug report

Is this a regression?

Yes, it was better in Angular 10.1.

Description

After upgrading the app to 11.1 I observed regression in build time – both initial and incremental. While intial build time increased by 10%% (150s to 165s), further incremental compilations in watch mode looks times slower.

There are two different cases I wanted to describe:

  • Making an edit in a page-level component – it has a number of dependencies on its own, but relatively few other modules depend on it; it’s imported directly by the routing module.
  • Making an edit into a low-level shared module that is used by virtually every component in the app.

Here is the timing breakdown:

Angular 10.1 Angular 11.1
initial 150s 165s
incremental – page-level edit 3s 8s
incremental – shared low-level edit 9s 40s ❗️

CPU profile (via node --inspect) shows something very strange for Angular 11: a better half of the time it spends in analyzingFileEmitter
Screen Shot 2021-01-29 at 2 52 37 PM

🌍 Your Environment

Angular Version:


Angular CLI: 11.1.2
Node: 12.14.1
OS: darwin x64

Angular: 11.1.1
... animations, cdk, common, compiler, compiler-cli, core, forms
... language-service, material, platform-browser
... platform-browser-dynamic, platform-server, router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1101.2
@angular-devkit/build-angular   0.1101.2
@angular-devkit/core            11.1.2
@angular-devkit/schematics      11.1.2
@angular/cli                    11.1.2
@nguniversal/builders           11.1.1
@nguniversal/express-engine     11.1.1
@schematics/angular             11.1.2
@schematics/update              0.1101.2
rxjs                            6.6.3
typescript                      4.1.3

Anything else relevant?

  • I tried disabling source maps. While it does speed up compilation, it’s still 24s (down from 40s with source maps) for the slow incremental case – way too long.
  • Also "showCircularDependencies": false
Complete `build` section from `angular.json`
"build": {
  "builder": "@angular-devkit/build-angular:browser",
  "options": {
    "outputPath": "dist/browser",
    "index": "src/index.html",
    "main": "src/main.ts",
    "tsConfig": "src/tsconfig.app.json",
    "polyfills": "src/polyfills.ts",
    "preserveSymlinks": true,
    "assets": [
      "src/assets",
      "src/views"
    ],
    "aot": true,
    "progress": false,
    "styles": [
      "src/app/app.scss"
    ],
    "stylePreprocessorOptions": {
      "includePaths": [
        "node_modules"
      ]
    },
    "scripts": [
      {
        "input": "node_modules/browser-update/update.npm.full.js",
        "inject": false,
        "bundleName": "browser-update-3.0"
      },
      {
        "input": "src/browser-update-config.js",
        "inject": false,
        "bundleName": "browser-update-3.0"
      }
    ],
    "namedChunks": true,
    "showCircularDependencies": false,
    "allowedCommonJsDependencies": [
      "lodash",
      "numeral",
      "@splitsoftware/splitio",
      "fuzzysort",
      "hellojs",
      "stale-lru-cache",
      "card-validator",
      "graphql-tag",
      "getstream",
      "@babel/runtime-corejs3",
      "core-js-pure",
      "axios",
      "zen-observable",
      "utfx",
      "jwt-decode"
    ]
  },
  "configurations": {
    "production": {
      "budgets": [
        {
          "type": "anyComponentStyle",
          "maximumWarning": "6kb"
        }
      ],
      "optimization": true,
      "outputHashing": "all",
      "sourceMap": {
        "scripts": true,
        "styles": false,
        "hidden": true,
        "vendor": false
      },
      "extractCss": true,
      "namedChunks": true,
      "extractLicenses": true,
      "vendorChunk": true,
      "buildOptimizer": true,
      "statsJson": true,
      "fileReplacements": [
        {
          "replace": "src/environments/environment.ts",
          "with": "src/environments/environment.prod.ts"
        },
        {
          "replace": "src/environments/development-modules.ts",
          "with": "src/environments/development-modules.prod.ts"
        },
        {
          "replace": "src/environments/development-keys.ts",
          "with": "src/environments/development-keys.prod.ts"
        },
        {
          "replace": "src/main.ts",
          "with": "src/main.prod.ts"
        }
      ]
    },
    "es5": {
      "tsConfig": "src/tsconfig.es5.json"
    }
  }
},

11 thoughts on “Incremental AOT compilation is 5 times slower in Angular 11.1 comparing to 10.1

  1. Thanks for reporting. I suspect the difference is due to the CLI’s new Webpack integration, as there haven’t been any relevant changes on the framework side. If you set environment variable NG_BUILD_IVY_LEGACY=1 the CLI will fallback to the previous Webpack integration implementation, perhaps you could try to see if using that would improve the times to what they were before. It could still be that the new CLI integration is limited by how the compiler operates, so I’ll not move this issue to the CLI repo right away.

    That said, do you think it would be possible to share a reproduction so that we can investigate further?

  2. Same here. Rebuild went up from 3s to 30s for our project in monorepo (10.1.4 -> 11.1.1 update). NG_BUILD_IVY_LEGACY=1 helps, 3s again.

    Some data for rebuild time difference between 11.1.1 default and NG_BUILD_IVY_LEGACY=1 modes (average of 4 passes)

    project1 3s -> 30s
    project2 4.7s -> 95s
    project3 10s -> 180s
    project4 3.4s -> 20s
    

    Timings for ng serve start time or ng build --prod has not changed between these modes. Source maps are disabled.

    packages versions

    Angular CLI: 11.1.2
    Node: 12.18.1
    OS: win32 x64
    
    Angular: 11.1.1
    ... animations, cdk, common, compiler, compiler-cli, core
    ... elements, forms, language-service, localize, material
    ... platform-browser, platform-browser-dynamic, platform-server
    ... router
    Ivy Workspace: Yes
    
    Package                         Version
    ---------------------------------------------------------
    @angular-devkit/architect       0.1101.2
    @angular-devkit/build-angular   0.1101.2
    @angular-devkit/core            11.1.2
    @angular-devkit/schematics      11.0.7
    @angular/cli                    11.1.2
    @angular/flex-layout            11.0.0-beta.33
    @ngtools/webpack                11.1.2
    @nguniversal/builders           11.1.1
    @nguniversal/express-engine     10.1.0
    @schematics/angular             11.1.2
    @schematics/update              0.1101.2
    rxjs                            6.6.2
    typescript                      4.1.3
    webpack                         4.46.0
    
  3. Have the same issue, but it seems that regression is in 11.1.1 specifically.
    I started experiencing the slowness after upgrading from 11.1.0 -> 11.1.1.

    After downgrading @angular-devkit/build-angular @angular/cli @angular/compiler-cli back to 11.1.0 slowness has disappeared.

    For me incremental build results for the same page level edit are:

      Angular Cli 11.1.0 Angular Cli 11.1.1
    lazy-module page level edit ~5s ~55s
  4. Thanks everyone for the reports. I did some profiling of my own using the ng9-aot-build-times repo. What seems to be happening in comparison to the prior Webpack integration pipeline is that now the files that need to be updated according to the compiler are actually incorporated into to rebuilt bundle. This was done to fix bugs where an incremental rebuild did not result in a fully up-to-date bundle.

    Unfortunately however, the Angular compiler is fairly pessimistic in its ability to determine which files need to be updated. I have some ideas to improve this situation but this effort is still underway.

  5. These increased build times have destroyed our productivity with angular.
    i cant believe its getting worse. if your not google it seems your not really relevant.
    this is a years long problem of continued increase in compile times.

    personally i’m done with angular now, it wasnt for the fact we have a huge investment in it, i’d dump the entire codebase.
    the cost of producing code with angular is unsustainable for us.

    frankly it seems that the angular team is more interested in its own goals, and probably serving googles needs than the community at large.

    i guess i was naive to think it would ever be any different.

  6. There’s literally an option to reduce the times back to what they were whilst they try and find a fix.

    To be fair, that option is hard to find. Angular should just release a patch with reverting to the old plugin by default.

  7. i wasnt talking specifically about the 10x – 11x but rather the fact that 10x itself is horribly slow. our compile times on older angular versions were far better. and we’ve spent so much time trying to optimize it. we’ve looked and spent time using nrwl nx, caching, buildable libraries, all of that. the fact we need to spend so much time on it is crazy.
    i had hoped that the 11x version would start to improve the build times.