Angular v10 – Excessive memory consumption at build

Since upgrading to Angular 10, the build process takes an excessive amount of RAM, making it impossible to deploy on Heroku (build fails with OOM error).

For the same code, just upgraded:

  • Angular 9 memory consumption at prod build: ~2 to 2.5 Gb.
  • Angular 10 memory consumption at prod build: ~8 to 9 Gb.

Heroku limit is 2.5 Gb, so it fails with Angular 10.

On my local computer, with no such memory limit, the build works ok.

🐞 bug report

Is this a regression?

Yes

Description

The memory spike seems to occur after ngcc has finished its pre-compilation steps es2015 to esm2015 and starts actual compilation of the code.

Angular Version:

     _                      _                 ____ _     ___
    / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
   / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
  / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
 /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                |___/
    

Angular CLI: 10.0.0
Node: 14.4.0
OS: linux x64

Angular: 10.0.1
... animations, cdk, common, compiler, compiler-cli, core, forms
... language-service, localize, material, platform-browser
... platform-browser-dynamic, router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.1000.0
@angular-devkit/build-angular     0.1000.0
@angular-devkit/build-optimizer   0.1000.0
@angular-devkit/build-webpack     0.1000.0
@angular-devkit/core              10.0.0
@angular-devkit/schematics        10.0.0
@angular/cli                      10.0.0
@angular/flex-layout              10.0.0-beta.32
@ngtools/webpack                  10.0.0
@schematics/angular               10.0.0
@schematics/update                0.1000.0
rxjs                              6.5.5
typescript                        3.9.5
webpack                           4.43.0

6 thoughts on “Angular v10 – Excessive memory consumption at build

  1. Alright, I found the culprit., and it was not what I would have ever expected.

    With the migration to Angular 10, the .browserlistrc was changed in a way where differential loading is no longer enabled (as none of the browsers that require ES5 are included in the browsers list). When differential loading is enabled, the CLI uses a pipeline that works a bit differently in certain areas, one of which is how it deals with asset files. With differential loading disabled, copying assets is done using the copy-webpack-plugin which is currently affected by a bug that’s been fixed in webpack-contrib/copy-webpack-plugin#507, but that’s not yet been released. Without the bug fix, the plugin is invoked for each Webpack compilation, of which there are 121 in the reproduction (I don’t know why it’s that many). Each of those loads ~58MB of asset files into memory, accounting for some 7 GB of memory usage 😱

    I applied the patch in webpack-contrib/copy-webpack-plugin#507 locally and it reduces memory usage by 120 times 58 MB, bringing it down to the levels pre-upgrade.

    Given that there’s no copy-webpack-plugin release yet, the simplest workaround for now would be to revert the changes to the .browserlistrc file to regain 7GB of RAM. And yes, I also find that hard to believe!

    Angular 9 is also affected here since #17867 (released 3 weeks ago in CLI 9.1.8), as the upgrade to v6 of copy-webpack-plugin introduces the regression. It’s only because the browserlist configuration triggered differential loading that it didn’t result in memory issues since 9.1.8.

  2. (Off-topic: Any tip on how I could investigate what seems to be the excessive number of webpack compiles?)

    I am not really familiar with Webpack’s compilation model so I can’t judge if that number is excessive. If every compilation consumes 60MB of RAM then surely it’s a problem, but now that that has been addressed I wouldn’t worry about it.

  3. Guys, I just updated from 9.1.9 to 10.0.9 and ….

    93%% after chunk asset optimization SourceMapDevToolPlugin default~some~modules-~9b5d21b2.js generate SourceMap
    <--- Last few GCs --->
    
    [18280:000001FE67057250]   189594 ms: Mark-sweep 2003.8 (2073.9) -> 1994.8 (2077.7) MB, 1782.6 / 0.2 ms  (average mu = 0.156, current mu = 0.053) allocation failure scavenge might not succeed
    [18280:000001FE67057250]   191210 ms: Mark-sweep 2007.2 (2077.7) -> 1998.0 (2080.4) MB, 1488.9 / 0.1 ms  (average mu = 0.121, current mu = 0.079) allocation failure scavenge might not succeed
    
    
    <--- JS stacktrace --->
    
    ==== JS stack trace =========================================
    
        0: ExitFrame [pc: 00007FF6DB19B3BD]
    Security context: 0x0013d6f808d1 <JSObject>
        1: addMappingWithCode [000002C14728B629] [C:\...\node_modules\webpack-sources\node_modules\source-map\lib\source-node.js:~150] [pc=000002566644FBF4](this=0x02ab
    d2a822c9 <GlobalObject Object map = 0000036EDC529A19>,0x021d48149d29 <Object map = 000003C02CEFC7F9>,0x01e3dc5533e1 <String[9]: isDefined>)
        2: /* anony...
    
    FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
     1: 00007FF6DA585EBF napi_wrap+114095
     2: 00007FF6DA530B46 v8::base::CPU::has_sse+66998
     3: 00007FF6DA531946 v8::base::CPU::has_sse+70582
     4: 00007FF6DAD46E4E v8::Isolate::ReportExternalAllocationLimitReached+94
     5: 00007FF6DAD2EF21 v8::SharedArrayBuffer::Externalize+833
     6: 00007FF6DABFB18C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1436
     7: 00007FF6DAC063C0 v8::internal::Heap::ProtectUnprotectedMemoryChunks+1312
     8: 00007FF6DAC02EE4 v8::internal::Heap::PageFlagsAreConsistent+3204
     9: 00007FF6DABF86E3 v8::internal::Heap::CollectGarbage+1283
    10: 00007FF6DABF6D54 v8::internal::Heap::AddRetainedMap+2452
    11: 00007FF6DAC1809D v8::internal::Factory::NewFillerObject+61
    12: 00007FF6DA97E1E1 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+1665
    13: 00007FF6DB19B3BD v8::internal::SetupIsolateDelegate::SetupHeap+546637
    14: 000002566644FBF4
    npm ERR! code ELIFECYCLE
    npm ERR! errno 134
    npm ERR! ...@0.0.0 ng: `ng "serve"`
    npm ERR! Exit status 134
    npm ERR!
    
    

    before everything was fine

    
         _                      _                 ____ _     ___
        / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
       / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
      / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
     /_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
                    |___/
        
    
    Angular CLI: 10.0.6
    Node: 12.18.3
    OS: win32 x64
    
    Angular: 10.0.9
    ... animations, common, compiler, compiler-cli, core, elements
    ... forms, language-service, platform-browser
    ... platform-browser-dynamic, router
    Ivy Workspace: <error>
    
    Package                            Version
    ------------------------------------------------------------
    @angular-devkit/architect          0.1000.6
    @angular-devkit/build-angular      0.1000.6
    @angular-devkit/build-ng-packagr   0.1000.6
    @angular-devkit/build-optimizer    0.1000.6
    @angular-devkit/build-webpack      0.1000.6
    @angular-devkit/core               8.3.23
    @angular-devkit/schematics         8.3.23
    @angular/cdk                       10.1.3
    @angular/cli                       10.0.6
    @angular/material                  10.1.3
    @ngtools/webpack                   10.0.6
    @schematics/angular                8.3.23
    @schematics/update                 0.1000.6
    ng-packagr                         10.0.4
    rxjs                               6.6.2
    typescript                         3.9.7
    webpack                            4.43.0
    
    
  4. I just upgraded to Angular v10.0.11 in my project where I was using Angular v9.1, I followed the official upgrade guide. I am using docker to build the project for production, and kubernetes to run it in development mode with the development server.

    In my docker container to build with ng build --prod had to increase the memory limit for docker from 2GB to 3GB otherwise the build would crash with “JavaScript heap out of memory”. The same for kubernetes I had to increase the pod memory limit to 3.5GB for it to work when running ng serve for development.

    In Angular v9 it was bad but in v10 it is worse… Could this issue be please reopened?