Extracting the tar content of undefined failed

  1. Clone https://github.com/codercom/code-server
  2. Checkout 3fbdb2e46c4f38268df42a03835b43151f9e55b2
  3. Run yarn
  4. You’ll often get something similar to:
ERROR Failed to install "browser" dependencies {"error":{"killed":false,"code":1,"signal":null,"cmd":"yarn --network-concurrency 1"},"stdout":"[1/4] Resolving packages...\n[2/4] Fetching packages...\ninfo Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.\n","stderr":"warning package.json: No license field\nwarning ../../package.json: No license field\nwarning @coder/app: No license field\nerror https://registry.yarnpkg.com/@material/base/-/base-0.41.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: \"ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v4/npm-@material-base-0.41.0-badadce711b4c25b1eb889a5e7581e32cd07c421/node_modules/@material/base/component.js'\"\n"}

Related #6312

I’m on the latest stable release of yarn, 1.13. And version 8.10 for node (yes it is outdated but shouldn’t affect this afaik).

Author: Fantashit

1 thought on “Extracting the tar content of undefined failed

  1. For me the issue was running concurrent yarn install for different project, that use the same yarn cache, as @nhooyr mentioned. I guess workaround is pass different folders for each install command --cache-folder but I haven’t tried.

    I had a script, similar to this which reproduces the problem

    (cd a && yarn install) &
    (cd b && yarn install) &
    
    wait %%1
    wait %%2

Comments are closed.

Extracting the tar content of undefined failed

Do you want to request a feature or report a bug?
Reporting a bug when running yarn install to install node dependencies. For severity, this bug seems critical considering it essentially prevents me from acquiring node dependencies.

What is the current behavior?
Fails sometimes with errors like the following:

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7/_cacheHas.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EEXIST: file already exists, mkdir '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/fbjs/-/fbjs-0.8.17.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-fbjs-0.8.17-c4d598ead6949112653d6588b01a5cdcd9f90fdd/lib/resolveImmediate.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command

The occurrence of this error is the challenging part. It does not always fail and does not always fail with the same dependency. The installation is sometimes successful after 3-5 tries.

If the current behavior is a bug, please provide the steps to reproduce.
I’ve attempted to install dependencies on bare-metal and in a node:8-alpine docker container. Both can sometimes encounter the error. I’ve tested this on my personal device in Montreal, Canada (Mac OS X10.13), on a AWS EC2 instance (Ubuntu 18.04), on a GCE instance (Ubuntu 16.04) and on a production server in France (Debian 8). Each of them can sometimes encounter this error. I’ve also tried installing with and without yarn.lock to no avail. Find a package.json that appears to sometimes reproduce the issue in this gist. The issue does not seem to happen with projects having fewer dependencies.

What is the expected behavior?
Successful installation of all packages like npm install or npm ci which deterministically succeeds without any tar or caching errors.

Please mention your node.js, yarn and operating system version.
Tested with the following version:
Node: 8 LTS, 10
Yarn: 1.9.2, 1.9.4
OS: Ubuntu 18.04 LTS, Ubuntu 16.04 LTS, Debian 8, Mac OSX 10.13
Registrie: registry.yarnpkg.com, registry.npmjs.org, private registry

If you require any additional information, don’t hesitate to request it. FWIW, reducing the network-concurrency seems to produce a slightly higher success ratio but not consistently enough to conclude the errors are related. It may be an area to investigate however. I’ve unfortunately exhausted all the time I could afford to spend on this after a few days of troubleshooting. I’ve reluctantly had to migrate all our CI builds back to using npm install/npm ci 🙁

Author: Fantashit

14 thoughts on “Extracting the tar content of undefined failed

  1. I’m seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it’s actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that’s because the other processes are kicked off by the root process rather than by different root processes. I don’t know if this information helps or if its actually way off base. But I figured I would share what I’ve found regardless.

  2. I’ve been tracking this down with a project we have and so far narrowed it down to the concurrent install the git-fetcher starts here. If the package that is being installed by the git-fetcher has any of the same dependencies of the currently installing package a race condition is created where the duplicated packages will be untarred to offline cache at the same time.

    I haven’t seen enough of the code base to understand where/what the correct fix is, but that’s the start of the issue.

  3. I spoke too soon. I had asked my team mate if we’d tried this, rather than actually trying myself and he was very confident that we had…he was mistaken and miss-read the previous post thinking it was related to the mutex flag, not network concurrency. We’ve since re-tried and can confirm this does seem to also address the issue for us.

  4. setting --network-concurrency 1 does not actually work for me.

    right now, the only workaround I’ve come across involves completely regenning yarn.lock. The error I get is:

    2.054 Performing "GET" request to "https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz".
    verbose 2.519 Error: https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
        at MessageError.ExtendableBuiltin (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:237:66)
        at new MessageError (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:266:123)
        at Extract.<anonymous> (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:59446:14)
        at emitOne (events.js:121:20)
        at Extract.emit (events.js:211:7)
        at Extract.module.exports.Extract.destroy (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135306:17)
        at Extract.module.exports.Extract._final (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135364:34)
        at callFinal (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:70270:10)
        at _combinedTickCallback (internal/process/next_tick.js:139:11)
        at process._tickCallback (internal/process/next_tick.js:181:9)

    Update: I’ve just found that using --skip-integrity-check allows me to bypass this error. While obviously that’s really a solution. This looks like kind of an important bug in the integrity check logic.

    Im using node@8.12.0, yarn@1.12.0

  5. There’s no ~/.npmrc created in my case. But regenerating yarn.lock worked for me.

    Simply,

    $ rm yarn.lock && yarn
    

    EDIT: Faced this issue twice only to end up landing here. 😄

  6. The problem is that the npm registry is generally unstable and returns errors (at a higher rate when multiple requests are firing apparently – maybe some kind of per-ip throttling?). For some reason they are not properly caught by Yarn, which blindly tries to hash them and compare them to the expected hash – which fails.

    So there is a bug in Yarn (we should print a more helpful error), but given that the real problem is how flaky the npm registry is, it’s not my priority at the moment (I’d definitely review a PR, though!).

    As for why it doesn’t happen with npm: they retry their requests until they work. Yarn has a mechanism to do that, but not on the part that specifically compute the hash.

    I’d suggest to use an offline mirror to stop relying on the npm registry for your installs.

  7. #6817 will “fix” that by showing the actual status code returned by the registry. I’d prefer it to be stable rather than blindly retry until it works so I haven’t added the retry code, but if there’s no improvements to the horizon we might have to do that.

    In the meantime I’ll close this discussion, as the error messages will change and this thread grown quite large (we can open new ones to discuss each status code individually).

Comments are closed.