Allow subdirectories within git repos in yarn install

23 thoughts on “Allow subdirectories within git repos in yarn install”

1. There are a couple of good ideas for syntax to use to specify subdirectories in the linked issue (npm/npm#2974)

   Use pip’s way of using a query parameter
   git+https://git.repo/some_repo.git?subdirectory=subdir_path#branch
   or
   git+https://git.repo/some_repo.git#branch?path=subdir_path
   Although I would assume most url parsers won’t parse this one.

   Use :: as it is not a valid token for a branch name
   git+https://git.repo/some_repo.git#branch::subdir_path
git+https://git.repo/some_repo.git#::subdir_path

2. Another idea: git+https://git.repo/some_repo.git/subdir_path#branch. If I were to assume this feature existed, that’s what I would try first. The other options would require someone to look up the syntax.

3. That might be ambiguous depending on how the git server urls are setup.

   The url might be something like

   git+https://github.com/yarnpkg/yarn/<subdir_path>#branch

   Then it become hard to say what is part of the repo path and what is the subdir_path

4. Not sure if this applies to every repo, but with Github both git+https://github.com/yarnpkg/yarn and git+https://github.com/yarnpkg/yarn.git work. You could assume anything after .git/ is a sub-directory.

   Regardless of syntax it would be nice to have this feature either way. It would be cool to make the syntax intuitive though :).

5. Long discussion about supporting this in npm:
   npm/npm#2974

   There are some suggested separators in the thread.

6. Yeah, rising popularity of monorepo architectures makes it a crucial feature. Is there any workaround for this, for now? Discussion about public API tend to be years long…

7. My # 1 use-case for this is pulling in bug-fixes from monorepo’s that haven’t been merged or released yet. I often want to keep most of the original packages, but pull in one of the sub-repos with a bug-fix. Currently I am unable to do this, and the workarounds are complex. Being able to reference a Github branch would greatly lower the resistance.

   I know there are some people at npm that are highly resistant to the idea of this, and monorepos in general. However, many of the contributors here work on projects that use monorepos. I would suspect that if yarn were to implement this feature and it became popular enough npm would follow suit. A lot of yarn ideas have been implemented in npm later on. yarn to me is the babel of npm.

8. @erykpiast couple workarounds that I can think of:

   1. clone the target repo somewhere and then use yarn add /path/to/repo/subdirectory
   2. or possibly (if it’s in your own monorepo) use yarn workspaces?

   Or — this is maybe even crazier — add the remote in question as a submodule and (optionally move your project files into a subdirectory) then use yarn workspaces?

   Disclaimer: I haven’t tried either of these approaches and they both add overhead for new developers & your CI builds…

9. Or — this is maybe even crazier — add the remote in question as a submodule and (optionally move your project files into a subdirectory) then use yarn workspaces?

   I can confirm this works.

   Upgraded to webpack 4. Create React App doesn’t have support yet but there’s a PR that’s waiting to be merged – unfortunately webpack 4 has changed it’s plugin API. Doubly unfortunate: create-react-app is repo containing multiple packages. Adding the forked PR as a submodule and adding the package using ‘file:’ syntax worked a charm.

   Literally saved an entire day or more of work with this work around.

10. It’s alarming that this isn’t on the roadmap for either npm or yarn due to the rise in popularity of monorepos. The git+... syntax makes contributing to single-package projects a breeze, and by contrast not having it for subdirectories makes contributing to monorepo projects a total headache.

11. @Frikki I don’t believe lerna has much submodule support, and isn’t a “consumption tool” anyways.

    @jwdunne For now I just npm publish a namespaced version of the fork and depend on it

12. Hello! I built https://github.com/ramasilveyra/gitpkg to “publish” packages from monorepos as git tags, it may help.

13. Hello! I built https://github.com/ramasilveyra/gitpkg to “publish” packages from monorepos as git tags, it may help.

    This may look a bit hacky at the beginning, but after some reasoning it seems to be a legitimate solution to the problem 👍

14. Seriously, gitpkg works! It creates tags inside the project repository, and can be used with repositories containing multiple packages.

    And this prompted me to ditch npm because only yarn can fetch nicely from Git.

15. +1

16. I built a service https://gitpkg.now.sh/ to download the sub folder of a git repo as a .tgz file, so that it can be installed by yarn and npm. With this service, package users can use the subdir directly and there is no need for package developers to publish the subdirs.

    For example, if you want to add babel-core in branch next-8-dev as dependency,
    just use:

    yarn add https://gitpkg.now.sh/babel/babel/packages/babel-core?next-8-dev
    

    also works for npm install.

    Note that I must say sorry to @ramasilveyra because my service’s name conflicts with the tool he develops but I didn’t know that before.

17. @EqualMa that’s even more than awesome! Thank you!

18. Any updates to this? Is it planned or anything?

19. I’m having an issue trying to fork a repo (https://github.com/ianstormtaylor/slate) that uses workspaces and has multiple packages in the same repo. I only want to modify and import one of the packages. Trying to yarn add pointing to the directory that holds the individual package.json file gives an error…

    yarn add https://github.com/kr-project/slate/tree/master/packages/slate#e73db650a332fc5ded3fd72cffb1fcd7279b85d7
    error https://github.com/kr-project/slate/tree/master/packages/slate: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?"
    

    I’ve also tried to check in the published files, but my understanding is this is a problem with yarn looking a tar only found at the root directory of a github repo?

    So is the only way to fork a project that uses workspaces to use something like gitpkg? I’m a bit surprised that there doesn’t seem to be more native support for this.

20. So @LucidDan and others.

    I’ve found the following to work:

    • You must be running yarn 2.1.X for this to work.
    • See this page for the newer syntax that allows this: https://yarnpkg.com/features/protocols
    • You have to add individual entries for each package in the monorepo to your package.json under either dependencies or, in my case, devDependencies:

    "package-1": "my-org-name/mono-repo-name#head=my-git-branch&workspace=package-1"
    

    It’s disappointing that there isn’t a way to just have yarn install the top-level monorepo and unroll all of the attendant packages into their proper place. Maybe there is and I’m just being stupid here? If so, someone enlighten me 😉

21. Try https://gitpkg.now.sh/

    yarn add https://gitpkg.now.sh/<user>/<repo>/<subdir>[?<branch>]

    OR

    "@react-native-community/cli-platform-ios":"https://gitpkg.now.sh/nomi9995/cli/packages/platform-ios?fix-ios-getProductName-regex"
    

Comments are closed.