Posted in Uncategorized

Virtual Module API

   February 23, 2021  1 Comment on Virtual Module API

Do you want to request a feature or report a bug?
Feature Reqeust

If this is a feature request, what is motivation or use case for changing the behavior?

The Problem

In vue-loader, we process Vue single file components that can contain multiple language blocks:

<template>/* ... */</template>
<script>/* ... */</script>
<style>/* ... */</style>

Each of these blocks need to be delegated to other webpack loaders for potential pre-processing. The way it currently works is by overloading the same source module with different inline loader chains:

// (output from vue-loader)
import template from '!!vue-template-compiler!selector?part=template!self.vue'
import script from '!!babel-loader?babelOptions!selector?part=script!self.vue'
import style from '!!style-loader!css-loader!selector?part=style!self.vue'

This does work and we’ve been using it for quite a long time. However, this leads to a number of problems:

  • Because all loader options must be inlined, it’s impossible to use non-serializable option values.

  • Because the requests all end in *.vue, we cannot rely on the configured rules in the main webpack config. The user has to duplicate the same config in both module.rules and vue-loader’s own loaders option, or we have to somehow infer it correctly.

    This can be circumvented if the pre-processor supports a fs-based config file, e.g. .babelrc or .postcssrc, but in some cases the user for some reason cannot use a config file, or the loader in question simply does not support config files.

  • If the user chains another loader before vue-loader, we have to respect that too. This causes the chained loader to be invoked many times (1 extra call for every language block in a vue file).

  • (cosmetic) these inline requests result in extremely long module names and makes error and stats output difficult to read.

Proposed API

@TheLarkInn had an experimental idea of a virtual-dependency-loader which is almost what we want. Except it doesn’t seem to work as intended. I tested it with a virtual dependency with a filename ending in .js, and a configured babel-loader does not apply to the loaded virtual module.

I propose a new loader context API loadVirtualModule that is similar to loadModule:

// in a loader
module.exports = function (source) {
  const cb = this.async()

  const descriptor = vueCompiler.parseComponent(source)

  this.loadVirtualModule({
    code: descriptor.script.content,
    map: descriptor.script.map,
    filename: './does-not-exist.js'
  }, (err, code, map) => {
    // the script part of the source, processed with all matching loaders
    // for does-not-exist.js
  })

  // even better if it returns Promise
  Promise.all([
    this.loadVirtualModule({ ... }), // script
    this.loadVirtualModule({ ... }),  // template
    this.loadVirtualModule({ ... }),  // style
  ]).then(parts => {
    const { code, map } = vueCompiler.assemble(parts)
    cb(null, code, map)
  })
}

This would greatly simplify the implementation of vue-loader and solve the problems we are facing above.

Author: Fantashit

1 thought on “Virtual Module API”

  1. Ok I have two possible workarounds for you:

    1

    Because all loader options must be inlined, it’s impossible to use non-serializable option values.

    You can “abuse” the system for complex options to pass complex options via inline loaders.

    The trick is that these complex queries need to be in the use option in the first place for this to work.

    const vueLoader = (options) => {
  // assuming options contains script, template, style and each of it is an array of loaders/use
  return [
    {
      loader: "vue-loader",
      options: {
        script: options.script.length, 
        template: options.template.length, 
        style: options.style.length
      }
    },
    ...options.script,
    ...options.template,
    ...options.style
  ]
}
    module.rules: [
  {
    test: /\.vue$/,
    use: vueLoader({
      script: [],
      template: ["vue-template-loader"],
      style: [
        {
          loader: "postcss-loader",
          options: { /* complex */ }
      ]
    })
  }
]

    In the pitching phase the loader API allows you to read and modify loaders via this.loaders this.currentLoaderIndex.

    // vue-loader
// Get the position of the style, template, and script loaders by the help of the length noted in the helper
let scriptLoaders = this.loaders.slice(this.currentLoaderIndex + 1, this.query.script);
let templateLoaders = this.loaders.slice(this.currentLoaderIndex + 1 + this.query.script, this.query.template);
const remainingLoaders = this.loaders.slice(this.currentLoaderIndex + 1 + this.query.script + this.query.template + this.query.style);
// ...
const remainingRequest = remainingLoaders.map(l => l.request + l.query).concat([this.request]).join("!");
const templateRequest = "-!" + templateLoaders.map(l => l.request + l.query).concat([
  "selector?part=template",
  remainingRequest
]).join("!");

// generate import
return `import template from ${loaderUtils.stringifyRequest(templateRequest)};\n....`;

    This would generate something like

    postcss-loader??ref-0-2 for complex queries.

    (cosmetic) these inline requests result in extremely long module names and makes error and stats output difficult to read.

    No more that long…

    If the user chains another loader before vue-loader, we have to respect that too. This causes the chained loader to be invoked many times (1 extra call for every language block in a vue file).

    That’s true. It’s maybe workaroundable when using this.loadModule in the selector-loader to load the remaining request.

    Because the requests all end in *.vue, we cannot rely on the configured rules in the main webpack config. The user has to duplicate the same config in both module.rules and vue-loader’s own loaders option, or we have to somehow infer it correctly.

    Technically users can use JS in the configuration to solve duplication.

    const postcssOptions = { ... }
module.rules: [
  { test: /\.vue$/, ..., options: { style: postcssOptions } },
  { test: /\.css$/, ..., options: postcssOptions }
]

    2

    With these rules:

    module.rules: [
  { test: /\.vue$/, resourceQuery: /template/, use: [
    "vue-template-compiler"
  ] },
  { test: /\.vue$/, resourceQuery: /script/, use: [
    "babel-loader"
  ] },
  { test: /\.vue$/, resourceQuery: /style/, use: [
    "css-loader"
  ] },
  { test: /\.vue$/, use: [
    "vue-loader"
  ] },
]

    You can generate this code when vue-loader is called with !this.resourceQuery:

    import template from 'self.vue?template'
import script from 'self.vue?script'
import style from 'self.vue?style'

    This expands to these loaders (because of the rules):

    self.vue?template -> vue-template-compiler!vue-loader!self.vue?template

    When the vue-loader is called with this.resourceQuery set, you can do the stuff the the selector did before and extract only the specified section.

    (cosmetic) these inline requests result in extremely long module names and makes error and stats output difficult to read.

    With this solution it would be very great, because the module names would be

    self.vue, self.vue?template, …

    because you are no longer using inline loaders.

    If the user chains another loader before vue-loader, we have to respect that too. This causes the chained loader to be invoked many times (1 extra call for every language block in a vue file).

    If this really is an issue, you could store the result of the remaining loaders when the vue-loader (without query) is invoked and restore it in the pitching phase of the vue-loader (with query). This would a small in-memory cache. This is safe, because without query is always invoked before with query, but to be extra sure you could add a content hash to the query: self.vue?4db28a2-style. Make sure to not leak memory with this cache…

    2 is nicer…

Comments are closed.