Preface

As many front-end developers probably know, more than a year since sindresorhus published the esm only manifesto last year, many projects have started to move to esm only, i.e., esm only and not cjs, in order to force the whole ecosystem to migrate to esm only faster.

Some popular projects already do this

  • thousands of npm packages maintained by sindresorhus
  • node-fetch
  • remark series
  • more.

They claim: you can still use the existing version without upgrading to the latest version, and that major version updates will not affect you. What are the facts?

I’ve had a few problems before with not being able to use esm only packages, whenever I tried to try esm only, there were always a few more problems, the most painful being that some packages were esm only and others were cjs only, always having to choose to drop one side. fuck esm only. the main problems were some cjs only packages, and packages that had to be compatible with typescript/jest/ts-jest/wallaby does not support esm properly. of course, I can choose to look for alternatives to esm only packages, such as globby => fast-glob, remark => markdown-it, node-fetch => node-fetch@2, and lodash-es => lodash, but this is not a permanent option after all, not to mention that some packages are hard to actually find replacements for, such as the remark family.

So, what are the problems with using older versions of packages? The main problem is that it is hard to find the correct version. Of course, if you are using a relatively independent package, such as node-fetch, you can just use the v2 version. But if you are using a project like vuepress/remark where monorepo contains many small packages, you will have a hard time finding the correct version of each subproject.

I recently had to do some conversions from markdown and manipulate ast to html when I was working on the epub generator, so I used remark again and decided to really try using esm. here are some of the steps I tried.

Objectives

There are several issues that must be addressed to use esm, otherwise it is not possible to use it in a production environment

  • typescript support - basically all web projects use ts and it is unacceptable not to support it
  • jest support - also a heavily used testing tool
    • wallaby support - a paid WYSIWYG testing tool
  • Allow references to cjs modules - requires support for existing packages
  • Dual module packages can still support both esm/cjs projects - requires support for cjs project references
  • Support for unpackaged modules - some private modules in monorepo will not bundle
  • esbuild support - esbuild is becoming a lib bundle standard

Modify package declaration

The first step is to modify the module type of the package, modify "type": "module" to declare the package as esm, all js code will run as esm module by default.

1
2
3

{
  "type": "module"
}

TypeScript support

NodeNext is supported since ts4.7, so you need to change tsconfig.json.

1
2
3
4
5
6

{
  "compilerOptions": {
    "module": "ESNext",
    "moduleResolution": "NodeNext"
  }
}

Also, importing other ts files within ts files must use the .js suffix.

This is a strange restriction, see ts 4.7 release documentation.

1
2

import { helper } from "./foo.js"; // works in ESM & CJS
helper();

Does it seem strange, but it’s the only way to write it now, and typescript will even prompt for it.

jest/wallaby support

For example, run the following code using the pnpm jest src/lodash.test.ts command.

1
2
3
4
5

import { uniq } from "lodash-es";

it("uniq", () => {
  console.log(uniq([1, 2, 1]));
});

An error occurred

1

Jest encountered an unexpected token

Experimental esm support is supported from jest 28 onwards, and wallaby/ts-jest are also supported by configuration, which can be handled by following the steps below.

  1. Configuring ts-jest

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

    {
"jest": {
    "preset": "ts-jest/presets/default-esm",
    "globals": {
    "ts-jest": {
        "useESM": true
    }
    },
    "moduleNameMapper": {
    "^(\\.{1,2}/.*)\\.js$": "$1"
    },
    "testMatch": ["<rootDir>/src/**/__tests__/*.test.ts"]
}
}

  2. change the command to node --experimental-vm-modules node_modules/jest/bin/jest.js src/lodash.test.ts

  3. Configure wallaby (This is where you can run the test files in the __tests__ directory, oddly enough.)

    1
2
3
4
5
6
7
8
9

    {
"wallaby": {
    "env": {
    "params": {
        "runner": "--experimental-vm-modules"
    }
    }
}
}

  4. Since the esm import is static, you also need to uninstall @types/jest and use the @jest/globals package to import functions needed for testing, such as it/expect/describe/beforeEach and so on.

    1
2
3
4
5

    import { it, expect } from "@jest/globals";

it("basic", () => {
expect(1 + 2).toBe(3);
});

nodejs support

nodejs has been supporting esm since 14, but the migration has not been smooth until now at 18, mainly due to the following issues.

Importing cjs only modules

Unfortunately, a large number of existing packages are for cjs only modules and it is not possible to migrate them anytime soon, and the interoperability between esm and cjs in nodejs is not very good, so it needs to be handled. Here is an example of fs-extra.

This is how it would normally be written before.

1
2
3
4

import { readdir } from "fs-extra";
import path from "path";

console.log(await readdir(path.resolve()));

An error occurs when running with tsx SyntaxError: The requested module 'fs-extra' does not provide an export named 'readdir', this seems to be a known error, refer to: https://github.com/esbuild-kit/tsx/issues/38

Now it needs to be modified as follows.

1
2
3
4

import fsExtra from "fs-extra";
import path from "path";

console.log(await fsExtra.readdir(path.resolve()));

Or modify it to the following code. Run with ts-node --esm <file> (this is not supported by tsx).

1
2
3
4

import fsExtra = require("fs-extra");
import path from "path";

console.log(await fsExtra.readdir(path.resolve()));

Using __dirname

Yes, you read that right, under the esm module __dirname is no longer available, it is replaced by import.meta.url, in short, it is now used in the following way.

1
2
3
4
5
6

import path from "path";
import { fileURLToPath } from "url";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
console.log(__dirname);

Refer to the article https://flaviocopes.com/fix-dirname-not-defined-es-module-scope/, and later, when we talk about esbuild, we will talk about how to handle import.meta.url in the cjs bundle (which is not supported in cjs, again, it’s a choice between (which is not supported in cjs and is again an option).

lib maintenance and usage

New esm and cjs dual package support configuration

Previously, we differentiated modules by the main/module field.

1
2
3
4
5

{
  "main": "dist/index.js",
  "module": "dist/index.esm.js",
  "types": "dist/index.d.ts"
}

However, an exception occurs when referencing in an esm project.

1

SyntaxError: The requested module 'cjs-and-esm-lib' does not provide an export named 'hello'

The esm project does not recognize this, it has a new exports field, so you need to add (note that the main field still needs to remain compatible with older node versions) the exports field.

1
2
3
4
5
6
7
8
9

{
  "exports": {
    ".": {
      "import": "./dist/index.esm.js",
      "require": "./dist/index.js",
      "types": "./dist/index.d.ts"
    }
  }
}

Refer to this answer: https://stackoverflow.com/a/7002098

esbuild support

I thought esbuild would be easy because it inherently supports esm, but I actually ran into quite a few problems.

Binding the following code to cjs gives an error.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import path from "path";
import { fileURLToPath } from "url";
import fsExtra from "fs-extra";

const { readdir } = fsExtra;

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
console.log(__dirname);
console.log(await readdir(__dirname));

Command

1
2

esbuild src/bin.ts --platform=node --outfile=dist/bin.esm.js --bundle --sourcemap --format=esm
esbuild src/bin.ts --platform=node --outfile=dist/bin.js --bundle --sourcemap --format=cjs

Error

1

[ERROR] Top-level await is currently not supported with the "cjs" output format

Here, because cjs cannot contain top-level await, it is modified as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

import path from "path";
import { fileURLToPath } from "url";
import fsExtra from "fs-extra";

const { readdir } = fsExtra;

(async () => {
  const __filename = fileURLToPath(import.meta.url);
  const __dirname = path.dirname(__filename);
  console.log(__dirname);
  console.log(await readdir(__dirname));
})();

There is no problem with the bundle, but it runs with errors.

1

node dist/bin.js

First is the first mistake.

1
2
3
4
5
6

var import_path = __toESM(require("path"), 1);
                  ^

ReferenceError: require is not defined in ES module scope, you can use import instead
This file is being treated as an ES module because it has a '.js' file extension and 'esm-demo\packages\esm-include-cjs-lib\package.json' contains "type": "module". To
treat it as a CommonJS script, rename it to use the '.cjs' file extension.

It says this is an esm package and the default code is the esm module, if you want it to be executed as a cjs module, you need to change it to the cjs suffix.

Modify the command.

1

esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --bundle --sourcemap --format=cjs

Then there was a second error.

1

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string or an instance of URL. Received undefined

Related Codes

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

// src/bin.ts
var import_path = __toESM(require("path"), 1);
var import_url = require("url");
var import_fs_extra = __toESM(require_lib(), 1);
var import_meta = {};
var { readdir } = import_fs_extra.default;
(async () => {
  const __filename = (0, import_url.fileURLToPath)(import_meta.url); // 这里是关键，因为 import.meta.url 在 cjs 代码中是空的
  const __dirname = import_path.default.dirname(__filename);
  console.log(__dirname);
  console.log(await readdir(__dirname));
})();

According to the author’s answer in this issue, modify the command as follows.

1

esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --inject:./import-meta-url.js --define:import.meta.url=import_meta_url --bundle --sourcemap --format=cjs

Unfortunately, this is not in effect and the bundle’s code is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

// import-meta-url.js
var import_meta_url2 = require("url").pathToFileURL(__filename);
console.log(import_meta_url2);

// src/bin.ts
var import_path = __toESM(require("path"), 1);
var import_url = require("url");
(async () => {
  const __filename2 = (0, import_url.fileURLToPath)(import_meta_url);
  const __dirname = import_path.default.dirname(__filename2);
  console.log(__dirname);
})();

You can clearly see that the variable name of the injected script is changed from import_meta_url => import_meta_url2, which is oddly problematic.

Maybe replace --inject => --banner.

1

esbuild src/bin.ts --platform=node --outfile=dist/bin.cjs --define:import.meta.url=import_meta_url --bundle --sourcemap --banner:js="var import_meta_url = require('url').pathToFileURL(__filename)" --format=cjs

This takes effect.

What about running the esm bundle?

It also gives an error.

1
2
3

throw new Error('Dynamic require of "' + x + '" is not supported')

Error: Dynamic require of "fs" is not supported

Modify the command according to the solution here.

1

esbuild src/bin.ts --platform=node --outfile=dist/bin.esm.js --bundle --sourcemap --banner:js="import { createRequire } from 'module';const require = createRequire(import.meta.url);" --format=esm

Now, the code after the bundle can finally run.

Conclusion

Maybe esm only looks good, and tree shaking looks like a great idea, but right now, it’s not even really available in production. This includes a number of important projects that have not been migrated, including react/vscode/electron/vite and so on. In fact, many people (including me) have used esm modules to write code before this, except that the final bundle product might be cjs, e.g. iife in the browser, cjs in nodejs, but the vast majority of application-level developers don’t care about that, only the lib maintainers do, and esm only shifts the complexity of using packages The complexity of using packages is also transferred to the user, and there is no real available solution for referencing esm only packages in cjs. The esm only movement is more of an orgy in the web front-end community than a project like esbuild/vite that solves a real problem.