ES Module Lexer

A JS/TS module syntax lexer used in es-module-shims.

Outputs the list and locations of exports and import specifiers, including dynamic import and import meta expressions.

Supports modern syntax features including import attributes, deferred evaluation, and source phase imports, as well as lexing type-only TypeScript imports and exports in the full build.

A very small single JS file (~7KiB Brotli-compressed for the minimal build) that includes inlined WebAssembly for very fast source analysis of ECMAScript module syntax only.

For an example of the performance, Angular 1 (720KiB) is fully parsed in 1ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.

Comprehensively handles the JS language grammar while remaining small and fast. - ~5ms per MB of JS cold and ~3ms per MB of JS warm.

Built with Chomp

Package Exports

Export	Build	Footprint	Parsing Speed (cold)	Parsing Speed (warm)
es-module-lexer	Full build, Wasm, JS & TypeScript	14.8KiB	4.7ms/MB	3.2ms/MB
es-module-lexer/js	Full build, CSP asm.js, JS & TypeScript	13.1KiB	14.9ms/MB	6.3ms/MB
es-module-lexer/minimal	Minimal build (v2-like API), Wasm, JS only	7.3KiB	4.5ms/MB	3.1ms/MB
es-module-lexer/minimal/js	Minimal build (v2-like API), CSP asm.js, JS only	6.5KiB	10.8ms/MB	5.7ms/MB

• Footprint is the Brotli-compressed size.
• Parse times are per MB of source, measured on the 3.1 MB test/samples set on a standard desktop machine (Node.js 26; warm averages 25 parses), via chomp bench.

Environment Support

The full build requires Node.js 18+ and engines with WebAssembly SIMD support (Chrome 91+, Firefox 89+, Safari 16.4+).

The minimal build (es-module-lexer/minimal) carries no SIMD requirement, running in all browsers with baseline ES modules support (Chrome 63+, Firefox 67+, Safari 11.1+ — the es-module-shims support matrix), with the asm.js builds covering those without WebAssembly.

CSP asm.js Build

The Wasm builds generate no code from strings, so they work with JavaScript eval disabled, including Node.js with --disallow-code-generation-from-strings, as long as the environment permits WebAssembly compilation.

For environments or CSP policies that disable WebAssembly compilation (script-src without wasm-unsafe-eval), use the es-module-lexer/js and es-module-lexer/minimal/js builds:

import { parse } from 'es-module-lexer/js';

Usage

npm install es-module-lexer

See src/lexer.ts for the type definitions.

With the full build:

import { init, parse } from 'es-module-lexer';

await init();

const source = `import { a } from './dep.js';\nexport var p = 5;`;
const [imports, exports] = parse(source);

imports[0].specifier; // "./dep.js"
source.slice(imports[0].importStart, imports[0].importEnd); // "import { a } from './dep.js'"
exports[0].name; // "p"

Or with the minimal (v2-like) build:

import { init, parse } from 'es-module-lexer/minimal';

await init();

const source = `import { a } from './dep.js';\nexport var p = 5;`;
const [imports, exports] = parse(source);

imports[0].n; // "./dep.js"
source.slice(imports[0].ss, imports[0].se); // "import { a } from './dep.js'"
exports[0].n; // "p"

While awaiting init() is always recommended since browser main threads restrict synchronous WebAssembly compilation, in Node.js and other environments it may be optional, since parse will rely on synchronous compilation otherwise. Calling parse before a pending init() has resolved will also fall back to synchronous compilation, so avoid mixing the two.

Minimal Build

See types/lexer.minimal.d.ts for the full interface definitions.

For size-sensitive embedders, the es-module-lexer/minimal build drops certain features to reduce the binary size. This is used for example by es-module-shims.

The minimal build keeps the terse v2-style record shapes rather than the full build's discriminated unions.

import { parse } from 'es-module-lexer/minimal';

const source = `
  import { name } from 'mod';
  import json from './json.json' with { type: 'json' };
  export var p = 5;
  import ('asdf');
  import.meta.url;
`;

const [imports, exports] = parse(source);

// Returns "mod"
imports[0].n
// Returns "mod"
source.slice(imports[0].s, imports[0].e);
// "s" = start
// "e" = end

// Returns "import { name } from 'mod'"
source.slice(imports[0].ss, imports[0].se);
// "ss" = statement start
// "se" = statement end

// Import type is provided by the numeric `t` value (see the ImportType type)
// Returns true
imports[0].t === 1;

// Returns "{ type: 'json' }"
source.slice(imports[1].a, imports[1].se);
// "a" = attribute start, -1 for no import attributes

// Dynamic imports have "d" as the start of the expression argument,
// with -1 for static imports and -2 for import.meta
// Returns "asdf" (only for string literal dynamic imports)
imports[2].n
// Returns "('asdf')"
source.slice(imports[2].d, imports[2].se);

// import.meta is indicated by d === -2
// Returns true
imports[3].d === -2;

// Exports keep the v2 flat { n, ln, s, e, ls, le } shape with no export
// classification, origins, statement starts, or `export *` records
// Returns "p"
source.slice(exports[0].s, exports[0].e);
// Returns "p"
source.slice(exports[0].ls, exports[0].le);

Upgrading from v2

The minimal build is a new entry point holding the v2-shaped API, with the following small differences from v2:

• init is a function returning a promise rather than a promise itself: await init becomes await init(). Calls are idempotent and share one compilation.
• parse returns [imports, exports] only; facade / hasModuleSyntax are dropped.
• at is dropped from import records; read attributes via source.slice(a, se - 1).
• ImportType is a type-only union of the numeric literals (StaticImportType = 1, DynamicImportType = 2, ...) with no runtime export.

Interpolated template specifiers are not globbed in the minimal build (n is undefined for them), and escape sequences in specifiers are decoded into n just as in the full build, including the parse error on invalid escape sequences.

Full Build

See types/lexer.d.ts for the full interface definitions.

The full build lexes both JavaScript and erasable type-only TypeScript syntax, reporting type-only imports and exports with a typeOnly flag.

The full build reports each import and export as a tagged union discriminated by type.

For example:

import { init, parse } from 'es-module-lexer';

await init();

const source = `
  import { name } from 'mod';
  import json from './json.json' with { type: 'json' };
  export var p = 5;
  export { x as 'external name' } from 'external';
  import ('asdf');
  import.meta.url;
  import source mod from './mod.wasm';
`;

const [imports, exports] = parse(source, 'optional-sourcename');

// { type: 'static', specifier: 'mod', phase: null, ... }
imports[0];
// Returns "import { name } from 'mod'"
source.slice(imports[0].importStart, imports[0].importEnd);

// Returns [['type', 'json']]
imports[1].attributes;
// Returns "{ type: 'json' }"
source.slice(imports[1].attributesStart, imports[1].importEnd);

// { type: 'static', specifier: 'external', ... } from the reexport
imports[2];
// { type: 'dynamic', specifier: 'asdf', dynamicStart, ... }
imports[3];
// { type: 'import-meta', ... } spanning "import.meta"
imports[4];
// { type: 'static', phase: 'source', specifier: './mod.wasm', ... }
imports[5];

// { type: 'direct', name: 'p', localName: 'p', ... }
exports[0];
// { type: 'reexport', name: 'external name', importName: 'x',
//   from: 'external', importIndex: 2, ... }
exports[1];

Export Analysis

Detached exports are resolved after the complete module is lexed. An imported binding is therefore classified as a reexport regardless of whether its import appears before or after the export:

const source = `
  export { value as publicValue };
  import { original as value } from 'dep';
`;
const [imports, exports] = parse(source);

exports[0].type === 'reexport';
exports[0].name === 'publicValue';
exports[0].importName === 'original';
exports[0].from === 'dep';
exports[0].importIndex === 0;
imports[0].specifier === 'dep';

When migrating a full-build consumer from v2, await init becomes await init() as in the minimal build, then switch on type before reading kind-specific fields: the terse v2 field names and numeric type tags are replaced by the descriptive names above, reexports no longer expose placeholder local-name properties, and bare star reexports now appear in the exports array with their origins available through importName and imports[importIndex] without rescanning source statements. A dynamic import whose argument is a template literal now reports a glob specifier (./locales/*.js) where v2 reported undefined, so a v2 if (specifier) check now admits globs; test glob to tell them apart (a literal import('a*b') and a template \a${x}b`both reporta*b). import.metarecords carryspecifier: nullandtypeOnly: falseso dependency loops need not narrow ontype` first.

TypeScript

The default parse lexes the type-only import and export syntax that Node.js type stripping erases, so the same lexer that handles your JavaScript also handles those TypeScript edges without a separate transform step:

const [imports, exports] = parse(`
  import type { Foo } from './foo';
  import { bar } from './bar';
  export type { Baz } from './baz';
`);

Type-only imports and exports are reported rather than elided, marked with the typeOnly field:

// import type { Foo } from './foo'  ->  { type: 'static', specifier: './foo', typeOnly: true, ... }
// import { bar } from './bar'       ->  { type: 'static', specifier: './bar', typeOnly: false, ... }
imports[0].typeOnly; // true
imports[1].typeOnly; // false

typeOnly is present on every static import and every export record. Inline modifiers are tracked per specifier, so export { type A, b } marks only A, and directly-exported export type Foo = ... / export interface Foo {} declarations are marked too. Plain JavaScript always reports typeOnly: false: every type-only form is a syntax error in JavaScript except import type from 'x', which is a value import of the default binding named type and keeps its runtime edge, so nothing changes for JavaScript consumers.

Both the Wasm and asm.js / CSP builds (es-module-lexer/js) lex TypeScript. The minimal build (es-module-lexer/minimal) lexes JavaScript only and omits typeOnly.

type and interface declarations are skipped whether exported or not, so an import(...) type buried in an alias right-hand side or an interface body (type T = import('x').Y, interface I { load(): import('x').Y }) is not reported as a runtime import. Ambient declare statements (declare const x: import('x').T;, declare module 'm' { ... }, declare global { ... }) are erased the same way.

export default interface Foo {} is reported as a type-only default export. Declarations with escaped names are erased but not reported as exports. At a line break after a complete alias right-hand side, a line-leading token that can only continue a type (|, &, ?, :, ., =>, extends) keeps the erased region open, so multi-line conditional and union types stay erased.

Caveats

All type-only imports and exports and type / interface declarations are grammar-certain and reported exactly via typeOnly. The one heuristic case is dynamic import() types: es-module-lexer is a lexer, not a full parser, so an import() type in an annotation position (annotations, generic arguments, as / satisfies) is classified best-effort by how its result is used, reported as probablyTypeOnly on the dynamic import record:

• typeof import('m') and exact unescaped non-promise members report probablyTypeOnly: true. Examples include import('m').T and import('m')['x']. Promise members (then, catch, finally), computed names, and escaped names stay runtime.
• await import('m') and promise member access always remain runtime imports.
• A bare unqualified import('m') annotation type is indistinguishable from a value use and reports probablyTypeOnly: false: runtime module graphs over-report rather than under-report.

export declare ambient declarations are erased with their names reported as type-only exports, and TS import-equals is lexed: import A = require('m') keeps a runtime import edge (type-only under import type), while a namespace alias right-hand side (import A = N.M) is erased.

Non-erasable TypeScript (enum, runtime namespace, parameter properties, legacy decorators) is out of scope, matching Node.js type stripping — though export enum E and export namespace N do report their declared value name as a runtime export.

Import Attributes

The attributesStart (a in the minimal build) field provides the index of the start of the { attributes bracket, or -1 for no attributes.

In the full build, the list of attribute key and value pairs of a static import is provided on the attributes field (dynamic import records report attributes: null, with attributesStart locating the options argument):

const [imports] = parse(`
  import json from './foo.json' with { type: 'json' };
  import './foo.css' with { type: 'css' };
  import pkg from 'pkg' with { type: 'json', integrity: 'sha384-...' };
`);

// Returns [['type', 'json']]
imports[0].attributes;

// Returns [['type', 'css']]
imports[1].attributes;

// Multiple attributes
// Returns [['type', 'json'], ['integrity', 'sha384-...']]
imports[2].attributes;

The attributes field is an array of [key, value] tuples, or null if there are no attributes.

Both keys and values support escape sequences:

const [imports] = parse(`
  import foo from './foo.js' with { "custom-key": "value" };
`);

// Quoted keys are unquoted, escape sequences are processed
// Returns [['custom-key', 'value']]
imports[0].attributes;

Escape Sequences

Escape sequences in specifier strings are decoded into the specifier field (n in the minimal build). A specifier that does not decode as a JS string (an invalid escape sequence) throws a parse error, just like the source would in a JS engine.

When the entire dynamic import argument is a single template literal, specifier is reported as a glob: each ${...} substitution is collapsed to a single * (for example import(`./locales/${locale}.js`) yields ./locales/*.js). A template concatenated with anything else, or any other expression, resolves to undefined. Substitutions are matched by the parser itself, so a / inside one is correctly disambiguated as regex or division and does not affect the glob.

The static parts are the raw specifier source: escape sequences are not cooked, and a literal * in the specifier is emitted as-is, so a consumer treating the glob specifier as a pattern has to apply its own escaping. Glob records report glob: true; a literal specifier that happens to contain * reports glob: false.

Star Re-exports

export * from 'module' is both a dependency on module and a re-export of its names. The minimal build reports it as an import only (t === 8), while the full build reports it on both sides:

const source = `export * from './core'`;
const [imports, exports] = parse(source);

// Returns "reexport-all"
exports[0].type;
// Returns "./core"
exports[0].from;

// The specifier is an import record typed 'reexport-star' so it is not
// confused with a side-effect `import './core'` (which is 'static').
imports[0].type === 'reexport-star';
imports[0].specifier;
// Returns "./core"

// The two halves share the same statement range.
source.slice(imports[0].importStart, imports[0].importEnd);
// Returns "export * from './core'"

export * as ns from 'module' is unchanged: it already reports the namespace name ns as a 'reexport', with the specifier as a normal static import.

Facade Detection

Facade modules that only use import / export syntax can be detected via the third return value (full build only):

const [,, facade] = parse(`
  export * from 'external';
  import * as ns from 'external2';
  export { a as b } from 'external3';
  export { ns };
`);
facade === true;

ESM Detection

Modules that uses ESM syntaxes can be detected via the fourth return value (full build only):

const [,,, hasModuleSyntax] = parse(`
  export {}
`);
hasModuleSyntax === true;

Dynamic imports are ignored since they can be used in Non-ESM files.

const [,,, hasModuleSyntax] = parse(`
  import('./foo.js')
`);
hasModuleSyntax === false;

Grammar Support

• Token state parses all line comments, block comments, strings, template strings, blocks, parens and punctuators.
• Division operator / regex token ambiguity is handled via backtracking checks against punctuator prefixes, including closing brace or paren backtracking.
• Always correctly parses valid JS source, but may parse invalid JS source without errors.

Limitations

The lexing approach is designed to deal with the full language grammar including RegEx / division operator ambiguity through backtracking and paren / brace tracking.

Because it lexes rather than fully parses, the analysis is not a validation pass: valid JS source is always analyzed correctly, but some invalid source is accepted without an error rather than rejected. For example export const = 1 lexes to an empty exports list instead of throwing. Callers that need to reject invalid source should run a validating parser separately.

Multiple exports per declaration (export var a = 'asdf', q = z) and renamed destructured exports (export var { a: b } = asdf) are detected correctly; earlier versions missed q and b in these forms.

Nesting is bounded: more than 1024 open brackets or template substitutions, or more than 512 nested dynamic import calls, throws a parse error at the overflowing token.

Building

This project uses Chomp for building.

With Chomp installed, two Emscripten SDKs are expected as sibling folders (or via the EMSDK_PATH and EMSDK_FASTCOMP_PATH environment variables): the upstream emsdk for the Wasm builds and a second checkout as emsdk-fastcomp for the asm.js builds; the build installs and activates the required versions (6.0.0 and 1.40.1-fastcomp) itself.

Example setup:

git clone https://github.com/guybedford/es-module-lexer
git clone https://github.com/emscripten-core/emsdk
git clone https://github.com/emscripten-core/emsdk emsdk-fastcomp
cargo install chompbuild
cd es-module-lexer
chomp test

License

MIT