Skip to content

Optimize esm exports by shifting the protocol burden to getters - #98932

Merged
lukesandberg merged 2 commits into
canaryfrom
fewer_esm_tags
Oct 2, 2026
Merged

Optimize esm exports by shifting the protocol burden to getters#98932
lukesandberg merged 2 commits into
canaryfrom
fewer_esm_tags

Conversation

@lukesandberg

@lukesandberg lukesandberg commented Sep 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

To support live bindings and cyclic references we bind esm exports as getter properties. However this is ambiguous with binding raw values, so we would always use 0 as a tag to flag that the next value was a value not a getter function.

In practice value exports account for 90% of exports and getters only 10% (from vercel-site). So this changes the protocol so the getter functions require tags instead saving a small number of bytes in the output.

@lukesandberg
lukesandberg added this pull request to stack #98899 September 19, 2026 06:23
@github-actions

github-actions Bot commented Sep 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: ad4cd34

@lukesandberg
lukesandberg marked this pull request as ready for review September 19, 2026 16:56
@sokra sokra mentioned this pull request Sep 21, 2026
Merged
@sokra

sokra commented Sep 21, 2026

Copy link
Copy Markdown
Member

In practice value exports account for 90% of exports and getters only 10%.

Is that a guess or from a real measurement?

@lukesandberg

lukesandberg commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

Is that a guess or from a real measurement?

it was measured on vercel-site vercel-marketing had a similar number. Actually live exports are very rare, so the main reason we bind getters is for cycles... which are more common but still rare.

@lukesandberg
lukesandberg removed this pull request from stack #98899 September 22, 2026 00:00
@lukesandberg
lukesandberg changed the base branch from fewer-circuit-breaker-modules to optimize-constant-reexport-bindings September 22, 2026 00:01
@lukesandberg
lukesandberg added this pull request to stack #99013 September 22, 2026 00:01
@lukesandberg
lukesandberg requested a review from sokra September 22, 2026 00:09
sokra added a commit that referenced this pull request Sep 22, 2026
@sokra @vercel-fleet-prod @lukesandberg
Turbopack: compact re-export registration (#98617)
812c88f 
## Summary

Adds a compact Turbopack runtime path for modules whose exports are
entirely forwarded from other modules. It replaces one generated wrapper
getter per binding with a flat re-export description, while preserving
live-binding semantics and source evaluation order.

This revision also makes the optimization generic for synthetic
re-export facades, rather than leaving facade integration in the stacked
export-mangling PR:

- synthetic facades use `ExportRegistrationMode::Reexport`;
- their explicit reference list supplies ordering because they have no
source `ImportMap`;
- an evaluation-only locals reference can be represented as an empty
group;
- references whose import is performed by the compact registration do
not emit it twice;
- pruned references and merged evaluation-only references are omitted,
because they deliberately produce no namespace binding and may have no
module factory in the chunk;
- async modules and known export-cycle circuit breakers retain the
general registration path.

The runtime now inspects source property descriptors. Constant exports
reuse the source value, live exports reuse the exact
receiver-independent Turbopack namespace getter, and
dynamic/proxy/missing properties fall back to a wrapper getter.
Destination descriptors remain enumerable, non-configurable, read-only
ESM namespace properties; setters are never forwarded.

## Output examples

A re-export-only module can suppress its redundant import and use a
module-id group:

```js
// before
var ns = __turbopack_context__.i("a.js")
__turbopack_context__.s(["a", () => ns.a, "b", () => ns.b])

// after
__turbopack_context__.S(["a.js", "a,a,b,b"])
```

Two source modules become groups separated by `0`:

```js
__turbopack_context__.S([
  "first.js", "a,a,b,b",
  0,
  "second.js", "c,c",
])
```

Encoding is selected independently per group. A comma-containing name
uses explicit pairs while another group can remain packed:

```js
__turbopack_context__.S([
  "first.js", "b", "b", "has,comma", "has,comma",
  0,
  "second.js", "c,c",
])
```

A synthetic facade includes its evaluation-only locals part as an empty
group, then its forwarded bindings in reference order:

```js
__turbopack_context__.S([
  "reexports.js <locals>",
  0,
  "x.js", "x,x",
  0,
  "y.js", "y,y",
])
```

Inside a scope-hoisted factory, imports are retained and existing
namespace objects are used as heads because another logical module in
the merged factory may read them:

```js
var locals = __turbopack_context__.i("reexports.js <locals>")
var x = __turbopack_context__.i("x.js")
__turbopack_context__.S([locals, 0, x, "x,x"], facadeId)
```

A merged evaluation-only locals reference is omitted from that list:
scope hoisting supplies only an in-factory ordering placeholder, not a
namespace variable that `S` could read.

## Runtime representation

For each forwarded property, `S` reads the source descriptor:

- data descriptor → capture the value with the runtime's value tag;
- getter descriptor → reuse that exact getter (Turbopack-generated
namespace getters are receiver-independent);
- no usable own descriptor → use `() => namespace[name]` as a defensive
fallback.

Tests pin constant primitives, frozen objects, function values, live
updates, exact getter identity, receiver independence, destination
descriptor flags, omitted setters, and the fallback path.

## Verification

- `cargo test -p turbopack-ecmascript -p turbopack-tests` — **590 unit,
302 execution, 3 auxiliary, 131 snapshot** tests passed on this layer
- Full stacked #97770 verification — **590 unit, 303 execution, 3
auxiliary, 131 snapshot** tests passed
- `cargo clippy -p turbopack-ecmascript -p turbopack-core -- -D
warnings`
- `cargo fmt --all -- --check`
- Runtime TypeScript check (`pnpm check` in
`turbopack-ecmascript-runtime/js`)
- `pnpm build-all`
- Dedicated coverage for ordinary import/re-export ordering, transformed
`DUMMY_SP` declarations, callable CommonJS namespaces, synthetic facade
ordering, scope-hoisted retained imports, pruned and merged evaluation
references, async fallback, and export-cycle circuit breakers

## Incremental size on top of #98932

Scratch comparison only, as requested: #98617 stayed based on `canary`.
Three apps were built with scope hoisting both disabled and enabled.
Client reports raw and deterministic gzip (`gzip -9 -n`, sorted JS
files); server reports raw only. Every row was repeated byte-for-byte.

| benchmark | scope hoisting | client raw | client gzip | server raw |
real `S` sites (client/server) |
| --- | --- | ---: | ---: | ---: | ---: |
| `basic-app` | off | +510 B | +214 B | **−6,114 B** | 0 / 56 |
| `basic-app` | on | +510 B | +164 B | +8,618 B | 0 / 0 |
| `heavy-npm-deps` *(lodash + Mantine + Mermaid)* | off | **−2,268 B** |
**−432 B** | +4,680 B | 3 / 8 |
| `heavy-npm-deps` *(lodash + Mantine + Mermaid)* | on | +522 B | +191 B
| +8,630 B | 2 / 2 |
| `module-cost` | off | +510 B | +263 B | +7,432 B | 0 / 6 |
| `module-cost` | on | +510 B | +265 B | +8,624 B | 0 / 1 |

Aggregate raw movement across the six rows is **+32,164 B**. The shared
helper is only amortized in configurations with many eligible facade
calls; scope hoisting commonly resolves forwarded bindings to locals,
leaving few calls while every runtime copy still carries the helper.

Descriptor reuse does **not** change generated module code or call-site
counts. Compared with the same facade integration using wrapper getters,
it adds **155 B client raw** and **527 B per runtime copy** (**1,054 B
server raw** in these apps). Its purpose is avoiding per-binding wrapper
allocation / access indirection where a source value or getter already
exists, not reducing bundle bytes. The size result was reported before
continuing, and descriptor work proceeded by explicit maintainer
direction.

<!-- NEXT_JS_LLM -->

<!-- fleet ecdfa248-cd54-41ac-b4a2-c9d49e2a67ee -->

---------

Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
@lukesandberg
lukesandberg force-pushed the fewer_esm_tags branch 2 times, most recently from 8285a66 to cda41c7 Compare October 1, 2026 22:56
@lukesandberg
lukesandberg removed this pull request from stack #99013 October 1, 2026 23:39
@lukesandberg
lukesandberg added this pull request to stack #99562 October 1, 2026 23:40
@lukesandberg
lukesandberg force-pushed the fewer_esm_tags branch 2 times, most recently from 97ff034 to af73a80 Compare October 2, 2026 17:14
@lukesandberg
lukesandberg removed this pull request from stack #99562 October 2, 2026 17:42
@lukesandberg
lukesandberg changed the base branch from optimize-constant-reexport-bindings to canary October 2, 2026 17:43
@lukesandberg
lukesandberg added this pull request to stack #99598 October 2, 2026 17:47
@lukesandberg
lukesandberg merged commit 680081d into canary Oct 2, 2026
101 checks passed
@lukesandberg
lukesandberg deleted the fewer_esm_tags branch October 2, 2026 22:58
lukesandberg added a commit that referenced this pull request Oct 3, 2026
@lukesandberg
Fix CI break caused by #98932 (#99621)
5964438 
No idea how that PR had a clean bill of health?

Some kind of race with export mangling getting enabled by default?
bgw added a commit that referenced this pull request Oct 5, 2026
@bgw
Revert "Optimize esm exports by shifting the protocol burden to gett…
263f682 
…ers (#98932)" (#99704)

Also reverts #99621

Discussion:
https://vercel.slack.com/archives/C04KC8A53T7/p1791231703532919?thread_ts=1791230213.615279&cid=C04KC8A53T7

This code depends on the exact format used for exports 😢😠:

https://github.com/vercel/remote-components/blob/201b460403d5813f632b687fab1ad2fc2ce4174b/packages/remote-components/src/runtime/turbopack/module.ts#L380-L401
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Oct 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@sokra sokra sokra approved these changes

Assignees

No one assigned

Labels

locked

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lukesandberg @sokra