Skip to content

Assemble catalogues in the bundler module pipeline so edits hot-reload - #78

Open
k0d13 wants to merge 8 commits into
mainfrom
fix/catalogue-module-pipeline
Open

Assemble catalogues in the bundler module pipeline so edits hot-reload#78
k0d13 wants to merge 8 commits into
mainfrom
fix/catalogue-module-pipeline

Conversation

@k0d13

@k0d13 k0d13 commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Fixes #71.

Catalogues were inlined into whoever imported them, which bakes the record into a module whose own bytes never change. No bundler can invalidate on that, so editing a catalogue did nothing until a cache-clearing restart (expo start --clear). Next.js was affected identically.

addExternalDependency could never have helped: Babel freezes the array the moment the plugin factory returns (config/helpers/deep-array.jsObject.freeze), so calling it from a visitor always threw into the catch {}. And neither Metro nor Next consumes externalDependencies anyway — Next declares it in its loader's config type and never turns it into a webpack dependency. Full analysis in the issue.

Approach

Catalogues stay real modules, assembled by each bundler's own module pipeline — the only layer with an invalidation signal.

packages/plugin-babel/src/
  catalogue.ts          shared: resolve fallback chain → assembled record
  metro/index.ts        withSayKit(metroConfig)
  metro/transformer.ts  transform worker wrapper
  next/index.ts         withSayKit(nextConfig)
  next/loader.ts        the loader those rules point at
  index.ts              parserOverride + the inlining visitor

Metro needs a transform worker rather than a loader because it routes .json straight through transformJSON and never runs Babel over it. withSayKit also registers non-JSON catalogue extensions with Metro's resolver, since a file Metro can't resolve isn't a module it can reload.

These two are the whole list of integrations, because they are the two bundlers not reachable any other way. On webpack, Vite, Rollup or esbuild proper, unplugin-saykit already does this through each bundler's own plugin API — there is no babel-plugin-saykit/webpack, and the loader is published only as ./next/loader so Turbopack can name it in a rule.

Assembly is opt-in

Inlining stays the default, so babel-plugin-saykit on its own behaves exactly as before and this is not a breaking change.

// Babel alone — inlines, no bundler config, no hot reload (the old behaviour)
plugins: ['saykit'];

// With a bundler integration — real modules, hot reload
plugins: [['saykit', { catalogues: 'module' }]];

The two are mutually exclusive by nature: if the plugin inlines the import, the integration is never asked for the module. Hence an explicit option rather than a default flip.

Next.js setup is one line

withSayKit derives the rules from saykit.config.* rather than asking you to hand-write them:

// next.config.mjs
import { withSayKit } from 'babel-plugin-saykit/next';
export default withSayKit({});

One rule per bucket, for Turbopack and for next --webpack, each targeting that bucket's output exactly — a Turbopack glob built from the output template, and a webpack predicate. That removes two footguns that were previously documentation warnings:

  • A JSON bucket needed its own rule, and silently served the raw file without one. Now generated automatically.
  • A .json rule would have swept up every JSON import in the app. Deriving from output means only catalogues match. The generated rules also declare the loader's output as JavaScript (type: 'javascript/auto', as: '*.js'), which webpack does not assume for .json.

Verified against running dev servers

Test Result
Next.js edit fr.po with the server running ✅ picked up
Next.js edit en.po while viewing /fr (fallback-only dependency) ✅ picked up via addDependency
Expo/Metro edit fr.json with Metro running, no --clear ✅ picked up
Expo/Metro edit en.json while fr falls back to it ❌ still stale

next build (Turbopack) prerenders /en, /fr, /pl with translations in place. next build --webpack fails in the example, but on @messageformat/parser being transpiled by Next's bundled Babel — a node_modules file no catalogue rule touches, and unrelated to this PR. So the webpack half of withSayKit is covered by unit tests (rule shape, and the predicate matching catalogues but not neighbouring source files) rather than a real build.

429 tests, pnpm check 19/19, lint and format clean.

Known gap (Metro only)

Metro keys its transform cache on each file's own bytes, and getCacheKey takes no filename — it is one global key — so there is no hook to hang a fallback chain's contents on. A fallback locale's contents baked into another locale's record can't be invalidated. Documented as a callout on the Babel integration page. Editing the locale you're viewing — the case in the issue — works.

Tracked as #81, with the analysis: the fix is emitting module.exports = Object.assign({}, require('./en.json'), {…}) so the merge happens over real module edges. Two things make it its own PR — metro-transform-worker routes .json into transformJSON, which never scans for dependencies, and each module has to emit its translations only so a fallback's msgid can't outrank a real translation.

Also in here

  • Metro upstream resolution bases off the config's projectRoot instead of process.cwd(), so --config and monorepo layouts resolve the project's own Metro. Falls back to this package's resolver for hoisted installs. withSayKit is now a no-op if it would wrap its own transformer, which would otherwise recurse until the worker died.
  • Metro cache key is salted with the saykit.config.* contents and this package's version. Previously, editing the fallback chain or a bucket formatter left every catalogue serving its cached record. The config path comes from the new resolveConfigFile, so it cannot silently disagree with the config that was actually loaded.
  • examples/babel — Babel and nothing else, babel src --out-dir dist then node dist/main.js. Every other example runs the plugin alongside a bundler, so this is the only one that would notice if the inlining default broke.
  • Catalogue loading is synchronous, one implementation shared by all three callers. Babel's visitor can't await, and a fallback chain is a handful of files in a pipeline that blocks on the result anyway.
  • @saykit/config exports resolveConfigFile.
  • Fixed the package's types paths, which pointed at .d.cjs while tsdown emits .d.cts — TS consumers were getting no types at all.

@changeset-bot

changeset-bot Bot commented Aug 3, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 25ee5b7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
babel-plugin-saykit Minor
@saykit/config Minor
@saykit/format-json Minor
@saykit/format-po Minor
unplugin-saykit Minor
@saykit/transform-js Minor
@saykit/transform-jsx Minor
saykit Minor
@saykit/carbon Minor
@saykit/react Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Aug 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
saykit Ready Ready Preview Aug 4, 2026 12:29am

@github-actions github-actions Bot added examples Updates or additions to example apps tests Modifications, additions, or fixes related to testing package: babel-plugin Related to babel-plugin-saykit website Updates to the documentation website labels Aug 3, 2026
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The PR adds inline and module catalogue modes to the Babel plugin. It adds Next.js, Turbopack, and Metro integrations, shared catalogue loading, cache invalidation, examples, documentation, package exports, and release changesets.

Changes

Catalogue loading and Babel boundary

Layer / File(s) Summary
Catalogue loading and Babel modes
packages/config/src/features/loader/*, packages/plugin-babel/src/catalogue.ts, packages/plugin-babel/src/index.ts, packages/plugin-babel/src/index.test.ts
The plugin now supports catalogues: 'inline' and catalogues: 'module'. Shared loading resolves catalogue sources and fallback files. Tests cover record assembly and both modes.
Configuration file resolution
.changeset/olive-pugs-repeat.md
The changeset records the exported resolveConfigFile helper.

Next.js and Turbopack integration

Layer / File(s) Summary
Next.js catalogue rules and loader
packages/plugin-babel/src/next/*, packages/plugin-babel/package.json, packages/plugin-babel/tsdown.config.ts
withSayKit adds Turbopack and webpack catalogue rules. The loader emits JavaScript records and tracks source files.
Next.js example and release metadata
examples/nextjs/*, website/content/integrations/babel.mdx, .changeset/fresh-owls-guess.md
The Next.js example enables module catalogues and wraps its configuration with withSayKit. Documentation and release metadata describe the integration.

Metro transformer integration

Layer / File(s) Summary
Metro wrapper and transformer
packages/plugin-babel/src/metro/*
withSayKit configures catalogue extensions and resolves the upstream transformer. The transformer emits catalogue JSON or JavaScript, delegates other files, and adds configuration-based cache data.
Expo integration
examples/expo/*, website/content/integrations/babel.mdx, .changeset/tidy-moons-agree.md
Expo uses module catalogue handling and the Metro wrapper. Documentation describes transformer-based catalogue handling and cache invalidation.

Babel example and integration documentation

Layer / File(s) Summary
Babel example
examples/babel/*
The example demonstrates inline catalogue compilation, English and French records, pluralisation, and fallback output.
Catalogue mode documentation
website/content/integrations/babel.mdx, examples/babel/README.md, .changeset/hot-donkeys-shave.md, .vscode/settings.json
Documentation covers inline and module handling, fallback chains, hot reload, dependency tracking, and static imports. The changeset records the module option.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

  • k0d13/saykit issue 79 — The issue proposes extending the Next.js catalogue loader to transform SayKit macros, which is implemented here as a Next.js and webpack loader foundation.

Possibly related PRs

  • k0d13/saykit#12 — The Babel changes use the configuration-driven transformer architecture introduced there.
  • k0d13/saykit#36 — The shared catalogue loading uses related fallback and record-assembly behaviour.
  • k0d13/saykit#42 — Both changes cover fallback-aware catalogue transformation and JSON output handling.

Poem

A rabbit packs records, locale by locale,
Through Metro and Next, each catalogue.
Babel keeps imports or builds them in flight,
Cache keys refresh when the files change right.
“Hop reload!” cries the hare with delight.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 70.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes moving catalogue assembly into the bundler pipeline to enable hot reload.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/catalogue-module-pipeline

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (2)
packages/plugin-babel/src/metro/index.ts (1)

39-44: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Guard against a double wrap.

If a caller applies withSayKit twice, metroConfig.transformerPath already points at this package's transformer.cjs. saykitTransformerPath then points at the SayKit transformer itself. upstream() in packages/plugin-babel/src/metro/transformer.ts line 26 loads that same module, so every transform recurses until the stack overflows. Add a check that returns the config unchanged when it is already wrapped.

🛡️ Proposed guard
+  const self = join(__dirname, 'transformer.cjs');
+  if (metroConfig.transformerPath === self) return metroConfig;
+
   const upstream = metroConfig.transformerPath ?? 'metro-transform-worker';
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/plugin-babel/src/metro/index.ts` around lines 39 - 44, Update the
wrapper logic around the transformerPath assignment in withSayKit to detect when
metroConfig.transformerPath already resolves to this package's transformer.cjs
and return metroConfig unchanged. Ensure the existing wrapping behavior remains
intact for unwrapped configurations and prevents transformer.ts upstream() from
loading the SayKit transformer recursively.
packages/plugin-babel/src/index.test.ts (1)

79-90: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Make the fallback test self-contained.

This test asserts farewell: 'Bye'. That key only exists because the previous test at lines 66-77 wrote en.json into the shared dir. The test then depends on file order and on the earlier test running. Write en.json inside this test as well, or move the shared fixture into a beforeEach.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/plugin-babel/src/index.test.ts` around lines 79 - 90, The fallback
test around loadCatalogue must create its own source-locale fixture instead of
relying on the preceding test's en.json. Write the required en.json data within
the test, or initialize the shared fixture in beforeEach, while preserving the
existing assertions for greeting, farewell, and sources.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.changeset/hot-donkeys-shave.md:
- Line 5: Update the release-note wording around the Metro integration to
qualify the hot-reload claim: state that active-locale catalogue edits
hot-reload, while fallback-locale edits may remain stale because Metro does not
register fallback files as dependencies.
- Line 2: Update the one-line release note for babel-plugin-saykit to include
the migration requirement: existing consumers must add the
babel-plugin-saykit/webpack loader or babel-plugin-saykit/metro configuration so
catalogue imports are transformed instead of remaining in source form.

In `@packages/plugin-babel/src/metro/index.ts`:
- Around line 36-49: Update the upstream resolution in the Metro configuration
flow to derive the createRequire base from the caller-provided config path
rather than process.cwd(), so --config paths and monorepos resolve the project’s
Metro dependency correctly. Preserve absolute upstream paths, and fall back to
this module’s own resolver when no usable config path is available.

In `@packages/plugin-babel/src/metro/transformer.ts`:
- Around line 55-57: Update getCacheKey to combine the upstream transformer key
with a SayKit-specific salt derived from this package’s version and stable
SayKit configuration fields, including fallback-chain and bucket-formatter
settings. Do not serialize the entire config when it may contain functions; use
a deterministic representation of only serializable stable fields so
configuration changes invalidate Metro’s cache.

In `@website/content/integrations/babel.mdx`:
- Around line 33-49: Update the Next.js configuration example around the
catalogue and webpack rules to document JSON catalogue handling: add a matching
*.json Turbopack rule and webpack loader rule, or explicitly state that JSON
catalogues require their own rule, while preserving the existing *.po
configuration.

---

Nitpick comments:
In `@packages/plugin-babel/src/index.test.ts`:
- Around line 79-90: The fallback test around loadCatalogue must create its own
source-locale fixture instead of relying on the preceding test's en.json. Write
the required en.json data within the test, or initialize the shared fixture in
beforeEach, while preserving the existing assertions for greeting, farewell, and
sources.

In `@packages/plugin-babel/src/metro/index.ts`:
- Around line 39-44: Update the wrapper logic around the transformerPath
assignment in withSayKit to detect when metroConfig.transformerPath already
resolves to this package's transformer.cjs and return metroConfig unchanged.
Ensure the existing wrapping behavior remains intact for unwrapped
configurations and prevents transformer.ts upstream() from loading the SayKit
transformer recursively.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 1d0db4ad-34f5-4afc-b573-7df751f9935c

📥 Commits

Reviewing files that changed from the base of the PR and between 77a2228 and 5ab74f9.

📒 Files selected for processing (12)
  • .changeset/hot-donkeys-shave.md
  • examples/expo/metro.config.js
  • examples/nextjs/next.config.mjs
  • packages/plugin-babel/package.json
  • packages/plugin-babel/src/catalogue.ts
  • packages/plugin-babel/src/index.test.ts
  • packages/plugin-babel/src/index.ts
  • packages/plugin-babel/src/metro/index.ts
  • packages/plugin-babel/src/metro/transformer.ts
  • packages/plugin-babel/src/webpack/index.ts
  • packages/plugin-babel/tsdown.config.ts
  • website/content/integrations/babel.mdx

Comment thread .changeset/hot-donkeys-shave.md
Comment thread .changeset/hot-donkeys-shave.md Outdated
Comment thread packages/plugin-babel/src/metro/index.ts Outdated
Comment thread packages/plugin-babel/src/metro/transformer.ts
Comment thread website/content/integrations/babel.mdx Outdated
@k0d13
k0d13 force-pushed the fix/catalogue-module-pipeline branch from 5ab74f9 to 8692d4d Compare August 3, 2026 08:56
@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Aug 3, 2026
@pkg-pr-new

pkg-pr-new Bot commented Aug 3, 2026

Copy link
Copy Markdown

Open in StackBlitz

@saykit/config

npm i https://pkg.pr.new/@saykit/config@78

@saykit/format-json

npm i https://pkg.pr.new/@saykit/format-json@78

@saykit/format-po

npm i https://pkg.pr.new/@saykit/format-po@78

saykit

npm i https://pkg.pr.new/saykit@78

@saykit/carbon

npm i https://pkg.pr.new/@saykit/carbon@78

@saykit/react

npm i https://pkg.pr.new/@saykit/react@78

babel-plugin-saykit

npm i https://pkg.pr.new/babel-plugin-saykit@78

unplugin-saykit

npm i https://pkg.pr.new/unplugin-saykit@78

@saykit/transform-js

npm i https://pkg.pr.new/@saykit/transform-js@78

@saykit/transform-jsx

npm i https://pkg.pr.new/@saykit/transform-jsx@78

commit: 25ee5b7

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

The preview build of 25ee5b7 is published. Open it in the playground to run this pull request against your own code, straight from the browser.

@github-actions github-actions Bot added package: config Related to @saykit/config and the CLI and removed preview Publish a preview build and link it from the pull request labels Aug 3, 2026
@k0d13
k0d13 force-pushed the fix/catalogue-module-pipeline branch from a109168 to 2a77e73 Compare August 3, 2026 23:15
@k0d13 k0d13 closed this Aug 3, 2026
@k0d13
k0d13 deleted the fix/catalogue-module-pipeline branch August 3, 2026 23:55
@github-actions github-actions Bot added the dependencies Updates or changes related to project dependencies label Aug 3, 2026
@k0d13
k0d13 restored the fix/catalogue-module-pipeline branch August 3, 2026 23:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/babel/README.md`:
- Line 17: Update the fenced output block in the README to specify the text
language, changing the opening fence to use the text identifier so it satisfies
markdownlint MD040.
- Around line 3-4: Update the build command in the Babel example README to use
the package.json build script, `pnpm build`, instead of the incomplete direct
Babel invocation; keep the subsequent `node dist/main.js` run command unchanged.

In `@packages/plugin-babel/src/index.ts`:
- Around line 49-65: Update the import handling around the specifier lookup to
require exactly one ImportDefaultSpecifier and reject declarations containing
any named or namespace specifiers, preserving the existing code-frame error
behavior. Add a regression test covering `import messages, { extra } from
'./messages.json'` and verify the mixed import is rejected rather than removing
the additional binding.

In `@packages/plugin-babel/src/metro/index.ts`:
- Line 52: Update resolveConfigFile and resolveConfig to accept an optional
project-directory parameter and use it for config discovery instead of always
relying on process.cwd(). In the Metro wrapper around the config resolution
call, pass metroConfig.projectRoot ?? process.cwd(), preserving the existing
fallback and ensuring the same root used by the upstream transformer is applied.

In `@packages/plugin-babel/src/metro/transformer.ts`:
- Line 27: Update the transformer’s resolve flow around config and cache
initialization to pass Metro’s projectRoot into resolveConfig and
resolveConfigFile, replacing implicit process.cwd() lookup. Key all
resolved-config results and cache-salt values by that same projectRoot so
separate projects cannot share configuration or caches.

In `@website/content/integrations/babel.mdx`:
- Around line 99-103: Update the Babel integration description around the
catalogue import paragraph to distinguish catalogue modes: explain that Babel
resolves and merges catalogue files only for inline mode, while module mode
preserves the import for the Next.js loader or Metro transformer to assemble.
Keep the surrounding transformer and runtime behavior description accurate.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f1789780-20e3-4a38-aee0-9f84ee601db1

📥 Commits

Reviewing files that changed from the base of the PR and between 5ab74f9 and a066fae.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (31)
  • .changeset/fresh-owls-guess.md
  • .changeset/hot-donkeys-shave.md
  • .changeset/olive-pugs-repeat.md
  • .changeset/tidy-moons-agree.md
  • .vscode/settings.json
  • examples/babel/README.md
  • examples/babel/babel.config.js
  • examples/babel/package.json
  • examples/babel/saykit.config.ts
  • examples/babel/src/locales/en.d.po.ts
  • examples/babel/src/locales/en.po
  • examples/babel/src/locales/fr.d.po.ts
  • examples/babel/src/locales/fr.po
  • examples/babel/src/main.ts
  • examples/babel/tsconfig.json
  • examples/expo/babel.config.js
  • examples/expo/metro.config.js
  • examples/nextjs/.babelrc
  • examples/nextjs/next.config.mjs
  • packages/config/src/features/loader/index.ts
  • packages/config/src/features/loader/resolve.ts
  • packages/plugin-babel/package.json
  • packages/plugin-babel/src/catalogue.ts
  • packages/plugin-babel/src/index.test.ts
  • packages/plugin-babel/src/index.ts
  • packages/plugin-babel/src/metro/index.ts
  • packages/plugin-babel/src/metro/transformer.ts
  • packages/plugin-babel/src/next/index.ts
  • packages/plugin-babel/src/next/loader.ts
  • packages/plugin-babel/tsdown.config.ts
  • website/content/integrations/babel.mdx
🚧 Files skipped from review as they are similar to previous changes (3)
  • .changeset/hot-donkeys-shave.md
  • packages/plugin-babel/tsdown.config.ts
  • examples/expo/metro.config.js

Comment thread examples/babel/README.md Outdated
Comment thread examples/babel/README.md Outdated
Comment thread packages/plugin-babel/src/index.ts Outdated
Comment thread packages/plugin-babel/src/metro/index.ts
Comment thread packages/plugin-babel/src/metro/transformer.ts
Comment thread website/content/integrations/babel.mdx
@k0d13 k0d13 added the preview Publish a preview build and link it from the pull request label Aug 4, 2026
@github-actions github-actions Bot removed the preview Publish a preview build and link it from the pull request label Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Updates or changes related to project dependencies examples Updates or additions to example apps package: babel-plugin Related to babel-plugin-saykit package: config Related to @saykit/config and the CLI tests Modifications, additions, or fixes related to testing website Updates to the documentation website

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HMR broken for expo/babel: adding new strings requires expo start --clear

1 participant