Skip to content

Add an interactive Web Platform Test runner - #631

Merged
andrewiggins merged 6 commits into
mainfrom
wpt-runner-interactive
Aug 31, 2026
Merged

andrewiggins merged 6 commits into
mainfrom
wpt-runner-interactive

Conversation

@andrewiggins

@andrewiggins andrewiggins commented Aug 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR is 2 of 3 (#630, #632) that adds a Web Platform Test runner to this repo, so we can validate our polyfills spec compliance against that test suite.

This PR contains the actual runner that executes the WPT test (main.ts). It's a webpage (index.html) that creates a Web Worker (run-worker.ts), loads our DOM polyfill into that worker (worker.ts), and parses the WPT test into JS instructions (adapter.ts) that it executes in that worker with the DOM polyfill. It uses the WPT test harness to execute the test inside the worker and get the results.

Specifically, this PR:

  • securely serve prepared WPT files with canonical path and symlink containment
  • adapt supported top-level HTML/SVG markup and classic scripts into generated source
  • execute each selected file in a fresh worker and polyfill Window
  • keep results on a private transferred MessagePort
  • make the first terminal event final and always clean up the worker
  • block worker networking and nested workers with a worker-specific CSP
  • provide an interactive page for running tests and inspecting source and results

Stacked dependency

This PR is stacked on wpt-runner-source-preparation, which prepares and validates the pinned WPT checkout.

Executable outcome

Run pnpm wpt:dev, enter a WPT path, and inspect the result, warnings, console output, original markup, harness source, and generated source.

This PR intentionally does not add capability policy, Playwright CLI automation, or WPT CI.

Validation

  • pnpm run lint
  • pnpm run type-check
  • pnpm exec vitest run
  • interactive three-case smoke execution
  • repeated-run fresh-worker validation

Changeset

An empty changeset records that this private test infrastructure releases no public package.

@@ -0,0 +1,64 @@
<!doctype html>

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is just a test file we can use to smoke test the runner. It demonstrates what a classic WPT test looks like.

@@ -0,0 +1,40 @@
const __WPT_SVG_NAMESPACE__ = 'http://www.w3.org/2000/svg';

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The WPT testharness needs some additional featues polyfill doesn't have yet so this adds the required shims so tests can run. Over time we might implement some of these in polyfill so this file might go away. Left that as out of scope for now.

@@ -0,0 +1,318 @@
const SVG_NAMESPACE = 'http://www.w3.org/2000/svg';

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file parses a WPT HTML test file and converts it to JS instructions we can execute in a worker against our Remote DOM polyfill. So a declaration like <svg viewBox> becomes a createElement('svg') and setAttribute(viewBox instruction.

It also handles downloading the script tags a test might reference as some times write their tests in a separate JS file. We download an execute those directly in the order they are in the HTML file.

Comment on lines +55 to +59
const AsyncFunction = Object.getPrototypeOf(async function () {})
.constructor as new (source: string) => () => Promise<void>;
await new AsyncFunction(
`${request.source}\n//# sourceURL=wpt:${request.path}`,
)();

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the meat. We take the generated bundle of createElement instructions that buildWptBundle in adapter.ts created, and execute them in an async function whose scope is this worker, upon which RemoteDom's polyfills are installed.

@andrewiggins
andrewiggins force-pushed the wpt-runner-interactive branch 2 times, most recently from ab43cd2 to 93a9744 Compare August 26, 2026 01:14
Base automatically changed from wpt-runner-source-preparation to main August 27, 2026 22:44
andrewiggins added a commit that referenced this pull request Aug 27, 2026
## Summary

This PR is 1 of 3 (#631, #632) that adds a Web Platform Test runner to this repo, so we can validate our polyfills spec compliance against that test suite.

This PR focuses on just downloading an archive of the WPT repo we can use to read test files. Future PRs setup the runner to execute them against our DOM polyfills. Since we are on NodeJS 24, all scripts are written in TypeScript and executed directly in Node.

Specifically, this PR:
- add a private workspace package for preparing upstream Web Platform Test sources
- pin the WPT revision and archive SHA-256 checksum
- validate archive paths and links before extraction
- publish revision-addressed caches atomically
- serialize concurrent preparation with abandoned-lock recovery

## Executable outcome

Run `pnpm wpt:prepare` to prepare or reuse the verified checkout. Set `WPT_ROOT` to use an existing checkout or `WPT_CACHE_DIR` to override the cache.

Commits:
* Add pinned WPT source preparation
* Add empty changeset for WPT preparation
* Migrate WPT preparation to TypeScript
* Enforce isolated declarations for WPT runner
@andrewiggins
andrewiggins force-pushed the wpt-runner-interactive branch 2 times, most recently from 90976f1 to 642f4d7 Compare August 27, 2026 22:47
@andrewiggins
andrewiggins force-pushed the wpt-runner-interactive branch from 642f4d7 to 49e1791 Compare August 31, 2026 19:22
@andrewiggins
andrewiggins merged commit a479a99 into main Aug 31, 2026
6 checks passed
@andrewiggins
andrewiggins deleted the wpt-runner-interactive branch August 31, 2026 20:20
andrewiggins added a commit that referenced this pull request Aug 31, 2026
## Summary

This PR is 3 of 3 (#630, #631) that adds a Web Platform Test runner to this repo, so we can validate our polyfills spec compliance against that test suite.

This PR adds a CLI to run the webpage from the previous PR in playwright. It also adds a way to specify which tests in WPT are expected to pass/fail, and why (if failure). So overtime we can document behaviors we support, ones we don't support *yet*, and which ones we explicitly don't want to support.

Specifically, this PR:
- add a canonical TSV inventory of supported and deferred WPT cases
- reject supported failures, missing results, unlisted results, duplicates, harness errors, worker errors, and timeouts
- report passing deferred cases as promotion candidates
- add a Playwright CLI that reuses one control page while creating a fresh worker per file
- add a dedicated cached WPT CI job

## Stacked dependency

This PR is stacked on `wpt-runner-interactive`, which provides secure serving and isolated worker execution.

## Executable outcome

Run `pnpm test:wpt` to execute every classified path and enforce the complete capability table. Use explicit paths for exploratory execution and `--capabilities` to enforce selected classified paths.

## Changeset

An empty changeset records that this private test infrastructure releases no public package.

Squashed commit messages:

* Add WPT capability enforcement
* Run WPT capabilities in CI
* Migrate WPT capability scripts to TypeScript
* Enforce native TypeScript syntax for WPT runner
* Separate WPT runner TypeScript environments
* Enforce isolated declarations for WPT capability runner
* Document WPT capability runner functions
* Align WPT capability runner tooling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants