Skip to content

feat: compare images with pixelmatch 8 - #1289

Merged
plum117 merged 2 commits into
mainfrom
chore/pixelmatch-8
Oct 9, 2026
Merged

plum117 merged 2 commits into
mainfrom
chore/pixelmatch-8

Conversation

@plum117

@plum117 plum117 commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Upgrades the comparison engine of @wdio/image-comparison-core from pixelmatch 7.2 to 8.0 (replaces Dependabot #1259), for v11.

pixelmatch 8 measures color differences in the OKLab color space with the HyAB distance instead of YIQ: closer to how people see colors, with fewer false positives and fewer missed changes. The API, the threshold scale (0–1, 1 = black vs white) and its default did not change, and it still supports our public checkerboard option. But mismatch numbers can change a little, so the changeset is a major (like the other v11 changes of these packages).

Changes

  • pixelmatch ^7.2.0 → ^8.0.0.
  • The ignore* presets keep their values (0.063, 0.13, 0). The comments and the README that described them in YIQ terms now describe the pixelmatch scale, and the README says that v11 uses OKLab/HyAB.
  • Changeset (major) for @wdio/image-comparison-core and @wdio/visual-service, with a note for users.

Why pixelmatch 8 and not @blazediff/core (#1195)

Benchmark on 5 real image pairs from our runs (same page with another clock, iPhone full page with a 1-px shift, the 2 Android display states, the Android black band, Linux vs Docker fonts), with our default and antialiasing settings:

pixelmatch 7 pixelmatch 8 @blazediff/core
Different pixels vs pixelmatch 7 — −2.5 % … +3 % (largest: 919 → 1161 px, 0.002 % → 0.003 %, 1-px shift with antialiasing) identical
Speed — about the same up to 2.7× faster on a 2202×18822 full page
checkerboard (public option since #1181) ✓ ✓ ✗ (silently ignored)
Metric YIQ OKLab + HyAB YIQ

@blazediff/core is a faithful, faster pixelmatch 7, but it would drop a public option and keep the old metric. Closing #1195 (thanks to the author) and #1259 after this PR.

Test

  • pnpm test (lint, types, 1077 unit tests, of which 784 in image-comparison-core): pass.
  • Local e2e: test.local.chrome.v10, .jasmine, .emulation, test.local.desktop.multi, test.local.desktop (setup run, then the real run, including the checks that expect a mismatch in a range): pass.
  • New compareImages.checkerboard.test.ts: the real pixelmatch on a transparent vs opaque-white image, checkerboard: false → 0 %, true → 100 %. It fails when compareImages stops passing the option; pixelmatch 7 gives the same results.
  • Cloud (run 37862194406, commit 0a30be1): all committed baselines pass with the new metric, with no failure and no retry:
Job Result
Desktop (LambdaTest) 8/8 spec files
iOS web (LambdaTest) 6/6 spec files
Android web (LambdaTest) 6/6 spec files
OCR (LambdaTest) pass
Mobile app (Sauce Labs) pass

🤖 Generated with Claude Code

pixelmatch 8 measures color differences with OKLab and HyAB instead of
YIQ. The API, the threshold scale and its default did not change, so the
ignore* presets keep their values; the comments and the README that
described them in YIQ terms are updated. Mismatch numbers can change a
little (about -3 % to +3 % on real screenshots), so the changeset is a
major for v11.

The public checkerboard option keeps working (pixelmatch 8 supports it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ceb67f0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@wdio/image-comparison-core Major
@wdio/visual-service Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@plum117
plum117 marked this pull request as ready for review October 9, 2026 00:00
@greptile-apps

greptile-apps Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[High impact] The PR appears safe to merge, with the previous test-coverage gap addressed.

Summary

Upgrades pixelmatch to version 8 and adds a major-release note for the changed color comparison.

  • Image comparisons now use pixelmatch 8's OKLab and HyAB color distance.

Reviews (2) · Last reviewed commit: "test: run the real pixelmatch with check..." · Reviewed by Greptile

Comment thread packages/image-comparison-core/package.json
The other checkerboard tests mock pixelmatch, so they only check that
the option is passed. A fully transparent image against an opaque white
image: with checkerboard false pixelmatch blends the transparent pixels
with white (no difference), with true it blends them with a checker
pattern (all 256 pixels differ). The first test fails when compareImages
stops passing the option; pixelmatch 7 gives the same results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@plum117
plum117 merged commit e9aa654 into main Oct 9, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants