Skip to content

test: give each rotated Android display state its own baseline, run the Android 16 rotation test again - #1308

Merged
plum117 merged 5 commits into
mainfrom
test/android16-rotation-viewport-tag
Oct 9, 2026
Merged

plum117 merged 5 commits into
mainfrom
test/android16-rotation-viewport-tag

Conversation

@plum117

@plum117 plum117 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Why

The test "should compare a screen after an orientation change" is skipped on LambdaTest Android 16 (#1274). After setOrientation, the emulator does not always come to the same display state: the rotated page is 952 or 903 pixels wide (display cutout used or left out), and this changes between sessions. The rotated screenshot was named with the viewport at the start of the session (-vp426x848), so one baseline had to match 2 different states (2.674 % mismatch in run 37824184770).

On a local Android 16 emulator the state after the rotation is stable (6 of 6 sessions: 900x295), so this happens only on LambdaTest.

What

  • tests/specs/mobile.web.spec.ts: on Android, the tag of the rotated screenshot is the viewport measured after the rotation (-vp${innerWidth}x${innerHeight}), so each rotated state has its own baseline. The Android 16 skip rule for the rotation test is removed.

  • New baselines in tests/lambdaTestBaseline/pixel_9_pro/, collected from 6 runs of the LambdaTest Android job (run 37937247476, attempts 1–6, with a temporary autoSaveBaseline: true and an upload, both reverted in this PR):

    Rotation Android 15 Android 16
    portrait → landscape vp903x322 (4×), vp952x322 (2×) vp903x322 (3×), vp952x322 (3×)
    landscape → portrait vp426x848 (3×), vp426x823 (3×) vp426x848 (5×), vp426x823 (1×)

    Images with the same name differ only in the status bar (clock, icons) and Chrome's page-load line under the address bar.

  • A third Android 16 start state, 426x827 (seen once in run 4): its 6 baselines are added. Without them, a session in that state fails all its checks on main too.

  • The 4 old Android 15 rotated baselines (named with the start viewport) are removed.

  • The spec comment about the display states names the new state.

Test

  • Local Android 16 emulator: pnpm test.local.emus.web setup and compare runs, 7/7 passing; the rotated baseline is named screenshot-landscape-vp900x295-….
  • Final run with the normal CI settings (autoSaveBaseline off, the temporary commit reverted), run 37963619201: all 5 jobs pass; the orientation change test passes on Android 15 and 16 in both orientations (Android 14 stays skipped, as before).

🤖 Generated with Claude Code

dprevost-LMI and others added 3 commits October 9, 2026 09:24
…rotation

On LambdaTest Android 16 the display state after setOrientation does not
always follow the state at the start of the session (the rotated page is
952 or 903 pixels wide). Use the viewport after the rotation in the tag,
so each rotated state has its own baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rotated screenshot now has its own baseline for each display state
after the rotation, so the skip rule is not needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove before merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 8f18fd0

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

plum117 and others added 2 commits October 9, 2026 13:03
…d 16 426x827 state

From 6 runs of the LambdaTest Android job (run 37937247476, attempts 1-6):
- 8 rotated states, named with the viewport after the rotation
  (Android 15 and 16, both directions, 2 states each); 7 were seen 2 to
  4 times with the same image except the status bar, the Android 16
  landscape-to-portrait 426x823 state once.
- A third Android 16 start state (426x827), seen once: its 6 baselines.
Remove the 4 old Android 15 rotated baselines, which had the viewport at
the start of the session in their names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts 433e928.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@plum117
plum117 marked this pull request as ready for review October 9, 2026 17:32
@greptile-apps

greptile-apps Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Low impact] The PR appears safe to merge; no actionable failures were found.

Summary

The PR names Android rotation screenshots using the viewport measured after rotation and enables the Android 16 rotation test again.

  • Android rotation checks match screenshots to the viewport after the phone turns.

No actionable issues were found. Image contents could not be inspected because this model does not support image input. No tests were run.

Reviews (1) · Last reviewed commit: "test: remove the TEMPORARY baseline uplo..." · Reviewed by Greptile

@plum117
plum117 merged commit 40d607f into main Oct 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants