test(e2e): Verify replay runtime controls drive native recording - #6704
Open
antonis wants to merge 6 commits into
Open
test(e2e): Verify replay runtime controls drive native recording#6704antonis wants to merge 6 commits into
antonis wants to merge 6 commits into
Conversation
Contributor
Semver Impact of This PR⚪ None (no version bump detected) 📋 Changelog PreviewThis is how your changes will appear in the changelog.
🤖 This preview updates automatically when you update the PR. |
Contributor
Android (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| f3215d3+dirty | 411.11 ms | 454.38 ms | 43.27 ms |
| af33f3b+dirty | 434.90 ms | 506.14 ms | 71.24 ms |
| c2e182c+dirty | 471.64 ms | 553.59 ms | 81.95 ms |
| 5ca03f9+dirty | 423.30 ms | 467.00 ms | 43.70 ms |
| a5d243c+dirty | 424.52 ms | 485.18 ms | 60.66 ms |
| bc8f61e+dirty | 442.98 ms | 506.56 ms | 63.58 ms |
| c823bb5+dirty | 409.87 ms | 478.57 ms | 68.70 ms |
| d387874+dirty | 438.69 ms | 453.77 ms | 15.08 ms |
| b0d3373+dirty | 557.66 ms | 579.42 ms | 21.76 ms |
| 0bd8916+dirty | 412.77 ms | 451.31 ms | 38.54 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| f3215d3+dirty | 48.30 MiB | 53.49 MiB | 5.19 MiB |
| af33f3b+dirty | 49.74 MiB | 55.09 MiB | 5.35 MiB |
| c2e182c+dirty | 49.74 MiB | 54.85 MiB | 5.11 MiB |
| 5ca03f9+dirty | 49.74 MiB | 55.26 MiB | 5.52 MiB |
| a5d243c+dirty | 48.30 MiB | 53.54 MiB | 5.23 MiB |
| bc8f61e+dirty | 49.74 MiB | 55.09 MiB | 5.35 MiB |
| c823bb5+dirty | 48.30 MiB | 53.58 MiB | 5.28 MiB |
| d387874+dirty | 49.74 MiB | 55.44 MiB | 5.70 MiB |
| b0d3373+dirty | 48.30 MiB | 53.58 MiB | 5.28 MiB |
| 0bd8916+dirty | 48.30 MiB | 53.57 MiB | 5.26 MiB |
Previous results on branch: test/replay-runtime-controls-e2e
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 426.52 ms | 438.90 ms | 12.37 ms |
| 3e493d9+dirty | 412.81 ms | 476.24 ms | 63.43 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 50.56 MiB | 56.49 MiB | 5.93 MiB |
| 3e493d9+dirty | 50.56 MiB | 56.49 MiB | 5.93 MiB |
Contributor
iOS (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7a89652+dirty | 3861.46 ms | 1229.61 ms | -2631.85 ms |
| 0b1b5e3+dirty | 3823.96 ms | 1220.12 ms | -2603.84 ms |
| eb55864+dirty | 3869.04 ms | 1241.75 ms | -2627.29 ms |
| 3e3142c+dirty | 3867.38 ms | 1232.48 ms | -2634.90 ms |
| 9210ae6+dirty | 3815.93 ms | 1214.14 ms | -2601.79 ms |
| 6177334+dirty | 3834.85 ms | 1217.58 ms | -2617.28 ms |
| 3a829f0+dirty | 3839.76 ms | 1214.69 ms | -2625.06 ms |
| 2cac31b+dirty | 3886.27 ms | 1230.85 ms | -2655.42 ms |
| a5d243c+dirty | 3842.35 ms | 1214.29 ms | -2628.06 ms |
| 244f6e8+dirty | 3833.36 ms | 1212.96 ms | -2620.41 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7a89652+dirty | 5.15 MiB | 6.70 MiB | 1.55 MiB |
| 0b1b5e3+dirty | 5.15 MiB | 6.70 MiB | 1.54 MiB |
| eb55864+dirty | 5.15 MiB | 6.88 MiB | 1.73 MiB |
| 3e3142c+dirty | 4.98 MiB | 6.56 MiB | 1.58 MiB |
| 9210ae6+dirty | 5.15 MiB | 6.68 MiB | 1.53 MiB |
| 6177334+dirty | 5.15 MiB | 6.68 MiB | 1.53 MiB |
| 3a829f0+dirty | 5.15 MiB | 6.70 MiB | 1.54 MiB |
| 2cac31b+dirty | 4.98 MiB | 6.55 MiB | 1.58 MiB |
| a5d243c+dirty | 5.15 MiB | 6.68 MiB | 1.53 MiB |
| 244f6e8+dirty | 4.98 MiB | 6.56 MiB | 1.58 MiB |
Previous results on branch: test/replay-runtime-controls-e2e
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 3855.98 ms | 1225.33 ms | -2630.64 ms |
| 3e493d9+dirty | 3859.54 ms | 1241.19 ms | -2618.35 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 5.15 MiB | 6.90 MiB | 1.75 MiB |
| 3e493d9+dirty | 5.15 MiB | 6.90 MiB | 1.75 MiB |
11 tasks
Contributor
iOS (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7a89652+dirty | 3827.11 ms | 1211.49 ms | -2615.62 ms |
| 0a147b2+dirty | 3845.11 ms | 1230.81 ms | -2614.30 ms |
| 5b7e8a7+dirty | 3855.46 ms | 1234.47 ms | -2620.99 ms |
| 5fe1c6c+dirty | 1201.36 ms | 1209.15 ms | 7.78 ms |
| 580fb5c+dirty | 3835.45 ms | 1218.61 ms | -2616.84 ms |
| 94af3bd+dirty | 3847.84 ms | 1230.33 ms | -2617.51 ms |
| f3215d3+dirty | 3846.08 ms | 1231.85 ms | -2614.23 ms |
| a636fa4+dirty | 3846.16 ms | 1210.10 ms | -2636.05 ms |
| 5a21b51+dirty | 3837.87 ms | 1223.47 ms | -2614.40 ms |
| bbb8d67+dirty | 3847.47 ms | 1225.13 ms | -2622.34 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 7a89652+dirty | 5.15 MiB | 6.70 MiB | 1.55 MiB |
| 0a147b2+dirty | 4.98 MiB | 6.51 MiB | 1.53 MiB |
| 5b7e8a7+dirty | 5.15 MiB | 6.70 MiB | 1.54 MiB |
| 5fe1c6c+dirty | 3.38 MiB | 4.77 MiB | 1.39 MiB |
| 580fb5c+dirty | 4.98 MiB | 6.46 MiB | 1.48 MiB |
| 94af3bd+dirty | 5.15 MiB | 6.69 MiB | 1.53 MiB |
| f3215d3+dirty | 5.15 MiB | 6.67 MiB | 1.52 MiB |
| a636fa4+dirty | 4.98 MiB | 6.55 MiB | 1.57 MiB |
| 5a21b51+dirty | 5.15 MiB | 6.67 MiB | 1.51 MiB |
| bbb8d67+dirty | 4.98 MiB | 6.55 MiB | 1.57 MiB |
Previous results on branch: test/replay-runtime-controls-e2e
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 3851.82 ms | 1233.59 ms | -2618.23 ms |
| 3e493d9+dirty | 3871.14 ms | 1234.98 ms | -2636.17 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 5.15 MiB | 6.90 MiB | 1.75 MiB |
| 3e493d9+dirty | 5.15 MiB | 6.90 MiB | 1.75 MiB |
Contributor
Android (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 6a3eb4c+dirty | 525.65 ms | 562.32 ms | 36.67 ms |
| 2cac31b+dirty | 416.45 ms | 449.45 ms | 33.00 ms |
| 9ad4522+dirty | 432.43 ms | 450.91 ms | 18.49 ms |
| 3ce5254+dirty | 373.90 ms | 427.84 ms | 53.94 ms |
| b04af96+dirty | 430.33 ms | 485.98 ms | 55.65 ms |
| 68ae91b+dirty | 515.04 ms | 578.08 ms | 63.04 ms |
| f3215d3+dirty | 396.53 ms | 436.66 ms | 40.13 ms |
| a736b76+dirty | 405.78 ms | 458.74 ms | 52.96 ms |
| 2363c0f+dirty | 419.48 ms | 463.92 ms | 44.44 ms |
| 038a6d7+dirty | 499.02 ms | 527.68 ms | 28.66 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 6a3eb4c+dirty | 49.74 MiB | 55.44 MiB | 5.70 MiB |
| 2cac31b+dirty | 49.74 MiB | 55.37 MiB | 5.63 MiB |
| 9ad4522+dirty | 49.74 MiB | 55.38 MiB | 5.63 MiB |
| 3ce5254+dirty | 43.94 MiB | 48.98 MiB | 5.04 MiB |
| b04af96+dirty | 49.74 MiB | 55.00 MiB | 5.26 MiB |
| 68ae91b+dirty | 49.74 MiB | 54.79 MiB | 5.05 MiB |
| f3215d3+dirty | 48.30 MiB | 53.49 MiB | 5.19 MiB |
| a736b76+dirty | 48.30 MiB | 53.48 MiB | 5.18 MiB |
| 2363c0f+dirty | 49.74 MiB | 54.84 MiB | 5.10 MiB |
| 038a6d7+dirty | 48.30 MiB | 53.60 MiB | 5.30 MiB |
Previous results on branch: test/replay-runtime-controls-e2e
Startup times
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 536.64 ms | 578.88 ms | 42.24 ms |
| 3e493d9+dirty | 521.27 ms | 568.02 ms | 46.75 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| dca994f+dirty | 50.56 MiB | 56.49 MiB | 5.93 MiB |
| 3e493d9+dirty | 50.56 MiB | 56.49 MiB | 5.93 MiB |
Contributor
|
antonis
added this pull request to stack #6705
September 11, 2026 12:52
antonis
marked this pull request as ready for review
September 14, 2026 09:49
Adds a manualReplay Maestro flow that exercises the Session Replay runtime controls end-to-end. It launches with buffer-mode replay off (replaysOnErrorSampleRate: 0) and no session sampling, taps a new "Start Replay" button that calls Sentry.getReplay().start(), then captures an exception and reuses assertReplay.yml to assert a real replay (replay_id, duration, segments, valid MP4) came back from the Sentry API. With both sample rates off, the replay can only exist if start() actually drove native recording across the bridge, so this closes the gap the Jest and native unit tests can't — they mock the bridge and stop there. The flow is auto-discovered by cli.mjs and runs on both platforms, reusing the existing prime + assert utilities. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a replayStopResume Maestro flow that exercises the remaining Session Replay runtime controls end-to-end, building on the start() coverage in manualReplay: - pause() + resume() must not crash and must leave recording active — after resuming, a captured error still carries a replay. - stop() must halt recording — a subsequent error carries no replay, asserted via a new "noReplay" sentryApi case + assertNoReplay utility (the positive assertReplay throws when a replay_id is absent, so it can't prove halting on its own). Buffer-mode replay is off and there is no session sampling, so a replay can only exist while a manually-started session is actively recording, which lets each assertion isolate what the control did. flush() and startBuffering() are left for a follow-up (no error event to key the assertion off standalone). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a no-error end-to-end path for the buffered-flush runtime controls: startBuffering() records a buffer regardless of sample rate, and flush() converts it to a session replay and uploads it immediately. The app surfaces getReplay().getReplayId(), so the replay is asserted by querying it directly (replayById), rather than discovering a replay_id on a captured error event. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The stop() "subsequent error carries no replay" assertion fails on iOS: sentry-cocoa's stop() does not clear scope.replayId, so an error captured after stop() still carries the stopped replay's id — unlike Android/sentry-java, which reset it to EMPTY_ID. Gate the assertion to Android via when.platform until the cocoa fix lands, then re-enable it cross-platform. pause()/resume() coverage stays on both platforms. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
antonis
force-pushed
the
test/replay-runtime-controls-e2e
branch
from
September 14, 2026 11:12
c9c0789 to
6f97d4f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on top of #6703 (Session Replay runtime controls). Review/merge that first; this PR's base auto-retargets to
mainonce it lands. The diff below is only the e2e delta.📢 Type of change
📜 Description
Adds e2e Maestro flows that verify the Session Replay runtime controls (added in #6703) actually drive native recording end-to-end — not just that the JS forwards across the bridge (which the Jest/native unit tests already cover by mocking the bridge and stopping there). With buffer-mode replay off (
replaysOnErrorSampleRate: 0) and no session sampling, a replay can only exist while a manually-started session is actively recording, so each assertion isolates what a control did.src/EndToEndTests.tsx— addStart,Pause,Resume,StopReplay buttons that call the matchingSentry.getReplay()?.*control.maestro/manualReplay.yml— tap Start Replay, prime the view, capture an exception, then reuseassertReplay.ymlto assert a real replay (replay_id, duration, segments, valid MP4) came back from the Sentry API. Provesstart()drives recording.maestro/replayStopResume.yml(new) —start()→pause()→resume()→ capture →assertReplay(pause/resume don't crash and recording survives); thenstop()→ capture →assertNoReplay(recording actually halted).maestro/utils/assertNoReplay.yml+ anoReplaycase insentryApi.js— the inverse ofassertReplay: asserts an event carries no replay association. Needed because the positiveassertReplaythrows whenreplay_idis absent, so it can't prove halting on its own.flush()andstartBuffering()are left for a follow-up — standalone they produce no error event to key the assertion off. The flows are auto-discovered bycli.mjsand run on both platforms, reusing theprimeReplayBuffer.yml+assertReplay.ymlutilities from #6683/#6684.💡 Motivation and Context
Runtime controls (#6703) had no e2e coverage. Unit tests mock
NATIVE, so a bridge-name mismatch, ABI/codegen drift, or a native no-op would pass them silently. This adds the missing through-the-bridge checks.💚 How did you test it?
yarn buildof the e2e package typechecks the newgetReplay()?.start/pause/resume/stop()buttons.ready-to-mergelabel.📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps
startBuffering()+flush()once feat(replay): Add Session Replay runtime controls #6703 lands (needs a flush-then-assert path without relying on an error event).