Repository navigation
Record why source checkpoints need bounded continuation - #1213
Draft
Floating-Y wants to merge 1 commit into
Draft
Floating-Y wants to merge 1 commit into
Floating-Y wants to merge 1 commit into
Conversation
Signed-off-by: Floating-Y <118035379+Floating-Y@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Sources currently import only the first 200 records yet advance the successful checkpoint over the unprocessed tail. Retaining the previous lower bound also cannot get past the readers' fixed ten-page caps.
Refs #1212. This ADR-only draft does not close the issue or implement its fix. The model proposal is under discussion.
What changes
Record ADR 0067 as Proposed and index it under sources. It explains bounded private continuation state, fixed window bounds, first-batch checkpointing, crash replay through existing document/version/job deduplication, pagination drift, and one bounded recovery scan for historical omissions.
The private field and incomplete-batch status remain discussion items. No migration, dependency, public API, production reader, or checkpoint implementation changes are included. A production implementation will follow once this ADR has landed.
How it was checked
Baseline: upstream dev
259931ac702623351c2ed4e6487474790e57a6b0. Windows; Rust/Cargo 1.98.1; dedicated PostgreSQL 16.15 with pgvector 0.8.6 andUTOPIA_TEST_REQUIRE_DB=1.cargo fmt --all --check,git diff --check, andcargo clippy --locked -p utopia-server -p utopia-store --all-targets -- -D warningspassed.Before review
b85855c).dev.