Skip to content

[#12957] fix(lance): Prevent stale schema repair writes - #13014

Open
IcebreakerSA wants to merge 1 commit into
apache:mainfrom
IcebreakerSA:codex/fix-lance-stale-schema-repair
Open

[#12957] fix(lance): Prevent stale schema repair writes#13014
IcebreakerSA wants to merge 1 commit into
apache:mainfrom
IcebreakerSA:codex/fix-lance-stale-schema-repair

Conversation

@IcebreakerSA

@IcebreakerSA IcebreakerSA commented Sep 8, 2026

Copy link
Copy Markdown

What changes were proposed in this pull request?

Reload the complete Lance table repair observation after an optimistic lock conflict instead of retrying with previously read schema data. Before updating metadata, verify that the observed table ID, location, and Lance version still match the current table.

Add deterministic regression tests for concurrent metadata updates and table identity or location changes, plus integration coverage for persisting a newer Lance dataset schema through the catalog API.

Why are the changes needed?

A schema repair can read an older Lance dataset version and later overwrite metadata that already points to a newer version. A version check alone is also insufficient when a table is replaced or its location changes.

Fix: #12957

Does this PR introduce any user-facing change?

No API or property changes. Lance schema repair no longer writes schema metadata from a stale table observation.

How was this patch tested?

  • Reproduced the regression against the original implementation, where a version 8 observation overwrote metadata at version 9.
  • Added deterministic interleaving tests that change stored metadata while the old dataset schema is being read, then verify the newer stored schema is preserved for version, table identity, and location changes.
  • Added an integration test that advances a real Lance dataset schema, loads it through Gravitino, and verifies the updated schema and lance.version remain persisted on a second read.
  • Ran the module unit checks and the new integration test on Linux: https://github.com/IcebreakerSA/gravitino/actions/runs/34242053525
  • Ran compile and Spotless checks locally with JDK 17.

@IcebreakerSA
IcebreakerSA force-pushed the codex/fix-lance-stale-schema-repair branch from 4dfe175 to 1936ad7 Compare September 8, 2026 14:36
@IcebreakerSA
IcebreakerSA force-pushed the codex/fix-lance-stale-schema-repair branch from 1936ad7 to f0ac9aa Compare September 8, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Prevent stale Lance schema repair from overwriting newer metadata

1 participant