Skip to content

[Bug] Time travel uses current VECTOR/BLOB field options after schema evolution #9696

Description

@zhoulii

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

master

Compute Engine

java & python api

Minimal reproduce step

Java

  1. Create a table with id and embedding ARRAY<FLOAT> COMMENT '__VECTOR_FIELD;3', with row tracking and data evolution enabled.
  2. Insert a row and create a tag named before_add.
  3. Add embedding_v2 using the same vector directive.
  4. Reload the table and read the before_add tag, projecting only id.

Python

  1. Create a data-evolution table with a __BLOB_DESCRIPTOR_FIELD or __BLOB_VIEW_FIELD column. Write a non-null reference and retain the snapshot.
  2. Drop that column, reload the table, and read the retained snapshot.

What doesn't meet your expectations?

Expected: historical reads should use the field declarations from the selected snapshot. Adding or dropping columns should not break earlier snapshots or change their returned values.

Actual:

  • Java fails before reading data with Some of the columns specified as vector-field are unknown. Similar validation failures occur after adding BLOB columns.
  • Python returns serialized descriptor/view bytes instead of resolving the original payload.

Both paths combine historical fields with current schema options, leaving the field declarations inconsistent.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions