Search before asking
Paimon version
master
Compute Engine
java & python api
Minimal reproduce step
Java
- Create a table with
id and embedding ARRAY<FLOAT> COMMENT '__VECTOR_FIELD;3', with row tracking and data evolution enabled.
- Insert a row and create a tag named
before_add.
- Add
embedding_v2 using the same vector directive.
- Reload the table and read the
before_add tag, projecting only id.
Python
- Create a data-evolution table with a
__BLOB_DESCRIPTOR_FIELD or __BLOB_VIEW_FIELD column. Write a non-null reference and retain the snapshot.
- Drop that column, reload the table, and read the retained snapshot.
What doesn't meet your expectations?
Expected: historical reads should use the field declarations from the selected snapshot. Adding or dropping columns should not break earlier snapshots or change their returned values.
Actual:
- Java fails before reading data with
Some of the columns specified as vector-field are unknown. Similar validation failures occur after adding BLOB columns.
- Python returns serialized descriptor/view bytes instead of resolving the original payload.
Both paths combine historical fields with current schema options, leaving the field declarations inconsistent.
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Paimon version
master
Compute Engine
java & python api
Minimal reproduce step
Java
idandembedding ARRAY<FLOAT> COMMENT '__VECTOR_FIELD;3', with row tracking and data evolution enabled.before_add.embedding_v2using the same vector directive.before_addtag, projecting onlyid.Python
__BLOB_DESCRIPTOR_FIELDor__BLOB_VIEW_FIELDcolumn. Write a non-null reference and retain the snapshot.What doesn't meet your expectations?
Expected: historical reads should use the field declarations from the selected snapshot. Adding or dropping columns should not break earlier snapshots or change their returned values.
Actual:
Some of the columns specified as vector-field are unknown.Similar validation failures occur after adding BLOB columns.Both paths combine historical fields with current schema options, leaving the field declarations inconsistent.
Anything else?
No response
Are you willing to submit a PR?