Skip to content

[python] Add as-of joins for multimodal scans - #9536

Draft
XiaoHongbo-Hope wants to merge 11 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/temporal-align
Draft

[python] Add as-of joins for multimodal scans#9536
XiaoHongbo-Hope wants to merge 11 commits into
apache:masterfrom
XiaoHongbo-Hope:codex/temporal-align

Conversation

@XiaoHongbo-Hope

@XiaoHongbo-Hope XiaoHongbo-Hope commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Purpose

Add a lazy binary as-of join for independently sampled multimodal streams.

The API follows pandas merge_asof, Polars join_asof, and DuckDB ASOF JOIN: preserve left rows, match at most one right row within by, and support backward, forward, or nearest matching with optional tolerance.

steps = join_asof(
    actions.scan(),
    images.scan().select("image"),
    on="event_time",
    by="episode_id",
    direction="nearest",
    tolerance=20,
).join_asof(
    states.scan().select("state"),
    direction="backward",
    tolerance=50,
)

Chained joins remain lazy. Paimon pins snapshots, plans compact metadata with _ROW_ID, fetches matched payloads by batch, and keeps BLOBs as descriptors.

Resampling, interpolation, windows, and streaming/watermark semantics are out of scope.

Tests

94 passed, 6 subtests passed

@XiaoHongbo-Hope XiaoHongbo-Hope changed the title [python] Add temporal alignment for multimodal scans [python] Add as-of joins for multimodal scans Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant