feat(python): add shared RaBitQ rotation for distributed IVF_RQ builds

[gstamatakis95]

Closes: #7012

What

Distributed IVF_RQ builds work in the Rust engine (#6359) but could not be driven from Python because the RaBitQ rotation could not be pinned across workers. Each per-fragment build generated its own random rotation, so segments rotated vectors differently, their binary codes were not comparable, and merging corrupted the index.

This adds a way to mint one rotation, broadcast it, and reuse it in every per-fragment build, mirroring how pq_codebook is injected.

Changes

• Add build_rq_rotation(dimension, num_bits=1, rotation_type="fast", dtype="float32") that returns one rotation as a JSON string.
• Add an rq_rotation parameter to create_index_uncommitted, parsed into a new transient RQBuildParams.rotation field and consumed by RabitQuantizer::build.
• build() reuses the supplied rotation instead of generating a random one, after validating num_bits, code_dim, and the signs length.

Notes

• Only the fast rotation is supported because its sign vector is JSON serializable.
• The matrix rotation keeps a dense matrix in a binary buffer that the JSON wire format drops, so it is rejected with a clear error.
• The params proto, the segment builder, and the merge and commit paths are unchanged.

Tests

• Rust unit tests for shared-rotation reuse, identical codes across builds, mismatch and bad-input rejection, and the matrix-via-JSON rejection.
• A Python integration test that builds two IVF_RQ segments on separate fragments with one shared rotation, merges, commits, and queries.

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[BubbleCal]

Just some naming comments

This is nice and is what on our roadmap! Thanks for the contribution!

[BubbleCal]

let's remove this
rotation_type is an internal param for compatibilty

[BubbleCal]

rename to rabitq_model?

[BubbleCal]

rename to build_rq_model

[codecov]

Codecov Report

❌ Patch coverage is 95.27559% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-index/src/vector/bq/builder.rs	96.63%	3 Missing and 1 partial ⚠️
rust/lance-index/src/vector/bq.rs	66.66%	1 Missing ⚠️
rust/lance/src/index/vector.rs	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[gstamatakis95]

@BubbleCal I believe we are ready.

[gstamatakis95]

Fixed linting @BubbleCal. Sorry for the delay

[gstamatakis95]

@BubbleCal Apologies for the spam, resolved conflicts, tests are passing and overall it looks OK. I don't know if I can trigger any of the workflows myself so please let me know.

[BubbleCal]

@BubbleCal Apologies for the spam, resolved conflicts, tests are passing and overall it looks OK. I don't know if I can trigger any of the workflows myself so please let me know.

No worries, it's a limitation that first-time contributor can't trigger the CI

Will keep eyes on it

[gstamatakis95]

Perfect, unsure if I need to do something specific so I'm just waiting for now

[BubbleCal]

@gstamatakis95 Thanks for the contribution!