Skip to content

GH-51268: [C++][Parquet] Unpack equal-width DELTA_BINARY_PACKED miniblocks in one call - #51250

Draft
prtkgaur wants to merge 1 commit into
apache:mainfrom
prtkgaur:delta-binary-packed-coalesce-miniblocks
Draft

GH-51268: [C++][Parquet] Unpack equal-width DELTA_BINARY_PACKED miniblocks in one call#51250
prtkgaur wants to merge 1 commit into
apache:mainfrom
prtkgaur:delta-binary-packed-coalesce-miniblocks

Conversation

@prtkgaur

@prtkgaur prtkgaur commented Sep 9, 2026

Copy link
Copy Markdown

Rationale for this change

The miniblocks of a DELTA_BINARY_PACKED block are packed back to back with no padding, so
consecutive miniblocks that share a bit width are bit-identical to one longer run at that
width. The decoder calls the bit unpacker once per miniblock all the same -- at the default
geometry one call per 32 values, which at narrow widths is mostly per-call setup.

What changes are included in this PR?

A look-ahead over the block's stored bit widths reports how many following miniblocks can be
folded into the current unpack call, so a run of four asks for 128 values instead of 32. A
miniblock joins the run only when its stored width equals the current one, which has already
been validated, so coalescing never depends on an unchecked width. The run also stops at the
end of the block and at what the caller has room for.

A zero bit width needs no unpack call, so the caller tests for it before the look-ahead.
That reads redundant, since the look-ahead declines anyway, but is not free to drop: without
it the two zero-width arms lose 17%. Both builds emit the same loops with the same
instruction counts, so this is how the compiler arranges the function rather than work saved
-- but it reproduces well outside build-to-build spread.

Are these changes tested?

A new typed test covers the width patterns that decide where a run starts and stops, and the
fixture gains a read batch size that stops partway through a coalesced run. Three mutations
-- ignoring the neighbour's width, ignoring the caller's room, and failing to advance the
block cursor -- each turn it red on both integer widths.

Benchmark

Graviton4, GCC 11.5, Release, one core, 9 repetitions, medians, 65,536 values. Both points
built twice with the builds interleaved; the two builds agree within 0.6%.

benchmark main this PR
Decode_Int32_Narrow 99.8 us 74.8 us 1.34x
Decode_Int32_Wide 101.9 us 83.7 us 1.22x
Decode_Int64_Narrow 82.1 us 70.9 us 1.17x
Decode_Int64_Wide 322.3 us 318.6 us 1.01x
Decode_Int32_Fixed 19.8 us 20.4 us 0.97x
Decode_Int64_Fixed 30.6 us 31.3 us 0.98x

The Fixed arms are the zero bit width path: they gain nothing here and give up 2-3% for
the check that keeps them out, stable across builds rather than noise. The wide arms gain
least, spending their time inside the unpacker rather than around it.

Are there any user-facing changes?

No. No API change, no format change, and decoded values are identical.

@prtkgaur prtkgaur changed the title Delta binary packed coalesce miniblocks [WIP][POC] Delta binary packed coalesce miniblocks Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

@github-actions github-actions Bot added the awaiting review Awaiting review label Sep 9, 2026
@prtkgaur
prtkgaur force-pushed the delta-binary-packed-coalesce-miniblocks branch from c16db5a to fa243dd Compare September 9, 2026 03:07
@prtkgaur prtkgaur changed the title [WIP][POC] Delta binary packed coalesce miniblocks GH-51268: [C++][Parquet] Unpack equal-width DELTA_BINARY_PACKED miniblocks in one call Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51268 has been automatically assigned in GitHub to PR creator.

…ne call

The miniblocks of a DELTA_BINARY_PACKED block are packed back to back
with no padding between them, so a run of miniblocks that share a bit
width is bit-identical to a single longer run at that width. GetInternal
called the bit unpacker once per miniblock all the same, which with the
default geometry is one call per 32 values - mostly per-call setup.

Look ahead over the block's stored bit widths and extend the current
call over each following miniblock that has the same width and that the
caller has room for in full. A miniblock joins the run only when its
width equals the current delta_bit_width_, which InitMiniBlock has
already validated, and the run also stops at the end of the block. Add a
test over the width patterns that decide where a run starts and stops,
and read at a batch size that stops partway through a coalesced run.

On the decode benchmarks already in the tree this is 1.17x to 1.33x on
top of the previous commit wherever the unpacker's per-call cost is a
meaningful share of the work. Decoded values are identical; no encoded
byte changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants