[branch-4.1][pick] Backport Paimon and Parquet scan improvements

be/benchmark/parquet/AGENTS.md

@@ -43,10 +43,10 @@ be/output/lib/benchmark_test --benchmark_list_tests \
   | grep -c '^ParquetDecoder/'  # currently 228
 
 be/output/lib/benchmark_test --benchmark_list_tests \
-  | grep -c '^ParquetKernel/'   # currently 80
+  | grep -c '^ParquetKernel/'   # currently 92
 
 be/output/lib/benchmark_test --benchmark_list_tests \
-  | grep -c '^ParquetReader/'   # currently 152
+  | grep -c '^ParquetReader/'   # currently 167
 ```
 
 When running the binary directly from `be/build_RELEASE/bin`, make sure the JVM and third-party
@@ -120,13 +120,16 @@ cache to manufacture a cold run.
 | DELTA_LENGTH_BYTE_ARRAY | BYTE_ARRAY |
 | DELTA_BYTE_ARRAY | BYTE_ARRAY |
 
-`ParquetKernel` contains 80 cases across five SIMD-sensitive stages: BYTE_STREAM_SPLIT,
-DELTA_PREFIX_SUM, DICTIONARY_GATHER, NULLABLE_EXPAND, and RAW_PREDICATE. It covers the applicable
-four- and eight-byte types, three dictionary working-set sizes, 0% through 90% null rates with both
-placement patterns, and 0% through 100% raw-predicate selectivities.
+`ParquetKernel` contains 92 cases across six decode and selection stages: BYTE_STREAM_SPLIT,
+DELTA_PREFIX_SUM, DICTIONARY_GATHER, NULLABLE_EXPAND, RAW_PREDICATE, and NESTED_SELECTION. It covers
+the applicable four- and eight-byte types, three dictionary working-set sizes, 0% through 90% null
+rates with both placement patterns, 0% through 100% raw-predicate selectivities, and 1%, 10%, and
+50% nested parent-row selectivities with both placement patterns. Nested selection registers the
+legacy and fused implementations in the same binary and validates both against an independent
+source-level oracle before timing.
 
 `ParquetReader` deliberately uses a single-variable matrix rather than a Cartesian product. After
-deduplication it contains 152 cases covering:
+deduplication it contains 167 cases covering:
 
 - operations: open-to-first-block, full scan, predicate scan, complex residual scan, limit 1, and
   limit 1000;
@@ -299,7 +302,7 @@ be simulated by silently changing the local reader benchmark.
 
 ## Current validation record
 
-The current expected registration counts are 228 decoder, 80 kernel, and 152 reader cases. A smoke
+The current expected registration counts are 228 decoder, 92 kernel, and 167 reader cases. A smoke
 run is an execution record only, not a reviewed performance baseline, because repetitions, host
 isolation, warmups, cache control, `perf` data, variance, and before/after comparison are not
 collected.

be/benchmark/parquet/README.md

@@ -39,20 +39,42 @@ be/output/lib/benchmark_test \
 
 ## SIMD kernel cases
 
-`ParquetKernel` isolates the five SIMD-sensitive stages from reader setup and virtual consumer
+`ParquetKernel` isolates six decode and selection stages from reader setup and virtual consumer
 overhead: byte-stream-split transpose, delta prefix sum, numeric dictionary gather, nullable
-expansion, and raw predicate evaluation. It covers the applicable 4-byte and 8-byte integer and
-floating-point physical types, raw-predicate selectivities from 0% through 100%, and nullable
-rates from 0% through 90% with clustered and alternating placement. Dictionary gather uses 32-,
-4,096-, and 262,144-entry working sets to separate cache-resident and cache-miss-dominated
-behavior.
+expansion, raw predicate evaluation, and repeated-level sparse selection. It covers the applicable
+4-byte and 8-byte integer and floating-point physical types, raw-predicate selectivities from 0%
+through 100%, and nullable rates from 0% through 90% with clustered and alternating placement.
+Nested selection covers 1%, 10%, and 50% surviving parent rows with both placement patterns.
+Each nested-selection scenario registers both `impl_legacy` and `impl_fused`; both paths use the
+same source levels and are checked against an independent oracle before timing.
+Dictionary gather uses 32-, 4,096-, and 262,144-entry working sets to separate cache-resident and
+cache-miss-dominated behavior.
 
 ```shell
 be/output/lib/benchmark_test \
   --benchmark_filter='^ParquetKernel/(dictionary_gather|nullable_expand)/' \
   --benchmark_min_time=0.1s
 ```
 
+For a reproducible nested-selection comparison, build once and run the two implementations from
+that same binary in ABBA order. Pin every command to the same otherwise-idle CPU:
+
+```shell
+taskset -c 8 be/output/lib/benchmark_test \
+  --benchmark_filter='^ParquetKernel/nested_selection/.*/impl_legacy$' \
+  --benchmark_min_time=1s --benchmark_repetitions=10 \
+  --benchmark_report_aggregates_only=true \
+  --benchmark_out=nested-legacy-a1.json --benchmark_out_format=json
+
+taskset -c 8 be/output/lib/benchmark_test \
+  --benchmark_filter='^ParquetKernel/nested_selection/.*/impl_fused$' \
+  --benchmark_min_time=1s --benchmark_repetitions=10 \
+  --benchmark_report_aggregates_only=true \
+  --benchmark_out=nested-fused-b1.json --benchmark_out_format=json
+
+# Repeat fused as B2, then legacy as A2, changing only --benchmark_out.
+```
+
 ## Local reader cases
 
 `ParquetReader` measures local open-to-first-block, full scan, predicate scan, complex residual