feat: rate-adaptive read batching with a bounded per-shard in-flight window

[merlimat]

Motivation

Read batching never adapts to the server's service rate: the batcher dispatches a ReadBatch as one RPC the moment its queue drains, and the read dispatch path has no in-flight limit — so under load the client floods the server with tiny ReadRequests (~4–5 gets each) instead of growing batches. Every few gets pay a full gRPC round: on saturated clusters, server read-path CPU is dominated by gRPC stream/header handling (~35%) and the per-connection loopyWriter (~25%) while the actual data work (pebble get) stays ~11%, and per-worker read throughput caps at the per-shard RPC round rate regardless of cluster size. Writes had the identical pathology before #360 — post-#360, writes on a 6-node cluster do 333–362k/s while reads (no replication, no fsync) manage only 189k/s.

Modification

Mirror #360 on the read path: at most maxReadBatchesInFlight (default 4) read batches may be outstanding per (client, shard).

• WriteWindow is generalized to DispatchWindow (nothing in it was write-specific; it's the same credit mechanism, and BatchFactory/BatchManager already called it "the dispatch window"). BatchFactory.getWriteWindow becomes getDispatchWindow. Both are package-private; no API impact.
• ReadBatchFactory now keeps a lazily-created per-shard DispatchWindow sized by the new maxReadBatchesInFlight config, and fails held-back batches on close via the existing failWindows path.
• ReadBatch frees its window slot in the RPC terminal callbacks (onError/onCompleted). GrpcRpcProvider.read routes every synchronous failure into the guarded observer and GuardedStreamObserver CASes on terminal delivery, so the release is exactly-once per dispatched batch, including across Failsafe retries.
• The batcher needs no logic changes — it already routes any factory's window generically; while a shard's window is exhausted, the open read batch keeps accumulating toward maxRequestsPerBatch, full batches queue FIFO in the window, and per-shard credits mean no cross-shard head-of-line blocking and no blocking of the batcher thread.
• Public API: OxiaClientBuilder.maxReadBatchesInFlight(int), naming consistent with maxWriteBatchesInFlight. Held-back operations remain bounded by the existing maxPendingBytes limiter.

Verifications

Unit tests mirror the #360 set: window release on response/failure/synchronous-failure for ReadBatch (via a single-slot window and a captured in-process server stream), plus the existing generic batcher tests for exhaustion-accumulation, FIFO dispatch on completion, per-shard independence, and close-fails-held-batches.

Loopback A/B (oxia standalone + oxia-benchmark, this branch vs 0.9.3, alternating runs with cool-downs, fresh server per run):

Fixed-rate 10k ops/s read-only (regression gate) — unchanged; the window does not delay dispatch when the server keeps up, and batches stay small on both sides:

run	p50	p95	p99	p99.9	gets/RPC
0.9.3 run 1	0.2 ms	0.5	1.2	12.6	1.4
0.9.3 run 2	0.3 ms	0.5	1.0	7.6	1.7
this PR run 1	0.2 ms	0.8	9.9¹	57.7¹	1.6
this PR run 2	0.2 ms	0.5	1.1	6.7	1.5

¹ first run of the suite; the tail comes from the first measured interval (JIT settling) — its steady-state intervals are p95 0.4 / p99 ≤1.5 ms.

Max-rate read-only (10k outstanding), directional on loopback — the adaptive mechanism works end-to-end: server-observed batch size pins at the maxRequestsPerBatch ceiling and read RPC count drops 34×, with better tail latencies (0.9.3: 1.50M RPCs at 33 gets/RPC, p99 36.5 ms; this PR: 43.6k RPCs at 1000 gets/RPC, p99 26.5 ms). Peak loopback throughput is not the decisive test here — against a many-core server over loopback, per-RPC overhead is near zero, which is the opposite of the saturated-cluster geometry this targets (gRPC per-RPC overhead dominating 2-CPU servers). The cluster expectation stays as in the write-side change: server-observed read batch size well beyond ~5 under load, per-worker reads far above ~19k/s, and 6-node aggregate reads exceeding the 333–362k/s write figure.