[v0.10.0-rc1] Binary String Support by chernser · Pull Request #2911 · ClickHouse/clickhouse-java

chernser · 2026-07-06T22:50:00Z

Summary

There are a few problems with reading binary strings:

they are not utf-8 strings and should not be decoded like that
reading big binary blobs may cause OOM because currently strings are created on read.

This PR introduces StringValue that acts as byte[] holder to create String ad-hoc instead of when read. So strings that suppose to be read via byte[] will stay as byte[].

This affects also JDBC where ResultSet has getBinaryStream for such cases. Now it is supported operation.

Part of #2263

Checklist

Delete items not relevant to your PR:

Closes #
Unit and integration tests covering the common scenarios were added
A human-readable description of the changes was provided to include in CHANGELOG
For significant changes, documentation in https://github.com/ClickHouse/clickhouse-docs was updated with further explanations or tutorials

…rted to StringValue

…ogic

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

extend AGENTS.md based on CONTIBUTING.md

github-actions · 2026-07-06T22:50:10Z

Repository collaborators can run the JMH benchmark suite against this PR by commenting:

/benchmark

Optional regression threshold override (Δ% on Time or Alloc/op; defaults to 10%):

/benchmark threshold=15

Only one benchmark run per PR is active at a time — issuing a new /benchmark comment cancels the previous run. After the run finishes a separate comment will be posted comparing it against the latest scheduled run on main; the PR check fails if any benchmark regresses by more than the threshold.

github-actions · 2026-07-06T23:24:47Z

Triage

Category: feature • Risk: high

Summary
This PR introduces opt-in binary string support for client-v2 and jdbc-v2 to fix two problems: String/FixedString columns were being decoded as UTF-8 at read time (losing non-UTF-8 binary data) and large binary blobs could cause OOM because strings were decoded eagerly. The core change is a new StringValue class (an internal read-time byte holder), a new binary_string_support configuration property (disabled by default, resolvable per-operation), and plumbing through BinaryStreamReader, AbstractBinaryFormatReader, MapBackedRecord, SerializerUtils (including ASM bytecode generation for POJO setters), and RowBinaryFormatWriter. On the JDBC side, ResultSet#getBinaryStream(int/String) moves from unsupported to implemented. The writer side gains setString(String, byte[]) and setString(int, byte[]) on the public ClickHouseBinaryFormatWriter interface.

What this impacts

client-v2: BinaryStreamReader.readValue() hot path now branches on binaryStringSupport; AbstractBinaryFormatReader, MapBackedRecord, SerializerUtils all updated to handle StringValue as a possible value type
jdbc-v2: ResultSetImpl.getBinaryStream() now functional; new JDBC tests added
Public API: ClickHouseBinaryFormatWriter interface gained two new methods — any third-party implementor that does not recompile will get AbstractMethodError at runtime (explicitly documented in CHANGELOG as a binary-incompatible change)
POJO binding: ASM bytecode generation in SerializerUtils.compilePOJOSetter extended with new String and byte[] target-type branches

Concerns

High — Public API shape changed: ClickHouseBinaryFormatWriter is a public interface; adding setString(String, byte[]) and setString(int, byte[]) breaks any downstream implementation at the binary level. The CHANGELOG acknowledges this but the reviewer should confirm this is acceptable for v0.10.0-rc1.
High — Readers and Writers: BinaryStreamReader.readValue() is the core deserialization hot path; the new stringAsBytes branching inside that method and the introduction of readNestedValue() for all container reads (Array, Map, Tuple, Nested, Variant, JSON) need careful review for correctness and performance.
High — Large diff: 2,039 additions + 44 deletions = 2,083 total lines; well above the 400-LOC reviewability threshold. Request split or flag if RC timeline requires proceeding as-is.
New connection-string setting: binary_string_support is added as an opt-in flag (default false), satisfying the feature-flag requirement, but the per-operation override path (merging client and query settings) should be verified against the existing settings-resolution logic.
ASM bytecode generation: two new branches in compilePOJOSetter emit static dispatch to stringValueToString/stringValueToByteArray; these are exercised by unit tests but are non-trivial to audit statically.

Required reviewer action

At least one human reviewer.

cursor

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit cbd500f. Configure here.}

cursor · 2026-07-06T23:28:31Z

-        featureManager.unsupportedFeatureThrow("getBinaryStream");
-
-        return null;
+        return getBinaryStream(getSchema().nameToColumnIndex(columnLabel));


JDBC Object getObject returns StringValue

Medium Severity

With binary_string_support enabled, getObject returns internal StringValue for String/FixedString columns without proper conversion. This causes issues when callers expect String (for Object.class) or byte[] (for byte[].class), as JdbcUtils.convert lacks StringValue handling. getString, getBytes, and getBinaryStream are unaffected.

^{Reviewed by Cursor Bugbot for commit cbd500f. Configure here.}

cursor · 2026-07-06T23:28:31Z

+    private Object readNestedValue(ClickHouseColumn column) throws IOException {
+        return readValue(column, null, false);
+    }
+


skipValue still loads string blobs

Medium Severity

skipValue still calls full readValue for every column. With binary_string_support enabled, skipped top-level String/FixedString columns still allocate a StringValue and backing bytes, so readToPOJO with unmapped large string fields can OOM despite the feature’s lazy-read goal.

^{Reviewed by Cursor Bugbot for commit cbd500f. Configure here.}

github-actions · 2026-07-13T16:56:17Z

📊 ClickHouse client benchmark results

commit cbd500f · mode pr-vs-main · run 29268438340

📊 Benchmark tables · ⚠️ regressions — click to expand

Timings (lower = better) — Δ vs baseline, + = slower on the PR build. cold = first iteration; warm = mean of the rest; σ = stddev of the warm iterations.

query_label	base cold	PR cold	base warm	PR warm	Δ warm	base σ	PR σ
SelectHitsCount (compression=on)	383	387	9.2	6.2	-32.4%	2.1	0.5
SelectHits (compression=on)	4536	4577	3460	3350	-3.2%	40.1	48.1
InsertHits (compression=on)	1040	1080	890	898	+0.8%	11.4	37.2
SelectHitsCount (compression=on)	201	193	5.5	4.8	-13.6%	0.6	1.0
SelectHits (compression=on)	4867	4883	3900	3869	-0.8%	28.9	29.2
InsertHits (compression=on)	1072	1089	887	878	-1.0%	44.8	34.0
SelectHitsCount (compression=on)	454	438	41.2	42.2	+2.4%	2.6	2.5
SelectHits (compression=on)	3234	3362	2102	2282	+8.6%	61.9	36.9
InsertHits (compression=on)	1926	1959	1671	1824	+9.2%	153	149
SelectHitsCount (compression=on)	419	422	18.0	18.8	+4.2%	2.7	1.7
SelectHits (compression=on)	5398	4646	3521	3466	-1.6%	105	91.2
InsertHits (compression=on)	3310	3327	2864	2722	-5.0%	219	256

Resource usage (lower = better) (per-iteration averages)

peak_memory_mb

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	116	116	+0.3%
clickhouse-java-v1	SelectHits (compression=on)	1245	1238	-0.5%
clickhouse-java-v1	InsertHits (compression=on)	1852	1857	+0.3%
clickhouse-java-v2	SelectHitsCount (compression=on)	118	117	-0.7%
clickhouse-java-v2	SelectHits (compression=on)	1234	1236	+0.2%
clickhouse-java-v2	InsertHits (compression=on)	1851	1859	+0.4%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	124	127	+1.9%
clickhouse-jdbc-v1	SelectHits (compression=on)	1243	1248	+0.4%
clickhouse-jdbc-v1	InsertHits (compression=on)	5061	6273	+23.9%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	131	131	+0.1%
clickhouse-jdbc-v2	SelectHits (compression=on)	1253	1243	-0.9%
clickhouse-jdbc-v2	InsertHits (compression=on)	9349	6939	-25.8%

user_cpu_ms

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	1211	1198	-1.1%
clickhouse-java-v1	SelectHits (compression=on)	19148	18722	-2.2%
clickhouse-java-v1	InsertHits (compression=on)	13292	13317	+0.2%
clickhouse-java-v2	SelectHitsCount (compression=on)	1389	1218	-12.3%
clickhouse-java-v2	SelectHits (compression=on)	22082	21726	-1.6%
clickhouse-java-v2	InsertHits (compression=on)	13281	13435	+1.2%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	1562	1616	+3.5%
clickhouse-jdbc-v1	SelectHits (compression=on)	12993	13005	+0.1%
clickhouse-jdbc-v1	InsertHits (compression=on)	18847	19671	+4.4%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	1489	1393	-6.4%
clickhouse-jdbc-v2	SelectHits (compression=on)	20105	20360	+1.3%
clickhouse-jdbc-v2	InsertHits (compression=on)	22200	21956	-1.1%

system_cpu_ms

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	277	353	+27.4%
clickhouse-java-v1	SelectHits (compression=on)	1792	1844	+2.9%
clickhouse-java-v1	InsertHits (compression=on)	1035	1154	+11.5%
clickhouse-java-v2	SelectHitsCount (compression=on)	257	323	+25.7%
clickhouse-java-v2	SelectHits (compression=on)	1041	1393	+33.8% ⚠️
clickhouse-java-v2	InsertHits (compression=on)	1164	1113	-4.4%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	354	238	-32.8%
clickhouse-jdbc-v1	SelectHits (compression=on)	1244	1633	+31.3% ⚠️
clickhouse-jdbc-v1	InsertHits (compression=on)	2591	3749	+44.7% ⚠️
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	369	519	+40.7% ⚠️
clickhouse-jdbc-v2	SelectHits (compression=on)	1382	1087	-21.3%
clickhouse-jdbc-v2	InsertHits (compression=on)	4134	3545	-14.2%

voluntary_context_switches

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	6500	8173	+25.7%
clickhouse-java-v1	SelectHits (compression=on)	30359	25492	-16.0%
clickhouse-java-v1	InsertHits (compression=on)	13647	14669	+7.5%
clickhouse-java-v2	SelectHitsCount (compression=on)	8937	8893	-0.5%
clickhouse-java-v2	SelectHits (compression=on)	19800	21304	+7.6%
clickhouse-java-v2	InsertHits (compression=on)	14776	15919	+7.7%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	9173	8584	-6.4%
clickhouse-jdbc-v1	SelectHits (compression=on)	23357	25982	+11.2%
clickhouse-jdbc-v1	InsertHits (compression=on)	20196	20023	-0.9%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	9826	10231	+4.1%
clickhouse-jdbc-v2	SelectHits (compression=on)	21863	21554	-1.4%
clickhouse-jdbc-v2	InsertHits (compression=on)	18178	17493	-3.8%

involuntary_context_switches

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	4	20	+400.0% ⚠️
clickhouse-java-v1	SelectHits (compression=on)	82	54	-34.1%
clickhouse-java-v1	InsertHits (compression=on)	27	41	+51.9% ⚠️
clickhouse-java-v2	SelectHitsCount (compression=on)	3	4	+33.3% ⚠️
clickhouse-java-v2	SelectHits (compression=on)	38	43	+13.2%
clickhouse-java-v2	InsertHits (compression=on)	39	42	+7.7%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	4	3	-25.0%
clickhouse-jdbc-v1	SelectHits (compression=on)	28	52	+85.7% ⚠️
clickhouse-jdbc-v1	InsertHits (compression=on)	42	64	+52.4% ⚠️
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	1	3	+200.0% ⚠️
clickhouse-jdbc-v2	SelectHits (compression=on)	87	67	-23.0%
clickhouse-jdbc-v2	InsertHits (compression=on)	60	52	-13.3%

major_page_faults

client	query_label	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	0	—
clickhouse-java-v1	SelectHits (compression=on)	0	—
clickhouse-java-v1	InsertHits (compression=on)	0	—
clickhouse-java-v2	SelectHitsCount (compression=on)	2	+∞% ⚠️
clickhouse-java-v2	SelectHits (compression=on)	0	—
clickhouse-java-v2	InsertHits (compression=on)	0	—
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	0	—
clickhouse-jdbc-v1	SelectHits (compression=on)	0	—
clickhouse-jdbc-v1	InsertHits (compression=on)	0	—
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	0	—
clickhouse-jdbc-v2	SelectHits (compression=on)	0	—
clickhouse-jdbc-v2	InsertHits (compression=on)	0	—

minor_page_faults

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	31201	31032	-0.5%
clickhouse-java-v1	SelectHits (compression=on)	332312	319479	-3.9%
clickhouse-java-v1	InsertHits (compression=on)	485290	483604	-0.3%
clickhouse-java-v2	SelectHitsCount (compression=on)	31748	31056	-2.2%
clickhouse-java-v2	SelectHits (compression=on)	320818	324021	+1.0%
clickhouse-java-v2	InsertHits (compression=on)	487384	490027	+0.5%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	33320	33859	+1.6%
clickhouse-jdbc-v1	SelectHits (compression=on)	328260	330354	+0.6%
clickhouse-jdbc-v1	InsertHits (compression=on)	1.3M	1.6M	+23.3%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	34896	35833	+2.7%
clickhouse-jdbc-v2	SelectHits (compression=on)	330829	327068	-1.1%
clickhouse-jdbc-v2	InsertHits (compression=on)	2.4M	1.8M	-24.5%

instructions

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	6.3G	6.3G	+1.1%
clickhouse-java-v1	SelectHits (compression=on)	215.5G	227.9G	+5.7%
clickhouse-java-v1	InsertHits (compression=on)	98.6G	99.1G	+0.5%
clickhouse-java-v2	SelectHitsCount (compression=on)	6.6G	6.3G	-3.4%
clickhouse-java-v2	SelectHits (compression=on)	271.6G	273.2G	+0.6%
clickhouse-java-v2	InsertHits (compression=on)	100.5G	99.8G	-0.7%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	7.8G	7.7G	-0.6%
clickhouse-jdbc-v1	SelectHits (compression=on)	151.1G	152.8G	+1.1%
clickhouse-jdbc-v1	InsertHits (compression=on)	166.3G	175.7G	+5.6%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	7.8G	8.0G	+2.6%
clickhouse-jdbc-v2	SelectHits (compression=on)	237.5G	239.9G	+1.0%
clickhouse-jdbc-v2	InsertHits (compression=on)	195.8G	196.3G	+0.3%

cpu_cycles

client	query_label	base	PR	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	3.9G	4.0G	+2.0%
clickhouse-java-v1	SelectHits (compression=on)	60.8G	60.3G	-0.9%
clickhouse-java-v1	InsertHits (compression=on)	41.7G	42.0G	+0.8%
clickhouse-java-v2	SelectHitsCount (compression=on)	4.2G	3.9G	-5.7%
clickhouse-java-v2	SelectHits (compression=on)	69.3G	69.1G	-0.3%
clickhouse-java-v2	InsertHits (compression=on)	42.3G	42.4G	+0.3%
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	5.0G	4.9G	-1.9%
clickhouse-jdbc-v1	SelectHits (compression=on)	40.7G	41.7G	+2.5%
clickhouse-jdbc-v1	InsertHits (compression=on)	59.5G	64.4G	+8.1%
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	4.8G	5.0G	+4.3%
clickhouse-jdbc-v2	SelectHits (compression=on)	63.9G	63.8G	-0.3%
clickhouse-jdbc-v2	InsertHits (compression=on)	71.4G	70.7G	-1.1%

cache_misses

client	query_label	Δ
clickhouse-java-v1	SelectHitsCount (compression=on)	—
clickhouse-java-v1	SelectHits (compression=on)	—
clickhouse-java-v1	InsertHits (compression=on)	—
clickhouse-java-v2	SelectHitsCount (compression=on)	—
clickhouse-java-v2	SelectHits (compression=on)	—
clickhouse-java-v2	InsertHits (compression=on)	—
clickhouse-jdbc-v1	SelectHitsCount (compression=on)	—
clickhouse-jdbc-v1	SelectHits (compression=on)	—
clickhouse-jdbc-v1	InsertHits (compression=on)	—
clickhouse-jdbc-v2	SelectHitsCount (compression=on)	—
clickhouse-jdbc-v2	SelectHits (compression=on)	—
clickhouse-jdbc-v2	InsertHits (compression=on)	—

chernser and others added 18 commits June 17, 2026 21:51

client v2 binary string initial impl

4e19e0d

Merge branch 'main' into 06/17/26/binary_string_support

2df5dd0

Merge branch 'main' into 06/17/26/binary_string_support

bdbf930

Cleaned code and added tests

3012f42

Merge branch 'main' into 06/17/26/binary_string_support

13c9987

implemented feature flag to enable only top level strings to be conve…

3318c97

…rted to StringValue

Added support of binary string to JDBC

b3b1275

updated changelog and features.md

e6b72ad

Updated implementation for MapBackedRecord to match abstract reader l…

5335e5d

…ogic

Added more tests

c5b3173

Fixed issue with POJO reading when binary strings are enabled

6262ef9

Redesigned to expose less. Added tests. Updated docs

b35806d

Fix cloud tests

2bbb606

extend AGENTS.md based on CONTIBUTING.md

571b6ee

Fix the file extension case.

c588bb5

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

26296c4

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge pull request #2908 from ClickHouse/agentmd-extend-contribmd

75e96ed

extend AGENTS.md based on CONTIBUTING.md

Merge branch 'main' into 06/17/26/binary_string_support

cbd500f

chernser changed the title ~~07/06/26/binary string support~~ [v0.10.0-rc1] Binary String Support Jul 6, 2026

chernser marked this pull request as ready for review July 6, 2026 23:22

chernser requested a review from mzitnik as a code owner July 6, 2026 23:22

github-actions Bot added triage:feature risk:high labels Jul 6, 2026

cursor Bot reviewed Jul 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[v0.10.0-rc1] Binary String Support #2911

[v0.10.0-rc1] Binary String Support #2911
chernser wants to merge 18 commits into
v0.10.0from
07/06/26/binary_string_support

chernser commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot Jul 6, 2026

Uh oh!

cursor Bot Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026 •

edited by workflow-authentication-public Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

chernser commented Jul 6, 2026

Summary

Checklist

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

github-actions Bot commented Jul 6, 2026

Triage

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Jul 6, 2026

Choose a reason for hiding this comment

JDBC Object getObject returns StringValue

Uh oh!

cursor Bot Jul 6, 2026

Choose a reason for hiding this comment

skipValue still loads string blobs

Uh oh!

github-actions Bot commented Jul 13, 2026 • edited by workflow-authentication-public Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 ClickHouse client benchmark results

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 13, 2026 •

edited by workflow-authentication-public Bot

Loading