[zhangstar333]

What problem does this PR solve?

Problem Summary:

mysql> desc blob_with_id;
+-------+--------+------+------+---------+-------+
| Field | Type   | Null | Key  | Default | Extra |
+-------+--------+------+------+---------+-------+
| id    | bigint | Yes  | true | NULL    |       |
| image | FILE   | Yes  | true | NULL    |       |
+-------+--------+------+------+---------+-------+
2 rows in set (0.00 sec)

mysql> select * from blob_with_id;
+------+--------------------------------------------------------------------------------------------+
| id   | image                                                                                      |
+------+--------------------------------------------------------------------------------------------+
|    1 | {"uri":null,"offset":null,"size":null,"content_type":null,"checksum":null,"inline":"AQID"} |
|    2 | {"uri":null,"offset":null,"size":null,"content_type":null,"checksum":null,"inline":"BAUG"} |
+------+--------------------------------------------------------------------------------------------+
2 rows in set (0.62 sec)

mysql> select * from blob_with_id@options("blob-as-descriptor"="true"); // return blob descriptor without data
+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id   | image                                                                                                                                                                                                      |
+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|    1 | {"uri":"file:/mnt/disk2/zhangsida/test_paimon/test_blob.db/blob_with_id/bucket-0/data-c94d7440-e3f3-49c9-8735-3e0dbf96e1df-1.blob","offset":4,"size":3,"content_type":null,"checksum":null,"inline":null}  |
|    2 | {"uri":"file:/mnt/disk2/zhangsida/test_paimon/test_blob.db/blob_with_id/bucket-0/data-c94d7440-e3f3-49c9-8735-3e0dbf96e1df-1.blob","offset":23,"size":3,"content_type":null,"checksum":null,"inline":null} |
+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.47 sec)

mysql> select cast(fl_get_binary(image) as string) from blob_table;
+--------------------------------------------+
| cast(fl_get_binary(image) as string) |
+--------------------------------------------+
| h                                          |
| hello world                                |
+--------------------------------------------+
2 rows in set (0.68 sec)


mysql> set paimon_blob_max_size = 2; default is 10M
Query OK, 0 rows affected (0.00 sec)

mysql> select id,cast(fl_get_binary(image) as string) from blob_table;
ERROR 1105 (HY000): errCode = 2, detailMessage = (10.26.20.3)[INVALID_ARGUMENT]Paimon BLOB column 'image' inline size 11 bytes exceeds paimon_blob_max_size=2 bytes. Increase paimon_blob_max_size or use blob-as-descriptor=true
mysql>

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:

• This is a refactor/code format and no logic has been changed.
  - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.

• Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[zhangstar333]

/review

[github-actions]

Static review of all 48 changed paths at 9b03e19 found two P2 issues: JNI descriptor/null/empty FILE values leak native allocations, and real Parquet ROW values lose the FileIO needed for decoding. Both findings are inline. There are no existing inline findings to retain or duplicate, and no additional user-provided focus.

Critical checkpoints:

• Goal and proof: Paimon BLOB maps to the canonical six-child FILE through native managed companions or JNI. Most intended paths are implemented; the two findings leave allocation cleanup and nested interoperability incomplete. Added tests were inspected, not executed.
• Scope and clarity: Physical BLOB framing, per-column lookup, table materialization, and JNI transport are separated; changes remain focused on this feature.
• Concurrency and locks: New state is scanner/split-local, cached table options are copied, and no new threads, locks, lock ordering, or heavy locked work were introduced.
• Lifecycle and initialization: Native readers own copied metadata and release per-split files/indexes; Java buffers are copied before release. The zero-length allocation exception is reported inline. No cross-translation-unit static initialization dependency was found.
• Configuration: paimon_blob_max_size is a session setting, defaults to 10 MiB, validates 1..Integer.MAX_VALUE, and reaches both reader paths per query without restart. Descriptor output is exempt.
• Compatibility: The reader consumes Paimon 1.4.2 framing without changing stored data. The new-FE/old-BE concern was dismissed because the supported upgrade order requires all BEs before FEs. See the Doris upgrade procedure.
• Thrift: TQueryOptions and TPaimonFileDesc definitions were checked on HEAD, master, branch-4.2, branch-4.1, branch-4.0, and branch-3.1. New IDs 237 and 21-23 are unused and optional; older preexisting ID differences are outside this PR.
• Parallel paths: Native/JNI, V1 rejection, Rust exclusion for projected FILE, fallback/system tables, Parquet versus ORC anchors, and nested projections were traced. The real nested JNI representation is the second finding.
• Special conditions: Native eligibility excludes unsupported merge/layout/delete/external-storage combinations. Full range validation distinguishes missing nullable fields from holes; inline limits precede payload reads, while descriptor mode reads metadata only.
• Test coverage: BE framing/corruption/ownership/range tests, Java ABI/projection/limit tests, and Spark-backed native/JNI regressions cover many NULL, empty, binary, filtering, schema, snapshot, compaction and rollover cases. Production native allocator cleanup and actual vectorized ROW fixtures are missing for the reported paths.
• Expected results: Both golden files and fixture SQL were inspected; static payload length/hash and rollover NULL-count arithmetic agreed. SQL results and golden-generation provenance were not independently executed or established.
• Observability: The materialization timer, tracing-reader footer/index/payload accounting, and contextual stream errors preserve useful attribution; no additional logging/metric defect was established.
• Transactions and persistence: No EditLog, transaction commit, master-failover, or persistence-writing logic changes. Snapshot-selected DataSplit metadata remains authoritative.
• Writes and crash behavior: No new Doris data-write or atomic publication path is introduced. Reader cleanup, errors, cancellation, split reset and EOF were inspected; the reported native allocation leak survives cleanup.
• FE/BE variables: The size option is forwarded through query options and JNI scanner parameters and read directly by native materialization; no missing active read-path handoff was found.
• Performance and memory: Native indexes/readers are reused and unprojected companions open lazily; large native buffers use tracked PODArray. The JNI zero-length ownership bug is actionable. No performance measurements were run.
• Other correctness checks: Original file positions survive filtering/splits; FILE predicates wait for final materialization; virtual BLOB mappings cannot take COUNT(column)/MIN/MAX metadata shortcuts; SDK range/bunch selection and historical field IDs preserve the accepted native layouts.
• Remaining limits: Static inspection only: no builds, tests, benchmarks, source edits, or submodule initialization. No unresolved review candidate or unreviewed changed path remains; runtime behavior and performance are unvalidated.

Review converged after 2 round(s).

[zhangstar333]

run buildall

[zhangstar333]

run buildall

[hello-stephen]

BE UT Coverage Report

Increment line coverage 87.42% (396/453) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	61.81% (27287/44147)
Line Coverage	46.50% (284689/612272)
Region Coverage	42.13% (225426/535094)
Branch Coverage	43.72% (105053/240312)

[hello-stephen]

FE UT Coverage Report

Increment line coverage 12.08% (18/149) 🎉
Increment coverage report
Complete coverage report

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 87.42% (396/453) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.29% (32326/42934)
Line Coverage	59.73% (362800/607377)
Region Coverage	56.35% (302098/536145)
Branch Coverage	57.38% (137713/240009)

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 22.16% (115/519) 🎉
Increment coverage report
Complete coverage report