Skip to content

fix(function): align CONV and BIN typed dispatch - #28650

Merged
XuPeng-SH merged 13 commits into
matrixorigin:mainfrom
XuPeng-SH:codex/issue-28392-28460-28461
Sep 11, 2026
Merged

fix(function): align CONV and BIN typed dispatch#28650
XuPeng-SH merged 13 commits into
matrixorigin:mainfrom
XuPeng-SH:codex/issue-28392-28460-28461

Conversation

@XuPeng-SH

@XuPeng-SH XuPeng-SH commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

#28392, #28460, #28461

What this PR does / why we need it

CONV/BIN accepted several typed domains at bind time but dispatched through
string-only or incompatible accessors at execution time. Prepared markers made
the gap observable because the runtime wire domain could change between
executions. Numeric prefixes and MySQL's FLOAT/DOUBLE textual representation
also had boundary mismatches.

The change keeps the existing integer/string fast paths and closes the common
typed dispatch boundary:

  • dispatch BOOL, BIT, DECIMAL, temporal, binary-string, and deferred prepared
    values through type-correct overloads;
  • preserve signed source-domain semantics for from_base, including integer,
    decimal, float, and temporal prefixes;
  • use bounded MySQL-compatible FLOAT/DOUBLE formatting;
  • treat raw HEX/BIT literals as numeric bit patterns when they fit in 64 bits,
    while keeping empty/over-wide literals bounded and deterministic;
  • reject unsupported fixed-width operands with a normal error instead of a
    varlena-accessor panic;
  • discover conversion marker positions once per prepared-plan generation and
    reuse the existing bounded runtime-plan cache without stale parameter
    references.

Review follow-up: binary protocol BIT domain

MYSQL_TYPE_BIT is an opaque bit-domain value. Its binary-protocol unsigned flag
describes wire encoding, not a semantic conversion to UINT64. The prepared
binary path now preserves T_bit for both flag variants. The packet-path
regression test exercises protocol parsing, execute-time binding, expression
execution, signed/unsigned BIT values, ordinary signed/unsigned LONGLONG, and
BIT rebinding after an integer execution.

Compatibility boundary

The final branch is rebased onto main at c4053fd (main protocol version 63)
and allocates the next cumulative version, MORPCVersion64, for the changed
typed BIN/CONV execution contract. MORPC versions are cumulative and must be
allocated in actual merge order: concurrent feature branches that provisionally
use v64 must rebase and take the next value before merging. This PR does not
claim interoperability between independently built sibling branches; sender
and receiver continue to fail closed below v64.

Validation / test plan

All commands use the repository CGo wrapper where required:

  • frontend full UT: mo-cgo-test -count=1 -timeout=900s ./pkg/frontend — PASS;
  • function and plan support race suite:
    mo-cgo-test -race -count=1 -timeout=600s ./pkg/sql/plan/function ./pkg/pb/plan — PASS;
  • compile/plan protocol and prepared-runtime race tests — PASS;
  • COM_STMT_EXECUTE BIT-domain regression race×5 — PASS;
  • incremental go vet for changed packages — PASS;
  • incremental golangci-lint --new-from-rev origin/main for the changed package
    closure — 0 issues;
  • gofmt and git diff --check — PASS.

The packet regression proves the protocol-parser -> execute binder ->
expression-executor contract; the retained prepared-cache tests separately
cover same-domain reuse, type replacement, parameter-reference restoration,
and cleanup. No new unbounded cache, wait, goroutine, or per-row scan was
introduced.

The final rebase also included main's #28671 test-infrastructure change; the
owning package suites were rerun on the final head. Fresh CI/BVT on the pushed
head remains the authoritative clean Linux and distributed validation and is
intentionally not awaited here.

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@matrix-meow matrix-meow added the size/L Denotes a PR that changes [500,999] lines label Sep 11, 2026
@matrix-meow matrix-meow added size/XL Denotes a PR that changes [1000, 1999] lines and removed size/L Denotes a PR that changes [500,999] lines labels Sep 11, 2026

@aptend aptend left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep re-review of exact head 219e8a6 covered every prior review, inline comment, reply/thread state, the delta from aptend previous reviewed head 5dcf95c, and the complete diff. The earlier temporal, integer-from_base, float-formatting, literal-width, and sender/receiver fencing findings are addressed, but the protocol-version collision, binary-protocol BIT dispatch, and checked-in BVT oracle still block this head.

Validation: make thirdparties and make cgo passed; owning suites for ./pkg/sql/plan/function, ./pkg/sql/plan, ./pkg/frontend, ./pkg/sql/compile, and ./pkg/pb/plan passed; focused race runs passed; git diff --check passed. A review-only COM_STMT_EXECUTE counterexample failed with expected 15 / actual 1 and was removed afterward. The current GitHub Compose + Pessimistic BVT also fails deterministically because func_string_bin.test cannot be matched to its result file and all 62 commands are marked abnormal.

Comment thread pkg/defines/const.go
MORPCVersion62 int64 = 62 // VARCHAR OCT overload identities
MORPCVersion63 int64 = 63 // INSERT IGNORE CHECK warning diagnostics
MORPCLatestVersion = MORPCVersion63
MORPCVersion64 int64 = 64 // typed BIN/CONV execution contracts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This protocol number is not uniquely owned. Current main ends at v63, but #28528 exact head ee2ddcb independently defines MORPCVersion64 for AUTO_ID_CACHE/PRE_INSERT, and #28272 exact head 3ac65f4 defines it for persistent data-branch identity. In a mixed cluster, a #28650 coordinator sees a #28528 node report 64 and dispatches typed BIN/CONV even though that receiver lacks these overloads; conversely, #28528 can send its auto-ID marker to this build, which lacks that contract. The >=64 check therefore does not prove either capability. Please coordinate a unique version after the real predecessor lands (or negotiate separate capabilities), rebase, and test against that predecessor.

prepareControl.Plan),
bitCountOverloadParamPositions: plan2.PreparedPlanBitCountFallbackParamPositions(
prepareControl.Plan),
conversionParamPositions: plan2.PreparedPlanConversionParamPositions(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Enabling conversion specialization for binary prepares exposes a BIT-domain regression when a client sets the protocol unsigned flag. ParseExecuteData decodes MYSQL_TYPE_BIT as uint64, while binaryProtocolPrepareParamDomains currently maps (MYSQL_TYPE_BIT, true) to T_uint64 instead of T_bit. Reproducer: prepare select conv(?, 2, 10), then COM_STMT_EXECUTE value 15 with type BIT and flag 0x80. This head chooses the uint64 overload, decimal-spells 15, parses it as base 2, and returns 1; the BIT contract (and SELECT conv(0b1111,2,10)) returns 15. Preserve T_bit regardless of this flag, as binaryProtocolPrepareParamConcreteType already does, and add signed/unsigned BIT packet tests.

conv_bit_value
15
SELECT BIN(X'') AS bin_empty_hex, CONV(X'', 2, 10) AS conv_empty_hex,
BIN(X'FFFFFFFFFFFFFFFFFF') AS bin_wide_hex,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] The checked-in oracle makes the current BVT fail before any query in this file executes. mo-tester normalizes continuation lines in recorded SQL without their leading indentation (see existing generated results such as func_string_oct.result), but this result retains seven spaces here and again on lines 61-63. The exact-head Compose + Pessimistic BVT reports that the command ending at func_string_bin.test:25 does not exist in the result file, then marks all 62 commands abnormal and exits 1. Please regenerate or normalize these multiline result commands so the added coverage actually runs.

@aptend aptend left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep re-review of exact head a1b717fb3f117672964c8b8daff4c6a83f124832 against base/merge-base b349fea56f1c16e3c947ac730cc114fca3c3f241. I read the complete review, inline-comment, reply, and thread history; used range-diff to compare the rebased series with my previous reviewed head 219e8a6fe4abc494c776678011ee7af63895bbd5; and rechecked the full 22-file diff. The BVT oracle blocker is fixed, and the four incremental commits correctly add and normalize the float cases. The earlier temporal/integer source-base findings also remain closed. Two blocking issues from the previous review are still present:

  1. pkg/defines/const.go:102 still assigns typed BIN/CONV to MORPC v64. Two independent open exact heads currently assign the same capability number to incompatible contracts: #28528 at 089c2e68cb712116cc584e6a3456dd782b321ef0 uses v64 for AUTO_ID_CACHE/PRE_INSERT, and #28624 at bfe9669e50982e9bc88576ee57cb295f0aed5e55 uses v64 for widened DECIMAL SUM state. Trigger: a mixed cluster combines either head with this one. Impact: version >= 64 passes even though the peer does not implement the advertised feature, so new overloads/physical vectors or unrelated v64 payloads can be sent to an incompatible receiver. Please coordinate a unique version after the actual predecessor or negotiate separate capabilities.

  2. pkg/frontend/computation_wrapper.go:1222 still maps (MYSQL_TYPE_BIT, unsigned=true) to T_uint64, unlike binaryProtocolPrepareParamConcreteType, which preserves T_bit. Trigger: prepare SELECT CONV(?, 2, 10) and COM_STMT_EXECUTE value 15 as MYSQL_TYPE_BIT with flag 0x80. Impact: runtime specialization selects the UINT64 overload, decimal-spells 15, reparses it in base 2, and returns 1 instead of the BIT result 15. A review-only exact-head test of binaryProtocolPrepareParamDomains(MYSQL_TYPE_BIT, true, "15") expected T_bit and deterministically got T_uint64; it was removed afterward. Please preserve BIT regardless of the unsigned flag and cover both packet flags through conversion execution.

Validation on this exact head: make thirdparties, make cgo, git diff --check, focused BIN/CONV/float tests, frontend protocol-domain tests, and remote-expression compatibility tests in pkg/sql/compile and pkg/pb/plan. The review-only BIT invariant test is the only intentional failure. GitHub Compose/Proxy BVT, coverage, and SCA are green; Ubuntu UT was still queued at the final check.

@XuPeng-SH

Copy link
Copy Markdown
Contributor Author

Follow-up pushed at f8f0e86, rebased onto main 438353c.

Addressed the remaining BIT protocol finding:

  • MYSQL_TYPE_BIT now preserves T_bit for both signed and unsigned wire flags; the unsigned flag is not treated as a semantic UINT64 conversion.
  • Added a real COM_STMT_EXECUTE packet-path regression covering signed/unsigned BIT, signed/unsigned LONGLONG, and BIT rebinding after an integer execution. Expected results distinguish the old UINT64 path (15/16 would become 1 for from_base=2) from the corrected BIT path (15/16).

The v64 fence remains intentionally cumulative: current main is v63, so this change takes v64 in the formal merge order. Concurrent branches that provisionally use v64 must rebase and allocate the next value before merging; this PR does not claim interoperability between independently built sibling branches.

Validation on the rebased head: frontend full UT; function/pb-plan race suite; compile/plan protocol and prepared-runtime race tests; BIT packet race x5; CGo-aware go vet; incremental golangci-lint (0 issues); and git diff --check all pass. The packet test proves parser -> execute binder -> expression executor; existing cache lifecycle tests remain the cache reuse/replacement evidence. Fresh CI/BVT on this head is intentionally not awaited.

@XuPeng-SH

Copy link
Copy Markdown
Contributor Author

Final rebase follow-up: the feature branch is now at c668423, rebased onto main c4053fd (including #28671). The same BIT fix and packet-path regression are unchanged; the final-head owning suites were rerun successfully:

  • frontend full UT;
  • function/pb-plan race suite;
  • compile/plan protocol and prepared-runtime race tests;
  • COM_STMT_EXECUTE BIT-domain race x5;
  • CGo-aware go vet;
  • incremental golangci-lint (0 issues);
  • git diff --check.

The PR body now records the final base/head and the v64 merge-order constraint. Fresh CI/BVT is left to the normal PR workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/XXL Denotes a PR that changes 2000+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants