Skip to content

fix(serde): reject table frames with mismatched schema/column counts - #444

Merged
singaraiona merged 1 commit into
RayforceDB:devfrom
belowzeroff:fix/serde-table-schema-mismatch
Aug 31, 2026
Merged

fix(serde): reject table frames with mismatched schema/column counts#444
singaraiona merged 1 commit into
RayforceDB:devfrom
belowzeroff:fix/serde-table-schema-mismatch

Conversation

@belowzeroff

Copy link
Copy Markdown
Contributor

How it looks from the user's side

de — and object file-load / IPC decode, which share the same path — accepts a serialized table whose schema-name count and column count disagree (the kind of frame a truncated read or a partially-written blob leaves behind) and silently loads it as a truncated table instead of erroring.

Reproduced with a hand-crafted frame carrying 2 schema names but 1 column:

Before — the decoder loops over min(names, columns), so the extra name is dropped and the corrupt/truncated frame deserializes into a plausible-but-wrong 1-column table, with no error:

(table [<] (list [1]))

After — the mismatch is rejected up front:

error: domain: deserialize table: schema/column count mismatch (2 names, 1 columns)

The danger is that the truncated table is indistinguishable from a real one, so it flows on into joins/queries/aggregations silently, rather than failing at the point the corruption entered.

Fix

ray_de_raw's table case set ncols = cols->len but bounded the rebuild loop by min(cols->len, schema->len), so a frame with unequal counts was accepted as the shorter of the two (extra columns dropped, or extra schema names ignored). Reject up front when schema->len != cols->len with a domain error, before building the table.

Tests

Adds a serde.rfl regression that splices a well-formed 2-name i64 schema with a well-formed 1-column list into one table payload and asserts de rejects it.

Full ASan+UBSan suite green: 3713 of 3714 passed (1 skipped, 0 failed).

The table decoder set ncols = cols->len but bounded the rebuild loop by
min(cols->len, schema->len), so a crafted frame whose schema-name count and
column count disagree was silently accepted as a truncated table (extra
columns dropped, or extra schema names ignored) instead of being rejected.

Reject up front when schema->len != cols->len with a domain error, before
building the table.

Adds a serde.rfl regression that splices a 2-name i64 schema with a 1-column
list into one table payload and asserts de rejects it.

@singaraiona singaraiona left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. The guard itself is correct, well-placed (after both decodes, before ray_table_new), and the error path releases both schema and cols symmetrically. CI is green — merging. A few notes, the first two are small in-PR cleanups that can go in a follow-up, the rest are adjacent gaps in the same threat model worth filing:

  1. Dead loop bound: with the guard in place, the && i < schema->len clause in the build loop (serde.c:814 on dev) is unreachable — ncols == schema->len always holds. Worth removing so the guard reads as the single mechanism; as it stands the min()-style bound suggests truncated frames are still tolerated here.

  2. Test hard-codes wire layout: the .rfl splice bakes in the 16-byte header, size at byte offset 8, payload-fits-in-one-byte, and little-endianness. If ray_ipc_header_t ever changes, (de _s74f) starts failing on a bad header and !- domain still passes — the test goes green while no longer exercising the mismatch. The C harness in test_store.c (test_serde_table_dict_de_errors) already builds frames via sizeof(ray_ipc_header_t) and would stay honest; consider moving/duplicating the case there.

  3. Same invariant, ragged rows: the guard enforces name/column count agreement but nothing checks the columns' lengths agree. A crafted frame with columns of length 1000 and 1 still decodes; ray_table_nrows reports column 0's length and row-wise ops read past the short column. The splayed loader explicitly rejects this (splay.c "torn overwrite?" check) — serde is now the odd decoder out.

  4. DICT branch has the sibling bug: the dict decode (serde.c:833) builds the 2-slot block directly with no keys->len == vals->len check (nor that vals is a LIST). A crafted dict with more keys than vals makes ray_dict_remove/sym probes index vals by a keys-bounded idx — OOB read/release. Strictly worse than the table case this PR fixes.

  5. Decode errors are swallowed and leaked on the IPC path: in ipc.c the dispatch is gated on if (msg && !RAY_IS_ERR(msg)) with no else — the new domain error object is never ray_error_free'd (ray_release is a no-op on errors) and the client gets a plain null instead of the error. Remote-repeatable leak, made trivially reachable by this PR. Same pattern in the response-frame branch (~ipc.c:918).

@singaraiona
singaraiona merged commit b033fd2 into RayforceDB:dev Aug 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants