Background
split_part(str, delimiter, partNum) currently falls back to Spark. It is RuntimeReplaceable and lowers to element_at(StringSplitSQL(str, delimiter), partNum). Comet already supports element_at, but it does not support the inner StringSplitSQL expression, so the whole function falls back with stringsplitsql is not supported.
StringSplitSQL differs from StringSplit (the split function) in that it splits on a literal string rather than a regex.
Proposal
Add a serde for StringSplitSQL (a native string split on a literal delimiter). This would enable split_part to run natively, since element_at over the resulting array is already supported.
Acceptance criteria
StringSplitSQL executes natively and matches Spark.split_part no longer falls back; add SQL file test coverage.
Background
split_part(str, delimiter, partNum)currently falls back to Spark. It isRuntimeReplaceableand lowers toelement_at(StringSplitSQL(str, delimiter), partNum). Comet already supportselement_at, but it does not support the innerStringSplitSQLexpression, so the whole function falls back withstringsplitsql is not supported.StringSplitSQLdiffers fromStringSplit(thesplitfunction) in that it splits on a literal string rather than a regex.Proposal
Add a serde for
StringSplitSQL(a native string split on a literal delimiter). This would enablesplit_partto run natively, sinceelement_atover the resulting array is already supported.Acceptance criteria
StringSplitSQLexecutes natively and matches Spark.split_partno longer falls back; add SQL file test coverage.