Skip to content

adbc: add CATALOG support with qualified scans and schema probe fallback - #92

Merged
adsharma merged 1 commit into
LadybugDB:mainfrom
adsharma:adbc-catalog-support
Oct 6, 2026
Merged

adsharma merged 1 commit into
LadybugDB:mainfrom
adsharma:adbc-catalog-support

Conversation

@adsharma

@adsharma adsharma commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Adds optional CATALOG attach option for 3-level namespaces (e.g. Databricks Unity Catalog).

  • Threads CATALOG through attachADBC → ADBCCatalog → qualifiedTableRef; builds a prequalified FROM clause once per table, propagated via ADBCTableScanInfo (fromClause + backtickIds).
  • Uses backtick quoting for catalog-qualified refs/columns (Spark/Databricks with ANSI off treat double-quotes as string literals).
  • Falls back to SELECT * WHERE 1=0 statement probe when AdbcConnectionGetTableSchema is unsupported, reading the Arrow stream schema.
  • Switches connector mutex to recursive_mutex since the probe re-enters via executeQuery.
  • Drops dead getTableSchemaViaQuery declaration; clang-format clean.

Legacy behavior unchanged when CATALOG is empty (bare double-quoted table).

Review notes / follow-ups:

  • schemaCache is still keyed by table name only; worth keying by catalog.schema.table in a follow-up.
  • backtickIds is tied to CATALOG presence; non-lakehouse drivers using CATALOG would also get backticks.

- Accept optional CATALOG attach option for 3-level namespaces
  (e.g. Databricks Unity Catalog), threaded through ADBCCatalog
  and ADBCConnector::qualifiedTableRef.
- Build a prequalified FROM clause once per table and propagate
  it via ADBCTableScanInfo (fromClause + backtickIds) through
  catalog entry and scan bind, avoiding double-quoting.
- Use backtick quoting for catalog-qualified refs and columns:
  with ANSI mode off (Spark/Databricks), double-quoted ids are
  string literals and fail with PARSE_SYNTAX_ERROR.
- Fall back to SELECT * WHERE 1=0 statement probe when
  AdbcConnectionGetTableSchema is unsupported (e.g. Databricks),
  reading the Arrow stream schema instead.
- Switch connector mutex to recursive_mutex since schema probe
  re-enters via executeQuery under the same lock.
@adsharma
adsharma merged commit 1123fb1 into LadybugDB:main Oct 6, 2026
2 checks passed
@adsharma
adsharma deleted the adbc-catalog-support branch October 6, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant