fix: isolate DataLakeCatalog databases from the schema panel's system.tables scan#163
Merged
Merged
Conversation
….tables scan (#162) A single system.tables query across every database, once ClickHouse resolves per-table metadata for a DataLakeCatalog (Iceberg/Glue/…) database, either aborts entirely or silently drops tables from other, healthy catalogs the instant one table in any catalog is broken — traced to a ClickHouse-side gap in DatabaseDataLake::getTablesIterator, reported upstream as ClickHouse/ClickHouse#110032. loadSchema now queries each DataLakeCatalog-engine database separately, requesting only database/name — the one column shape ClickHouse resolves via system.tables' fast path without opening each table's storage object, so one broken catalog can no longer take down any other. total_rows/total_bytes/ comment for catalog tables are zero/empty rather than fetched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LxWjVvVDCQ1MksmKjtVzAL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
system.tablesquery across every database, once ClickHouse resolves per-table metadata for aDataLakeCatalog(Iceberg/Glue/…) database, either aborts entirely or silently drops tables from other, healthy catalogs the instant a single table in any catalog is broken. Traced to a ClickHouse-side gap inDatabaseDataLake::getTablesIterator, reported upstream as ClickHouse/ClickHouse#110032.loadSchemanow queries eachDataLakeCatalog-engine database separately, requesting onlydatabase, name— the one column shape ClickHouse resolves viasystem.tables' fast path without opening each table's storage object, so one broken catalog can no longer take down any other.total_rows/total_bytes/commentfor catalog tables show as zero/empty rather than being fetched — not obtainable without re-triggering the underlying bug.querySystemAware's error handling: only an actual "Unknown setting" error latchesdataLakeCatalogSettingUnsupportednow; any other error (e.g. a per-table catalog failure) is no longer mistaken for "the setting itself is unsupported," which used to disable data-lake-catalog visibility for the rest of the session.Closes #162.
Test plan
npm test— 1472/1472 passing,ch-client.jsat 100/98.3/100/100 (remaining branch gaps are pre-existing inloadSchemaLineage, unrelated to this change)node build/build.mjs— bundles cleanlyDataLakeCatalogdatabases, two of which have genuinely broken tables (missing Icebergtable_type, an inaccessible S3 access point): all 5 now list their tables correctly via the new query shape, where they previously failed or came back empty/incomplete.🤖 Generated with Claude Code
https://claude.ai/code/session_01LxWjVvVDCQ1MksmKjtVzAL