Repository navigation
Conversation
HasProfileData delegated to ProfileTypes with a zero time range, which skips the time filter and runs a DISTINCT over every row in the table, only for the result to be reduced to a boolean. A LIMIT 1 probe returns the same answer after reading a single granule. HasProfileData backs the UI's empty-state check and runs on every page load.
|
Confirming this from production, with numbers — and closing #6417, which I opened a month after this without searching first and which makes the same change. Running a self-hosted Parca on the ClickHouse backend with 72h of retention, the 108,591,120 rows / 7.9 GB read to return 9 rows. 1.3s warm, and 61s cold — so the "seconds to minutes" in the description is accurate, and the cold case is the one users feel, since this runs on page load. Two details that may be useful for review:
Your version is tighter than mine was. The one thing mine had that this doesn't is a regression test: executing the query needs a live ClickHouse, while the regression is entirely visible in the SQL, so I pulled the statement into a small |
What does this PR do?
HasProfileDatadelegated toProfileTypeswith a zero time range.ProfileTypesskips its time filter when start and end are zero, so every call executed:
with no WHERE and no LIMIT — and the result was immediately reduced to
len(types) > 0. DISTINCT cannot terminate early, so this scanned the entiretable on every call just to answer a boolean.
HasProfileDatabacks the UI'sempty-state check and runs on every page load; with hundreds of millions of rows
this made the UI landing screen take seconds to minutes.
This replaces it with
SELECT 1 FROM <table> LIMIT 1, which reads one granuleand stops. Semantics are unchanged: both answer "does the table contain at
least one row", since any row necessarily has profile-type values (non-nullable
columns).
Why is it needed?
Observed in production with ~426M rows: the empty-state check was the single
most expensive query hitting ClickHouse, and under load it held connection
pool slots long enough to starve all other queries.
How to test it?
go test ./pkg/clickhouse/"no data yet" prompt; with an empty store, the prompt still shows. The
response is now immediate regardless of table size.