Skip to content

pkg/clickhouse: answer HasProfileData with one row, not a scan - #6417

Closed
guettlibot wants to merge 1 commit into
parca-dev:mainfrom
guettli:upstream-pr-hasprofiledata
Closed

guettlibot wants to merge 1 commit into
parca-dev:mainfrom
guettli:upstream-pr-hasprofiledata

Conversation

@guettlibot

Copy link
Copy Markdown

Independent of #6415 and #6416 — branches from main, touches only pkg/clickhouse/querier.go.

What

HasProfileData asks whether the table holds any data at all. It answered by calling ProfileTypes(time.UnixMilli(0), time.UnixMilli(0)):

func (q *Querier) HasProfileData(ctx context.Context) (bool, error) {
	types, err := q.ProfileTypes(ctx, time.UnixMilli(0), time.UnixMilli(0))
	...
	return len(types) > 0, nil
}

and ProfileTypes applies its time filter only when both bounds are non-zero:

// Only apply time filter if both start and end are non-zero
if start.Unix() != 0 && end.Unix() != 0 {
	query += " WHERE time_nanos > ? AND time_nanos < ?"

With (0, 0) the filter is skipped, so the cheapest question in the API ran an unbounded SELECT DISTINCT over six columns — a full scan of a table that grows without bound.

Measured

On a server with 72 hours of retention:

X-ClickHouse-Summary: {"read_rows":"108591120","read_bytes":"7930500128",
                       "result_rows":"9","elapsed_ns":"1436439879"}

108,591,120 rows / 7.9 GB read to return 9 rows — 1.3s warm, 61s cold. The UI calls this to decide whether to show its empty state, so the cost lands on page loads rather than on anything a user asked for.

Fix

Existence needs one row:

SELECT 1 FROM <table> LIMIT 1

The statement lives in a small function so a test can pin its shape without a server — executing it needs a live ClickHouse, while the regression is entirely visible in the SQL. The test asserts LIMIT 1 and the absence of DISTINCT/GROUP BY/ORDER BY.

ProfileTypes itself is unchanged; its zero-bounds behaviour is left exactly as it is, since other callers may rely on the all-time semantics.

HasProfileData asks whether the table holds any data at all. It answered by
calling ProfileTypes(UnixMilli(0), UnixMilli(0)), and ProfileTypes applies its
time filter only when both bounds are non-zero -- so the cheapest question in
the API ran an unbounded SELECT DISTINCT over six columns, a full scan of a
table that grows without bound.

Measured on a server with 72 hours of retention: 108,591,120 rows and 7.9 GB
read to return 9 rows, 1.3 seconds warm and 61 seconds cold. The UI calls this
to decide whether to show its empty state, so the cost was paid on page loads
rather than on anything a user asked for.

Existence needs one row. The query is a small function so a test can pin its
shape without a server: this needs a live ClickHouse to execute, while the
regression is entirely visible in the statement.
@guettlibot

Copy link
Copy Markdown
Author

Closing as a duplicate of #6402, which makes the same change a month earlier and more concisely. I filed this without searching existing PRs first — my mistake.

I have moved the production measurements (108,591,120 rows / 7.9 GB read to return 9 rows; 1.3s warm, 61s cold) and a note about ProfileTypes' zero-bounds behaviour over to #6402, where they are more useful. Please review that one instead.

@guettlibot guettlibot closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant