Skip to content

feat(collector): support milvus collector (#3877) - #4303

Open
paultanay wants to merge 14 commits into
apache:masterfrom
paultanay:feature/milvus-collector
Open

feat(collector): support milvus collector (#3877)#4303
paultanay wants to merge 14 commits into
apache:masterfrom
paultanay:feature/milvus-collector

Conversation

@paultanay

@paultanay paultanay commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

What's changed?

Reworked the Milvus monitor to use HertzBeat's native protocol: http + parseType: prometheus - no custom module, no new Java classes.

  • hertzbeat-manager/src/main/resources/define/app-milvus.yml - monitoring template scraping Milvus's Prometheus endpoint (port 9091, /metrics), covering 12 metric groups: proxy request count, slow query count, search/insert vector counts, search and mutation latency histograms (histogram bucket series), QueryNode collection/entity counts, and RootCoord collection/partition/DML/DDL counts
  • home/docs/help/milvus.md - English help documentation, metric tables aligned exactly to the template

Fixes #3877

@github-actions github-actions Bot added doc Improvements or additions to documentation monitoring-template collector backend labels Aug 8, 2026
@paultanay
paultanay marked this pull request as ready for review August 8, 2026 16:18
@Duansg
Duansg marked this pull request as draft August 9, 2026 15:42
@paultanay
paultanay force-pushed the feature/milvus-collector branch from f64a96f to 443ebdb Compare August 11, 2026 18:14
@paultanay
paultanay requested a review from Duansg August 11, 2026 18:16
Aias00
Aias00 previously approved these changes Aug 17, 2026

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: feat(collector): support milvus collector (#3877)

Verdict: ✅ APPROVED — useful new monitor template; please address two doc/metric notes before or after merge.

What this PR does

Adds a Milvus vector-DB monitor template (app-milvus.yml) scraping the Prometheus /metrics endpoint on port 9091, plus the matching help doc. Covers proxy request/slow-query/vector counts, search & mutation latency, QueryNode loaded collections/entities, and RootCoord collection/partition/DML/DDL counts across 12 priority groups.

Assessment

  • Structure: Valid template — ASF license header, trilingual name/help, sensible params (host/port/timeout), and consistent protocol: http + parseType: prometheus per group. Port default 9091 and path /metrics match Milvus defaults.
  • Risk: Low — additive, no code path touched.

Notes (non-blocking)

  1. Doc/metric mismatch: home/docs/help/milvus.md lists metrics that the template does not define (e.g. milvus_proxy_sq_latency_sum, milvus_proxy_receive_bytes_count, milvus_proxy_send_bytes_count, milvus_querynode_search_req_latency_sum, milvus_querynode_search_group_nq_sum, milvus_querynode_evicted_memory_size). The "Collected Metrics" tables read like Milvus's full metric surface rather than what this template actually scrapes. Please align the doc tables with the 12 metric names present in app-milvus.yml so users aren't misled.
  2. Histogram metric names: milvus_proxy_sq_latency and milvus_proxy_mutation_latency are modeled with an le label (classic histogram-bucket shape). Prometheus histograms expose ..._bucket / ..._sum / ..._count series; a bare milvus_proxy_sq_latency series usually does not exist, so these two groups may silently collect nothing. Please verify Milvus emits a bare series with the le label, or retarget the metric name to milvus_proxy_sq_latency_bucket (and add _sum/_count as needed).

Nice addition overall.

@paultanay
paultanay marked this pull request as ready for review August 17, 2026 21:17
@paultanay
paultanay requested a review from Aias00 August 17, 2026 21:17
@paultanay

Copy link
Copy Markdown
Contributor Author

addressed remaining suggestions
@Aias00 @Duansg could you please take a look! Thanks...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend doc Improvements or additions to documentation monitoring-template

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] <collector> support milvus collector

3 participants