impala: add container-based config discovery support#24320
Draft
vitkyrka wants to merge 3 commits into
Draft
Conversation
Add an autodiscovery `discovery:` strategy to the impala OpenMetrics check scoped to the impalad daemon (the role the default conf.yaml.example and main E2E suite already target), keyed off the container's exposed port 25000 rather than the shared `apache/impala` image tag so the statestored/catalogd/hive-metastore roles are not misidentified. Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 1e35465 | Docs | Datadog PR Page | Give us feedback! |
Adds the changelog.d entry for PR #24320 (impala container-based config discovery support). Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…pose ps assert_all_discovery_candidates_stable() shells out to a bare `docker compose ps -q <service>`, which re-renders every service's volume interpolations in the compose file, not just the target service's. STATESTORED_LOG_FOLDER, CATALOGD_LOG_FOLDER, and IMPALAD_LOG_FOLDER are only set for the duration of the `docker_run` env_vars context and are unset by the time this later ad-hoc `ps` call runs, so compose failed with "invalid spec: :/opt/impala/logs: empty section between colons". Add `:-<default>` fallbacks so the volume specs stay valid even when the vars are unset. Environment: Datadog workspace Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Validation ReportAll 21 validations passed. Show details
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds container-based config discovery support to the impala integration. When the Datadog Agent detects a container with the
impalaAD identifier, it probes the container's exposed ports — starting with 25000 if it is among them — until it finds the metrics endpoint, then schedules an OpenMetrics check against it automatically — no manualconf.yamlneeded.Scope: the
apache/impalaDocker image is shared by all four Impala daemon roles (impalad,statestored,catalogd,hms), but onlyimpaladexposes port 25000. Usingport_hints: [25000]in thefrom_portsstrategy ensures only the daemon (service_type: daemon) role is ever discovered, so the ambiguous shared image never produces a false-positive candidate for the other roles. This PR intentionally scopes discovery to thedaemonrole only, matching the integration's defaultconf.yaml.exampleand its primary E2E test target — discovery forstatestore/catalogis left as a follow-up since those roles aren't exposed on a stable, image-distinguishable port.Known gap: the new
test_e2e_discovery/test_e2e_discovery_all_candidatestests could not be run to completion in this sandbox because theapache/impalaDocker images arelinux/amd64-only and crash under QEMU user-mode emulation on this host'saarch64architecture (qemu: uncaught target signal 4 (Illegal instruction) - core dumped). This was confirmed to be a pure environment limitation, not a code defect, because the same crash also broke a pre-existing, unrelated E2E test during the identical session-fixture container startup. Unit tests, formatting, andddev validate all impalaall pass cleanly; the discovery E2E tests should be verified in CI or a native amd64 environment before merge.Motivation
Part of DSCVR-548. Follows the same pattern established by the krakend integration.
Port selection rationale: Port 25000 is Impala's official documented default metrics port for the daemon (
impalad) role, and is the port used by this integration's ownconf.yaml.exampleand default E2E test instance.Review checklist (to be filled by reviewers)
qa/requiredif this PR needs QA validation, orqa/skip-qaif it does not. Exactly one of the two is required.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged