Skip to content

impala: add container-based config discovery support#24320

Draft
vitkyrka wants to merge 3 commits into
masterfrom
discovery/impala
Draft

impala: add container-based config discovery support#24320
vitkyrka wants to merge 3 commits into
masterfrom
discovery/impala

Conversation

@vitkyrka

@vitkyrka vitkyrka commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds container-based config discovery support to the impala integration. When the Datadog Agent detects a container with the impala AD identifier, it probes the container's exposed ports — starting with 25000 if it is among them — until it finds the metrics endpoint, then schedules an OpenMetrics check against it automatically — no manual conf.yaml needed.

Scope: the apache/impala Docker image is shared by all four Impala daemon roles (impalad, statestored, catalogd, hms), but only impalad exposes port 25000. Using port_hints: [25000] in the from_ports strategy ensures only the daemon (service_type: daemon) role is ever discovered, so the ambiguous shared image never produces a false-positive candidate for the other roles. This PR intentionally scopes discovery to the daemon role only, matching the integration's default conf.yaml.example and its primary E2E test target — discovery for statestore/catalog is left as a follow-up since those roles aren't exposed on a stable, image-distinguishable port.

Known gap: the new test_e2e_discovery / test_e2e_discovery_all_candidates tests could not be run to completion in this sandbox because the apache/impala Docker images are linux/amd64-only and crash under QEMU user-mode emulation on this host's aarch64 architecture (qemu: uncaught target signal 4 (Illegal instruction) - core dumped). This was confirmed to be a pure environment limitation, not a code defect, because the same crash also broke a pre-existing, unrelated E2E test during the identical session-fixture container startup. Unit tests, formatting, and ddev validate all impala all pass cleanly; the discovery E2E tests should be verified in CI or a native amd64 environment before merge.

Motivation

Part of DSCVR-548. Follows the same pattern established by the krakend integration.

Port selection rationale: Port 25000 is Impala's official documented default metrics port for the daemon (impalad) role, and is the port used by this integration's own conf.yaml.example and default E2E test instance.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Add an autodiscovery `discovery:` strategy to the impala OpenMetrics
check scoped to the impalad daemon (the role the default conf.yaml.example
and main E2E suite already target), keyed off the container's exposed
port 25000 rather than the shared `apache/impala` image tag so the
statestored/catalogd/hive-metastore roles are not misidentified.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 3, 2026

Copy link
Copy Markdown

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 97.98% (+9.78%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1e35465 | Docs | Datadog PR Page | Give us feedback!

vitkyrka and others added 2 commits July 3, 2026 18:56
Adds the changelog.d entry for PR #24320 (impala container-based config discovery support).

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…pose ps

assert_all_discovery_candidates_stable() shells out to a bare
`docker compose ps -q <service>`, which re-renders every service's
volume interpolations in the compose file, not just the target
service's. STATESTORED_LOG_FOLDER, CATALOGD_LOG_FOLDER, and
IMPALAD_LOG_FOLDER are only set for the duration of the `docker_run`
env_vars context and are unset by the time this later ad-hoc `ps`
call runs, so compose failed with "invalid spec: :/opt/impala/logs:
empty section between colons". Add `:-<default>` fallbacks so the
volume specs stay valid even when the vars are unset.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant