Skip to content

fix(observability): exclude the extension's JwtMetricsCollector so it stops failing every 10s (#173) - #184

Merged
cuioss-oliver merged 2 commits into
mainfrom
fix/173-jwt-metrics-collector-exclusion
Aug 6, 2026
Merged

fix(observability): exclude the extension's JwtMetricsCollector so it stops failing every 10s (#173)#184
cuioss-oliver merged 2 commits into
mainfrom
fix/173-jwt-metrics-collector-exclusion

Conversation

@cuioss-oliver

@cuioss-oliver cuioss-oliver commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Fixes #173.

The defect

de.cuioss.sheriff.token.quarkus.metrics.JwtMetricsCollector constructor-injects the SecurityEventCounter produced by the token-sheriff-validation extension's unqualified TokenValidatorProducer. That producer resolves issuers from sheriff.token.issuers.<name>.* — a namespace this gateway never populates, because the request path runs off the @GatewayValidator-qualified validator built from gateway.yaml.

The collector carries @Scheduled(every = "10s"). So bean creation failed on every tick, and the scheduler logged a full stack trace each time — roughly sixty lines every ten seconds. Readiness and token validation were unaffected; the log was not.

This is exactly the mismatch ADR-0027 already decided, with one bean left outside its scope.

Reproduced, then fixed

The new test was written first and run against unmodified main. It failed with the trigger identity from the issue report verbatim:

Registered extension jobs: [1_de.cuioss.sheriff.token.quarkus.metrics.JwtMetricsCollector#updateCounters
                            (de.cuioss.sheriff.token.quarkus.metrics.JwtMetricsCollector#updateCounters)]

Both guard legs passed in that run, so the failures were genuine rather than a container that had contributed nothing.

The change

quarkus.arc.exclude-types now names the extension's metrics package alongside its health package:

quarkus.arc.exclude-types=de.cuioss.sheriff.token.quarkus.health.*,de.cuioss.sheriff.token.quarkus.metrics.*

Deliberately not a de.cuioss.sheriff.token.quarkus.* sweep: that tree also carries beans a future change here may legitimately want (the @BearerToken producer, the claim-mapper registry), and a sweep would remove them at the moment someone reached for one. Deliberately not quarkus.scheduler.enabled=false either — that would pre-emptively remove a mechanism this gateway has no reason to forbid.

ADR-0027 broadened

The ADR's title described a subset of what it now decides, so it is restated over beans rather than probes: one structural mismatch, presenting as two symptoms depending on what drives each bean (health endpoint → aggregate readiness DOWN; scheduler → a stack trace every ten seconds). Renamed to 0027-The_token-validation_extensions_unqualified_beans_are_excluded_not_accommodated.adoc, with all four link sites updated.

The ADR's own risk section now records the honest weakness the second symptom exposed: the exclusion list grows by discovery, and each entry so far was added after someone observed a symptom in a running deployment.

The test

ExtensionUnqualifiedBeanExclusionTest carries two assertions plus a control:

  • SpecificJwtMetricsCollector is not a bean. Paired with SheriffMetrics resolving, so a run in which CDI contributed nothing cannot masquerade as a successful exclusion.
  • General (the fitness function) — no scheduled job may name the extension's Quarkus wiring. A bean-by-bean assertion only ever catches beans someone already knew about; this one fails the build when the next eagerly-driven newcomer lands.
  • Matched positive/negative control over the filter. With the exclusion in place nothing schedules anything, so Quarkus never starts the scheduler and the general assertion passes over an empty input. The control is what keeps a filter that had stopped matching from passing for a gateway that schedules nothing. Its near miss is de.cuioss.sheriff.token.validation.* — the library this gateway genuinely uses, which a prefix trimmed one segment too short would over-select.

Not in scope

Publishing the qualified validator's SecurityEventCounter as meters is genuinely absent and is separate work — the seam is SheriffMetrics.bindSecurityEventCounter, which already does this for the cui-http counter. The excluded collector never published a meter either (it failed inside its own dependency resolution), so this PR removes stack traces, not JWT metrics. Recorded in the ADR and in the properties comment rather than left to be rediscovered.

The issue's second suggestion — that a metrics collector finding no property-based issuers should warn once and go idle rather than fail hard on every tick — is a token-sheriff-side observation and is worth raising upstream. It is noted in the ADR's alternatives as complementary to this exclusion, not a substitute.

Verification

  • verify -Ppre-commit — green
  • verify — green

🤖 Generated with Claude Code

https://claude.ai/code/session_01QQM5D6d5HeMWfKta3hRQje

Summary by CodeRabbit

  • Bug Fixes

    • Prevented extension-owned health checks and scheduled metrics collection from affecting gateway readiness or triggering failures.
    • Preserved gateway-owned JWKS readiness reporting and core scheduler functionality.
  • Documentation

    • Updated architecture, configuration, environment-variable, and integration-test documentation to reflect the revised behavior.
    • Added an architectural decision record describing bean exclusions, readiness behavior, risks, and alternatives.
  • Tests

    • Added integration coverage confirming excluded metrics collectors and scheduled jobs are not available, while gateway functionality remains active.

… stops failing every 10s (#173)

The token-sheriff-validation-quarkus extension's `metrics.JwtMetricsCollector`
constructor-injects the `SecurityEventCounter` produced by the extension's
UNQUALIFIED `TokenValidatorProducer`, which resolves issuers from the
`sheriff.token.issuers.<name>.*` namespace this gateway never populates — the
request path runs off the `@GatewayValidator`-qualified validator built from
`gateway.yaml`. The collector carries `@Scheduled(every = "10s")`, so bean
creation failed on every tick and the scheduler wrote roughly sixty stack-trace
lines every ten seconds. Readiness and token validation were unaffected; the log
was not, and unreadable logs are a real cost during integration work.

This is the same structural mismatch ADR-0027 already decided, with one bean left
outside its scope. `quarkus.arc.exclude-types` now names the extension's metrics
package alongside its health package.

- Broaden ADR-0027 from "unqualified health probes" to "unqualified beans": one
  mismatch presenting as two symptoms depending on what drives each bean, and the
  decision restated over beans rather than probes. Renamed accordingly, with the
  four link sites updated.
- Stop short of a `de.cuioss.sheriff.token.quarkus.*` sweep: that tree also carries
  beans a future change may legitimately want, and a sweep would remove them
  silently.
- Add `ExtensionUnqualifiedBeanExclusionTest`. Beyond pinning the specific bean, it
  asserts the general form — no scheduled job may name the extension's Quarkus
  wiring — so the next bean of this shape fails the build instead of being found in
  a production log. Both legs carry the paired guards, and the filter carries a
  matched positive/negative control because the resting state is an empty input.

Publishing the QUALIFIED validator's SecurityEventCounter as meters remains absent
and is separate work; the collector never published a meter either, so this removes
stack traces, not JWT metrics.

Closes #173

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQM5D6d5HeMWfKta3hRQje
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@cuioss-oliver, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 28 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository: cuioss/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a709e0d-0df6-40e5-b8cd-93129dc4f52a

📥 Commits

Reviewing files that changed from the base of the PR and between aeb6922 and 8e7df20.

📒 Files selected for processing (2)
  • doc/adr/0027-The_token-validation_extensions_unqualified_beans_are_excluded_not_accommodated.adoc
  • doc/adr/0032-The_shipped_artifact_declares_nothing_test-shaped_the_deployment_supplies_what_it_names.adoc
📝 Walkthrough

Walkthrough

The gateway now excludes the token-validation extension’s unqualified health probes and JwtMetricsCollector through package-scoped CDI patterns. A Quarkus test verifies bean and scheduled-job exclusion. ADR and operational documentation describe the decision and readiness behavior.

Changes

Token-validation extension exclusion

Layer / File(s) Summary
Package-scoped CDI exclusions
api-sheriff/src/main/resources/application.properties
The configuration excludes the extension’s health.* and metrics.* types while retaining unrelated Quarkus beans.
Bean and scheduler verification
api-sheriff/src/test/java/de/cuioss/sheriff/gateway/quarkus/ExtensionUnqualifiedBeanExclusionTest.java
The integration test confirms that JwtMetricsCollector and extension-owned scheduled jobs are absent while gateway metrics and the scheduler remain available.
Decision and documentation updates
doc/adr/0027-..., doc/adr/0032-..., doc/architecture.adoc, doc/development/integration-test-topology.adoc, doc/user/environment-variable-overrides.adoc
The ADR and documentation describe the expanded exclusion, retained readiness behavior, and renamed references.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Assessment against linked issues

Objective Addressed Explanation
Exclude JwtMetricsCollector and preserve the ADR-0027 exclusion posture [#173]
Keep gateway token validation and gateway-owned readiness behavior unaffected [#173]

Possibly related PRs

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly states the main change: excluding JwtMetricsCollector to stop its recurring failures.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cuioss-review-bot

cuioss-review-bot Bot commented Aug 6, 2026

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 8e7df20)

🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The ExtensionUnqualifiedBeanExclusionTest relies on hard-coded trigger description strings (including IDs and formatting), which makes it brittle to internal scheduler formatting changes; consider asserting on parsed components (bean class + method) or mocking a Trigger instead of using full literal identities.
  • The extensionJobsAmong filter uses String.contains(EXTENSION_QUARKUS_PACKAGE) on the entire trigger description, which could produce false positives if any other job description happens to include that substring; you might tighten this to match on the bean class portion specifically rather than the full description.
  • Both the application properties comments and the test class Javadoc are quite long and narrative; it may be easier to maintain if you move some of the background/rationale to the ADR and keep the in-code comments focused on what is being pinned and how, rather than the full history and justification.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `ExtensionUnqualifiedBeanExclusionTest` relies on hard-coded trigger description strings (including IDs and formatting), which makes it brittle to internal scheduler formatting changes; consider asserting on parsed components (bean class + method) or mocking a `Trigger` instead of using full literal identities.
- The `extensionJobsAmong` filter uses `String.contains(EXTENSION_QUARKUS_PACKAGE)` on the entire trigger description, which could produce false positives if any other job description happens to include that substring; you might tighten this to match on the bean class portion specifically rather than the full description.
- Both the application properties comments and the test class Javadoc are quite long and narrative; it may be easier to maintain if you move some of the background/rationale to the ADR and keep the in-code comments focused on what is being pinned and how, rather than the full history and justification.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: cuioss/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: ef8f7aca-b970-47fa-b9af-d2588a2613ed

📥 Commits

Reviewing files that changed from the base of the PR and between ec2d903 and aeb6922.

📒 Files selected for processing (8)
  • api-sheriff/src/main/resources/application.properties
  • api-sheriff/src/test/java/de/cuioss/sheriff/gateway/quarkus/ExtensionUnqualifiedBeanExclusionTest.java
  • doc/adr/0027-The_token-validation_extensions_unqualified_beans_are_excluded_not_accommodated.adoc
  • doc/adr/0027-The_token-validation_extensions_unqualified_health_probes_are_excluded_not_accommodated.adoc
  • doc/adr/0032-The_shipped_artifact_declares_nothing_test-shaped_the_deployment_supplies_what_it_names.adoc
  • doc/architecture.adoc
  • doc/development/integration-test-topology.adoc
  • doc/user/environment-variable-overrides.adoc
💤 Files with no reviewable changes (1)
  • doc/adr/0027-The_token-validation_extensions_unqualified_health_probes_are_excluded_not_accommodated.adoc

…the metrics bean in ADR-0032

Review feedback on #184.

- ADR-0032 §5 said the exclusion key removes "the unused extension health probes";
  it now removes the metrics collector too, so the sentence names both.
- ADR-0027 §2 records the stronger check that was tried and rejected, so it is not
  attempted again from scratch. Asserting the extension's unqualified TokenValidator
  is unresolvable looks driver-agnostic but does not work: measured, it stays
  resolvable with every reaching bean excluded, because the producer is retained
  regardless. Retained is not instantiated — nothing calls a method on it, which is
  why the ten-second failure is genuinely gone — but that leaves resolvability unable
  to tell the healthy state from the broken one. Enumerating the container's beans
  and diffing against the exclusion list is the other candidate, and it is a
  membership snapshot, which ADR-0030 forbids.

What stays derivable is the symptom, not the scope. The gap is carried in the ADR's
Risks rather than papered over.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQM5D6d5HeMWfKta3hRQje
@cuioss-oliver

Copy link
Copy Markdown
Collaborator Author

Thanks @sourcery-ai — replying to all three points. Two are declined with reasons, one is worth being precise about.

1. Hard-coded trigger description strings are brittle.

Declined, because the literals are not in the production assertion — they are the fixture of a matched positive/negative control, which is a deliberate pattern here rather than an oversight. shouldRegisterNoScheduledJobFromTheExtension reads real Trigger objects from the running Scheduler and never parses a literal. filterFlagsExtensionJobsAndIgnoresGatewayJobs feeds the filter a synthetic list precisely so the filter is exercised independently of the container, and it exists because of a real vacuity hazard: with the fix in place nothing schedules anything, Quarkus never starts the scheduler, and the production assertion runs over an empty input. Without a control over the filter itself, a filter that had stopped matching would be indistinguishable from a gateway that schedules nothing.

If the scheduler's identity format changes, the control keeps passing (it tests the filter, not the format) and the production assertion keeps working (it reads getId() / getMethodDescription() through the API). Neither is coupled to the literal. The same idiom is used in ShippedApplicationPropertiesTest.detectorFlagsProfileKeysAndIgnoresPercentInsideValues in this module.

Mocking a Trigger is not available: Mockito is forbidden by this project's standards.

2. contains(EXTENSION_QUARKUS_PACKAGE) could false-positive.

Fair to raise, and the near miss you are pointing at is covered explicitly — the control's third fixture is de.cuioss.sheriff.token.validation.SomeJob#run, the validation library this gateway genuinely does use, which a prefix trimmed one segment too short would over-select. That is the collision that would actually matter.

Beyond that, the input is Trigger.getId() plus getMethodDescription(), both of which are <bean class>#<method> with a generated prefix — not free text — so a substring match over them is bounded to class names. And the failure direction is safe: a false positive fails the build with the offending identity printed, which someone reads and corrects. A false negative would be the dangerous one, and tightening to a parsed bean-class field would trade a loud, self-explaining failure for a parser that can silently stop matching when the identity format shifts.

3. Comments and Javadoc are long and narrative.

Declined — this is house style in this repository, not incidental. application.properties and the neighbouring test classes (DefaultProfileReadinessTest, ShippedApplicationPropertiesTest) carry the same density, and it is load-bearing here: the previous, shorter version of this exclusion is precisely what let a second bean sit outside its scope until it turned up in a production log.

The split you suggest is the one already in place. Rationale, alternatives and the residual gap live in ADR-0027 — including, as of 8e7df20, why a derived scope check is not available. The in-code text states what is pinned and why an assertion is not vacuous, which is the part a reader needs at the assertion rather than one file away.

@cuioss-oliver

Copy link
Copy Markdown
Collaborator Author

/review

@cuioss-oliver
cuioss-oliver added this pull request to the merge queue Aug 6, 2026
Merged via the queue into main with commit bcfe3c9 Aug 6, 2026
34 checks passed
@cuioss-oliver
cuioss-oliver deleted the fix/173-jwt-metrics-collector-exclusion branch August 6, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JwtMetricsCollector logs a stack trace every 10s — the ADR-0027 bean exclusion misses it

1 participant