Repository navigation
feat(fluentbit): add the prometheus_remote_write input - #2145
Open
NopeCharon wants to merge 1 commit into
Open
NopeCharon wants to merge 1 commit into
NopeCharon wants to merge 1 commit into
Conversation
Add a prometheusRemoteWrite field to the ClusterInput spec for Fluent Bit's prometheus_remote_write input plugin, which receives metrics sent with the Prometheus remote write protocol. Until now it was only usable through customPlugin. The input renders tag, listen, port, uri, buffer_max_size, buffer_chunk_size, successful_response_code and threaded, and reuses the shared TLS settings like the http input, so the listener can serve HTTPS with a key password read from a Secret. The CRD restricts successfulResponseCode to 200, 201 and 204, the only codes Fluent Bit accepts, and requires uri to start with a slash, since Fluent Bit compares it with the request path and rejects every request otherwise. tag_from_uri is left out because the plugin registers it but never reads it. Regenerate the deepcopy code, CRDs, setup.yaml and API docs, and list the input in docs/plugins/fluentbit/index.md. Fixes fluent#2144 Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Adds a
prometheusRemoteWriteinput to ClusterInput, for Fluent Bit'sprometheus_remote_writeinput plugin. It receives metrics pushed with the Prometheus remote write protocol, for example by Prometheus, Grafana Alloy or Fluent Bit's ownprometheus_remote_writeoutput. Until now it was only usable throughcustomPlugin.This renders as:
The input covers
listen,port,uri,bufferMaxSize,bufferChunkSize,successfulResponseCodeandthreaded. It also takes the sharedtlssettings that thehttpinput uses, so the listener can serve HTTPS, withkeyPasswordread from a Secret.The CRD catches two mistakes that otherwise only show up at runtime:
successfulResponseCodeis limited to 200, 201 and 204. Fluent Bit logs an error for any other value and falls back to 201.urimust start with/. Fluent Bit compares it exactly against the request path, so without the slash every request is rejected with400 invalid endpoint.I left out
tag_from_uri: in 5.2.0 the plugin registers it but never reads it. Theportdescription reminds users to add the port to the FluentBitportstoo, so the Service exposes it.The deepcopy code, CRDs,
setup.yamland API docs are regenerated, and the input is listed indocs/plugins/fluentbit/index.md.Which issue(s) this PR fixes:
Fixes #2144
Does this PR introduced a user-facing change?
Additional documentation, usage docs, etc.:
Testing
TestInput_PrometheusRemoteWrite_ParamsandTestInput_PrometheusRemoteWrite_ParamsOnlySetFieldscheck the rendered parameters with every field set, with none and with some.TestInput_PrometheusRemoteWrite_ParamsTLSchecks the TLS parameters, including a key password read from a Secret and the error when that key is missing.TestClusterInputList_Load_PrometheusRemoteWritechecks the classic and YAML output of a ClusterInput.make generate manifests docs-updateleaves no diff,make lintreports 0 issues, andgo test ./apis/fluentbit/...passes.I ran the rendered configs on
fluent/fluent-bit:5.2.0, with a second Fluent Bit sendingfluentbit_metricsthrough itsprometheus_remote_writeoutput:successfulResponseCode: 204andthreaded: true.In both cases the receiver printed the incoming metrics (e.g.
fluentbit_uptime{hostname="..."} = 1) and the sender logged no errors.This PR and #2143 (kafka input) each add a field next to
execWasiinInputSpec. Whichever merges second will need its generated files regenerated.