Repository navigation
Add bounded direct ProxySQL capacity probes - #203
Draft
pradeepkintali wants to merge 4 commits into
Draft
pradeepkintali wants to merge 4 commits into
pradeepkintali wants to merge 4 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: dea961fa-5766-415f-a0f5-2562c55a0385
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fbc83df2-d1af-406f-b97b-51cf0aaa570d
Carry shadow graph evaluation, isolated HTTP and background counters, startup notices, and public rollout guidance into the capacity stack while preserving direct capacity sampling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fbc83df2-d1af-406f-b97b-51cf0aaa570d
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fbc83df2-d1af-406f-b97b-51cf0aaa570d
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Add direct ProxySQL frontend and writer-backend capacity probes without extending the existing replica-lag/primary-concurrency rollout PR.
Stacked on #196. The base is
pkintali/a1-freno-primary-protectionso the diff contains only this capacity feature. It relies on that PR's composite dependencies, fail-closed checks, and recovery behavior. Retarget to the default branch after #196 merges. Nothing is merged or deployed by this PR.Behavior and load bounds
statsinterface using an explicit stats-only account; never follow discovered backend addresses to run capacity SQL on MySQL.Safety / limitations
No listeners, firewall rules, production credentials, runtime limits, or enforcement are changed here. Paired configuration PR: https://github.com/github/puppet/pull/44385 (separate, disabled-by-default rollout).
Stats-only credentials cannot read runtime configuration tables. Limits must be verified against runtime for every endpoint and kept synchronized when membership/configuration changes; there is no runtime-limit drift detection. Example limits/thresholds are not production recommendations.
No backend SQL or connections are added by these probes, but colocated ProxySQL still shares CPU/memory with MySQL. Production CPU/latency overhead has not been benchmarked and must be reviewed in shadow mode before enforcement.
Validation
script/cibuildpassed in the Go 1.20 container: binary build and race-enabled tests for all packages.Rollout / rollback
Deploy the binary with existing configuration first. Collection remains absent unless
ProxySQLCapacitySettingsis explicitly configured. Approve admin access/credential transport, verify limits/endpoint identity, observe query load, validate recovery/client defer behavior, then canary one cluster's required dependencies.Remove both capacity dependencies to roll back enforcement, retaining lag and primary concurrency. Remove the separately configured probes to stop capacity collection; coordinate configuration restarts and startup recovery.