Skip to content

K8SPG-1053 make PMM sidecar work with readOnlyRootFilesystem#1639

Open
hors wants to merge 2 commits into
mainfrom
K8SPG-1053
Open

K8SPG-1053 make PMM sidecar work with readOnlyRootFilesystem#1639
hors wants to merge 2 commits into
mainfrom
K8SPG-1053

Conversation

@hors

@hors hors commented Jun 13, 2026

Copy link
Copy Markdown
Collaborator

The pmm-agent config (pmm-agent.yaml) was written to the container root filesystem, so the sidecar failed on pods enforcing readOnlyRootFilesystem: true (e.g. OpenShift):
Config file /usr/local/percona/pmm/config/pmm-agent.yaml is not writable: read-only file system.

The config is stateless — pmm-agent regenerates it on every start (PMM_AGENT_SETUP_FORCE=1) — so from crVersion 3.1.0 relocate it to the writable /tmp/pmm path (inside the /tmp emptyDir already mounted into the sidecar) for both PMM2 and PMM3, and align the PMM2 tempdir to /tmp/pmm. Older crVersions keep the previous paths to avoid changing running pods.

CHECKLIST

Jira

  • Is the Jira ticket created and referenced properly?
  • Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
  • Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

  • Is an E2E test/test case added for the new feature/change?
  • Are unit tests added where appropriate?

Config/Logging/Testability

  • Are all needed new/changed options added to default YAML files?
  • Are all needed new/changed options added to the Helm Chart?
  • Did we add proper logging messages for operator actions?
  • Did we ensure compatibility with the previous version or cluster upgrade process?
  • Does the change support oldest and newest supported PG version?
  • Does the change support oldest and newest supported Kubernetes version?

The pmm-agent config (pmm-agent.yaml) was written to the container root
filesystem, so the sidecar failed on pods enforcing
readOnlyRootFilesystem: true (e.g. OpenShift):
  Config file /usr/local/percona/pmm/config/pmm-agent.yaml is not
  writable: read-only file system.

The config is stateless — pmm-agent regenerates it on every start
(PMM_AGENT_SETUP_FORCE=1) — so from crVersion 3.1.0 relocate it to the
writable /tmp/pmm path (inside the /tmp emptyDir already mounted into the
sidecar) for both PMM2 and PMM3, and align the PMM2 tempdir to /tmp/pmm.
Older crVersions keep the previous paths to avoid changing running pods.
Copilot AI review requested due to automatic review settings June 13, 2026 16:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the PMM sidecar container configuration so that the pmm-agent.yaml file is written to a writable location (/tmp/pmm) when readOnlyRootFilesystem: true is enforced, while preserving legacy paths for older CR versions.

Changes:

  • Added version-gated helpers to select a writable PMM agent config path and (for PMM2) an aligned temp directory starting from CR version 3.1.0.
  • Updated PMM2 and PMM3 sidecar environment variables to use the new config path logic (and PMM2 tempdir logic).
  • Extended/unit-updated tests to validate the new config path behavior and updated expected env var values.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
percona/pmm/pmm.go Introduces version-based selection of PMM_AGENT_CONFIG_FILE (and PMM2 tempdir) to support read-only root filesystems.
percona/pmm/pmm_test.go Adds coverage for config path selection and updates expected env var values for PMM sidecars.

egegunes
egegunes previously approved these changes Jun 15, 2026

@egegunes egegunes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but please check monitoring tests

@JNKPercona

Copy link
Copy Markdown
Collaborator
Test Name Result Time
backup-enable-disable passed 00:12:28
builtin-extensions passed 00:06:00
cert-manager-tls passed 00:09:21
custom-envs passed 00:18:45
custom-tls passed 00:05:49
database-init-sql passed 00:02:28
demand-backup passed 00:23:16
demand-backup-offline-snapshot passed 00:12:49
dynamic-configuration passed 00:03:37
finalizers passed 00:06:00
init-deploy passed 00:03:16
huge-pages passed 00:02:55
major-upgrade-14-to-15 passed 00:10:21
major-upgrade-15-to-16 passed 00:10:31
major-upgrade-16-to-17 passed 00:10:38
major-upgrade-17-to-18 passed 00:10:45
ldap passed 00:03:52
ldap-tls passed 00:05:40
monitoring passed 00:08:12
monitoring-pmm3 passed 00:09:14
one-pod passed 00:05:56
operator-self-healing passed 00:10:27
pitr passed 00:11:30
scaling passed 00:05:23
scheduled-backup passed 00:27:02
self-healing passed 00:09:24
sidecars passed 00:02:43
standby-pgbackrest passed 00:19:44
standby-streaming passed 00:12:48
start-from-backup passed 00:11:39
tablespaces passed 00:06:48
telemetry-transfer passed 00:04:35
upgrade-consistency passed 00:06:34
upgrade-minor passed 00:06:55
users passed 00:04:54
Summary Value
Tests Run 35/35
Job Duration 01:41:30
Total Test Time 05:22:36

commit: 808852d
image: perconalab/percona-postgresql-operator:PR-1639-808852d24

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants