Skip to content

[Bug]: file resource with an http(s) source and no cache-validation headers is treated as changed on every run #581

Description

@miharp

Summary

A file resource whose source is an http(s) URL, pointed at a server that sends no Last-Modified, ETag, or checksum header at all — or sends an ETag that's never consulted because checksum => etag wasn't requested — is treated as "changed" on every single puppet apply/agent run, forever. This causes the file to be rewritten and any notify/subscribe relationship to fire every run (e.g. restarting a service on every Puppet run), plus constant log/report noise.

This is a more precise, OpenVox-specific write-up of puppetlabs/puppet#9553, which reports the same symptom but frames it as a missing feature (staging_location) rather than identifying the actual root cause below. It's filed separately here because the root cause and fix are specific to this codebase's current HTTP-checksum logic (which already diverges from upstream via checksum => etag support, #328/#329).

Root cause

Puppet::FileServing::HttpMetadata#initialize fabricates the current wall-clock time as a fake mtime whenever the HTTP response gives no consulted signal:

@checksums[:mtime] = "{mtime}#{Time.now}"

Since :mtime is always the last resort in the checksum-type fallback chain in #collect, and the local file's on-disk mtime can never be newer than "right now," this comparison is guaranteed to conclude "changed" on every request — independent of whether the remote content actually changed. Note this also affects servers that do send an ETag (see the GitHub example below): #collect only checks @checksums[:etag] when the resource's checksum parameter is literally :etag, so a default file resource never benefits from an ETag at all unless the user knows to opt in.

Steps to reproduce

Two real-world origins that trigger this today:

  1. Artifactory behind a caching proxy — sends neither header at all:

    $ curl -sI https://packages.adoptium.net/artifactory/api/gpg/key/public
    HTTP/2 200
    date: ...
    content-type: text/plain
    content-length: 1793
    ...
    

    (no etag, no last-modified)

  2. raw.githubusercontent.com — sends an ETag, but it's never consulted without checksum => etag:

    $ curl -sI https://raw.githubusercontent.com/torvalds/linux/master/README
    HTTP/2 200
    etag: "f31342d556cea9c5785f1e4b48d9954cbc68ef21914dc817dbb009cb63b9ee4d"
    ...
    

    (no last-modified)

file { '/etc/example-key':
  source => 'https://raw.githubusercontent.com/<owner>/<repo>/<branch>/<path>',
  notify => Service['example'],
}

Service['example'] restarts on every Puppet run against either source, indefinitely — with no checksum => etag workaround available for the Artifactory case, since it sends no ETag either.

Self-contained, runnable repro — see miharp/openvox#1, which runs a real puppet apply in Docker against a hand-rolled HTTP server that can withhold or misuse cache-validation headers on demand. It includes console output from both an unpatched run (scenarios 2 and 4 spuriously fire notify on unchanged content — this bug, in both the Artifactory and raw.githubusercontent.com shapes) and the fix branch (same scenarios, no spurious fire).

Impact

  • Unwanted service restarts on every run for any resource sourced this way.
  • Report/log noise on every compile.
  • Affects any origin that omits both ETag and Last-Modified, and — more broadly — any origin whose ETag isn't explicitly opted into via checksum => etag, which is most manifests in practice.

Proposed fix

#582

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions