[samuel-williams-shopify]

When reading a chunked body, trailer fields were added to the message headers without marking where the trailer section begins, so they ended up indistinguishable from header fields. This change marks the trailer section when the first trailer field is received, so headers.header and headers.trailer remain distinct.

Problem

Protocol::HTTP1::Body::Chunked#read_trailer called @headers.add(name, value, trailer: true) for each received trailer field, but never called Headers#trailer! to record where the trailer section starts. After the body was read, headers.trailer? was false, headers.trailer was empty, and headers.header included the trailer fields.

Anything inspecting or forwarding the headers after the body has been read would therefore treat trailer fields as header fields. RFC 9110, Section 6.5.2 says a recipient MUST NOT merge trailer fields into the header section unless it knows the field semantics permit it. This is more significant because the default Protocol::HTTP::Header::Generic policy permits arbitrary unlisted fields (e.g. x-forwarded-host) in trailers, and request trailers are controlled by the client.

For example, a chunked message with trailer x-forwarded-host: evil previously yielded headers.trailer? => false and headers.header.to_a => [["etag", "\"t\""], ["x-forwarded-host", "evil"]].

Solution

read_trailer now calls @headers.trailer! immediately before adding the first trailer field. Since trailer! is @tail ||= @fields.size, this is a no-op if the trailer section was already marked, so existing callers which mark the trailer section themselves continue to work. Validation is unchanged, as add was already invoked with trailer: true.

Marking is deliberately lazy: the overwhelmingly common case is an empty trailer section (0\r\n\r\n), and in that case the headers are left untouched. This means headers.trailer? stays false, later calls to headers.add are not switched into trailer validation (which would otherwise reject e.g. content-type), frozen headers do not raise FrozenError, and the behaviour matches socketry/async-http#259, which only marks the trailer section when a trailing HEADERS frame actually arrives.

After this change, the same message yields headers.trailer? => true, headers.header.to_a => [["etag", "\"t\""]] and headers.trailer.to_a => [["x-forwarded-host", "evil"]].

Residual exposure

Only the header and trailer enumerators are separated. The indexed lookup API, including headers["x-forwarded-host"], headers.include?, and headers.to_h, still covers all fields, so code which looks up a field by name after the body has been read will still see trailer values. Callers which need to ignore trailers for a given field should consult headers.header, and headers.flatten remains available for callers which explicitly want to merge trailers. Separately, Headers#delete and #extract do not adjust @tail when removing fields from the header section, which is addressed in socketry/protocol-http#125.

Tests

Tests cover the body reader directly (announced and unannounced trailers, an empty trailer section including frozen headers and subsequent add, and the case where the trailer section was already marked and contained a field), and end-to-end cases through both Connection#read_response and Connection#read_request.

[samuel-williams-shopify]

Closing after discussion: this PR models the wrong thing.

Headers#add(key, value, trailer: true) already answers the question that matters on receive — may this field be a trailer? — and main applies it to every received trailer field. A field whose policy permits trailers is one whose semantics do not depend on arriving before the body, so once received it is simply a field; the consumer may treat it as a header.

@tail (set by trailer!) answers a different, sender-side question — from this point, fields will be sent as trailers — and setting it on receive conflates "where did this arrive" with "how will this be sent". A buffering proxy shows why that is wrong: it receives the whole message, forwards with a content-length body, and write_fixed_length_body ignores the trailer section, so fields this PR moved into the trailer section would be dropped on the way out. On main they are forwarded as headers, which is correct.

The remaining question is whether the policy is right — in particular Protocol::HTTP::Header::Generic.trailer? returning true, which permits every unknown field as a trailer. That is tracked in socketry/protocol-http (issue linked below).

See socketry/protocol-http#127.