Skip to content

Keep received HTTP/2 trailers distinct from the header section. - #259

Closed
samuel-williams-shopify wants to merge 2 commits into
mainfrom
http2-received-trailer-section
Closed

samuel-williams-shopify wants to merge 2 commits into
mainfrom
http2-received-trailer-section

Conversation

@samuel-williams-shopify

@samuel-williams-shopify samuel-williams-shopify commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

When an HTTP/2 stream received trailing headers, they were appended to the received message's Protocol::HTTP::Headers without marking where the trailer section begins, so they were merged into the header section. This change calls Headers#trailer! before adding the first received trailer field, keeping headers.header and headers.trailer distinct.

Problem

Async::HTTP::Protocol::HTTP2::Stream#receive_trailing_headers called @headers.add(key, value, trailer: true) for each received trailer field, which validates the field against the trailer policy, but never called @headers.trailer! to record the boundary. As a result, after reading the body, headers.trailer? was false, headers.header included the trailer fields, and headers.trailer was empty. RFC 9110 §6.5.2 says a recipient must not merge trailer fields into the header section unless the field semantics permit it.

Solution

receive_trailing_headers now calls @headers.trailer! immediately before adding each trailer field. trailer! is idempotent (@tail ||= @fields.size), so this is safe if trailers are delivered in multiple frames, and a trailing HEADERS frame with no fields leaves trailer? false rather than creating an empty trailer section. The sending side already calls trailer! on its own, separate, headers object before writing, so it is unaffected.

The same defect in protocol-http1's chunked body reader is being fixed separately in socketry/protocol-http1#64, which marks the trailer section lazily in the same way; this is the HTTP/2 counterpart. The new tests are HTTP/2-specific rather than in the shared AProtocol fixture because HTTP/1 will continue to fail them until the protocol-http1 fix is released, at which point the assertions can be moved into the shared fixture to cover every protocol.

Behavioural note

Once a received message's headers have a trailer section, Headers#add without an explicit trailer: argument runs in trailer mode and raises Protocol::HTTP::InvalidTrailerError for fields whose policy forbids trailers. Code that mutates a received message's headers after consuming the body (for example, a buffering proxy adding x-forwarded-for after reading the request body) must do so before reading the body, or build a fresh Headers instance. Previously such mutations silently succeeded because the trailer section was never marked.

socketry/protocol-http#125 fixes Headers#delete and Headers#extract to preserve the trailer boundary (@tail) when removing fields; it should ship before this change, otherwise deleting a header field from a received message with trailers can shift the boundary and reclassify fields.

Not addressed here

A second HEADERS frame without END_STREAM on a stream that already has headers falls through to receive_initial_headers rather than being treated as a malformed trailer. If it carries only regular fields, the request still passes valid? (from the first frame), @headers is replaced with a fresh instance, and prepare_input raises ArgumentError ("Input body already prepared!"), which process_headers does not rescue, so the whole connection is torn down instead of the stream being reset with RST_STREAM(PROTOCOL_ERROR). This is pre-existing and orthogonal to the trailer section boundary, so it is left for a separate change.

@samuel-williams-shopify

Copy link
Copy Markdown
Contributor Author

Closing after discussion, for the same reason as socketry/protocol-http1#64.

Headers#add(..., trailer: true) already enforces whether a received field may be a trailer. @tail is a sender-side marker ("fields from here will be sent as trailers"); setting it on received HTTP/2 trailing HEADERS conflates arrival position with send framing, and would cause a buffering proxy that forwards with a known-length body to drop the fields. The policy question (Generic.trailer?) is tracked in socketry/protocol-http.

See socketry/protocol-http#127.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant