XML: keep HTML void element support from changing how malformed XML recovers - #8563
XML: keep HTML void element support from changing how malformed XML recovers#8563timtebeek wants to merge 1 commit into
Conversation
…ecovers The `element` rule was left-factored when void element support was added in #7906, so strict XML and HTML-like sources began sharing one shape. The shape of the rule, not just which alternative matches, determines the resynchronization sets ANTLR's error recovery uses, so the change altered recovery for XML too -- despite the void element predicate itself being gated on `htmlMode`. `XMLLexer` ends a `STRING` at the first embedded quote, because an attribute value may contain neither `"` nor `<`. An attribute whose value carries an unescaped quote -- common in embedded expressions such as Azure API Management policies -- therefore re-lexes the rest of the value as further attributes. Under the new shape those recovered into an `attribute` with a synthesized `=`, which the printer cannot distinguish from a real one and wrote back into the value: - list += "AAA,"; + list += "AAA=,"; One level of nesting deeper the same input instead abandoned the rule with a null `EQUALS`, and `visitAttribute` threw a `NullPointerException`. Give strict XML back the two plain alternatives it had before void element support existed, gated by `isHtmlMode()`, and reject attributes and elements that error recovery left incomplete or invented so such input falls back to a `ParseError` preserving the original text, as #7555 established for other malformed input. The C# grammar sources are left as they are, matching #7906, which also did not regenerate them.
|
Closing as a duplicate of #8562, which covers this and does it better. I ran both against a shared corpus. #8562 alone already resolves the two symptoms this PR was opened for — the invented
Everything else in the corpus was identical, including void element handling for The #7906 attribution is already captured in #8562's description via the 8.84.4 bisect, so nothing here is additive. |
What's wrong
elementrule, so strict XML and HTML-like sources now share one shape:That PR argued this was safe for XML because the predicate is gated on
htmlMode. The predicate is — but the rule shape isn't, and the shape is what determines the resynchronization sets ANTLR'sDefaultErrorStrategyuses. Recovery changed for every source the XML parser touches, including strict XML, where the void alternative is never taken.The input that exposes it:
XMLLexerhasSTRING : '"' ~[<"]* '"', so an attribute value may contain neither"nor<. A value carrying an unescaped quote — routine in embedded expressions such as Azure API Management policies, whose bodies span many lines — ends itsSTRINGat the first embedded quote, and the remainder re-lexes as further attributes.Under the new shape those recover into an
attributewith a synthesized=. Once built, the printer cannot tell it from a real one and writes it back into the source:<set-variable name="allowList" value="@{ string list = ""; - list += "AAA,"; + list += "AAA=,";One level of nesting deeper, the same input instead abandons the rule with a null
EQUALS, andvisitAttributethrows:Two symptoms, one cause; nesting alone decides which you get. Neither reproduces on any released version — I checked 8.20.0, 8.40.0 and 8.56.1, and confirmed by bisect and by reverting only
rewrite-xml/onto currentmain.The fix
Give strict XML back its original rule, gated by
isHtmlMode(), so HTML keeps the left-factored shape and XML keeps the recovery behaviour it had before rewrite-xml: support HTML void elements in JSP/HTML parsing #7906. This is what makes "HTML support does not affect XML parsing" true of recovery and not only of matching.Reject attributes and elements that recovery left incomplete or invented.
visitAttributenow rejects a context with a missingName/STRINGor a synthesizedEQUALS, andvisitElementrejects a nameless element, so the document falls back to aParseErrorthat preserves the original text — the contract XML: harden parser against malformed input crashes #7555 established for other malformed input. A synthesized token is detectable (getTokenIndex() < 0); a synthesized=in a builtXml.Attributeis not, which is why this has to happen at the parse tree.These files genuinely are not well-formed XML —
xmllintrejects them — so aParseErroris the right outcome. What isn't right is a crash, or a reprint that differs from the input.I deliberately did not blanket-null-guard
convert(TerminalNode, ...). On its own that converts the crash into a silently wrong LST and hides the grammar problem, which is strictly worse; the print-idempotency check only catches corruption that happens to be visible in the reprint.Tests
Three tests in
XmlParserTest, each verified to fail without the main-source change and pass with it:attributeValueWithUnescapedQuoteIsNotGivenAnExtraEquals— round-trips again instead of gaining an=policyExpressionAttributeValueIsNeverRewritten— a policy-style document;ParseError, text byte for byte, and asserts noAAA=in the reported diffnestedAttributeValueWithUnescapedQuoteAndAngleBracketDoesNotThrow—ParseErrorrather than an NPE:rewrite-xml:testand:rewrite-maven:testare green, including the void element and JSP tests from rewrite-xml: support HTML void elements in JSP/HTML parsing #7906.Notes
elementrule onisHtmlMode()versus keeping one shape and constraining recovery some other way. Duplicating the alternatives grows the ATN, but it is the only version I found that leaves XML recovery provably untouched.rewrite-csharp/are left alone, matching rewrite-xml: support HTML void elements in JSP/HTML parsing #7906, which also did not regenerate them;XMLParserBasehas no C# counterpart yet.