Skip to content

Java: fix two comment-scanning defects that drop modifiers from the LST - #8564

Open
timtebeek wants to merge 2 commits into
mainfrom
tim/java-print-idempotency-defects
Open

Java: fix two comment-scanning defects that drop modifiers from the LST#8564
timtebeek wants to merge 2 commits into
mainfrom
tim/java-print-idempotency-defects

Conversation

@timtebeek

@timtebeek timtebeek commented Aug 19, 2026

Copy link
Copy Markdown
Member

Two Java parser print-idempotency defects, found while triaging ParseFailures rows from a large ingest. Both surface as:

java.lang.IllegalStateException: <file> is not print idempotent.
	at org.openrewrite.Parser.requirePrintEqualsInput(Parser.java:52)

Both are broken comment-state tracking in the same two scanners, reached from opposite directions, so they're one PR with a commit each.

A third, unrelated defect from the same triage is split out to #8565.


1. /* inside a line comment latches the modifier scanner

sortedModifiersAndAnnotations and collectAnnotations walk the source between an annotation and the declaration that follows, to recover modifiers in source order. Their comment scanner sets inMultilineComment on /* without checking whether it is already inside a // comment:

char c = source.charAt(i);
if (c == '/' && source.length() > i + 1) {
    char next = source.charAt(i + 1);
    if (next == '*') {
        inMultilineComment = true;   // fires on the second '/' of "//*"
    } else if (next == '/') {
        inComment = true;
    }
}

Walking //*x: the first / sets inComment, the second sees next == '*' and sets inMultilineComment. At the newline inComment is cleared but inMultilineComment is not, and nothing clears it short of a literal */. The scan finds no modifier keywords, so the modifiers never enter the LST and the declaration's type is printed at the modifier's offset.

Reproducer:

class Test {
    @Deprecated  //*not a block comment
    public String value() {
        return null;
    }
}

prints as:

class Test {
    @Deprecated  //*not a block comment
    String String value() {
        return null;
    }
}

String and public are both 6 characters, so the clobber lands cleanly here. With a shorter type the leftover tail of public is emitted as the name's prefix — a Step return type prints as Stepic Step, and a 7-character Tasklet swallows the following space and prints as TaskletTasklet.

The /* can appear anywhere in the comment, not just immediately after the //. Fields are affected too (there the modifier is simply lost). A real /* ... */ later on clears the state and masks the bug.

Scope: not a regression — the scanner dates back to a59e6e970f (2021) and this reproduces on every release since. Present identically in the Java 8, 11, 17, 21 and 25 parsers; all five are fixed here, and the test is in the TCK so it runs against each.

Fix: guard the opener against re-entering comment state.

2. /*/ closes a block comment it shouldn't

Same two scanners, opposite direction from (1). The close condition only checked that the previous character was *:

if (inMultilineComment && c == '/' && source.charAt(i - 1) == '*') {

The third character of /*/ satisfies that, so the scanner leaves comment state while still inside the comment, then reads the comment body as modifiers:

class Test {
    @Deprecated /*/ not the end of the comment */ public String value() {
        return null;
    }
}

prints as ... String String value() — the same corruption as (1), reached the other way.

Fix: record where the opener started and require the closing / to be at least three characters past it, so the opener's own / can never close the comment. This also subsumes the i > 0 bounds check, which was only present in collectAnnotationssortedModifiersAndAnnotations could read source.charAt(-1).

Scope: pre-existing, same five parser modules.


Testing

Five new TCK tests in CommentTest; all five fail on main and pass with the fix, and being in the TCK they run against every parser module. Beyond the reproducers above, the /*/ change was checked against /**/, /***/, /*/*/, /* /* */, back-to-back /*/ a */ /*/ b */, a multi-line /*/-opened comment, and no-whitespace @Deprecated/*/ x */public. Full compatibilityTest green across rewrite-java-8/11/17/21/25, plus the rewrite-java, rewrite-java-test and rewrite-java-lombok test suites.

Aside, not touched here: StringUtils (rewrite-core) has a scanner of the same shape that can also latch inMultilineComment from inside a // comment, e.g. on // /*. It consumes two characters per delimiter so //* doesn't trip it, and I have no failing case for it, so I've left it alone rather than widen this PR.

`sortedModifiersAndAnnotations` and `collectAnnotations` scan the source
between an annotation and the declaration it precedes to recover modifiers
in source order. Their hand-rolled comment scanner sets `inMultilineComment`
on `/*` without first checking whether it is already inside a `//` comment,
so a line comment containing `/*` latches the flag; nothing clears it short
of a literal `*/`.

The scan then finds no modifier keywords, so the modifiers never make it
into the LST and the declaration's type is printed at the modifier's offset:

    @deprecated  //*not a block comment
    public String value() { ... }

prints as `String String value()`, and with a type shorter than `public`
the leftover characters are emitted as the name's prefix (`Step` over
`public` yields `Stepic`).

Guard both scanners against re-entering comment state, in all five parser
modules.
@timtebeek
timtebeek force-pushed the tim/java-print-idempotency-defects branch from e45ae28 to 0235435 Compare August 19, 2026 19:42
@timtebeek
timtebeek marked this pull request as ready for review August 19, 2026 19:49
@timtebeek timtebeek changed the title Java: fix two parser print-idempotency defects (line-comment /*, markdown doc comment EOI sentinel) Java: fix three parser print-idempotency defects in comment scanning Aug 19, 2026
The close condition in `sortedModifiersAndAnnotations` and
`collectAnnotations` only checked that the previous character was `*`, so
the third character of `/*/` satisfied it and the scanner left comment
state while still inside the comment. It then read the comment body as
modifiers and dropped the real ones:

    @deprecated /*/ not the end of the comment */ public String value()

printed as `... String String value()` -- the same silent corruption as the
`//*` case, from the opposite direction.

Track where the opener started and require the closing `/` to be at least
three characters past it, so the opener's own `/` can never close the
comment. This also subsumes the `i > 0` bounds check, which was present
only in `collectAnnotations`; `sortedModifiersAndAnnotations` could read
`source.charAt(-1)`.

Adds a regression test for the inverse case too, `/* // */` before
modifiers, which the preceding commit fixes but nothing covered.
@timtebeek
timtebeek force-pushed the tim/java-print-idempotency-defects branch from e5df27e to 611efaf Compare August 19, 2026 20:23
@timtebeek timtebeek changed the title Java: fix three parser print-idempotency defects in comment scanning Java: fix two comment-scanning defects that drop modifiers from the LST Aug 19, 2026
@timtebeek
timtebeek marked this pull request as draft August 19, 2026 20:24
@timtebeek timtebeek added bug Something isn't working java parser labels Aug 19, 2026
@timtebeek timtebeek moved this from In Progress to Ready to Review in OpenRewrite Aug 19, 2026
@timtebeek
timtebeek marked this pull request as ready for review August 19, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working java parser

Projects

Status: Ready to Review

Development

Successfully merging this pull request may close these issues.

2 participants