fix: reject messages whose parallel repeated fields disagree in length - #3507
Open
pjfanning wants to merge 1 commit into
Open
fix: reject messages whose parallel repeated fields disagree in length#3507pjfanning wants to merge 1 commit into
pjfanning wants to merge 1 commit into
Conversation
Motivation: Three places read repeated protobuf fields that are written in lockstep but read as if their lengths were guaranteed to agree. DaemonMsgCreateSerializer drives its loop over the constructor arguments by getSerializerIdsCount and indexes args, manifests and hasManifest with it, so a message where those disagree raises IndexOutOfBoundsException. The pre-2.4 branch zips args with manifests, which silently drops the tail of the longer one. ArteryMessageSerializer zips the keys and values of a compression table advertisement, so a mismatch silently builds a table the sender did not advertise, which is then acknowledged back to the sender as accepted. It also narrows the advertised table version, and the ack's version, from int to byte with byteValue, so versions 256 apart are indistinguishable. Modification: Check the lengths agree before indexing, and check the table version fits in a byte before narrowing it. Report either as NotSerializableException. Both are conditions no toBinary produces. Result: A malformed message is reported as a serialization failure rather than raising IndexOutOfBoundsException or being silently accepted as something other than what it said.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Three places read repeated protobuf fields that
toBinarywrites in lockstep, but read them asif the lengths were guaranteed to agree.
DaemonMsgCreateSerializer:191-201drives its loop over the constructor arguments bygetSerializerIdsCountand then indexesgetArgs(idx),getManifests(idx)andgetHasManifest(idx)with it. A message where those disagree raisesIndexOutOfBoundsExceptionout of the generated accessor. The pre-2.4 branch below it zipsargs with manifests, which silently drops the tail of the longer list instead.
ArteryMessageSerializer:185-190zips the keys and values of a compression tableadvertisement. A mismatch silently builds a table the sender did not advertise — and this table
becomes the node's outbound compression table (
ArteryTransport.scala:523), which is thenacknowledged back to the sender as accepted at the advertised version.
ArteryMessageSerializer:190and:205narrow the advertised table version, and the ack'sversion, from
inttobytewithbyteValue/toByte.CompressionTable.versionis aByteon both sides, so anything outside that range is not a version a peer advertised, andnarrowing makes values 256 apart indistinguishable. I only spotted the ack site while writing
the fix; it has the same problem.
Modification
Check the lengths agree before indexing, and check the version fits in a byte before narrowing,
reporting either as
NotSerializableException. Both are conditions notoBinaryproduces:DaemonMsgCreateSerializer.toBinaryappends to all four lists per argument, andserializeCompressionAdvertisementadds a key and a value per entry from aByteversion.Result
A malformed message is reported as a serialization failure rather than raising
IndexOutOfBoundsExceptionor being silently accepted as something other than what it said.Tests
sbt "remote/testOnly org.apache.pekko.remote.serialization.*"— 199 passed, 1 pendingFive new tests, four of which were checked to discriminate by reverting the two production files
and re-running. The pre-fix failures are worth recording because they confirm each diagnosis:
IndexOutOfBoundsExceptionClassNotFoundException— it zipped down to one pair, then tried to load the string manifest as a class nameThe fifth,
accept the whole byte range of compression table versions, passes either way bydesign: it is the no-regression guard for
Byte.MinValue,-1,0,1andByte.MaxValue,since
-1means "disabled" and the range is documented onCompressionTable.sbt "remote/mimaReportBinaryIssues"— no issuessbt "remote/scalafmtCheckAll" headerCreateAll— cleanReferences
None.