Repository navigation
[mypyc] Fix range() loops with native int bounds and step > 1 near the type's maximum - #22470
Closed
Jahongir-Qurbonov wants to merge 1 commit into
Closed
Jahongir-Qurbonov wants to merge 1 commit into
Jahongir-Qurbonov wants to merge 1 commit into
Conversation
…e type's maximum When the end value of a range() for loop has a native int type, the loop index uses the same type. With a step whose magnitude is greater than 1, incrementing the index could overflow past the bounds of the type, so for example `for i in range(u64(2**64 - 2), u64(2**64 - 1), 2)` never terminated, and with signed types this was undefined behavior. Now when |step| > 1, the step block first checks whether stepping would go past the bounds of the type (by comparing the index against a constant). If so, it would also go past the end value, so the index is set to the end value instead, which ends the loop. Negative steps with unsigned end values were also rejected at compile time (`Value -1 is out of range for "u64"`), since the step was added as a negative constant. Subtract the magnitude of the step instead. If the magnitude of the step doesn't fit in the native int type, fall back to using an arbitrary precision int index.
Collaborator
|
I have an alternative implementation in a branch I'm working on. I will compare the two -- possibly an ideal implementation shared properties between both. |
JukkaL
added a commit
that referenced
this pull request
Nov 17, 2026
A for loop over range(a, b, step) with a literal step other than 1 or -1 could overflow the index near the limits of its type, and keep looping with a wrapped-around index. For example, with u8 bounds, range(250, 255, 2) produced 250, 252, 254, 0, 2, ... For a native int index, exit the loop before the increment if index + step wouldn't fit in the type. Such a value would be past the end anyway. This is a single compare against a constant. It's skipped if the step is 1 or -1, or if the end is a constant that rules out overflow. For a short int index, fall back to an int index if the step could overflow a short int. This happens only for literal bounds near +-2**62 or huge steps. Also load a step that doesn't fit in a short int as an int literal. Previously this generated an out-of-range C literal, which either broke the build or silently wrapped the step (so the loop could hang). Alternative to #22470. This generates a bit simpler IR and appears to be faster on my mac laptop (perf is similar on x86-64 Linux).
Collaborator
|
I went with #22501 instead, which was more efficient on Apple Silicon in my experiments. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the end value of a
range()for loop has a native int type, mypyc uses that type for the loop index register. The loop checksi < endand then doesi += step. With step 1 this can't overflow, but with a larger step the increment can wrap around past the bounds of the type before the condition gets a chance to stop the loop:The same happens with u8 etc., and with signed types (i8/i16/i32/i64) near the maximum value it's signed overflow (undefined behavior in C for i64).
Also, a negative step with an unsigned end value was rejected at compile time, since the step was added as a negative constant of the index type:
Now, for a fixed-width index, the step block adds or subtracts the magnitude of the step, so negative steps work with unsigned types. When the magnitude of the step is greater than 1, it first compares the index against a constant (e.g.
MAX - step). If stepping would go past the bounds of the type, it would also go past the end value, so the index is set to the end value instead, which makes the loop condition fail. The common step ±1 case generates the same IR as before (apart fromi - 1instead ofi + -1). If the step magnitude doesn't even fit in the native int type (e.g.range(i8(...), i8(...), 200), which used to be a compile error too), the loop falls back to an arbitrary precision int index.Added a run test that compares loops over u8, u64, i8, i16, i32 and i64 bounds near the minimum and maximum values with various positive and negative steps against Python's
range(), plus irbuild tests for a step > 1 (i64) and a negative step with an unsigned end (u64).