Add CIRCUITPY_HEAP_SRAM_SIZE: opt-in internal-RAM heap start segment - #11176
Add CIRCUITPY_HEAP_SRAM_SIZE: opt-in internal-RAM heap start segment#11176lynt-smitka wants to merge 1 commit into
Conversation
On boards whose python heap defaults to external PSRAM (adafruit#10240), all python allocations pay external-memory latency. Measured on an Adafruit Fruit Jam (RP2350 + 8 MB PSRAM): span writes reach ~8.7 MB/s vs ~64+ MB/s in internal SRAM; clearing/filling a 38 KB RGB565 buffer takes 4.2 ms from PSRAM vs 0.23 ms from SRAM (18x), and a bytecode-heavy program's frame time improved ~10% overall with its early allocations in SRAM - the interpreter working set (module bytecode, young objects) pays the PSRAM tax too. This answers the open review question on adafruit#10240 about how much the PSRAM heap costs. Since the heap is already segmented (start segment + auto-grown splits), a minimal opt-in helps a lot: the new settings.toml key allocates just the START segment from the internal (dma-capable) pool, while growth segments keep coming from PSRAM - allocations made early in a program run at internal-RAM speed and everything else spills over transparently. Unset (default) keeps today's behavior. An oversized request simply fails the internal allocation and falls back to the PSRAM path. The documented tradeoff: the internal pool is shared with display/audio DMA buffers, so a large segment can prevent later large DMA allocations on that board - an explicit per-device choice.
tannewt
left a comment
tannewt
left a comment
There was a problem hiding this comment.
I'm not sure about this. I don't really want another settings.toml knob, especially one like this where a very specific value would be needed for a specific case.
Instead, I'd prefer a general speed up based on analysis of what memory reads are actually being done that are too slow on PSRAM. Maybe the heap metadata needs to be in internal ram? Do you have a more detailed analysis of what memory accesses are slow?
|
I benchmarked this A/B on this branch. What real matters is data size relative to the 16 KB XIP cache: work on large blocks gives the best results (a magnitude better) - whether streamed whole (fills, copies, decoding, ulab) or accessed all over (FFT, the GC's object walk). The ordinary Python code gains around 15% (dict 14%,
Heap metadata in internal RAM would speed up gc.collect() but not the others (and by less than the 3.6x in the table, since that was measured with the whole segment in SRAM). The numbers apply to buffers that actually end up in the segment SRAM of course. What a program allocates early lands there, what doesn't fit goes to PSRAM as today (measured; an oversized value simply boots with stock behavior). I like tuning knobs 😊 But I get the point. What about letting the existing |
|
What if we just started doing all port_malloc allocations internally first and fallback to PSRAM when no internal space is available and it doesn't need to be dma_capable? It risks not being able to allocate DMAable buffers but maybe they are small enough to fit. |
Does that include heap chunk additions, or just non-heap allocations? I'm a little confused. |
|
@dhalbert From the measurements' side: everything in the table above is heap data, so heap chunk additions would need to be included for the benefit to show. @tannewt My view is biased toward the game engine, and your proposal would help Fruit Jam performance a lot. But I can't guess the impact on the rest of CircuitPython. That's why I shaped the PR as opt-in, so everyone can tune it to their own needs. Framebuffers are allocated at boot, so they'd be fully served either way. I'm not sure about audiocore and the other DMA-demanding parts. Maybe keep a per-port reserve of internal RAM for those, ~64 KB? |
3 participants
Since #10240 the VM heap on PSRAM boards lives entirely in external PSRAM. In that PR's review @dhalbert noted: "it will reduce performance on boards with PSRAM -- I wonder by how much." I measured it while working on a game engine, and the cost is significant enough to deserve an opt-out for performance-sensitive workloads:
The last row is the interesting one: it's not just big buffers - the interpreter's working set (module bytecode, young objects) pays the PSRAM latency too.
What this does
Adds one opt-in settings.toml key:
CIRCUITPY_HEAP_SRAM_SIZE = <bytes>allocates only the heap's start segment from the internal (dma-capable) pool. Growth segments still come from PSRAM viaMP_PLAT_ALLOC_HEAP, so allocations made early in a program run at internal-RAM speed while total capacity is unchanged (everything else spills over transparently).CIRCUITPY_HEAP_START_SIZE(which sizes the initial PSRAM segment): when the new key is satisfied it fully defines the start segment andCIRCUITPY_HEAP_START_SIZEis not consulted; if the new key is unset or its internal-RAM allocation fails,CIRCUITPY_HEAP_START_SIZEapplies as usual.Tradeoff (documented in
docs/environment.rst)The internal pool is shared with DMA buffers (displays, audio). Allocations made at boot coexist fine; in the rare case of re-allocating a large DMA buffer at runtime (e.g. switching a framebuffer to a higher resolution), the request may no longer fit and raises a regular catchable MemoryError.