Repository navigation
FYI/Clarification for *_like(arr) functions using arr.device #1013
Description
This may be more of a CuPy question about how comfortable CuPy is with changing behavior and pushing through the change in v15. But it does come from the Array API so just bringing it up here once for awareness.
The standard states that empty_like(arr) should allocate as if empty_like(arr, device=arr.device) was passed. And that is actually what JAX and torch do:
import cupy as cp
arr = cp.arange(1000) # arr.device == Device(0)
with cp.cuda.Device(1)
print(cp.empty_like(arr).device) # Device(1) for cupy, 0 for JAX/torch
CuPy does currently not do this (I half suspect this was missed at the time), CuPy completely relies on context managers right now (to the point that even a NumPy array works there).
CC @leofang
Activity
betatim commentedon Sep 14, 2026
What is the thing to clarify? I think the standard is clear on what
empty_likeshould do, but it doesn't concern itself with context managers/other ways to select the device.For a small snippet of cupy it seems like the context manager expresses the intent of the user that they want device 1 for the code inside the
withblock. A bit likeempty_like(arr, device=Device(1))would express my intention that I want something different from the default. But if you imagine awithblock of more than a few lines it might be less clear what the user's intent was. But that seems a bit like a discussion for cupy and/or a question of opinion about whether using a context manager for device placement is a ergonomic design or notseberg commentedon Sep 14, 2026
The thing to clarify would be if CuPy can't easily make the decision to re-write everything so that:
The computation device is inferred from the inputs for
empty_like, etc. but... I think also for all ufunc/kernel calls probably.And maybe to confirm that inferring computation device from inputs is actually what we want by default (this is obvious for GPU vs. CPU or so, I think).
Sure, you can just say "that is clearly what the specs say and CuPy violates it". But if CuPy can't change this easily, that could become a problem.
Yes, the
with:statement becomes unclear if large, but I am not sure it is quite a simple as it seems:will happily ignore both device (and thus also stream) if
datais on a different device, once that change happens. I.e. the mental model about what device execution happens on is completely changed.I guess in practice that may be possible to pull off because cross device data access isn't all that common, but I am honestly not sure how easy this is for CuPy.
betatim commentedon Sep 14, 2026
I'm not sure if we agree or disagree. I think the spec is clear: "use the device of
arr, unless the user says differently". One way the user can "say differently" is to pass thedevice=argument toemtpy_like. Within the cupy universe/school of thought using the context manager is another way to "say differently". So I think it is fine that the context manager overrides things?Two follow on thoughts: (1) can cupy also support
device=as an argument so that it is easy to write code that works across array libraries and (2) in an array consuming library you should probably not use the context manager?