Skip to content

GPUTracking: place the remaining cluster-finder constants in the constant address space - #15818

Merged
ktf merged 17 commits into
AliceO2Group:devfrom
ktf:pr15818
Sep 21, 2026
Merged

ktf merged 17 commits into
AliceO2Group:devfrom
ktf:pr15818

Conversation

@ktf

@ktf ktf commented Sep 19, 2026

Copy link
Copy Markdown
Member

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.

@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for 15fab9a at 2026-09-19 20:04:

## sw/BUILD/O2-latest/log
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Common/Constants/include/CommonConstants/LHCConstants.h:19:10: fatal error: GPUCommonDef.h: No such file or directory
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Common/Constants/include/CommonConstants/LHCConstants.h:19:10: fatal error: GPUCommonDef.h: No such file or directory
ninja: build stopped: subcommand failed.

Full log here.

@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for 94f37bc at 2026-09-19 20:06:

## sw/BUILD/O2-latest/log
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Common/Constants/include/CommonConstants/LHCConstants.h:19:10: fatal error: GPUCommonDef.h: No such file or directory
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Common/Constants/include/CommonConstants/LHCConstants.h:19:10: fatal error: GPUCommonDef.h: No such file or directory
ninja: build stopped: subcommand failed.

Full log here.

@ktf

ktf commented Sep 19, 2026

Copy link
Copy Markdown
Member Author

@davidrohr first bunch. I need to disentangle the rest from the migration to GPUdouble*. Do you want me to fix the remaining formatting issue? AFAICT, it's preexisting.

I have also opened FB24852288 with the Apple support. I think I still have some bug report open from 2008, though, so I wouldn't hold my breath. I would call it a victory they added generic address space "only" a year later.

@ktf ktf changed the title GPUTracking: place shared constants in the constant address space GPUTracking: place the remaining cluster-finder constants in the constant address space Sep 19, 2026
@alibuild

alibuild commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for e59b7ff at 2026-09-20 01:26:

## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
grep: error-log.txt: binary file matches
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Common/DCAFitter/GPU/cuda/GPUInterface.cu:55:15: error: use '= default' to define a trivial destructor [modernize-use-equals-default]
++ [[ 0 == 0 ]]
++ exit 1
--

Full log here.

@davidrohr

Copy link
Copy Markdown
Collaborator

@davidrohr first bunch. I need to disentangle the rest from the migration to GPUdouble*. Do you want me to fix the remaining formatting issue? AFAICT, it's preexisting.

If there are genuine formatting issues, sure, please fix them. But I think there is also some bogus clang-format behavior in some cases with the GPU macros.

For double: I'd prefer to use a typedef for double instead of messing with all the code and replacing double with GPUdouble. Generally, I'd like to leave the code standard c++ where possible.

ktf added 12 commits September 20, 2026 09:58
Same treatment the TPC constants already had. MSL requires every variable at
program scope to name an address space, and diagnoses it at the declaration,
so a header full of plain constexpr breaks any device translation unit that
merely includes it -- whether or not the constants are used. GPUglobalconstexpr()
expands to constexpr everywhere except Metal, where it adds constant.

Class-scope static members need it too: MSL counts them as program scope.
constexpr functions do not, and are left alone.

The six vDrift and ExB calibration defaults are double, which does not exist in
MSL at all. They are host-only -- nothing under GPU/ refers to them -- so they
are now compiled out of device code rather than converted, which would have
changed their precision on CUDA and HIP.

Preprocessed output is unchanged for host, CUDA, HIP and cling; device code
sees the same constants minus those six doubles. Together this takes the TRD
headers from 144 errors to 0 in a Metal translation unit.
MSL requires every variable at program scope to name an address space and
diagnoses it at the declaration, so these headers broke any device translation
unit that included them. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, where it adds constant.

Preprocessed declarations are unchanged for host, CUDA, HIP and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
…ress space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
…ress space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
… space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.
MSL requires every variable at program scope, class-scope statics included, to
name an address space and diagnoses it at the declaration, so these headers
broke any device translation unit that included them regardless of whether the
constants were used. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, where it adds constant.

Last of the series: with this the Metal translation unit has no program-scope
diagnostics left, down from 208.
…address space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
ktf added 5 commits September 20, 2026 10:03
…s space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
… in the constant address space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
…tant address space

MSL requires every variable at program scope, class-scope statics included, to
name an address space. GPUglobalconstexpr() expands to constexpr everywhere
except Metal, so this is a no-op for host, CUDA, HIP, OpenCL and cling.

These were missed by the earlier pass: the header they sit in only started
being compiled once the errors ahead of it were gone.
@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for f6ea55c at 2026-09-20 10:36:

No log files found

Full log here.

@ktf

ktf commented Sep 20, 2026

Copy link
Copy Markdown
Member Author

Ok, I think I fixed all the reported clang format issues.

That said, regarding double, the code actually needs three different types, all of which are aliased to doubles in backends which support double as fast as float (or thereabouts).

For hardware lacking double support (i.e. M5) or with extremely slow double support (some gaming card, as I understand it) the code I have separates the different semantics of the double in three:

  • GPUdoubleStore: a double you only store and transfer (keeps 8 bytes so host and device layouts match);
  • GPUdoubleValue: a double you compute with where the hardware has one (plain float on Metal);
  • GPUdoubleCalc: a double you compute with because the arithmetic cancels and float would lose it (the compensated two-float on Metal).

Of the three, only GPUdoubleValue could be dropped if we aliased float to double on Metal. The first one is needed to get the proper layout when converting host / device and the third one allows for an implementation which is numerically comparable to the double one. That said, I would still keep all three, just to avoid confusion and catch mismatches at compile time.

@davidrohr

Copy link
Copy Markdown
Collaborator

Ok, I think I fixed all the reported clang format issues.

That said, regarding double, the code actually needs three different types, all of which are aliased to doubles in backends which support double as fast as float (or thereabouts).

For hardware lacking double support (i.e. M5) or with extremely slow double support (some gaming card, as I understand it) the code I have separates the different semantics of the double in three:

* `GPUdoubleStore`: a double you only store and transfer (keeps 8 bytes so host and device layouts match);

* `GPUdoubleValue`: a double you compute with where the hardware has one (plain float on Metal);

* `GPUdoubleCalc`: a double you compute with because the arithmetic cancels and float would lose it (the compensated two-float on Metal).

Of the three, only GPUdoubleValue could be dropped if we aliased float to double on Metal. The first one is needed to get the proper layout when converting host / device and the third one allows for an implementation which is numerically comparable to the double one. That said, I would still keep all three, just to avoid confusion and catch mismatches at compile time.

To be honest, I don't like this. With 3 types, it makes the code even more complicated, and it will be spread all over O2. And I don't see a benefit at all. All our GPUs can do double precision. It is true that some gaming GPUs have very reduced performance (1/8th to 1/64th of FP32). But then, we have << 1 per mille double precision operations, so it does not play any role performance-wise. It can play a role if we use double precision instructions accidentally as was done in the ITS code, but that was a bug on our side and we better spot and fix it. The only thing is: we need double-precision for the deterministic mode, since there we need full precision for the trigonometric / exp / log / etc. functions. This will obviously not work on MacOS then.

IMHO, I would implement a class that stores a real double, to be compatible, and then for computation converts it to 2 floats on the fly. Then we can just use a typedef, and we don't need to mess with the code.
I would only complicate the code if there is a real benefit. If at some point we want to start to use MacOS for GPU computing heavily and we need to have more double precision in the code, we can revisit this.

And for the constexpr: I think we should file a bug report to apple, to automatically place all global constexpr values in the constant address space. There is absolutely no reason not to do so. CUDA, HIP, and OpenCL can do this. Even if they do not react soon, we should still do it, since they need to get feedback what aspects of their language are annoying.,

@ktf

ktf commented Sep 21, 2026

Copy link
Copy Markdown
Member Author

For what concerns the global constexpr, I did file a bug report with Apple already (FB24852288 is the private ID). I doubt they will address it in less than two years, though. So if you do not mind, I would go ahead and merge this. I promise I will do the cleanup myself when they fix it, even if retired at that point. ;-)

For what concerns the floating point. I understood the prominence of double was much higher. The other PR I have also has some full IEEE-754 emulation path (much slower, something like 30x), I need to check if that one can be made just a drop in replacement for double.

I suggest we merge this PR (global constexpr fixes only) and we continue on the rebase of the big one.

@davidrohr

Copy link
Copy Markdown
Collaborator

Sure, let's merge this one. I think there is no way around.
If possible, can we comment on the apple PR to try to increase priority?

And for double, let's try to avoid invasive changes.

@ktf
ktf merged commit 1b917c4 into AliceO2Group:dev Sep 21, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants