Skip to content

Implement pools chaining + example and test macros - #15688

Open
jackal1-66 wants to merge 3 commits into
AliceO2Group:devfrom
jackal1-66:chainingPools
Open

jackal1-66 wants to merge 3 commits into
AliceO2Group:devfrom
jackal1-66:chainingPools

Conversation

@jackal1-66

Copy link
Copy Markdown
Collaborator

Multiple things will be changed by this PR in the usage of the event pools:

  • The pools are now chained so that when all the events are used in a file, another file will be pulled → no more repeated files when randomisation is enabled
  • The development is a first step to use event pools on hyperloop
  • Both an example and the test macro were implemented
    Fixes implemented along the way:
  • RoundRobin not considered when using evtpools → now it is forwarded and the events are reused in case all the files have been opened and completely used
  • The driver seed was being thrown away → with rngseed = 0 (the default) the generator called gRandom->SetSeed(0), discarding the driver seed. Now with 0 the seed is taken directly from o2-sim

@jackal1-66
jackal1-66 requested review from a team and sawenzel as code owners August 18, 2026 10:35
@github-actions

Copy link
Copy Markdown
Contributor

This PR did not have any update in the last 30 days. Is it still needed? Unless further action in will be closed in 5 days.

@github-actions github-actions Bot added the stale label Sep 18, 2026
@jackal1-66

Copy link
Copy Markdown
Collaborator Author

It is needed, but to fix a few parts and solve the conflicts

@github-actions github-actions Bot removed the stale label Sep 22, 2026
@jackal1-66 jackal1-66 changed the title Implement pools chaining via TChain + example and test macro Implement pools chaining + example and test macros Oct 9, 2026
@jackal1-66

Copy link
Copy Markdown
Collaborator Author

The code-review by Sandro has just been implemented. This covers:

  • GeneratorFromO2Kine: reset the original event header when a file is
    closed, so no stale header leaks across a file boundary
  • GeneratorFromO2Kine: close the current file only once the next one is
    known to be usable (a failed open no longer drops a good file)
  • Expected number of events per generator (Generator::setExpectedNEvents,
    falling back to the job-wide count): inside a hybrid each sub-generator
    is told its share, computed from the fractions; the event pool passes
    it on to its internal reader. Used in the "not enough events" warning
    and in the "ran out of events" message (also in GeneratorHepMC)
  • GeneratorHybrid: no lookahead event is scheduled after the last event
    of the job, and with fixed fractions a sub-generator is never asked for
    more than its share (request budget). Avoids a rare fatal "ran out of
    events" for pools holding exactly the requested events
  • GeneratorHybrid::Init: a sub-generator whose Init() fails stops the job
  • Unusable input (no readable file for extkinO2, no pool file for
    evtpool) is fatal at initialisation again; evtpool used to segfault
  • rundpl.sh: set -o pipefail; README: roundRobin=false default and the
    behaviour above documented

Additional improvements were also done while fixing the issues before

  • GeneratorHybrid: a failure of a sub-generator in a worker thread is
    handed to the main thread and raised there only if the event is used
    (with more than one worker the job used to hang in some of the validation tests I ran locally)
  • GeneratorHybrid: negative fractions are now rejected
  • GeneratorHybrid: call ROOT::EnableThreadSafety() before starting the
    workers; o2-sim-dpl-eventgen with several workers and two pools
    crashed. o2-sim already has that. Since this development is useful also on HY,
    having this on the hybrid generator is important. To be discussed if it's actually needed though

More tests with various scenarios were implemented, handling a few exceptional cases:

  • test_EventPoolChaining: install a throwing fatal handler, new cases
    for the header reset, failed open, expected events, unusable input
  • new test_GeneratorHybridEventPool (+ EventPoolTestUtils.h): hybrid
    scenarios for the lookahead, shares, budget, Init failures, HepMC,
    failure forwarding, cocktail and parallel modes

@alibuild

alibuild commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for a36ec38 at 2026-10-10 10:37:

## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -a -v clang-diagnostic-error error-log.txt
++ grep -a ' error:'
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Generators/test/test_GeneratorHybridEventPool.cxx:141:10: error: escaped string literal can be written as a raw string literal [modernize-raw-string-literal]
++ [[ 0 == 0 ]]
++ exit 1
--

Full log here.

@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for ea60b56 at 2026-10-10 12:28:

## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -a -v clang-diagnostic-error error-log.txt
++ grep -a ' error:'
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Generators/test/test_GeneratorHybridEventPool.cxx:141:10: error: escaped string literal can be written as a raw string literal [modernize-raw-string-literal]
++ [[ 0 == 0 ]]
++ exit 1
--

Full log here.

@alibuild

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for 1cadd4a at 2026-10-10 16:24:

## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -a -v clang-diagnostic-error error-log.txt
++ grep -a ' error:'
/sw/SOURCES/O2/slc9_x86-64-slc9_x86-64/0/Generators/test/test_GeneratorHybridEventPool.cxx:141:10: error: escaped string literal can be written as a raw string literal [modernize-raw-string-literal]
++ [[ 0 == 0 ]]
++ exit 1
--

Full log here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants