Skip to content

CI: validate OpenBLAS and select a compatible kernel for ASE tests - #8348

Open
Stardust0831 wants to merge 2 commits into
deepmodeling:developfrom
Stardust0831:ci/ase-openblas-core-selection
Open

Stardust0831 wants to merge 2 commits into
deepmodeling:developfrom
Stardust0831:ci/ase-openblas-core-selection

Conversation

@Stardust0831

Copy link
Copy Markdown
Collaborator

The GNU CI image's OpenBLAS 0.3.20 Cooperlake kernel returns incorrect DSYMM results. In the Gamma-only Si calculator and restart tests, this corrupts the density and aborts the fourth SCF iteration with a charge integral of -17.6846 and Can't find even an electron!.

Check the OpenBLAS library linked to ABACUS before running the ASE tests. Preserve a working kernel selection; when the regression probe fails, select and verify an alternative supported by the CPU.

Linked Issue

Related: #7794.

What's changed?

  • Add a standard-library Python helper in interfaces/ASE_interface/tests/ that resolves ABACUS's OpenBLAS dependency and compares 30 small DSYMM cases against scalar matrix products.
  • Keep the current kernel when it passes. Otherwise, try SkylakeX when the required AVX-512 features are available, then Haswell when AVX2/FMA are available. Use features common to all listed CPUs.
  • Probe each candidate in a fresh process with one thread and a timeout. Require both the requested kernel identity and a finite maximum absolute error at most 1e-12 before exporting OPENBLAS_CORETYPE through GITHUB_ENV.
  • Stop with diagnostics when no compatible kernel passes. Other BLAS providers receive no override; other architectures retain a working default. A corrected Cooperlake implementation also retains its current selection.
  • Keep the helper and its 15 policy tests alongside the ASE test utilities. Run them from that directory before the existing ASE suites.

This changes the ASE CI environment. It does not modify the numerical engine, solver, scientific assertions, or reference data.

Unit Tests and/or Case Tests for my changes

Validation used upstream eba09c937c73, built in the failing GNU image (sha256:b195875e7f3262d26eb4e07960afc0e7a193861699cd281eb08f65562880b17d), on an AMD Ryzen 9 9950X3D. Runtime tests used OMP_NUM_THREADS=1, OPENBLAS_NUM_THREADS=1, and MKL_NUM_THREADS=1.

Failure isolation: selecting Cooperlake reproduced both original Si failures in three repetitions, including the original CI's first three energies and fourth-step density diagnostics. Replacing only DSYMM with its scalar definition made both tests pass while other BLAS operations retained Cooperlake. The standalone DSYMM maximum error was 2.786 for Cooperlake, versus at most 4.44e-16 for the passing kernels.

Selector results:

Initial kernel CPU features supplied to the selector Outcome Original two Si tests
Default Barcelona Actual host flags Preserved Both pass
Cooperlake Actual host flags SkylakeX Both pass
Cooperlake Masked to AVX/AVX2/FMA Haswell Both pass
Haswell Actual host flags Preserved Probe passed; suites not repeated for this scenario

Commands run inside the image, with OPENBLAS_CORETYPE=Cooperlake and GITHUB_ENV=/evidence/selector-relocation/cooperlake.env set for the workflow step:

cd interfaces/ASE_interface/tests
python3 -m unittest discover -s . -p test_select_openblas_core.py -v
# 15 passed.
python3 select_openblas_core.py --abacus abacus --github-env "$GITHUB_ENV"
# Rejected Cooperlake and emitted OPENBLAS_CORETYPE=SkylakeX.

The selector also preserved the healthy default without writing an override. The workflow working directory and subprocess invocation were exercised in the original image; the helper and policy-test contents match those used for the complete numerical validation.

With exactly the emitted environment applied to the following processes (commands from the repository root):

(cd interfaces/ASE_interface/abacuslite && bash xtest.sh)
# 56 passed, 2 existing skips.
(cd interfaces/ASE_interface/tests && bash xtest.sh)
# 7 passed.

Staged governance and whitespace checks passed:

python3 tools/03_code_analysis/agent_governance_check.py --staged
git diff --cached --check

Validation limits: the AVX2 feature mask was exercised on the same physical AVX-512-capable host, not on separate AVX2-only hardware. Other architectures have policy-test coverage only. The probe covers this DSYMM regression, not every BLAS operation or shape. The original hosted logs do not record the selected OpenBLAS kernel, so its Cooperlake selection is inferred from the host and exact reproduced numerical signature. GitHub-hosted validation of the selector remains pending; a particular newer OpenBLAS release has not been established as the dependency fix.

Governance Notes

  • INPUT/docs changes: no INPUT or scientific behavior changes; no documentation update required for this CI helper.
  • Core module impact: no source/ changes. The Python helpers are invoked directly by the workflow and need no CMake linkage.
  • Exceptions requested: none.

Reminder

  • I have read AGENTS.md and docs/developers_guide/agent_governance.md.
  • I have linked an issue or explained why this PR does not need one.
  • I have added adequate unit tests and/or case tests, or explained why not.
  • I have listed the exact verification commands run and their results.
  • I have described user-visible behavior changes, including INPUT parameter changes.
  • I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
  • I have requested any needed governance exception above (none needed).

@Growl1234

Growl1234 commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

That adds quite a few complexity. Would it be possible to simply choose the generic kernel for GitHub CI? Btw I'm surprised that such fault can be related to system OpenBLAS...

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants