Repository navigation
Distribute DeePMD inference through MDCell for UnitCell workflows - #8330
Merged
Merged
Conversation
added 10 commits
November 18, 2026 20:42
ABACUS-PR-BOT
left a comment
Collaborator
There was a problem hiding this comment.
Source inspection identifies a conditional MPI-backend compatibility risk in the new UnitCell inference route. The inline comments also include three non-blocking follow-ups. This was a static review; no builds or runtime tests were run.
ABACUS-PR-BOT
left a comment
Collaborator
There was a problem hiding this comment.
Static follow-up of 7fd9105b243f; no builds or runtime tests were run. The checked component_count is now reused for allocation and reduction.
ABACUS-PR-BOT
left a comment
Collaborator
There was a problem hiding this comment.
Incremental static review of b75ca502f260; builds and tests were not run.
mohanchen
approved these changes
Nov 20, 2026
mohanchen
approved these changes
Nov 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reminder
AGENTS.mdanddocs/developers_guide/agent_governance.md.Linked Issue
Refs #8310. The original 8192-atom run on two A800 80 GB GPUs with DeePMD-kit 3.1.1 remains to be verified before claiming the issue is fully fixed.
What's changed?
source_cellMDCell adapter that initializes a domain-decomposed MDCell from STRU or UnitCell, synchronizes coordinates and lattice geometry, returns ghost forces to owners, and gathers a full force matrix for UnitCell optimizers. The driver passes concrete INPUT values to the adapter; the adapter handles restart replication, skin conversion, world-domain construction, atom-count logging, and the UnitCell backing association. It does not receiveParameteror use globalPARAM.Run_MD::prepare_mdcelland its MDCell friend declaration were removed.relaxandcell-relaxkeep their existing UnitCell-facing optimizer interface. Energy and virial use the MDCell cross-rank reductions.dim_aparam > 0by mapping either one parameter vector or globalnat * dim_aparamvalues to owned atoms and, when the model requires it, ghosts. Keepfparamframe-scoped.InputNlist; reject multi-rank inference when the latter is unavailable. Handle ranks with no local atoms.docs/advanced/md.md.Unit Tests and/or Case Tests for my changes
cmake --build /tmp/abacus-dp-mdcell-build-dpa --target abacus_pw_para MODULE_CELL_mdcell_adapter MODULE_MD_dp_aparam -j8— passed with local DeePMD 3.2.1.dev26 PyTorch build.ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^(MODULE_CELL_mdcell_adapter(_np[24])?|MODULE_MD_dp_aparam|MODULE_MD_run)$'— 5/5 passed outside the restricted sandbox.mdanddomain_decomposition— passed. DeePMD 2.2.11 C-API fallbackmdcompile — passed.relaxandcell-relaxeach completed two steps. A sparse two-atom case gave identical energy, forces, and pressure at one and four ranks, including completely empty ranks.dim_aparam=1DeePMD model: per-atom values0.1 0.2and uniform value0.1each gave matching one- and two-rank results; changing the parameters changed the energy.python3 tools/03_code_analysis/agent_governance_check.py --base origin/develop --head HEAD --format text— no blockers, only four header-include review warnings.git diff --check origin/develop..HEAD— passed. Code-quality score: 8/8 changed C++ files passed.Checks not completed or passed:
The original 8192-atom/two-A800/DeePMD 3.1.1 configuration is unavailable locally. This workstation has one 11 GB RTX 2080 Ti. At 1024 atoms, two ranks sharing that GPU exhausted its memory; this does not establish the memory limit when ranks use separate A800s.
Existing
MODULE_MD_DP_potentialusesAl-SCAN.pband failed because this local DeePMD installation lackslibdeepmd_backend_tf.so. The older 2.2.11 installation also lacks required TensorFlow link dependencies. Neither result was counted as a pass.The reporter model and the locally generated atomic-parameter model are not committed to the repository. A committed manual rank-comparison script and fixture now reproduce the reporter-model check; no model-backed Autotest case is registered because the model is an issue attachment.
Follow-up initialization refactor:
source /home/fy19/abacus-develop/toolchain/install/setup && cmake --build /tmp/abacus-dp-mdcell-build-dpa --target abacus_pw_para MODULE_MD_run MODULE_CELL_mdcell_adapter -j4— passed.OMP_NUM_THREADS=1 ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R 'MODULE_MD_run|MODULE_CELL_mdcell_adapter'— 4/4 passed (adapter at 1, 2, and 4 ranks, plus Run_MD).After moving setup decisions into the adapter, the executable and
MODULE_CELL_mdcell_adapterrebuilt successfully. The adapter regression test checks normal replication (2 atoms), restart without repeated replication (1 atom), and retention of the atom-count log;OMP_NUM_THREADS=1 ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R 'MODULE_MD_run|MODULE_CELL_mdcell_adapter'passed 4/4, including 1/2/4-rank adapter runs.Temporary runtime cases with this build: two-rank LJ Anderson MD with
cell_replica 2 1 1andmd_neighbor_skin 1.0(64 atoms), PW NVT restart (2 atoms), and two-rank LJ NVE restart withcell_replica 2 1 1(checkpoint remained 32 atoms) — all exited successfully; the atom-count log remained present. These smoke cases were run from/tmp, not committed.python3 tools/03_code_analysis/agent_governance_check.py --stagedandgit diff --cached --check— passed with no blockers. The final adapter revision has no blockers: the checker reported a reduced global-reference count (1 added, 3 removed), a review warning for<iosfwd>required by thestd::ofstream&declaration, and a no-docs-needed warning. This follow-up changes no INPUT keyword, log format, or public user behavior.Governance Notes
dp_aparamnow works in the MDCell inference path with its existing documented input shapes;docs/advanced/md.mddescribes the parallel behavior.source_cellowns the adapter and empty-domain neighbor handling;source_mdDP uses it. The external ESolver and UnitCell optimizer interfaces are unchanged. New source files are listed in CMake andsource/Makefile.Objects.std::string,std::vector, andstd::int64_tin their public declarations; concrete cell types are forward-declared.Latest follow-up verification
a04ba4094unified MDCell lattice updates throughset_lattice_geometry(latvec, lat0); NHC and MSST pass the existinglat0. The MPI decomposition, adapter, NHC, MSST, and LJ tests passed (6 tests total) after the affected targets built.e442366b6inlined the output/restart cadence checks without changing their predicates.source /home/fy19/abacus-develop/toolchain/install/setup && cmake --build /tmp/abacus-dp-mdcell-build-dpa --target abacus_pw_para MODULE_MD_func -j 8passed;ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^MODULE_MD_func$'passed. A seven-step LJ MD smoke run without_freq_ion=3andmd_restartfreq=5producedSTRU1, STRU6forout_stru=0, andSTRU1, STRU4, STRU6, STRU7forout_stru=1; both wrote restart state at step 6. The earlier predicate-onlyMODULE_MD_runtest target was removed with those predicates. An initial CTest command also selected three targets not built in the DPA build directory, so they were Not Run; this was a build-selection issue, not a test failure.python3 tools/03_code_analysis/agent_governance_check.py --staged --format textpassed with only a documentation-sync review warning.git diff --cached --checkpassed. These follow-up changes do not alter INPUT behavior or user-facing documentation.e0c6d61a1shortened only comments inrun_md.cpp(11 comment lines added, 60 removed); executable statements are unchanged.git diff --cached --checkand the staged governance check passed without blockers. No new runtime test or documentation is needed for this comment-only follow-up; the earlier MD runtime checks remain the behavior evidence.50d03d908removes redundantinttostd::size_tconversions only from standard-container subscripts in the DP inference and MDCell adapter paths. Bounds checks, size comparisons, and non-index conversions remain.source /home/fy19/abacus-develop/toolchain/install/setup && cmake --build /tmp/abacus-dp-mdcell-build-dpa --target abacus_pw_para MODULE_CELL_mdcell_adapter MODULE_MD_dp_aparam -j 8passed;ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^(MODULE_CELL_mdcell_adapter(_np[24])?|MODULE_MD_dp_aparam)$'passed 4/4. The staged governance checker had no blockers, andgit diff --cached --checkpassed. This is an indexing-only cleanup with no INPUT or numerical behavior change, so no new docs or tests are needed.Conflict resolution and review follow-up (2026-11-19)
develop(d75733074), resolving the forcefield source rename and migrated unit-test paths. GitHub reports the PR mergeable against currentdevelop.d11038e21verifies thatlibdeepmd_op_pt.sodirectly depends onlibmpi.sobefore enabling distributed DeePMD inference in both CMake and Make builds. A non-MPI operator configuration now warns and excludes the communication-capable compile definition; runtime rejects multi-rank inference. The MD guide requires the operator and ABACUS to use the same MPI runtime. The resolved MPI library was checked locally withldd.out_stru=0/1; exact file inventory cannot be expressed by the numerical Autotest reference checks. Added a manual model-backed rank-comparison script and fixture, with reproduction instructions intests/04_FF/22_DP_rank_consistency/README.md. It checks energy, all forces and stress after coordinate and lattice changes, including empty rank partitions. The model binary remains the issue DPA3 relax runs out of GPU memory with two GPUs allocated #8310 attachment.component_countfor adapter force allocation and reduction.Fresh verification after review changes:
cmake --build /tmp/abacus-dp-mdcell-build-dpa --target abacus_pw_para MODULE_CELL_mdcell_adapter MODULE_MD_dp_aparam -j 8passed with local DeePMD 3.2 development build.ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^(04_FF_md_output_cadence|MODULE_CELL_mdcell_adapter(_np[24])?|MODULE_MD_dp_aparam)$'passed 5/5.python3 tests/integrate/validation_tools/check_dp_rank_consistency.py --abacus /tmp/abacus-dp-mdcell-build-dpa/abacus_pw_para --mpi /home/fy19/abacus-develop/toolchain/install/openmpi-5.0.10/bin/mpirun --model /tmp/issue8310_repro/frozen_model_op.pth --fixture tests/04_FF/22_DP_rank_consistency/STRUpassed for original, displaced and expanded 2-atom cells at 1 and 4 ranks. The displaced cell produced a nonzero force (max 0.7566 eV/Angstrom). Comparison tolerances match the integration-suite values: 1e-5 eV, 1e-4 eV/Angstrom and 1e-3 kbar. The model run required execution outside the restricted sandbox because OpenMPI could not open its interfaces inside it.libmpi.sodependency emitted the expected distributed-inference-disabled warning, and the resulting flags omitted__DPMD_COMM_NLIST_ARGS; the real MPI-enabled operator retained__DPMD_COMM_NLIST_ARGS=13.python3 tools/03_code_analysis/agent_governance_check.py --stagedreported no findings;git diff --cached --checkpassed. The full PR diff checker reports only review warnings for migration-neutral global use and header includes, with no blockers.CI build fix (2026-11-19)
test_domain_decomposition.cpp: two tests still called removedMDCell::set_lattice_vectors. Commit7fd9105b2updates those calls toset_lattice_geometry(lattice, lat0)while preserving the original lattice constant. This is a test-only call-site correction; no user-facing behavior or documentation changes are needed.cmake --build /tmp/abacus-dp-mdcell-build-dpa --target MODULE_CELL_NEIGHBOR_mdcell_migrate_mpi -j 8passed.ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^MODULE_CELL_NEIGHBOR_(mdcell_migrate_mpi|domain_decomposition_np[14])$'passed 3/3 at 2, 1, and 4 MPI ranks.python3 tools/03_code_analysis/agent_governance_check.py --stagedhad only the expected documentation-sync review warning for a test-only change;git diff --cached --checkpassed.Review follow-up and documentation check (2026-11-19)
b75ca502frestoresmdcell_adapter.cppandmdcell_reader.cpptoMODULE_MD_dp_forcefield's serial test sources; preserves ordinary distributed TensorFlow.pbinference when the PyTorch MPI operator is unavailable; checks the LJ restart's actual step index; and extends the model comparison to two-steprelaxandcell-relaxon persistent DP/MDCell instances. The MD guide now distinguishes PyTorch and TensorFlow requirements.cmake --build /tmp/abacus-dp-mdcell-build-dpa --target MODULE_MD_dp_forcefield abacus_pw_para -j 8passed with DeePMD enabled.ctest --test-dir /tmp/abacus-dp-mdcell-build-dpa --output-on-failure -R '^04_FF_md_output_cadence$'passed 1/1.cmake --build /tmp/abacus-dp-mdcell-build-no-dp --target MODULE_MD_dp_forcefield -j 8passed without DeePMD; its CTest passed 1/1. The same target compiled and linked against a synthetic DeePMD installation without an MPI-enabled PyTorch operator, exercising the compile-time guard branch.python3 tests/integrate/validation_tools/check_dp_rank_consistency.py --abacus /tmp/abacus-dp-mdcell-build-dpa/abacus_pw_para --mpi /home/fy19/abacus-develop/toolchain/install/openmpi-5.0.10/bin/mpirun --model /tmp/issue8310_repro/frozen_model_op.pth --fixture tests/04_FF/22_DP_rank_consistency/STRUpassed all three SCF geometries plus two consecutive evaluations each forrelaxandcell-relaxat one and four ranks. A TensorFlow.pbruntime check was unavailable because the local DeePMD package has no TensorFlow backend library.python3 tools/03_code_analysis/agent_governance_check.py --stagedandgit diff --cached --checkpassed with no findings.The
docs/readthedocs.org:abacus-rtdcheck on7fd9105b2failed at the 900-second platform limit before any project documentation build command ran. Its public build record stopped afterapt-get updateand beforeapt-get installbegan; the same check had passed ond11038e21. This is recorded as a documentation service timeout, not a Sphinx diagnostic. The new commit retriggers it.On
b75ca502f, both Read the Docs contexts (abacus-developandabacus-rtd) completed successfully, confirming the previous 900-secondabacus-rtdfailure was transient. The four review threads addressed by this follow-up were replied to and resolved.