Repository navigation
fix(ase): backport bundled calculator and parser corrections - #8319
QuantumMisaka wants to merge 19 commits into
Conversation
ABACUS-PR-BOT
left a comment
There was a problem hiding this comment.
Source inspection identified two compatibility issues and two non-blocking follow-ups below. Reviewed the complete diff and affected callers, including ASE 3.23/3.29 source APIs. Governance and Python syntax checks passed; the full whitespace check flagged preserved fixture formatting only. Builds, tests and the engine were not run; author-reported results remain unverified, and CI diagnostics were unavailable.
| # the parser; otherwise a malformed mapping leaks a bare IndexError. | ||
| reference = _read_stru_reference( | ||
| directory, stru_file, self.atomorder, log_path=log | ||
| ) |
There was a problem hiding this comment.
The new mandatory STRU reread rejects valid noncollinear input such as mag 1.0 -1.0 0.0: _parse_coord_line() recognizes a vector only when the y component starts with a digit, then rejects the remaining negative token. This prevents ordinary SCF result collection before either output reader runs. Please accept signed vector components and cover a negative-y moment through the public SCF handoff.
There was a problem hiding this comment.
Fixed in 770926e. _parse_coord_line() now parses all three Cartesian moment components with float(), including negative values. Added a public SCF handoff regression with mag 0.0 -1.0 0.0; the offline suite passes on ASE 3.23 and 3.29.
There was a problem hiding this comment.
Follow-up: signed Cartesian moments remain supported after the keyword-parser cleanup, including the negative-y public SCF handoff case. Its regression is maintained in the standalone suite and run externally against the bundled implementation; the imported test directory has been removed from this PR (329bdf8).
There was a problem hiding this comment.
Confirmed in the current source: keyword-delimited parsing consumes signed vector components before the SCF STRU reread, including the original negative-y case.
| if vector_moments: | ||
| results.setdefault('natoms', len(atoms)) | ||
| if 'energy_without_entropy' in atoms.info: | ||
| results['energy_without_entropy'] = atoms.info['energy_without_entropy'] |
There was a problem hiding this comment.
After a successful calculation with Fermi or entropy data, inherited calc.export_properties() now raises: ASE 3.23/3.29 reconstructs Properties(self.results), which accepts neither efermi nor energy_without_entropy; vector magmoms also violates its shape schema. This affects export and the experimental atoms.get_properties(["energy"]) path, while ordinary energy getters remain usable. Please preserve the extra metadata while providing a schema-compatible export and regression.
There was a problem hiding this comment.
Fixed in 770926e. Abacus.export_properties() now filters unknown metadata and skips vector magnetic moments that do not match ASE’s versioned scalar schema; efermi, energy_without_entropy and vector magmoms remain in calc.results. Added coverage for both export_properties() and Atoms.get_properties(["energy"]).
There was a problem hiding this comment.
Follow-up: export_properties still projects through ASE Properties, while read_results reuses the already-validated SinglePointDFTCalculator properties without projecting twice. Extra metadata and vector moments remain in calc.results. The public export/get_properties regressions pass; their source now stays in the standalone suite (329bdf8).
There was a problem hiding this comment.
The current export override validates a separate schema-compatible mapping while calc.results retains the extra metadata and vector moments.
| Both ASE `energy` and `free_energy` retain ABACUS's force-consistent | ||
| `E_KohnSham`. When an entropy term is present, `energy_without_entropy` is | ||
| provided separately; a missing term remains unknown. Noncollinear vector | ||
| moments are retained without modifying ASE's global property definitions. |
There was a problem hiding this comment.
Non-blocking: the recommended examples/soc.py and tests/magnetic.py still replace ASE’s global magnetic-property definitions. The new handoff no longer needs that workaround; removing it would align these consumers with the documented behavior and avoid rejecting later collinear results in the same process.
There was a problem hiding this comment.
Addressed in 770926e: removed the global ASE magnetic-property mutation from examples/soc.py and tests/magnetic.py. The calculator now handles vector moments locally.
There was a problem hiding this comment.
The global magnetic-property mutations remain removed from both examples/soc.py and tests/magnetic.py. This consumer correction is retained in the reduced diff; it is not part of the test-directory removal (329bdf8).
There was a problem hiding this comment.
Both global magnetic-schema overrides are removed from the SOC example and magnetic integration test, so this consumer cleanup is addressed.
There was a problem hiding this comment.
Non-blocking: tests/magnetic.py again overrides ASE's global magmom/magmoms schema at 83667f8f, while the SOC example retains the earlier cleanup. Please remove the test override too so the noncollinear case can catch regressions in the calculator's handling of ASE's standard schema.
Static source review only; no tests or CI results verified.
|
|
||
| @pytest.mark.parametrize("scale", [1.234564321, 1.2345652]) | ||
| def test_lts_rounded_cell_nonunit_alat_and_nonorthogonal_vectors(tmp_path, scale): | ||
| """The precise Cartesian header, rather than its rounded Angstrom line, sets alat.""" |
There was a problem hiding this comment.
Non-blocking: this reader still takes alat from the rounded lattice constant (Angstrom) line, so this test establishes tolerance acceptance rather than precise-header selection. Likewise, lines 414–415 describe a second spin channel although their fixture declares nspin=1 and one E_Fermi. Please align both descriptions with the paths the assertions actually exercise.
There was a problem hiding this comment.
Corrected in 770926e: the fixture is an nspin=1 log with one Fermi value; the comment now describes that value directly.
There was a problem hiding this comment.
The imported test file has now been removed from the ABACUS PR. The standalone counterpart describes tolerance acceptance with rounded lattice output and the single Fermi value in its nspin=1 fixture; it does not claim precise-header selection or a second spin channel. The bundled README states the fixed-tolerance limitation (329bdf8).
There was a problem hiding this comment.
The affected test file has been removed from this PR, so neither misleading description remains in the bundled change. The README states the fixed-tolerance limitation; the standalone wording remains unverified here.
|
The compatibility findings are fixed in 770926e and the four inline replies document the changes. The standalone counterpart is tracked separately as QuantumMisaka/abacuslite#16 and remains draft pending the ABACUS-first maintenance decision. |
kirk0830
left a comment
There was a problem hiding this comment.
I have only reviewed a very limited of this PR, there are 7000 net increase line, a detailed review seems not plausible. However, as for as what I have found, there are still several lines, designs need to be re-considered, for a long-term maintainability.
| values = list(atomorder) | ||
| except (TypeError, ValueError) as exc: | ||
| raise RuntimeError( | ||
| f"scf atomorder 无法解析(STRU: {stru_path};期望 {natoms} 个整数索引;" |
There was a problem hiding this comment.
please use English instead of Chinese, thanks
There was a problem hiding this comment.
Converted this diagnostic to English. The SCF errors now identify the relevant STRU/log and the rejected condition; the language cleanup is included in 329bdf8.
| f"fail-closed)" | ||
| ) from exc | ||
|
|
||
| valid_integer = all( |
There was a problem hiding this comment.
for this boolean is used only once, it is usually not recommended to explicitly create such a variable, instead, use the expression at the rhs of the = operator to keep the code clean
There was a problem hiding this comment.
Removed the one-use valid_integer variable and placed the check directly in the permutation condition. The adjacent comment explains the integer, range and uniqueness requirements (329bdf8).
| ) as exc: | ||
| suffix = f";log: {log_path}" if log_path is not None else "" | ||
| raise RuntimeError( | ||
| f"scf STRU 身份信息无效(STRU: {stru_path}{suffix};{exc};fail-closed)" |
There was a problem hiding this comment.
Converted the STRU/coordinate diagnostics here to English and kept the filename and failing condition in the error (329bdf8).
| cell_inverse = _invert_cell(cell, f"STRU {stru_path}") | ||
|
|
||
| if coord_type.startswith('d'): | ||
| positions_species = coordinates @ cell |
There was a problem hiding this comment.
what's the meaning of the variable "positions_species"? you use the "_" as "of"?
There was a problem hiding this comment.
Renamed this to species_positions: Cartesian positions in the species-grouped STRU order, before applying the mapping back to ASE order. The unused return fields and redundant array copies have also been removed (329bdf8).
| 'path': stru_path, | ||
| 'positions': np.asarray(positions_species, dtype=float)[mapping].copy(), | ||
| 'cell': np.asarray(cell, dtype=float).copy(), | ||
| 'cell_inverse': np.asarray(cell_inverse, dtype=float).copy(), |
There was a problem hiding this comment.
what is the necessity to pass the inverse of the cell in this function? cell is also passed, the inverse should be calculated elsewhere
There was a problem hiding this comment.
Removed cell_inverse from the reference dictionary because no caller consumed it. The reference cell is still checked for singularity; the matching routine computes the candidate inverse where the periodic residual needs it (329bdf8).
| scale = float(traj[0]['alat_in_angstrom']) | ||
| if not np.isfinite(scale) or scale <= 0: | ||
| raise ValueError(f"Invalid lattice constant in Angstrom: {scale!r}") | ||
| return direct, scale |
There was a problem hiding this comment.
this is not what a commonly metadata like
There was a problem hiding this comment.
Renamed the helper to _log_coordinate_system. Its two results are the logged fractional/Cartesian convention and the Cartesian scale in Angstrom. It reuses the already-validated raw trajectory during SCF collection (329bdf8).
| raise | ||
| except (AssertionError, KeyError, OSError, TypeError, ValueError, IndexError) as exc: | ||
| raise RuntimeError( | ||
| f"scf running log 原始帧校验失败:{log_path}({exc};fail-closed)" |
There was a problem hiding this comment.
The frame-matching diagnostics are now in English, including the last-frame coordinate and cell differences. The cleanup is included in 329bdf8.
| return _frame_coordinate_metadata(log_path, legacyio=legacyio)[0] | ||
|
|
||
|
|
||
| def _validate_raw_scf_trajectory(log_path, expected, *, legacyio=None): |
There was a problem hiding this comment.
this function has a non-trivial cognitive complexity due to too deep conditional statements...what does this function do?
There was a problem hiding this comment.
This check prevents atom reordering from silently dropping an extra atom in malformed output. I separated the per-frame checks from the trajectory loop and reduced duplicate matching bookkeeping and result conversion. The raw cardinality check remains before indexing; malformed-frame and atom-order regressions still pass (329bdf8).
There was a problem hiding this comment.
Follow-up in 4d24f6324: the raw-frame validation and SCF matching now live in private _scf_frames.py. The calculator selects the reader and passes it explicitly; validation still checks each raw frame before atom reordering can discard an extra atom. f31344ce0 adds coverage in the existing inline suite for eight engine coordinate modes and ensures valid per-atom options do not prevent geometry matching.
| result['mag'] = ('Cartesian', | ||
| list(map(float, fields[idx+1:idx+4]))) | ||
| idx += 4 | ||
| elif idx + 3 < len(fields): |
There was a problem hiding this comment.
actually, this is a quite fragile parsing strategy if rely on the number of lines and this is where the ase-abacus project always failed in the past, which furtherly caused the endless bugfix... Do you have any better idea on it to enhance the robustness of these lines?
There was a problem hiding this comment.
Replaced the positional look-ahead with keyword-delimited option parsing. Counts now validate the declared grammar: three coordinates, mobility/velocity triplets, one or three magnetic components, and paired angle1/angle2 values. Signed/scientific values and valid trailing bare mobility triplets are covered, as are truncated or ambiguous records. This uses small standalone cases and existing fixtures (329bdf8).
| trajectory = read_traj_from_running_log(abacus_lines) \ | ||
| if fileobj.name != 'running_md.log' else \ | ||
| read_traj_from_md_dump(fileobj.parent / 'MD_dump') | ||
| is_md_dump = fileobj.name == 'running_md.log' |
There was a problem hiding this comment.
we do not always use the Java like "is_" as the boolean prefix in this project.
There was a problem hiding this comment.
Replaced is_md_dump with md_calculation in the latest reader. The legacy reader separately names geometry_from_dump, so the variable identifies the selected data source rather than suggesting that it defines periodicity (329bdf8).
| positions=frame['coords'][ind], | ||
| cell=frame['cell'], | ||
| cell=cell, | ||
| pbc=is_md_dump, |
There was a problem hiding this comment.
confusing in semantics: pbc or not depends on whether is MD dumped?
There was a problem hiding this comment.
Agreed: periodicity does not depend on the output file. Both readers now set pbc=True for the ABACUS periodic simulation cell, including SCF output. Existing SCF/MD fixtures verify this independently of whether MD_dump supplies the trajectory (329bdf8).
| ind = ind or list(range(len(frame['elem']))) | ||
| positions = frame['coords'][ind] | ||
| if is_md_log: | ||
| if uses_md_dump: |
There was a problem hiding this comment.
confusing semantics. it is not the md dump directly determines the unit. either use another variable name or add in-line annotations
There was a problem hiding this comment.
Separated md_calculation from geometry_from_dump and documented the format units at the conversion: MD_dump lattice vectors are in alat and its Cartesian positions are in Angstrom; running-log cells are already in Angstrom, while positions follow the logged coordinate convention. The conversion and non-unit-alat regressions pass (329bdf8).
| cell=frame['cell'], | ||
| positions=positions, | ||
| cell=cell, | ||
| pbc=is_md_log, |
There was a problem hiding this comment.
The pbc expression here no longer depends on the calculation/file branch: both readers return pbc=True for the periodic ABACUS cell. Geometry source selection and unit conversion are handled separately (329bdf8).
| @@ -0,0 +1,33 @@ | |||
| INPUT_PARAMETERS | |||
There was a problem hiding this comment.
can't the testfiles in the integration tests folder satisfy the functionalities being tested? adding quite many additional files without maximizing the reuse is strongly discouraged.
There was a problem hiding this comment.
Removed the newly imported tests/unit and tests/fixtures trees and their added CI step. ABACUS keeps its existing inline and engine integration harness. The standalone suite can be run externally against the bundled source; its own duplicate tests/data were consolidated. The resulting PR changes 8 existing files, +760/-178, with no new recorded logs (329bdf8).
There was a problem hiding this comment.
Non-blocking: please reuse the inline suites for numerical Direct/nonunit-alat position expectations in both public readers, plus signed/scientific moments and bare mobility after a complete option. The current reader mocks check magnetic ordering at alat=1, and the input tests cover leading bare flags but not those option combinations. A pinned external check against the bundled source would also cover this gap without restoring the fixture trees.
There was a problem hiding this comment.
Non-blocking: please add compact rejection cases to the existing inline suite: an unmatched SCF frame, a malformed frame after an earlier valid match, and an incomplete option triplet through public read_stru(). The new eight-mode case and existing option round trips cover successful paths; these additions would protect the extracted validation paths.
Static review at 034d4fba; no runtime or CI results verified.
There was a problem hiding this comment.
The geometry-rejection part of my earlier request is obsolete at 83667f8f. The updated standalone tests at QuantumMisaka/abacuslite@8910a348 now exercise empty/multiple SCF results, collection without STRU, and final relax/MD frames. Those sources address the revised contract; I have not verified their execution against this bundled revision.
|
Following the maintainability review, the PR is now limited to 8 existing files (+760/-178) at 329bdf8. The imported offline regression directories, recorded logs and added CI step are removed; ABACUS keeps its existing harness. Standalone tests and documentation were also consolidated in QuantumMisaka/abacuslite#16, and release artifacts contain no tests, logs or development history. The follow-up removes redundant SCF matching state, array copies and property conversion, and leaves downstream abacus.json markers untouched. Keyword parsing, named streams, periodicity, units and the ASE minimum are addressed individually in the review threads. Verification: 130 external regressions against the bundled source on ASE 3.23 and 3.29, 55 existing inline tests, and one native SCF integration passed; the standalone source and rebuilt installed wheel each passed 210 tests. The PR description records skipped cases and the pre-existing energy-reference mismatch. Hosted checks for this new head are still running. Threads are left open for reviewer confirmation. |
ABACUS-PR-BOT
left a comment
There was a problem hiding this comment.
The earlier signed-moment and property-export issues are addressed in source. The consumer schema overrides are removed, and the misleading test descriptions are no longer in the bundled diff. A focused test-reuse suggestion remains non-blocking.
Static incremental review at cc3f2f91ca6934b684adac6e3eb9a1cf688af3f0: target-base governance, Python syntax and contributor whitespace checks passed. No candidate imports, builds, engine tests or CI results were verified; external test reports remain author-reported.
kirk0830
left a comment
There was a problem hiding this comment.
Thanks for your contribution.
The lines of codes changed have been largely eliminated due to the reuse of reference files, that's good.
I have made my first complete scan of the code changes, now I have a better understanding on this PR.
This PR makes the ase interface more robust, but there are still comments regarding the readability and maintainability, as what I have pointed out inline and, another suggestion is, please consider split several helper functions out of the core.py, it may be good to have a new file to place those functions, because now the file is too long. A very long file can be harmful for both human and agentic programmer in the same way: occupying too much the context window.
Regards,
Yike
| def energy_components(ener: Mapping[str, float]) -> Dict[str, float]: | ||
| """Return optional energy metadata from one ABACUS energy table. | ||
|
|
||
| ``E_KohnSham`` already includes ``E_entropy(-TS)`` and remains the | ||
| force-consistent ASE energy. The separate no-entropy value is metadata, | ||
| not an ASE ``free_energy`` replacement. Missing entropy stays unknown. | ||
|
|
||
| Args: | ||
| ener: Frame-local ABACUS energies in eV. | ||
|
|
||
| Returns: | ||
| The no-entropy energy in eV when its entropy term is available. | ||
| """ | ||
| if 'E_entropy(-TS)' not in ener: | ||
| return {} | ||
| return { | ||
| 'energy_without_entropy': float(ener['E_KohnSham']) - float(ener['E_entropy(-TS)']), | ||
| } | ||
|
|
There was a problem hiding this comment.
the function name doesn't match its behavior
There was a problem hiding this comment.
Renamed this helper to _energy_without_entropy_metadata() in 4d24f6324. It returns the single optional metadata field indicated by its name, rather than a breakdown of the energy components. Both readers use it; the ASE energy and free-energy values are unchanged.
|
@kirk0830 Thanks for the review. The energy helper now names its single optional field, and the SCF validation and matching helpers have moved out of |
kirk0830
left a comment
There was a problem hiding this comment.
only one suggestion in this round: rename the _scf_frames.py to something like atomorder_bridge.py, because the semantics of scf collides with the frame of MD, they two both are not expressive enough for the functions in the file.
|
@QuantumMisaka why do you choose the _self_geometry_check.py? what is the significance of the term "scf"? |
|
Thanks to @kirk0830 for raising this. I removed the helper in |
This PR updates result handling in the bundled ASE calculator. It fixes magnetic moments, output units, and ASE property export, and requires exactly one parsed result for single-point SCF.
Related issues and provenance
Related: #8165, #8175. Adapted from
QuantumMisaka/abacuslite@39ab1d5and5d3da5d, with related standalone follow-ups inQuantumMisaka/abacuslite#16and#17. This PR follows ABACUS's bundled interface workflow.What's changed?
abacus.json, and writing leaves existing JSON files untouched.The earlier STRU/log geometry comparison has been removed. Rerunning SCF in the same directory with a displaced structure produced one log record per run on v3.10.1. In a fault-injection check, the previous comparison accepted stale output when the k-point mesh changed at fixed geometry. Output provenance belongs to the execution layer and is outside this PR's scope.
Unit and case tests
The diff changes seven files in
interfaces/ASE_interface. Bundled integration test files match the PR base. Focused new package regressions are maintained in QuantumMisaka/abacuslite#17.From the candidate root, with
PYTHONPATH=interfaces/ASE_interface,OMP_NUM_THREADS=1, and Python 3.12:python -m pytest -q interfaces/ASE_interface/abacuslite/{core.py,io/generalio.py,io/latestio.py,io/legacyio.py,utils/ksampling.py} -k 'not test_calculator_results and not test_restart' python3 tools/03_code_analysis/agent_governance_check.py --base f05ff9f2 --head HEAD --format text git diff --check f05ff9f2...HEADgit archive HEAD interfaces/ASE_interfacewithpython -m pip wheel --no-deps --no-build-isolationimported successfully. The wheel excludes the removed geometry module.With the ABACUS v3.10.1 LTS toolchain loaded:
test_calculator_resultsenergy mismatch reproduced at bothb1ba5e4d2and83667f8f7(-194.9530533315vs-194.953050937eV). The hosted ASE plugin job covers the develop engine.Governance notes
All changes are confined to the bundled Python interface. The ASE README documents the result contract and dependency floor.