Experimental: Nix packaging - #1935
Open
franzpoeschel wants to merge 17 commits into
Open
Conversation
Move the overlay definition to nix/overlay.nix and split the pybind11 and python3 overrides into dedicated files under nix/, matching the existing per-package layout (nix/adios2, nix/openpmd_api).
openpmd_api/default.nix declares mpi ? null and filters empty entries out of propagatedBuildInputs, but mpi.dev was dereferenced before the filter, crashing when mpi is null. Only propagate mpi.dev when mpi is present.
Drop the hand-maintained ADIOS2 fork and the pybind11/python3 override machinery in favor of the nixpkgs-provided packages, which already satisfy the version requirements. Clean up the openpmd_api package: - propagate mpi only once instead of both mpi and mpi.dev - use lib.cmakeBool and lowercase pname - set proper meta (homepage, description, changelog, license, maintainers) and build for all of platforms.unix instead of x86 only, fixing builds on aarch64
- derive the package version from the CMake project version instead of hardcoding it - expose the openpmd-ls and openpmd-pipe command line tools as apps - add a development shell with the project toolchain and linters - make 'nix fmt' work by providing a formatter (nixfmt-tree) - drop leftover nixosModules scaffolding from the flake template
Move the apps and devShells definitions into their own files (nix/apps.nix, nix/devShells.nix). Derive the package version from the OPENPMDAPI_VERSION_* macros in include/openPMD/version.hpp, appending the version label (e.g. 'dev') only when it is non-empty instead of always suffixing '-dev'.
Add an openpmd-example-datasets derivation that mirrors share/openPMD/download_samples.sh but fetches the pinned example datasets as fixed-output derivations, and feed it into the check so the data-dependent tests run. Fix checks.x86_64-linux.test, which previously always failed by design (postBuild ran 'ctest ... false' and no sample data was available): - run the suite via doInstallCheck with CTest in serial mode (-j 1), since CMake 4 defaults to parallel execution which exhausts the sandbox process limit (cgroup pids.max) and makes MPI/PmiX thread creation fail intermittently - copy the sample data into the CMake build directory (nixpkgs configures out-of-source into cmakeBuildDir) so the example, tool and Python-unittest tests are registered - skip the MPI-based tests in the sandbox: the C++ parallel append-mode tests read back iterations in an order that depends on MPI write scheduling (hard-coded iterationOrder assertion), and other tests launch mpiexec / import the MPI-enabled Python binding (CLI.pipe.py, Example.py.*_parallel), which is order-, timing- and resource- sensitive on a single node Also ignore the /result symlinks created by 'nix build'.
Silences the 'app lacks attribute meta' warnings from 'nix flake check'.
Add a GitHub Actions workflow that installs Nix with the Determinate Systems installer, caches the Nix store, and runs 'nix flake check'.
Do not let the generic cmake checkPhase drive ctest when building through plain stdenv.mkDerivation (i.e. python = null): buildPythonPackage has no checkPhase and remaps doCheck to installCheckPhase itself, but the stdenv path used to run ctest both in checkPhase and installCheckPhase when doCheck = true. Keep the caller's check intent under a separate binding and route all testing through the (restricted) installCheckPhase.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pros:
nix developnix run <flake-location>#ls(or pipe)Todo: