Repository navigation
Conversation
…ce_monitor Bundles the SWIG Python bindings (ndcctools.taskvine, work_queue, chirp, resource_monitor) together with the C binaries and helper scripts that packaging/build-conda installs for these subsystems (vine_worker, vine_factory, work_queue_worker, resource_monitor, chirp client tools, dttools utilities, etc.), exposed as console_scripts that exec the bundled executables. The build compiles a plain dynamic build against the target's system toolchain (no static musl linking) and relies on cibuildwheel's default auditwheel/delocate repair step to vendor the one non-baseline runtime dependency (libz); OpenSSL is already statically linked by configure. Published under btovar's personal PyPI account for now, since the cooperative-computing-lab organization has not been authorized to publish there yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Pin setuptools>=77 in [build-system]: our SPDX-style `license = "GPL-2.0-only"` requires it, but with only setuptools>=68 pinned, cibuildwheel's isolated build env resolved an older setuptools/packaging combo that can't parse it (ImportError: Cannot import packaging.licenses). Masked locally because --no-isolation reuses the ambient environment. - Tolerate missing resource_monitor artifacts in build_py._stage_bin(): upstream's resource_monitor/src/Makefile only builds resource_monitor, piggybacker, rmonitor_poll_example, rmonitor_snapshot, and the two preload libraries on native Linux/x86_64 (CCTOOLS_LINUX_NATIVE_X86_64); they're empty TARGETS/PROGRAMS elsewhere (linux/aarch64, all of macOS). The generated console_script shim now exits with a clear "not available on this platform" message instead of a bare exec failure when invoked on a wheel that doesn't have the binary. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
setuptools>=77's SPDX license-expression validation needs packaging>=24.2 (for packaging.licenses), but without an explicit constraint pip was free to satisfy wheel's looser packaging>=24.0 with 24.1, which lacks that submodule -- reproduced the macOS CI failure locally once isolation was enabled. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The SPDX license = "GPL-2.0-only" string form makes setuptools>=77
require packaging>=24.2 (for packaging.licenses) to validate it.
cibuildwheel's cp39 constraints file pins packaging==24.1 exactly, so
requiring packaging>=24.2 in [build-system] deadlocks pip's resolver
for that target ("packaging>=24.2" vs "packaging==24.1").
Simplest fix given we don't need SPDX-expression validation here: go
back to the classic (deprecated but functional) license = {text = ...}
table, and drop both the setuptools>=77 and packaging>=24.2 pins that
were only needed for the SPDX form.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…s-builds macos-14 (Apple Silicon) runners cross-build the x86_64 wheel variant by using an x86_64-mode python3-config for the SWIG extension modules, while cibuildwheel signals the target arch via the ARCHFLAGS env var -- a mechanism only distutils/setuptools build_ext consumes automatically. Our raw `./configure && make` bypasses that machinery, so libdttools.a and friends were compiled for the host's native arm64 while the SWIG .so (via CCTOOLS_PYTHON3_CCFLAGS/LDFLAGS, derived from that x86_64 venv's python3-config) targeted x86_64, and linking the two failed: "found architecture 'arm64', required architecture 'x86_64'". configure already seeds its ccflags/ldflags from $CFLAGS/$LDFLAGS, so folding ARCHFLAGS into those env vars before invoking configure/make makes the whole build consistently target the arch cibuildwheel wants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Drop --strict (which adds -Werror): cibuildwheel's macOS wheel builds target an older MACOSX_DEPLOYMENT_TARGET than the runner's SDK, which turns availability warnings (utimensat needs macOS 10.13+, used in chirp_fs_local.c) into hard errors. --strict is meant to enforce this project's own lint discipline and is already applied by its native/ conda CI jobs -- not needed for a downstream packaging build, and not worth chasing every possible warning across every CI platform/SDK combination. - Force a clean rebuild when ARCHFLAGS changes between build_py runs: cibuildwheel reuses the same checkout across every (Python version, arch) combo on a macOS runner. Our "skip configure/make if config.mk exists" fast path (for repeated python-version builds of the *same* arch) doesn't know when the target arch changed, so stale objects from a previous arch lingered and got linked alongside freshly compiled ones for the new arch. Track the last-used ARCHFLAGS in a marker file and `make clean` first when it differs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Move wheel/sdist building into testing.yml as build-wheels/build-sdist jobs on the same triggers as the rest of the regression suite (push to master, PR to master/release/*, workflow_dispatch) -- build-only, no publish, so a PR that breaks packaging shows up as a failing check like any other regression, not just on PRs that happen to touch pyproject.toml. - Replace wheels.yml with publish-wheels.yml, triggered only by pushing a release/* *tag* (this repo has both release/* tags and release/* branches -- this intentionally matches only the tag ref). Builds wheels/sdist fresh from that tag and publishes to PyPI via a trusted publisher (OIDC, id-token: write) -- no API token/secret stored in the repo. Requires one-time setup on pypi.org: register a trusted publisher for the ndcctools project with owner=btovar, repository=cctools, workflow filename=publish-wheels.yml, environment=pypi. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
Give an overall description of the changes, along with the context and motivation.
Mention relevant issues and pull requests as needed.
Merge Checklist
The following items must be completed before PRs can be merged.
Check these off to verify you have completed all steps.
make testRun local tests prior to pushing.make formatFormat source code to comply with lint policies. Note that some lint errors can only be resolved manually (e.g., Python)make lintRun lint on source code prior to pushing.