Harden CLI exit behavior, CI/CD guardrails, bundled dep updates#203
Closed
lelia wants to merge 15 commits into
Closed
Harden CLI exit behavior, CI/CD guardrails, bundled dep updates#203lelia wants to merge 15 commits into
lelia wants to merge 15 commits into
Conversation
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
…sions Bundles the following Dependabot PRs into uv.lock (regenerated): - urllib3 2.6.3 -> 2.7.0 (closes #200) - gitpython 3.1.46 -> 3.1.50 (closes #198) - python-dotenv 1.2.1 -> 1.2.2 (closes #190) - pytest 9.0.2 -> 9.0.3 (closes #188) - uv 0.9.21 -> 0.11.6 (closes #184) - cryptography 46.0.5 -> 46.0.7 (closes #181) - pygments 2.19.2 -> 2.20.0 (closes #177) - requests 2.32.5 -> 2.33.0 (closes #175) All eight target versions were verified through Socket Firewall (sfw) on the full transitive dependency tree (15 packages including transitive deps fetched clean; no malware/typosquat/supply-chain alerts). Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Bundles three Dependabot PRs that target the e2e test fixtures used by the Socket scan/reachability validation suite: - tests/e2e/fixtures/simple-npm: axios 1.15.0 -> 1.15.2 (closes #196) - tests/e2e/fixtures/simple-pypi: requests 2.31.0 -> 2.33.0 (closes #187) - tests/e2e/fixtures/simple-pypi: flask 3.0.0 -> 3.1.3 (closes #186) These fixtures were stale (not intentionally pinned), so it's safe to bring them current. Socket Firewall verified install paths for both: - npm fixture: 228 packages fetched clean - pypi fixture: install path clean Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
The --commit-message flag passes its value directly into the API request URL as a query parameter with no length limit. AI-generated commit messages and the common CI pattern of concatenating $BUILDKITE_BUILD_NUMBER + $BUILDKITE_MESSAGE can easily exceed URL length limits, producing HTTP 413 errors. The 413 originates from an infrastructure-layer URL length limit (nginx/Cloudflare), not application-level validation -- confirmed via inspection of the Socket API route handler, which has no constraint on commit_message (unlike committers, which enforces <= 200 chars and returns a clean 400). 200 chars chosen as a conservative defensive ceiling given URL encoding can 2-3x raw character count. No customer should ever want a 2000-character commit message in their scan metadata. A backend-side validation (returning 400 instead of 413) is filed as a follow-on for the depscan API team. Motivated by customer incidents (Plaid). Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Adds explicit handling and structured logging for API/infrastructure failures
so they are distinguishable from real blocking security findings in CI.
Exit code semantics (NEW for 2.3.0):
0 - Clean scan, no blocking issues (or --disable-blocking)
1 - Blocking security finding(s) detected
2 - Process interrupted (SIGINT) -- already in place
3 - Infrastructure / API error -- NEW
5 - Warning-level alerts only -- preserved
This is a BREAKING change for any pipeline that previously caught exit 1 to
mean "anything went wrong." Such pipelines now need to handle 3 separately
for infrastructure failures, or use --exit-code-on-api-error to remap.
Changes:
- socketsecurity/core/__init__.py
* Import RequestTimeoutExceeded and `requests`
* Wrap fullscans.stream_diff with requests.exceptions.Timeout ->
RequestTimeoutExceeded
* Wrap fullscans.post (create_full_scan) with the same pattern
- socketsecurity/socketcli.py
* Import RequestTimeoutExceeded + APIFailure
* IS_BUILDKITE constant (gates BK-specific markers per spec §3)
* New _emit_infrastructure_error helper emits Buildkite log section
markers (^^^ +++ and ---) when BUILDKITE=true, plus a soft_fail hint;
bare log.error otherwise. Markers go to stdout via print() so they
aren't prefixed with log formatting; markers are literal strings on
other CI platforms so the gate is required.
* Explicit RequestTimeoutExceeded and APIFailure handlers added before
the generic Exception handler, all using config.exit_code_on_api_error
- socketsecurity/config.py
* New CliConfig field: exit_code_on_api_error (default 3)
* New flag: --exit-code-on-api-error <int>
Customers can remap to 0 (swallow), 100 (Buildkite soft_fail), etc.
Note: --disable-blocking now only zeroes out exit 1 (security findings),
not exit 3 (infrastructure). This separation is the whole point of the
new code -- callers who want to also swallow infra errors should use
--exit-code-on-api-error 0.
Motivated by customer incidents (Plaid 413s and timeouts; Anthropic
'other' SocketCategory crash).
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Adds and updates unit tests for the v2.3.0 behavior changes: tests/unit/test_cli_config.py - Commit message under 200 chars passes through unchanged - Commit message over 200 chars truncated to exactly 200 - Quote-strip runs BEFORE truncation (250-char inner -> 200) - --exit-code-on-api-error defaults to 3 - --exit-code-on-api-error accepts custom and zero values tests/unit/test_socketcli.py - APIFailure -> exit 3 by default - APIFailure -> exit 3 EVEN with --disable-blocking (breaking change for 2.3.0: --disable-blocking only affects security findings now) - RequestTimeoutExceeded -> exit 3 by default - --exit-code-on-api-error 100 remaps timeout to exit 100 - --exit-code-on-api-error 0 swallows infrastructure errors - Generic RuntimeError uses exit_code_on_api_error too - _emit_infrastructure_error: * BUILDKITE=true emits "^^^ +++" and "---⚠️ " markers (stdout) plus a soft_fail hint (logger) * Without BUILDKITE, no markers and no soft_fail hint * Traceback only included when include_traceback=True Replaces the old test_cli_honors_disable_blocking_for_api_failures test which encoded the pre-2.3.0 coupling between --disable-blocking and infra errors -- that coupling is gone by design. Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
The repo had no explicit Dependabot config, so Dependabot ran on full
defaults: one PR per package per manifest, across every manifest in
the tree -- including the e2e test fixtures that are intentionally
crafted to exercise Socket's scanner. The cumulative result was the
"PR pileup" this PR is consolidating.
New config:
- uv ecosystem (main app): grouped weekly into ONE minor/patch PR and
one major PR; matches the existing python:uv labeling
- github-actions: grouped weekly into ONE minor/patch PR
- docker: separate weekly PR per Dockerfile change
- 7-day cooldown across all ecosystems to give upstream time to pull
bad releases
- e2e fixtures (tests/e2e/fixtures/{simple-npm,simple-pypi}) are
INTENTIONALLY excluded -- their pins should be chosen for supply-
chain signal, not auto-bumped (this is why we had three fixture
PRs in the cleanup)
Pattern adapted from SocketDev/socket-basics.
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
For every Dependabot-authored PR, inspect what changed and conditionally
run Socket Firewall (sfw) install smoke jobs against the affected
manifests. Because sfw uses the anonymous Socket public-data API it
needs NO secret, so this runs cleanly under the standard `pull_request`
context -- no pull_request_target, no token-leak surface.
Jobs (all conditional on file diff):
- python-sfw-smoke: pyproject.toml / uv.lock -> `sfw uv sync` plus
an import smoke on the modules that depend on
the upgraded packages (cryptography, gitpython,
requests, ...). Catches API-removal breaks
from minor/patch deprecations.
- fixture-npm-sfw-smoke: tests/e2e/fixtures/simple-npm/** -> `sfw npm
install` in a clean cwd.
- fixture-pypi-sfw-smoke: tests/e2e/fixtures/simple-pypi/** -> `sfw pip
install -r requirements.txt` in a clean venv.
- dockerfile-smoke: `docker build --pull` (no push) when the
Dockerfile changes.
- workflow-notice: Flag Dependabot PRs that touch workflow or
dependabot config files for explicit human
review (anti-supply-chain-confusion guardrail).
Pattern adapted from SocketDev/socket-basics dependabot-review.yml.
Action SHAs match the pins already in python-tests.yml and e2e-test.yml
so zizmor stays happy.
Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
python-tests.yml: - `uv lock --locked` -- fails if uv.lock has drifted from pyproject.toml. Prevents the "forgot to commit the lockfile" class of mistake. - Import smoke step that loads every top-level module touching the upgraded packages (cryptography, gitpython, requests, urllib3, ...). Catches API-removal breaks from minor/patch deprecations that the unit suite alone wouldn't surface. - `uvx pip-audit --strict` against the synced env -- light CVE check on the resolved transitive tree. Runs in seconds via uv's caching. e2e-test.yml: - Skip e2e on Dependabot PRs. They don't have access to the Socket API secret so e2e would always fail on them, polluting the PR check UI. Supply-chain risk for dep bumps is covered by dependabot-review.yml's Socket Firewall smoke jobs, which need no secrets. Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
Bumps the project version from 2.2.86 to 2.3.0 (minor) to signal the breaking exit-code change in this PR: Exit 1 == blocking security finding (previously: anything) Exit 3 == infrastructure / API error (NEW) CHANGELOG.md: - Breaking-change callout for the exit code semantics shift - --exit-code-on-api-error documentation - Commit message auto-truncation note - Buildkite log formatting note - Bundled Dependabot bumps roll-up - CI hardening summary (dependabot.yml, dependabot-review.yml, python-tests guards, e2e skip-on-dependabot) README.md: - New "Exit codes" section with the canonical table - Buildkite soft_fail examples (default exit 3 and custom 100) pyproject.toml + socketsecurity/__init__.py + uv.lock: - 2.2.87 (from the in-flight branch) -> 2.3.0 Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
`uvx pip-audit --disable-pip` requires `-r` plus either hashed requirements or `--no-deps`. The previous invocation crashed at start. Now: export the locked deps via `uv export --no-hashes --no-emit-project` into a tmp requirements file (skipping the local editable install of the project itself), then feed that to pip-audit with `--disable-pip --no-deps`. Verified locally -- no known vulnerabilities found across the 85 locked transitive deps. Signed-off-by: lelia <2418071+lelia@users.noreply.github.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
lelia
changed the title
lelia
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
exit 1for security findingsexit 3--exit-code-on-api-errorto map to Buildkite'ssoft_failsfw0--disable-blocking)12SIGINT)3Implementation notes
3can be remapped with--exit-code-on-api-error <N>if your pipeline needs a different value--disable-blockingno longer zeroes out infrastructure errorss, it only affects exit1--exit-code-on-api-error 01as "anything went wrong" will need to handle3separately, or remap with--exit-code-on-api-error <N>Adds
413errors from oversize URL query params--exit-code-on-api-errorflag to remap exit codes; can be helpful for Buildkite'ssoft_failattributedependabot.ymlconfig with batch grouping optionsdependabot-review.ymlto scan Dependabot PRs withsfwTest plan
uv lock --lockeddrift checkuvx pip-auditagainst locked deps — no known CVEsPending verification:
python-tests+e2e-test+dependabot-reviewworkflows on the PR3by default--exit-code-on-api-error 100+ forced timeout → exit100BUILDKITE=trueenv →^^^ +++markers appear in output--disable-blockingon a forced infra error → still exits3(not0)