Skip to content

Propagate gain correction to L.A.Cosmic uncertainties - #958

Merged
mwcraig merged 3 commits into
astropy:mainfrom
nomad3:fix-729-scale-uncertainty
Jul 30, 2026
Merged

Propagate gain correction to L.A.Cosmic uncertainties#958
mwcraig merged 3 commits into
astropy:mainfrom
nomad3:fix-729-scale-uncertainty

Conversation

@nomad3

@nomad3 nomad3 commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Summary

  • propagate L.A.Cosmic gain correction to CCDData uncertainties using the existing uncertainty arithmetic
  • preserve the reported ADU data and uncertainty units when gain_apply=False
  • cover standard deviation, variance, and inverse-variance representations without changing cosmic-ray detection or replacement-pixel semantics

This deliberately does not claim support for every equivalent scaled uncertainty unit; that is an existing Array API wrapper boundary outside #729.

Fixes #729

Validation

  • python -m pytest ccdproc/tests/test_cosmicray.py -q — 35 passed
  • python -m pytest ccdproc -q — 352 passed, 29 skipped
  • JAX_ENABLE_X64=1 CCDPROC_ARRAY_LIBRARY=jax python -m pytest ccdproc -q — 345 passed, 29 skipped, 7 expected failures
  • CCDPROC_ARRAY_LIBRARY=dask python -m pytest ccdproc -q — 347 passed, 34 skipped
  • Black, Ruff, and git diff --check

The non-default array-api-strict L.A.Cosmic tests remain expected-failed because Astro-SCRAPPY requires NumPy.

Checklist

For documentation changes:

  • For documentation changes: Does your commit message include a [skip ci]?

For bugfixes:

  • Did you add an entry to the CHANGES.rst file?
  • Did you add a regression test?
  • Does the commit message include a Fixes #issue_number?
  • Does this PR add, rename, move or remove any existing functions or parameters?

AI assistance disclosure

OpenAI Codex was used for source inspection, test design, implementation, and automated review. The contributor has completed the final review and takes responsibility for the contribution's correctness and maintenance.

@nomad3
nomad3 force-pushed the fix-729-scale-uncertainty branch from 63ee3e8 to 98106bb Compare July 27, 2026 04:19
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.71%. Comparing base (3ea1af9) to head (d33d7cf).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #958      +/-   ##
==========================================
+ Coverage   96.70%   96.71%   +0.01%     
==========================================
  Files           8        8              
  Lines        1576     1581       +5     
==========================================
+ Hits         1524     1529       +5     
  Misses         52       52              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nomad3
nomad3 marked this pull request as ready for review July 27, 2026 13:00
@mwcraig
mwcraig requested a review from Copilot July 29, 2026 14:48

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking on #729 — routing the gain through the wrapper arithmetic so it propagates into the uncertainty is the right approach, and the gain_apply=False unit fix addresses real mislabeling. Two confirmed issues to resolve before this is ready, plus a couple of test notes.

1. Variance/InverseVariance uncertainty arrays silently escape to numpy on jax. Only _StdDevUncertaintyWrapper has an xp-aware _propagate_multiply; the Variance and InverseVariance wrappers fall through to astropy's numpy propagation path. Verified under jax: after cosmicray_lacosmic with gain applied, result.uncertainty.array is a numpy array for Variance and InverseVariance, while StdDev correctly stays jax. The wrapper gap pre-exists this PR, but this adds the first public call site that exercises it while the PR claims coverage of all three uncertainty types — and the tests can't see it because assert_allclose coerces to numpy before comparing. Please either fix the escape (xp-aware _propagate_multiply for the other two wrappers) or explicitly track it: add a namespace assertion to the test and mark the jax case xfail (or record it as a known escape) so the gap is visible rather than silently passing.

2. Dead code — the unit-reassignment block is a no-op. This line (and the surrounding 4-line block):

nccd.uncertainty.unit = nccd.uncertainty._data_unit_to_uncertainty_unit(nccd.unit)

has no effect: _unwrap_uncertainty rebuilds the uncertainty from .array alone, dropping the unit, and astropy then re-derives it from the parent CCDData. I deleted the whole block and got byte-identical results across every case tested. It also reaches into a private astropy API (_data_unit_to_uncertainty_unit) for nothing. Please delete it, along with the redundant second is not None guard.

Smaller test points:

  • The tests derive their expected units via the same private _data_unit_to_uncertainty_unit the code calls — that makes the assertion circular. Please state the expected units literally (e.g. electron for StdDev, electron**2 for Variance).
  • The backend_xfail reason says "uses astroscrappy", but astroscrappy is monkeypatched out in these tests — please update the reason to the actual limitation.

The numerics, units, and non-mutation coverage are otherwise good — with the escape either fixed or explicitly tracked and the dead block removed, this will be in good shape.

This review comment was written by Claude (via Claude Code) following Matt's review of the PR; Matt has reviewed and approved its content before posting.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Ready to approve

The changes directly address the reported unit/uncertainty inconsistency with targeted tests and appear consistent with existing array-API and uncertainty-handling patterns.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR fixes cosmicray_lacosmic so that when gain correction is applied to the output data, the associated CCDData.uncertainty is scaled consistently using the existing uncertainty-propagation arithmetic, and when gain_apply=False the original ADU units are preserved for both data and uncertainties (addressing #729).

Changes:

  • Update cosmicray_lacosmic to propagate gain scaling into CCDData uncertainty objects via the existing arithmetic propagation paths.
  • Ensure output units are only changed when gain_apply=True (preserving ADU units when disabled).
  • Add regression tests covering StdDev, Variance, and InverseVariance uncertainty representations.
File summaries
File Description
CHANGES.rst Adds a bug-fix entry documenting the uncertainty + unit behavior change.
ccdproc/tests/test_cosmicray.py Adds regression coverage for gain scaling across multiple uncertainty types and gain_apply modes.
ccdproc/core.py Implements gain-aware uncertainty propagation and conditional unit updates in cosmicray_lacosmic.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Low

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

nomad3 added 3 commits July 29, 2026 21:05
Fixes astropy#729

Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
Signed-off-by: Simon Aguilera <saguilera1608@gmail.com>
@nomad3
nomad3 force-pushed the fix-729-scale-uncertainty branch from 98106bb to d33d7cf Compare July 30, 2026 01:20

@mwcraig mwcraig left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nomad3!

@mwcraig
mwcraig merged commit 736555d into astropy:main Jul 30, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

lacosmic corrects for gain in data array but not uncertainties

3 participants