Fix CI coverage aggregation by normalizing LCOV source paths#1614
Merged
Conversation
The Merge-LcovFile function was treating the same source file as different entries when LCOV files came from different platforms (Linux, macOS, Windows) because each platform uses different absolute paths: - Linux: /home/runner/work/DSC/DSC/dsc_lib/src/foo.rs - macOS: /Users/runner/work/DSC/DSC/dsc_lib/src/foo.rs - Windows: D:\a\DSC\DSC\dsc_lib\src\foo.rs This caused the same source lines to be counted 15+ times (3 platforms x 5 test groups) in the total, with Pester coverage runs contributing many DA:line,0 entries that dragged the overall percentage down from ~80% to ~48%. Fix: Add ConvertTo-NormalizedLcovSourcePath that strips platform-specific workspace prefixes and normalizes path separators before using them as merge keys. This ensures coverage data from all platforms and test groups is correctly combined for each source file. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses inaccurate CI code coverage aggregation by normalizing LCOV SF: (source file) paths so coverage from different platforms merges into a single entry per source file, matching local .\build.ps1 -codecoverage results more closely.
Changes:
- Added
ConvertTo-NormalizedLcovSourcePathto normalize LCOVSF:paths (workspace prefix stripping + separator normalization). - Updated
Merge-LcovFileto merge coverage keyed by normalized source paths. - Added Pester tests validating path normalization and cross-platform LCOV merge behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| helpers.build.psm1 | Adds LCOV source-path normalization and uses it during LCOV merge to fix cross-platform aggregation. |
| tests/helpers.build.coverage.tests.ps1 | Adds Pester coverage tests for normalization and merge aggregation correctness. |
…nchor test assertions - Remove generic ADO prefix patterns that could produce inconsistent keys for non-CI absolute paths across platforms. - Expand the marker-based fallback to also handle drive-letter absolute paths (not just paths starting with /). - Anchor test regex assertions with (?m)^ and $ to prevent false positives (e.g., DA:1,8 matching DA:1,80). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Code Coverage ReportNo Rust files were changed in this PR. 🔵 Full Codebase Coverage81% (good)
|
The Export-PesterCodeCoverageReport function searches for binaries using Get-ChildItem without -Recurse. Since CI builds without -Release, binaries are placed in bin/debug/ (not bin/ directly). The workflow was passing 'Join-Path $PWD bin' which only searched the top-level bin/ directory, causing 'No executable binaries found' on all platforms. Fix: Use Get-ArtifactDirectoryPath (which returns bin/debug for non-Release builds) to resolve the correct binary directory path. This ensures all 12 Pester coverage LCOV files are generated and included in the merge, adding the integration test coverage that accounts for ~30% of the total. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
On Unix, PowerShell's .Mode property shows Windows-style attributes (e.g., '-----') rather than Unix permissions. The code checked $_.Mode -match 'x' which always evaluates to false on Linux/macOS, causing 'No executable binaries found' in all Unix Pester coverage jobs. Fix: On Unix, identify Rust binaries by the absence of a file extension (Rust produces extensionless binaries on Unix). Files with extensions are only included if they have execute permission via .UnixMode. This approach is robust against artifact upload/download stripping execute permissions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merging coverage from all 3 platforms inflates the total line count because each platform has platform-specific source files (Windows has ~4500 extra lines from registry, service, DISM resources; macOS has some unique files). These platform-specific lines are only partially covered by their respective platform tests, dragging down the reported percentage. Local build.ps1 -codecoverage runs on a single platform and reports ~81%. The CI report should do the same. Use only Linux coverage data (Rust tests + Pester integration tests) which matches the local workflow and accurately reflects codebase coverage on the primary platform. Also reduces the coverage-report job's dependencies to just linux-build and linux-pester, allowing it to complete sooner without waiting for all platforms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ationale - Add \r? before $ anchors in test regex assertions to handle CRLF line endings when tests run on Windows. - Add workflow comment explaining why coverage uses Linux-only artifacts: cross-platform merging inflates total lines due to platform-specific source files, producing misleadingly low percentages vs local results. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The CI coverage report shows ~42% code coverage, while running
.\build.ps1 -codecoveragelocally reports >80%. Three issues were identified:Pester coverage never generated:
Export-PesterCodeCoverageReportsearchedbin/for executables but they live inbin/debug/(non-Release build). All 12 Pester jobs emitted "No executable binaries found".Unix binary detection broken: On Linux/macOS, the code checked
$_.Mode -match 'x'but PowerShell's.Modeon Unix shows Windows-style attributes (-----), not Unix permissions. The execute bit is in.UnixMode.Cross-platform merging inflated totals: Merging LCOV from all 3 platforms inflates
totalLinesbecause Windows has ~4,500 extra platform-specific source lines (registry, service, DISM resources) that are only partially covered, dragging down the percentage.Fix
Workflow: Use
Get-ArtifactDirectoryPathto resolve the correctbin/debugpath for Pester coverage generation.Binary detection: On Unix, identify Rust binaries by absence of file extension (robust against artifact upload stripping execute permissions).
Single-platform coverage: Use only Linux coverage data (Rust tests + Pester integration tests) which matches the local
build.ps1 -codecoveragesingle-platform workflow. Path normalization is still applied to strip workspace prefixes.Path normalization: Added
ConvertTo-NormalizedLcovSourcePathto strip CI workspace prefixes and normalize separators, ensuring correct LCOV merging regardless of source.Verification
Testing
Added Pester tests in
tests/helpers.build.coverage.tests.ps1covering: