changed rule to new rule Bls_cpi_Category#2044
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the validation configuration for the US BLS CPI category import by replacing the deleted records count validator with a percentage-based validator. A critical issue was identified where the threshold value was set to 0.1, which translates to an extremely restrictive 0.1% threshold due to the validator's implementation using a 0-100 scale. It is recommended to change this threshold to 10 to represent a 10% limit.
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request updates the test runner script to install uv if it is missing, adds several new CPI-U and CPI-W category mappings, introduces a golden summary report, and updates the validation configuration to use a percentage-based deleted records check and a goldens check. Feedback was provided regarding an incorrect relative path for the golden files in the validation configuration, which would lead to validation failures.
hareesh-ms
left a comment
There was a problem hiding this comment.
Lets revisit the logic to chose golden
This PR includes:
Fix: Added the relevant rule ID and set the threshold to 0.6% (representing 1,230 rows out of 211,256).
1230/211256 x 100 ~ 0.58223% ~ 0.6%Keeping the threshold at this level ensures we capture a meaningful baseline of the data without letting minor anomalies skew the results.
Pvmap fix as per new Data additions
The Problem
The CI build (python_install step in Cloud Build) recently started failing with:
1 pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000
Because dependencies in requirements_all.txt are unpinned, standard pip3 attempts to fetch the latest versions. Recent upstream updates on
PyPI (within the google-cloud-* / protobuf ecosystems) introduced a complex version conflict. Standard pip3's backtracking resolver got
stuck in an infinite loop trying to reconcile these versions, exceeding its 200,000-round limit and crashing.
Fix:
Updated the setup_python function in run_tests.sh to:
environment using pip3 (takes < 1 second).
Why it works: uv is powered by a high-performance Rust resolver (PubGrub) that handles deep backtracking effortlessly. It resolves and installs the entire unpinned dependency graph in under 2 seconds.