Skip to content

changed rule to new rule Bls_cpi_Category #2044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
balit-raibot merged 10 commits into from
Jul 10, 2026
Merged

changed rule to new rule Bls_cpi_Category #2044

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
46c1caf
changed rule to new rule
niveditasing May 28, 2026
ab5f2af
increased threhsold
niveditasing May 29, 2026
5a89768
Merge branch 'master' into rule_change_validation_config
niveditasing May 29, 2026
96acb45
Merge branch 'master' into rule_change_validation_config
balit-raibot Jul 8, 2026
d34de1c
adding goldens
balit-raibot Jul 8, 2026
cca2708
increased deletion threshold
balit-raibot Jul 9, 2026
7389ba5
increased deletion threshold
balit-raibot Jul 9, 2026
cfd863e
added missing pvmap keys
balit-raibot Jul 9, 2026
cf2ee91
adding goldens
balit-raibot Jul 9, 2026
9b7e8bb
install uv inside virtualenv to resolve pip resolution errors
niveditasing Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions run_tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ function setup_python {
if [[ "$PYTHON_REQUIREMENTS_INSTALLED" = false ]]
then
echo "Installing Python requirements"
if command -v uv &> /dev/null; then
echo "Using uv for package installation"
uv pip install -r requirements_all.txt -q
else
pip3 install -r requirements_all.txt -q
if ! command -v uv &> /dev/null; then
echo "Installing uv"
pip3 install uv -q
fi
echo "Using uv for package installation"
uv pip install -r requirements_all.txt -q
PYTHON_REQUIREMENTS_INSTALLED=true
fi
}
Expand Down
16 changes: 16 additions & 0 deletions statvar_imports/us_bls/cpi_category/cpi_u_pvmap.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,7 @@ Club dues and fees for participant sports and group exercises(5),consumerGoodsCa
Coffee,consumerGoodsCategory,Coffee,,,,,,,,,,
College textbooks(4)(14)(6),consumerGoodsCategory,CollegeTextbooks,unit,IndexPointBasePeriodDecember2001Equals100,,,,,,,,
College textbooks(4)(15)(6),consumerGoodsCategory,CollegeTextbooks,unit,IndexPointBasePeriodDecember2001Equals100,,,,,,,,
College textbooks(4)(6)(15),consumerGoodsCategory,CollegeTextbooks,unit,IndexPointBasePeriodDecember2001Equals100,,,,,,,,
College textbooks(4)(6)(14),consumerGoodsCategory,CollegeTextbooks,unit,IndexPointBasePeriodDecember2001Equals100,,,,,,,,
College textbooks(5)(15)(7),consumerGoodsCategory,CollegeTextbooks,unit,IndexPointBasePeriodDecember2001Equals100,,,,,,,,
College tuition and fees,consumerGoodsCategory,CollegeTuitionAndFees,,,,,,,,,,
Expand Down Expand Up @@ -319,6 +320,7 @@ Eggs(4),consumerGoodsCategory,Eggs,,,,,,,,,,
Electricity,consumerGoodsCategory,Electricity,,,,,,,,,,
Electricity(11),consumerGoodsCategory,Electricity,,,,,,,,,,
Elementary and high school tuition and fees,consumerGoodsCategory,ElementaryAndHighSchoolTuitionAndFees,,,,,,,,,,
Elementary and high school tuition and fees(14),consumerGoodsCategory,ElementaryAndHighSchoolTuitionAndFees,,,,,,,,,,
Energy,consumerGoodsCategory,Energy,,,,,,,,,,
Energy commodities,consumerGoodsCategory,EnergyCommodities,,,,,,,,,,
Energy services,consumerGoodsCategory,EnergyServices,,,,,,,,,,
Expand Down Expand Up @@ -435,10 +437,12 @@ Health insurance(4)(8),consumerGoodsCategory,HealthInsurance,unit,IndexPointBase
Health insurance(5)(9),consumerGoodsCategory,HealthInsurance,unit,IndexPointBasePeriodDecember2005Equals100,,,,,,,,
Hospital and related services,consumerGoodsCategory,HospitalAndRelatedServices,,,,,,,,,,
Hospital and related services(4),consumerGoodsCategory,HospitalAndRelatedServices,,,,,,,,,,
Hospital and related services(4)(14),consumerGoodsCategory,HospitalAndRelatedServices,,,,,,,,,,
Hospital services(11)(17),consumerGoodsCategory,HospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Hospital services(16),consumerGoodsCategory,HospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Hospital services(17),consumerGoodsCategory,HospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Hospital services(4)(17),consumerGoodsCategory,HospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Hospital services(4)(18),consumerGoodsCategory,HospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Household cleaning products(4)(5),consumerGoodsCategory,HouseholdCleaningProducts,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Household cleaning products(5),consumerGoodsCategory,HouseholdCleaningProducts,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Household cleaning products(5)(6),consumerGoodsCategory,HouseholdCleaningProducts,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand Down Expand Up @@ -471,12 +475,14 @@ Information technology commodities(11),consumerGoodsCategory,InformationTechnolo
Information technology commodities(12),consumerGoodsCategory,InformationTechnologyCommodities,unit,IndexPointBasePeriodDecember2009Equals100,,,,,,,,
"Information technology, hardware and services(17)",consumerGoodsCategory,InformationTechnologyHardwareAndServices,unit,IndexPointBasePeriodDecember1988Equals100,,,,,,,,
"Information technology, hardware and services(18)",consumerGoodsCategory,InformationTechnologyHardwareAndServices,unit,IndexPointBasePeriodDecember1988Equals100,,,,,,,,
"Information technology, hardware and services(14)",consumerGoodsCategory,InformationTechnologyHardwareAndServices,unit,IndexPointBasePeriodDecember1988Equals100,,,,,,,,
Inpatient hospital services(11)(17)(6),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(16)(6),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(17)(6),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(17)(7),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(4)(17)(6),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(4)(6)(17),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Inpatient hospital services(4)(6)(18),consumerGoodsCategory,InpatientHospitalServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Instant and freeze dried coffee(4)(6),consumerGoodsCategory,InstantAndFreezeDriedCoffee,,,,,,,,,,
Instant coffee(4)(6),consumerGoodsCategory,InstantCoffee,,,,,,,,,,
Instant coffee(5)(7),consumerGoodsCategory,InstantCoffee,,,,,,,,,,
Expand Down Expand Up @@ -627,6 +633,7 @@ Nursing homes and adult day services(4)(17),consumerGoodsCategory,NursingHomesAn
Nursing homes and adult day services(11)(17),consumerGoodsCategory,NursingHomesAndAdultDayServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Nursing homes and adult day services(16),consumerGoodsCategory,NursingHomesAndAdultDayServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Nursing homes and adult day services(17),consumerGoodsCategory,NursingHomesAndAdultDayServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Nursing homes and adult day services(4)(18),consumerGoodsCategory,NursingHomesAndAdultDayServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
"Olives, pickles, relishes(4)(5)(6)",consumerGoodsCategory,OlivesPicklesRelishes,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Olives, pickles, relishes(5)(6)",consumerGoodsCategory,OlivesPicklesRelishes,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Olives, pickles, relishes(5)(6)(7)",consumerGoodsCategory,OlivesPicklesRelishes,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand Down Expand Up @@ -700,6 +707,7 @@ Other uncooked poultry including turkey(6),consumerGoodsCategory,OtherUncookedPo
Other video equipment(4)(5),consumerGoodsCategory,OtherVideoEquipment,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Other video equipment(5),consumerGoodsCategory,OtherVideoEquipment,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Other video equipment(6),consumerGoodsCategory,OtherVideoEquipment,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Other video equipment(14),consumerGoodsCategory,OtherVideoEquipment,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Outdoor equipment and supplies(4)(5),consumerGoodsCategory,OutdoorEquipmentAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Outdoor equipment and supplies(5),consumerGoodsCategory,OutdoorEquipmentAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Outdoor equipment and supplies(6),consumerGoodsCategory,OutdoorEquipmentAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand All @@ -711,9 +719,11 @@ Outpatient hospital services(9)(6),consumerGoodsCategory,OutpatientHospitalServi
Owners' equivalent rent of primary residence(11)(16),consumerGoodsCategory,OwnersEquivalentRentOfPrimaryResidence,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of primary residence(15),consumerGoodsCategory,OwnersEquivalentRentOfPrimaryResidence,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of primary residence(16),consumerGoodsCategory,OwnersEquivalentRentOfPrimaryResidence,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of primary residence(17),consumerGoodsCategory,OwnersEquivalentRentOfPrimaryResidence,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of residences(11)(16),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of residences(15),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of residences(16),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of residences(17),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Parking and other fees(4)(5),consumerGoodsCategory,ParkingAndOtherFees,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Parking and other fees(5),consumerGoodsCategory,ParkingAndOtherFees,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Parking and other fees(5)(6),consumerGoodsCategory,ParkingAndOtherFees,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand Down Expand Up @@ -810,6 +820,7 @@ Rent of primary residence,consumerGoodsCategory,RentOfPrimaryResidence,,,,,,,,,,
Rent of primary residence(11),consumerGoodsCategory,RentOfPrimaryResidence,,,,,,,,,,
Rent of shelter(15),consumerGoodsCategory,RentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Rent of shelter(16),consumerGoodsCategory,RentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Rent of shelter(17),consumerGoodsCategory,RentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Rental of video discs and other media(4)(5)(6),consumerGoodsCategory,VideoDiscAndOtherMediaRental,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Rental of video discs and other media(5)(6)(7),consumerGoodsCategory,VideoDiscAndOtherMediaRental,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Rental of video or audio discs and other media(4)(5)(6),consumerGoodsCategory,VideoAndAudioDiscAndOtherMediaRental,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand Down Expand Up @@ -839,6 +850,7 @@ Services less energy services,consumerGoodsCategory,ServicesLessEnergyServices,,
Services less medical care services,consumerGoodsCategory,ServicesLessMedicalCareServices,,,,,,,,,,
Services less rent of shelter(15),consumerGoodsCategory,ServicesLessRentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Services less rent of shelter(16),consumerGoodsCategory,ServicesLessRentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Services less rent of shelter(17),consumerGoodsCategory,ServicesLessRentOfShelter,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
"Sewing machines, fabric and supplies(4)(5)",consumerGoodsCategory,SewingMachinesFabricAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Sewing machines, fabric and supplies(5)",consumerGoodsCategory,SewingMachinesFabricAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Sewing machines, fabric and supplies(5)(6)",consumerGoodsCategory,SewingMachinesFabricAndSupplies,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Expand All @@ -849,6 +861,7 @@ Shelter,consumerGoodsCategory,Shelter,,,,,,,,,,
Ship fare(4)(5)(6),consumerGoodsCategory,ShipFare,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Ship fare(5)(6)(7),consumerGoodsCategory,ShipFare,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Smartphones(4)(6)(15),consumerGoodsCategory,Smartphones,unit,IndexPointBasePeriodDecember2019Equals100,,,,,,,,
Smartphones(4)(6)(16),consumerGoodsCategory,Smartphones,unit,IndexPointBasePeriodDecember2019Equals100,,,,,,,,
Snacks,consumerGoodsCategory,Snacks,,,,,,,,,,
Snacks(4),consumerGoodsCategory,Snacks,,,,,,,,,,
Snacks(5),consumerGoodsCategory,Snacks,,,,,,,,,,
Expand Down Expand Up @@ -881,15 +894,18 @@ Technical and business school tuition and fees(6),consumerGoodsCategory,Technica
"Telephone hardware, calculators, and other consumer information items(4)(5)",consumerGoodsCategory,TelephoneHardwareCalculatorsAndOtherConsumerInformationItems,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Telephone hardware, calculators, and other consumer information items(5)",consumerGoodsCategory,TelephoneHardwareCalculatorsAndOtherConsumerInformationItems,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Telephone hardware, calculators, and other consumer information items(5)(6)",consumerGoodsCategory,TelephoneHardwareCalculatorsAndOtherConsumerInformationItems,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
"Telephone hardware, calculators, and other consumer information items(14)",consumerGoodsCategory,TelephoneHardwareCalculatorsAndOtherConsumerInformationItems,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Telephone services(4)(5),consumerGoodsCategory,TelephoneServices,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Telephone services(5)(6),consumerGoodsCategory,TelephoneServices,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Televisions,consumerGoodsCategory,Televisions,,,,,,,,,,
Televisions(14),consumerGoodsCategory,Televisions,,,,,,,,,,
Tenants' and household insurance(4)(5),consumerGoodsCategory,TenantsAndHouseholdInsurance,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Tenants' and household insurance(5)(6),consumerGoodsCategory,TenantsAndHouseholdInsurance,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Tires(4),consumerGoodsCategory,Tires,,,,,,,,,,
Tires(5),consumerGoodsCategory,Tires,,,,,,,,,,
Tobacco and smoking products,consumerGoodsCategory,TobaccoAndSmokingProducts,,,,,,,,,,
Tobacco and smoking products(4),consumerGoodsCategory,TobaccoAndSmokingProducts,,,,,,,,,,
Tobacco and smoking products(4)(14),consumerGoodsCategory,TobaccoAndSmokingProducts,,,,,,,,,,
Tobacco products other than cigarettes(4)(5),consumerGoodsCategory,TobaccoProductsOtherThanCigarettes,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Tobacco products other than cigarettes(5)(6),consumerGoodsCategory,TobaccoProductsOtherThanCigarettes,unit,IndexPointBasePeriodDecember1997Equals100,,,,,,,,
Tomatoes,consumerGoodsCategory,Tomatoes,,,,,,,,,,
Expand Down
2 changes: 2 additions & 0 deletions statvar_imports/us_bls/cpi_category/cpi_w_pvmap.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,5 +196,7 @@ Electricity,consumerGoodsCategory,Electricity,,,,,,,,,,
Utility (piped) gas service,consumerGoodsCategory,UtilityPipedGasService,,,,,,,,,,
Rent of primary residence,consumerGoodsCategory,RentOfPrimaryResidence,,,,,,,,,,
Owners' equivalent rent of residences(1),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Owners' equivalent rent of residences(2),consumerGoodsCategory,OwnersEquivalentRentOfResidences,unit,IndexPointBasePeriodDecember1982Equals100,,,,,,,,
Physicians' services,consumerGoodsCategory,PhysiciansServices,,,,,,,,,,
Hospital services(2),consumerGoodsCategory,HospitalAndRelatedServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
Hospital services(3),consumerGoodsCategory,HospitalAndRelatedServices,unit,IndexPointBasePeriodDecember1996Equals100,,,,,,,,
2 changes: 2 additions & 0 deletions statvar_imports/us_bls/cpi_category/golden_data/golden_summary_report.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
"observationPeriods","MinDate","ScalingFactors","StatVar","Units","NumPlaces","MeasurementMethods"
"[P1M]","2025-01","[]","dc/x6l3mg60hpe7c","[IndexPointBasePeriod1982_1984Equals100]","1","[BLSUnchained]"
13 changes: 10 additions & 3 deletions statvar_imports/us_bls/cpi_category/validation_config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,17 @@
"schema_version": "1.0",
"rules": [
{
"rule_id": "check_deleted_records_count",
"validator": "DELETED_RECORDS_COUNT",
"rule_id": "check_deleted_records_percent",
"validator": "DELETED_RECORDS_PERCENT",
Comment thread
balit-raibot marked this conversation as resolved.
"params": {
"threshold": 1230
"threshold": 0.6
}
},
{
"rule_id": "check_goldens_summary_report",
"validator": "GOLDENS_CHECK",
"params": {
"golden_files": "../../../../golden_data/golden_summary_report.csv"
Comment thread
niveditasing marked this conversation as resolved.
}
}
]
Expand Down
Loading