Skip to content

adding goldens for BLS imports#2090

Open
balit-raibot wants to merge 16 commits into
datacommonsorg:masterfrom
balit-raibot:adding_goldens_bls_imports
Open

adding goldens for BLS imports#2090
balit-raibot wants to merge 16 commits into
datacommonsorg:masterfrom
balit-raibot:adding_goldens_bls_imports

Conversation

@balit-raibot

@balit-raibot balit-raibot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

This PR adds golden data for BLS imports: BLS_CPI, BLS_CES_State, BLS_JOLTS, BLS_CPI_Category and US_CPI.

NOTE: BLS_CPI and BLS_JOLTS don't have observationAbout column in the observations output file.
The imports have only one observation about which is being used by .tmcf file.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds golden summary reports for the c_cpi_u and cpi_u datasets. The reviewer pointed out that the golden summary report for cpi_w is missing and should be added to ensure complete test coverage for all datasets defined in the manifest.

Comment thread scripts/us_bls/cpi/golden_data/golden_summary_report_cpi_u.csv Outdated
@niveditasing
niveditasing self-requested a review July 10, 2026 06:07
@balit-raibot balit-raibot changed the title adding goldens for BLS_CPI adding goldens for BLS imports Jul 10, 2026
@balit-raibot

Copy link
Copy Markdown
Contributor Author

@gemini-code-assist review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces validation configurations and golden summary reports for the US BLS CPI and CES State data imports, updating their manifests to reference the new validation files. The reviewer identified that the relative paths to the golden summary reports in both validation_config.json files are incorrect, pointing to non-existent directories by going up four levels instead of three. Correcting these paths is necessary to ensure the validation runner can locate the golden files.

Comment thread scripts/us_bls/cpi/validation_config.json
Comment thread scripts/us_bls/cpi/validation_config.json
Comment thread statvar_imports/us_bls/bls_ces_state/validation_config.json
@balit-raibot

Copy link
Copy Markdown
Contributor Author

@gemini-code-assist review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds validation configurations and golden summary reports for several US Bureau of Labor Statistics (BLS) datasets, including CPI, JOLTS, and CES State. Specifically, it introduces validation_config.json files to define validation rules (such as deleted records percentage thresholds and golden data checks) and updates the respective manifest.json files to reference them. It also adds the corresponding golden summary report CSV files. As there are no review comments, I have no feedback to provide.

@pulkeet-wq

Copy link
Copy Markdown
Contributor

LGTM

{
"schema_version": "1.0",
"rules": [
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Goldens check for places are missing from this file. Please add the same

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @pulkeet-wq this import doesn't have observationAbout column in the observations output file.
The import has only one observation about which is being used by .tmcf file.

"schema_version": "1.0",
"rules": [
{
"rule_id": "check_deleted_records_percent",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Golden checks for places are missing from this file. please add the same

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @pulkeet-wq this import doesn't have observationAbout column in the observations output file.
The import has only one observation about which is being used by .tmcf file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants