added golden USCensusPEP_PopulationEstimatebyRace#2048
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds several golden data CSV files and introduces a validation configuration file (validation_config.json) along with its reference in the manifest to enable automated validation checks for the US Census PEP population estimate by race import. Feedback on the changes highlights a critical filename mismatch in the validation configuration that would cause a file-not-found error, as well as recommendations to ensure all validation rules have unique rule_id values for clearer reporting.
…ing persistent Session and retries
…er imports to module level
…d format preprocess.py
…hs in validation_config.json
|
The refactoring of |
|
@gemini-code-assist review |
There was a problem hiding this comment.
Code Review
This pull request introduces validation configurations and golden summary reports for US Census PEP population estimates by race. It updates preprocess.py to use a robust requests.Session with retry logic, fixes a bug in a string length check, sorts input files, and updates pandas operations to avoid deprecation warnings. Feedback is provided to extend the retry adapter configuration to cover both HTTP and HTTPS protocols.
…ocess changes and keep delim_whitespace fix
eaade27 to
2d31fdd
Compare
This PR includes:
To eliminate risk of unexpected behavior, performance side-effects, and standard Pandas warnings (such as SettingWithCopyWarning):
df.iloc[j, 0] = ....
column's underlying data type.
recent and future Pandas versions.
Testing results: https://storage.mtls.cloud.google.com/datcom-import-test/scripts/us_census/pep/population_estimate_by_race/USCensusPEP_PopulationEstimatebyRace/2026_07_17T00_22_16_589130_07_00/input0/validation/validation_output.csv
PR checklist : https://docs.google.com/spreadsheets/d/1BzweR9Sj58j0H2_BweGTmfE4Z1lrjPZL8u1FS0kzCeg/edit?pli=1&gid=0#gid=0