fix: CodeQL CDS extractor support for private @sap/cds-indexer package installs via .npmrc#347
fix: CodeQL CDS extractor support for private @sap/cds-indexer package installs via .npmrc#347data-douser merged 12 commits intomainfrom
@sap/cds-indexer package installs via .npmrc#347Conversation
Adds a full complement of .github/** files for leveraging AI agents with the ql-mcp server for more efficient development of features and troubleshooting of problems related to SAP frameworks such as CAP / CDS, UI5, and XSJS. Sets up the 'advanced-security/codeql-sap-js' repo for ongoing "agentic maintenance", aka Resolves #259
…nd UI5 prompt outer fences (#336) * Initial plan * Address review feedback: fix yamllint commands, agent structure diagram, and UI5 prompt fences Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> Agent-Logs-Url: https://github.com/advanced-security/codeql-sap-js/sessions/404b2862-c982-4730-894b-d434ad2bb1ed --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…e in copilot-setup-steps workflow Agent-Logs-Url: https://github.com/advanced-security/codeql-sap-js/sessions/baa532ae-dce3-46f4-8f65-9200ae19135c Co-authored-by: mbaluda <5237080+mbaluda@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes CDS extractor dependency installation so private @sap/cds-indexer installs can respect project npm configuration (e.g., custom registries via .npmrc), and extends JS extractor pre-finalization indexing to include common UI5 XML view/fragment files.
Changes:
- Add
.npmrcdiscovery/copy helpers and integrate them into CDS dependency cache installation. - Export the new helpers from the CDS package manager entrypoint.
- Extend
pre-finalize.cmdto index UI5.view.xml/.fragment.xmlfiles (in addition to.cdsindexing).
Reviewed changes
Copilot reviewed 3 out of 5 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| extractors/javascript/tools/pre-finalize.cmd | Adds UI5 XML indexing and extra logging around index-files steps. |
| extractors/cds/tools/src/packageManager/index.ts | Re-exports new .npmrc helper utilities from the package manager module. |
| extractors/cds/tools/src/packageManager/cacheInstaller.ts | Implements .npmrc lookup/copy and wires it into cache-based dependency installation. |
| extractors/cds/tools/dist/cds-extractor.bundle.js | Regenerated bundle reflecting the package manager changes. |
Comments suppressed due to low confidence (3)
extractors/cds/tools/src/packageManager/cacheInstaller.ts:76
findNearestNpmrcwalks all the way up to the filesystem root. Since it’s called with a path undersourceRoot, this can unintentionally pick up an.npmrcoutside the analyzed project (e.g., a parent directory or user home), which may contain credentials and different registry settings. Consider bounding the search tosourceRoot(or another explicit stop directory) and ignore any.npmrcfound outside that boundary.
export function findNearestNpmrc(startDir: string): string | undefined {
let current = resolve(startDir);
// Walk up the directory tree until we find an .npmrc or reach the root
while (true) {
const candidate = join(current, '.npmrc');
if (existsSync(candidate)) {
return candidate;
}
const parent = dirname(current);
if (parent === current) {
// Reached filesystem root without finding .npmrc
return undefined;
}
current = parent;
}
extractors/cds/tools/src/packageManager/cacheInstaller.ts:103
- Copying a project
.npmrcinto the cache directory can persist authentication tokens under${sourceRoot}/.cds-extractor-cache/...beyond the install, increasing the risk of credential exposure (e.g., via workspace artifacts or later inspection). Prefer passing npm config vianpm --userconfig/NPM_CONFIG_USERCONFIG, or ensure the copied.npmrcis removed immediately after the install completes (success or failure).
export function copyNpmrcToCache(cacheDir: string, projectDir: string): void {
const npmrcPath = findNearestNpmrc(projectDir);
if (!npmrcPath) {
return;
}
const dest = join(cacheDir, '.npmrc');
try {
copyFileSync(npmrcPath, dest);
cdsExtractorLog('info', `Copied .npmrc from '${npmrcPath}' to cache directory '${cacheDir}'`);
} catch (err) {
cdsExtractorLog(
'warn',
`Failed to copy .npmrc to cache directory: ${err instanceof Error ? err.message : String(err)}`,
);
}
extractors/cds/tools/src/packageManager/cacheInstaller.ts:268
copyNpmrcToCacheis only called when the cache directory is first created. If the cache directory already exists butnode_modulesis missing/partial (e.g., previous failed run), the subsequentnpm installwill run without the intended.npmrcand can still fail to reach private registries. Invoke the npmrc setup right before runningnpm install(whennodeModulesExistsis false), not only during initial directory creation.
// Copy the project's .npmrc (if any) so npm respects custom registries
const firstProjectDir = Array.from(dependencyGraph.projects.keys())[0];
if (firstProjectDir) {
copyNpmrcToCache(cacheDir, join(sourceRoot, firstProjectDir));
}
}
|
@copilot apply changes based on the comments in this thread |
Agent-Logs-Url: https://github.com/advanced-security/codeql-sap-js/sessions/8906d85d-2830-47c2-b48a-447b735a4538 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…ve accidentally committed .codeql-version Agent-Logs-Url: https://github.com/advanced-security/codeql-sap-js/sessions/8906d85d-2830-47c2-b48a-447b735a4538 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
…eql-version Agent-Logs-Url: https://github.com/advanced-security/codeql-sap-js/sessions/8906d85d-2830-47c2-b48a-447b735a4538 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Done in commits 5fda254 and 7dac699:
|
What This PR Contributes
Addresses a bug in the CDS extractor where
.npmrcconfig files were not being copied or respected for CDS extractor installs of scanned CAP project dependencies, meaning that our "best effort" support for@sap/cds-indexerpackages was not working as the CDS extractor was only trying to download packages from npmjs.org -- regardless of a project's.npmrcconfig.Summary of Changes
This pull request introduces improvements to the CDS package manager's dependency installation process and enhances the JavaScript extractor's pre-finalization script. The main focus is on ensuring that custom npm configuration is respected during dependency installation and on expanding file indexing for better CodeQL analysis.
Outline of Changes
Dependency installation robustness:
findNearestNpmrcandcopyNpmrcToCachefunctions incacheInstaller.tsto locate and copy the nearest.npmrcfile into the cache directory. This ensures thatnpm installin the cache respects custom registry settings, authentication tokens, and SSL configurations.cacheInstallDependenciesfunction to invokecopyNpmrcToCachebefore installing dependencies, improving support for projects with custom npm configurations.index.tsfor use in other modules.findNearestNpmrcandcopyNpmrcToCache(including error handling when the copy fails), and for the.npmrcpropagation path incacheInstallDependencies.Extractor pre-finalization enhancements:
pre-finalize.cmdto add informative logging around the indexing of.cdsfiles and to include new steps for indexing UI5.view.xmland.fragment.xmlfiles, ensuring these are included in the CodeQL database.pre-finalize.cmdto checkERRORLEVELafter eachcodeql database index-filesinvocation and short-circuit with a non-zero exit code if any step fails, so a failure in CDS indexing is no longer masked by subsequent UI5 indexing.Future Works
📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.