From 43dcad54f1ddf60f016a34025739faa301389e2f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 20 Nov 2025 18:04:04 +0000 Subject: [PATCH 01/19] Initial plan From fc47c6a4a3b0e114cd32bfe8e187734d737335a4 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 20 Nov 2025 18:18:23 +0000 Subject: [PATCH 02/19] Implement agentic maintenance infrastructure with agents, instructions, prompts, and issue templates Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/ISSUE_TEMPLATE/cap-modeling-task.yml | 98 +++++++ .github/ISSUE_TEMPLATE/cds-extractor-task.yml | 82 ++++++ .github/ISSUE_TEMPLATE/dependency-upgrade.yml | 106 ++++++++ .github/ISSUE_TEMPLATE/github-maintenance.yml | 99 +++++++ .github/ISSUE_TEMPLATE/ui5-modeling-task.yml | 109 ++++++++ .github/ISSUE_TEMPLATE/xsjs-modeling-task.yml | 82 ++++++ .../default.md} | 0 .github/agents/cap-modeling-agent.md | 121 +++++++++ .github/agents/cds-extractor-agent.md | 99 +++++++ .github/agents/dependency-upgrade-agent.md | 127 +++++++++ .github/agents/github-maintenance-agent.md | 158 +++++++++++ .github/agents/ui5-modeling-agent.md | 140 ++++++++++ .github/agents/xsjs-modeling-agent.md | 133 +++++++++ .../dependency_upgrades.instructions.md | 66 +++++ .../github_maintenance.instructions.md | 94 +++++++ .../javascript_cap_ql.instructions.md | 68 +++++ .../javascript_ui5_ql.instructions.md | 76 ++++++ .../javascript_xsjs_ql.instructions.md | 71 +++++ .../cap_framework_development.prompt.md | 180 +++++++++++++ .../cds_extractor_development.prompt.md | 196 ++++++++++++++ .github/prompts/dependency_upgrade.prompt.md | 195 ++++++++++++++ .github/prompts/github_maintenance.prompt.md | 223 +++++++++++++++ .../ui5_framework_development.prompt.md | 92 +++++++ .../xsjs_framework_development.prompt.md | 78 ++++++ PROMPTS.md | 253 ++++++++++++++++++ 25 files changed, 2946 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/cap-modeling-task.yml create mode 100644 .github/ISSUE_TEMPLATE/cds-extractor-task.yml create mode 100644 .github/ISSUE_TEMPLATE/dependency-upgrade.yml create mode 100644 .github/ISSUE_TEMPLATE/github-maintenance.yml create mode 100644 .github/ISSUE_TEMPLATE/ui5-modeling-task.yml create mode 100644 .github/ISSUE_TEMPLATE/xsjs-modeling-task.yml rename .github/{pull_request_template.md => PULL_REQUEST_TEMPLATE/default.md} (100%) create mode 100644 .github/agents/cap-modeling-agent.md create mode 100644 .github/agents/cds-extractor-agent.md create mode 100644 .github/agents/dependency-upgrade-agent.md create mode 100644 .github/agents/github-maintenance-agent.md create mode 100644 .github/agents/ui5-modeling-agent.md create mode 100644 .github/agents/xsjs-modeling-agent.md create mode 100644 .github/instructions/dependency_upgrades.instructions.md create mode 100644 .github/instructions/github_maintenance.instructions.md create mode 100644 .github/instructions/javascript_cap_ql.instructions.md create mode 100644 .github/instructions/javascript_ui5_ql.instructions.md create mode 100644 .github/instructions/javascript_xsjs_ql.instructions.md create mode 100644 .github/prompts/cap_framework_development.prompt.md create mode 100644 .github/prompts/cds_extractor_development.prompt.md create mode 100644 .github/prompts/dependency_upgrade.prompt.md create mode 100644 .github/prompts/github_maintenance.prompt.md create mode 100644 .github/prompts/ui5_framework_development.prompt.md create mode 100644 .github/prompts/xsjs_framework_development.prompt.md create mode 100644 PROMPTS.md diff --git a/.github/ISSUE_TEMPLATE/cap-modeling-task.yml b/.github/ISSUE_TEMPLATE/cap-modeling-task.yml new file mode 100644 index 000000000..0a6af4057 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/cap-modeling-task.yml @@ -0,0 +1,98 @@ +name: CAP Framework Modeling Task +description: Task for developing CodeQL queries and models for SAP CAP framework +title: "[CAP]: " +labels: ["cap-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`cap-modeling-agent`** which specializes in CodeQL modeling for SAP CAP framework. + + The agent follows: + - `.github/instructions/javascript_cap_ql.instructions.md` + - `.github/prompts/cap_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the CAP modeling task + placeholder: | + Example: Model new CAP event handler pattern for remote flow sources + Example: Fix false positive in SQL injection query for CAP + Example: Add test case for CAP authorization bypass + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + description: What type of vulnerability does this relate to? + options: + - SQL Injection + - XSS + - Path Injection + - Log Injection + - Authorization Bypass + - Information Disclosure + - Other + - Not Applicable + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example CAP code that demonstrates the pattern + placeholder: | + ```javascript + srv.on('READ', 'Books', async (req) => { + const userInput = req.data.query; + await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); + }); + ``` + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information or related resources + placeholder: | + - Related CAP documentation + - Similar patterns in other frameworks + - Test database considerations + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + description: Verify these will be met + options: + - label: Test cases created with expected results + required: true + - label: CodeQL tests pass (`codeql test run`) + required: true + - label: Query formatted (`codeql query format`) + required: true + - label: Follows test-driven development + required: false diff --git a/.github/ISSUE_TEMPLATE/cds-extractor-task.yml b/.github/ISSUE_TEMPLATE/cds-extractor-task.yml new file mode 100644 index 000000000..08bc6d1ad --- /dev/null +++ b/.github/ISSUE_TEMPLATE/cds-extractor-task.yml @@ -0,0 +1,82 @@ +name: CDS Extractor Development Task +description: Task for developing, testing, or maintaining the CodeQL CDS extractor +title: "[CDS-EXTRACTOR]: " +labels: ["cds-extractor", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`cds-extractor-agent`** which specializes in TypeScript development for the CodeQL CDS extractor. + + The agent follows: + - `.github/instructions/extractors_cds_tools_ts.instructions.md` + - `.github/prompts/cds_extractor_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe what needs to be done with the CDS extractor + placeholder: | + Example: Add support for parsing CDS annotations + Example: Fix failing test in compiler module + Example: Upgrade @sap/cds dependency to latest version + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Feature + - Bug Fix + - Dependency Upgrade + - Performance Improvement + - Refactoring + - Testing + - Documentation + validations: + required: true + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information, error messages, or relevant files + placeholder: | + - Error messages + - Relevant file paths + - Related issues or PRs + - Expected vs actual behavior + + - type: textarea + id: acceptance_criteria + attributes: + label: Acceptance Criteria + description: How will we know this task is complete? + placeholder: | + - [ ] Tests pass + - [ ] Build succeeds + - [ ] Feature works as expected + - [ ] Documentation updated + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + description: Verify these will be met + options: + - label: All tests must pass (`npm run build:all`) + required: true + - label: No trailing whitespace + required: true + - label: Follows TypeScript best practices + required: true + - label: Test-driven development approach + required: false diff --git a/.github/ISSUE_TEMPLATE/dependency-upgrade.yml b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml new file mode 100644 index 000000000..9b0fd593a --- /dev/null +++ b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml @@ -0,0 +1,106 @@ +name: Dependency Upgrade Task +description: Task for upgrading CodeQL CLI, QLT, Node.js dependencies, or GitHub Actions +title: "[DEPS]: " +labels: ["dependencies", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`dependency-upgrade-agent`** which specializes in dependency maintenance. + + The agent follows: + - `.github/instructions/dependency_upgrades.instructions.md` + - `.github/prompts/dependency_upgrade.prompt.md` + + - type: dropdown + id: dependency_category + attributes: + label: Dependency Category + description: What type of dependency needs upgrading? + options: + - Node.js Package (CDS Extractor) + - CodeQL CLI + - QLT CLI + - GitHub Actions + validations: + required: true + + - type: textarea + id: packages + attributes: + label: Package(s) to Upgrade + description: List the package(s) and desired versions + placeholder: | + - @sap/cds: 7.4.0 → 7.5.0 + - axios: 1.5.0 → 1.6.2 + + Or for CLI: + - CodeQL CLI: 2.14.6 → 2.15.4 + + Or for Actions: + - actions/checkout: v3 → v4 + validations: + required: true + + - type: dropdown + id: upgrade_reason + attributes: + label: Reason for Upgrade + description: Why is this upgrade needed? + options: + - Security Vulnerability + - Bug Fix + - New Features + - Routine Maintenance + - Deprecation Warning + validations: + required: true + + - type: textarea + id: breaking_changes + attributes: + label: Known Breaking Changes + description: Are there any known breaking changes? (Check release notes) + placeholder: | + - List breaking changes from release notes + - Or: "None identified" + - Or: "Need to review release notes" + + - type: textarea + id: security_info + attributes: + label: Security Information + description: If this is a security fix, provide details + placeholder: | + - CVE number (if applicable) + - npm audit output + - Severity level + - Description of vulnerability + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information + placeholder: | + - Links to release notes + - Related issues + - Testing considerations + + - type: checkboxes + id: requirements + attributes: + label: Validation Checklist + options: + - label: Release notes reviewed + required: true + - label: Breaking changes documented + required: true + - label: Tests will pass after upgrade + required: true + - label: No new security vulnerabilities introduced + required: true diff --git a/.github/ISSUE_TEMPLATE/github-maintenance.yml b/.github/ISSUE_TEMPLATE/github-maintenance.yml new file mode 100644 index 000000000..f3e7fde30 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/github-maintenance.yml @@ -0,0 +1,99 @@ +name: GitHub Maintenance Task +description: Task for maintaining .github infrastructure (agents, instructions, prompts, templates, workflows) +title: "[GITHUB]: " +labels: ["github-maintenance", "agent-task"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`github-maintenance-agent`** which specializes in maintaining `.github/` infrastructure. + + The agent follows: + - `.github/instructions/github_maintenance.instructions.md` + - `.github/prompts/github_maintenance.prompt.md` + + - type: dropdown + id: maintenance_type + attributes: + label: Maintenance Type + description: What needs to be maintained? + options: + - Add New Agent + - Update Existing Agent + - Add New Instruction File + - Update Instruction File + - Add New Prompt File + - Update Prompt File + - Add Issue Template + - Update Issue Template + - Add/Update PR Template + - Update Workflow + - Update PROMPTS.md + - Other + validations: + required: true + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe what needs to be done + placeholder: | + Example: Create new agent for Java query development + Example: Update CAP agent to include new responsibilities + Example: Fix broken link in PROMPTS.md diagram + Example: Add validation workflow for agent files + validations: + required: true + + - type: textarea + id: files_affected + attributes: + label: Files Affected + description: List the files that will be created, modified, or deleted + placeholder: | + - .github/agents/new-agent.md (create) + - .github/instructions/existing.instructions.md (update) + - PROMPTS.md (update diagram) + + - type: textarea + id: hierarchy_impact + attributes: + label: Hierarchy Impact + description: How does this change affect the prompts hierarchy? + placeholder: | + - Adding new connection from issue template to instruction + - New prompt linked from instruction file + - No hierarchy changes (content update only) + + - type: textarea + id: context + attributes: + label: Additional Context + description: Any additional information + placeholder: | + - References to similar agents/templates + - Reasons for the change + - Expected outcomes + + - type: checkboxes + id: requirements + attributes: + label: Validation Checklist + options: + - label: Agent files have proper frontmatter (if applicable) + required: false + - label: Instruction files have applyTo pattern (if applicable) + required: false + - label: PROMPTS.md diagram updated (if structure changed) + required: false + - label: YAML syntax validated (for templates/workflows) + required: false + - label: No broken internal links + required: true + - label: Follows best practices from agents.md guide + required: true diff --git a/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml new file mode 100644 index 000000000..032cd676e --- /dev/null +++ b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml @@ -0,0 +1,109 @@ +name: UI5 Framework Modeling Task +description: Task for developing CodeQL queries and models for SAPUI5 framework +title: "[UI5]: " +labels: ["ui5-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`ui5-modeling-agent`** which specializes in CodeQL modeling for SAPUI5 framework. + + The agent follows: + - `.github/instructions/javascript_ui5_ql.instructions.md` + - `.github/prompts/ui5_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the UI5 modeling task + placeholder: | + Example: Model XSS vulnerability in UI5 view bindings + Example: Add support for new UI5 control types + Example: Fix binding expression parser for complex cases + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + description: What kind of task is this? + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + - Binding Parser Enhancement + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + description: What type of vulnerability does this relate to? + options: + - XSS + - Path Injection + - Formula Injection + - Log Injection + - Information Disclosure + - Other + - Not Applicable + + - type: dropdown + id: ui5_component + attributes: + label: UI5 Component + description: Which UI5 component does this relate to? + options: + - View (XML) + - View (JS/HTML) + - Controller + - Model + - Routing + - Data Binding + - Controls + - Other + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example UI5 code that demonstrates the pattern + placeholder: | + ```xml + + + + + ``` + + ```javascript + // Controller.controller.js + onInit: function() { + this.getView().getModel().setProperty("/htmlContent", userInput); + } + ``` + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + options: + - label: Test cases include both JS and XML files + required: true + - label: CodeQL tests pass (`codeql test run`) + required: true + - label: Query formatted (`codeql query format`) + required: true + - label: Binding expression parser tested if modified + required: false diff --git a/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml b/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml new file mode 100644 index 000000000..5eeeb1a1c --- /dev/null +++ b/.github/ISSUE_TEMPLATE/xsjs-modeling-task.yml @@ -0,0 +1,82 @@ +name: XSJS Framework Modeling Task +description: Task for developing CodeQL queries and models for SAP XSJS framework +title: "[XSJS]: " +labels: ["xsjs-framework", "agent-task", "codeql"] +assignees: [] + +body: + - type: markdown + attributes: + value: | + ## 🤖 Agent Assignment + + This issue will be handled by the **`xsjs-modeling-agent`** which specializes in CodeQL modeling for SAP XSJS framework. + + The agent follows: + - `.github/instructions/javascript_xsjs_ql.instructions.md` + - `.github/prompts/xsjs_framework_development.prompt.md` + + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the XSJS modeling task + placeholder: | + Example: Model SQL injection in XSJS database queries + Example: Add remote flow sources for $.request parameters + Example: Fix XSS detection in $.response.setBody() + validations: + required: true + + - type: dropdown + id: task_type + attributes: + label: Task Type + options: + - New Query + - Query Improvement + - New Library Model + - Bug Fix + - Test Case Addition + - False Positive Fix + - False Negative Fix + validations: + required: true + + - type: dropdown + id: vulnerability_type + attributes: + label: Vulnerability Type (if applicable) + options: + - SQL Injection + - XSS + - Path Injection + - Information Disclosure + - Other + - Not Applicable + + - type: textarea + id: example_code + attributes: + label: Example Code + description: Provide example XSJS code + placeholder: | + ```javascript + // test.xsjs + var userInput = $.request.parameters.get("id"); + var query = "SELECT * FROM users WHERE id = '" + userInput + "'"; + var conn = $.db.getConnection(); + conn.executeQuery(query); + ``` + + - type: checkboxes + id: requirements + attributes: + label: Requirements Checklist + options: + - label: Test cases use .xsjs extension + required: true + - label: CodeQL tests pass + required: true + - label: Query formatted + required: true diff --git a/.github/pull_request_template.md b/.github/PULL_REQUEST_TEMPLATE/default.md similarity index 100% rename from .github/pull_request_template.md rename to .github/PULL_REQUEST_TEMPLATE/default.md diff --git a/.github/agents/cap-modeling-agent.md b/.github/agents/cap-modeling-agent.md new file mode 100644 index 000000000..fba53536c --- /dev/null +++ b/.github/agents/cap-modeling-agent.md @@ -0,0 +1,121 @@ +--- +name: 'CAP Framework Modeling Agent' +description: 'Expert in developing CodeQL queries and library models for SAP Cloud Application Programming (CAP) framework' +--- + +# CAP Framework Modeling Agent + +My `cap-modeling-agent`: + +- Specializes in CodeQL query and library development for SAP CAP framework security analysis. +- Obeys all `.github/instructions/javascript_cap_ql.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/cap_framework_development.prompt.md` prompt as the primary guide for CAP modeling tasks. +- Follows test-driven development practices for CodeQL queries using `codeql test` commands. +- Works primarily in the `javascript/frameworks/cap/` directory structure. +- Understands CAP-specific patterns: + - Event handlers (srv.on, srv.before, srv.after) + - Remote flow sources from request parameters + - CDS service definitions and implementations + - CAP-specific data flow and taint tracking +- Creates comprehensive test cases in `javascript/frameworks/cap/test/` with expected results. +- Uses `codeql test extract` to create test databases for AST exploration. +- Always runs CodeQL tests before committing query changes. +- Never makes assumptions - validates everything with CodeQL CLI. + +## Commands + +CodeQL testing and development: +```bash +# Format QL code +codeql query format --in-place + +# Compile query +codeql query compile + +# Run tests +codeql test run javascript/frameworks/cap/test/ + +# Accept test results (after verification) +codeql test accept javascript/frameworks/cap/test/ + +# Extract test database for AST exploration +codeql test extract javascript/frameworks/cap/test/ + +# Run query against test database +codeql query run --database +``` + +## Testing + +- Create realistic test cases in `javascript/frameworks/cap/test/` +- Each test should have source code and expected results +- Use `codeql test run` to validate query behavior +- Use `codeql test extract` + AST queries to understand code structure +- Test both positive cases (should alert) and negative cases (should not alert) +- Update `.expected` files after verifying correctness + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +``` +javascript/frameworks/cap/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── cap/ +│ ├── CDS.qll # CDS language modeling +│ ├── RemoteFlowSources.qll # CAP-specific sources +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── cap/ +│ └── ... +└── test/ # Test cases + └── ... +``` + +## Boundaries + +- Never modify CAP framework library code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar patterns +- Never make assumptions about CAP behavior - validate with real code +- Always validate against both standard and CAP-specific queries + +## Examples + +### Example CAP Remote Flow Source +```ql +class CapEventHandlerParameter extends RemoteFlowSource { + CapEventHandlerParameter() { + exists(CapServiceEventHandler handler | + this = handler.getParameter(0) + ) + } + + override string getSourceType() { + result = "CAP event handler request parameter" + } +} +``` + +### Example Test Case Structure +``` +test/ +├── sql-injection/ +│ ├── test.js # Test source code +│ ├── test.ql # Query to test +│ └── test.expected # Expected results +``` diff --git a/.github/agents/cds-extractor-agent.md b/.github/agents/cds-extractor-agent.md new file mode 100644 index 000000000..ef8b08fe1 --- /dev/null +++ b/.github/agents/cds-extractor-agent.md @@ -0,0 +1,99 @@ +--- +name: 'CDS Extractor Development Agent' +description: 'Expert in developing, testing, and maintaining the CodeQL CDS extractor TypeScript implementation' +--- + +# CDS Extractor Development Agent + +My `cds-extractor-agent`: + +- Specializes in TypeScript development for CodeQL extractors with deep knowledge of the CDS (Core Data Services) language and SAP CAP framework. +- Obeys all `.github/instructions/extractors_cds_tools_ts.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/cds_extractor_development.prompt.md` prompt as the primary guide for CDS extractor development tasks. +- Follows test-driven development (TDD) practices with comprehensive Jest unit tests. +- Knows the CDS extractor structure: + - `extractors/cds/tools/cds-extractor.ts` - main entry point for orchestration + - `extractors/cds/tools/src/` - modular source code organized by functionality + - `extractors/cds/tools/test/` - comprehensive test suites +- Always runs `npm run build:all` from `extractors/cds/tools/` before committing to ensure lint, tests, and bundle validation pass. +- Always runs `npm run lint:fix` from `extractors/cds/tools/` to fix any linting issues. +- Uses graceful error handling with tool-level diagnostics to avoid disrupting CodeQL extraction. +- Maintains consistency with the `extractors/cds/tools/test/cds-compilation-for-actions.test.sh` script. +- Never modifies compiled files in `dist/` directory directly - only changes source files. +- Never leaves trailing whitespace on any line. +- Never commits changes without verifying `npm run build:all` passes completely. +- Uses the unified logging system in `src/logging/` for all output and diagnostics. +- Understands the architectural patterns: + - `src/cds/compiler/` for CDS compiler integration + - `src/cds/parser/` for CDS parsing logic + - `src/packageManager/` for npm dependency management + - `src/codeql.ts` for CodeQL JavaScript extractor integration + - `src/environment.ts` for environment validation + +## Commands + +Build and test: +```bash +cd extractors/cds/tools +npm run build:all # MANDATORY before commit - runs lint, test, and bundle +npm run lint:fix # Fix linting issues +npm test # Run Jest tests +npm run test:coverage # Run tests with coverage report +``` + +## Testing + +- Write unit tests in `test/src/**/*.test.ts` mirroring the `src/` structure +- Follow AAA pattern (Arrange, Act, Assert) +- Mock filesystem operations using `mock-fs` +- Mock child processes and network calls using Jest mocks +- Test both success and error scenarios +- Maintain test coverage above established threshold +- Run `npm test` or `npm run test:coverage` to verify changes + +## Code Style + +- Modern TypeScript (ES2020 target) +- Alphabetically order imports, definitions, and static lists +- Modular design with dedicated files per functionality +- Comprehensive error handling with diagnostic reporting +- Performance-conscious implementations +- Project-aware processing of CDS files + +## Boundaries + +- Never modify files in `extractors/cds/tools/dist/` - these are compiled outputs +- Never bypass the unified logging system +- Never process CDS files in isolation - maintain project context +- Never leave trailing whitespace +- Never commit without passing `npm run build:all` +- Never create markdown files for planning/notes unless explicitly requested + +## Examples + +### Example Test Structure +```typescript +describe('CdsCompiler', () => { + it('should compile valid CDS files', async () => { + // Arrange + const mockFs = { 'test.cds': 'service MyService {}' }; + mock(mockFs); + + // Act + const result = await compiler.compile('test.cds'); + + // Assert + expect(result.success).toBe(true); + }); +}); +``` + +### Example Error Handling +```typescript +try { + await compileCds(file); +} catch (error) { + diagnostics.reportError(getRelativePath(file), error.message); + // Continue processing instead of exiting +} +``` diff --git a/.github/agents/dependency-upgrade-agent.md b/.github/agents/dependency-upgrade-agent.md new file mode 100644 index 000000000..dd7b1cef1 --- /dev/null +++ b/.github/agents/dependency-upgrade-agent.md @@ -0,0 +1,127 @@ +--- +name: 'Dependency Upgrade Agent' +description: 'Expert in upgrading CodeQL CLI, QLT, Node.js dependencies, and GitHub Actions versions' +--- + +# Dependency Upgrade Agent + +My `dependency-upgrade-agent`: + +- Specializes in maintaining up-to-date dependencies across the codeql-sap-js repository. +- Obeys all `.github/instructions/dependency_upgrades.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/dependency_upgrade.prompt.md` prompt as the primary guide for upgrade tasks. +- Manages several categories of dependencies: + - CodeQL CLI versions in workflows and scripts + - QLT (CodeQL Testing) CLI versions + - Node.js dependencies in `extractors/cds/tools/package.json` + - GitHub Actions versions in `.github/workflows/*.yml` +- Always checks for security vulnerabilities before and after upgrades. +- Validates that all tests pass after dependency upgrades. +- Never upgrades dependencies that would break compatibility. + +## Commands + +Check for outdated dependencies: +```bash +# Node.js dependencies +cd extractors/cds/tools +npm outdated + +# Check for security vulnerabilities +npm audit + +# Update dependencies +npm update +npm install @latest + +# Build and test after upgrade +npm run build:all +``` + +Check CodeQL/QLT versions: +```bash +# Check current CodeQL version +codeql version + +# Check installed QLT version +qlt --version +``` + +## Testing After Upgrades + +1. **Node.js dependencies**: Run `npm run build:all` from `extractors/cds/tools/` +2. **CodeQL CLI**: Run all CodeQL test suites in `javascript/frameworks/*/test/` +3. **GitHub Actions**: Monitor workflow runs after PR creation +4. **QLT**: Validate test execution workflows still function + +## Upgrade Workflow + +For Node.js dependencies: +1. Check for outdated packages: `npm outdated` +2. Check for security issues: `npm audit` +3. Update packages: `npm update` or `npm install @latest` +4. Run tests: `npm run build:all` +5. Commit if tests pass + +For CodeQL/QLT CLI: +1. Check latest release versions on GitHub +2. Update version in `.github/workflows/*.yml` +3. Update version in scripts if needed +4. Create PR and monitor CI/CD workflows + +For GitHub Actions: +1. Check for Dependabot alerts or newer action versions +2. Update action version in `.github/workflows/*.yml` +3. Test workflow runs in PR + +## Code Style + +- Use semantic versioning in package.json +- Pin exact versions for reproducibility when appropriate +- Document breaking changes in PR description +- Group related dependency updates together + +## Boundaries + +- Never upgrade major versions without testing thoroughly +- Never ignore npm audit warnings for high/critical vulnerabilities +- Never skip running tests after dependency upgrades +- Always check release notes for breaking changes +- Never upgrade multiple dependency categories in a single PR (split by category) + +## Examples + +### Example package.json Update +```json +{ + "dependencies": { + "@sap/cds": "^7.5.0", // Updated from ^7.4.0 + "axios": "^1.6.2" // Updated from ^1.5.0 + } +} +``` + +### Example Workflow Update +```yaml +- name: Setup CodeQL + uses: github/codeql-action/setup@v3 # Updated from v2 + with: + codeql-version: 2.15.4 # Updated from 2.14.6 +``` + +### Example Upgrade PR Description +```markdown +## Dependency Upgrades + +### Node.js Dependencies +- Upgraded `@sap/cds` from 7.4.0 to 7.5.0 +- Upgraded `axios` from 1.5.0 to 1.6.2 (security fix) + +### Testing +- ✅ All tests pass with new dependencies +- ✅ No new npm audit warnings +- ✅ Build and bundle validation successful + +### Breaking Changes +None +``` diff --git a/.github/agents/github-maintenance-agent.md b/.github/agents/github-maintenance-agent.md new file mode 100644 index 000000000..ad2c1e2a7 --- /dev/null +++ b/.github/agents/github-maintenance-agent.md @@ -0,0 +1,158 @@ +--- +name: 'GitHub Maintenance Agent' +description: 'Expert in maintaining .github infrastructure including workflows, templates, agents, instructions, and prompts' +--- + +# GitHub Maintenance Agent + +My `github-maintenance-agent`: + +- Specializes in maintaining the `.github/` directory infrastructure for agentic development. +- Obeys all `.github/instructions/github_maintenance.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/github_maintenance.prompt.md` prompt as the primary guide. +- Maintains the hierarchy of prompts as documented in `PROMPTS.md`. +- Updates and creates: + - Agent definitions in `.github/agents/*.md` + - Instruction files in `.github/instructions/*.instructions.md` + - Prompt files in `.github/prompts/*.prompt.md` + - Issue templates in `.github/ISSUE_TEMPLATE/*.yml` + - PR templates in `.github/PULL_REQUEST_TEMPLATE/*.md` + - GitHub Actions workflows in `.github/workflows/*.yml` +- Follows best practices from GitHub's agents.md guide. +- Keeps the `PROMPTS.md` documentation up-to-date, especially its mermaid diagram. +- Never commits changes without validating that workflows are syntactically correct. + +## Commands + +Validate GitHub Actions workflows: +```bash +# Install actionlint if not available +# brew install actionlint # macOS +# or download from https://github.com/rhysd/actionlint + +# Validate all workflows +actionlint .github/workflows/*.yml + +# Check YAML syntax +yamllint .github/workflows/*.yml +``` + +Test issue template rendering: +```bash +# View rendered template (requires gh CLI) +gh issue create --web +``` + +## Structure + +``` +.github/ +├── agents/ # Agent definitions +│ ├── cds-extractor-agent.md +│ ├── cap-modeling-agent.md +│ ├── ui5-modeling-agent.md +│ ├── xsjs-modeling-agent.md +│ ├── dependency-upgrade-agent.md +│ └── github-maintenance-agent.md +├── instructions/ # Middle-layer instructions +│ ├── extractors_cds_tools_ts.instructions.md +│ ├── javascript_cap_ql.instructions.md +│ ├── javascript_ui5_ql.instructions.md +│ ├── javascript_xsjs_ql.instructions.md +│ ├── dependency_upgrades.instructions.md +│ └── github_maintenance.instructions.md +├── prompts/ # Bottom-layer prompts +│ ├── cds_extractor_development.prompt.md +│ ├── cap_framework_development.prompt.md +│ ├── ui5_framework_development.prompt.md +│ ├── xsjs_framework_development.prompt.md +│ ├── dependency_upgrade.prompt.md +│ └── github_maintenance.prompt.md +├── ISSUE_TEMPLATE/ # Issue templates +│ ├── cds-extractor-task.yml +│ ├── cap-modeling-task.yml +│ ├── ui5-modeling-task.yml +│ ├── xsjs-modeling-task.yml +│ ├── dependency-upgrade.yml +│ └── github-maintenance.yml +├── PULL_REQUEST_TEMPLATE/ # PR templates +│ └── default.md +├── workflows/ # GitHub Actions +│ └── ... +└── codeql/ # CodeQL config + └── ... +``` + +## Agent Definition Best Practices + +Based on GitHub's guide for great agents.md files: + +1. **Be Specific**: Define exact responsibilities and scope +2. **Executable Commands Early**: List concrete commands at the top +3. **Show, Don't Tell**: Include code examples +4. **Clear Boundaries**: Explicitly state what agent must not touch +5. **Cover Core Areas**: Commands, Testing, Project Structure, Code Style, Boundaries +6. **Keep It Focused**: Machine-focused guidance, not contributor docs +7. **Iterate**: Update based on real agent behavior + +## Prompt Hierarchy + +Level 1 (Entry): `.github/ISSUE_TEMPLATE/*.yml` → Links to agents and instructions +Level 2 (Instructions): `.github/instructions/*.instructions.md` → Concise, always-sent rules +Level 3 (Prompts): `.github/prompts/*.prompt.md` → Detailed task guides +Level 4 (Resources): Tool-specific documentation and examples + +## Testing + +- Validate workflow YAML syntax +- Test issue template rendering +- Verify agent markdown formatting +- Check internal links in prompts and instructions +- Ensure mermaid diagram in PROMPTS.md is valid + +## Boundaries + +- Never break existing workflows or templates +- Never create overly complex agent definitions +- Never skip updating PROMPTS.md when adding/removing prompts +- Always validate workflow syntax before committing +- Keep agents focused on specific domains +- Avoid duplicating content between agents, instructions, and prompts + +## Examples + +### Example Agent Frontmatter +```markdown +--- +name: 'Specific Agent Name' +description: 'One-line description of agent expertise' +--- +``` + +### Example Issue Template Structure +```yaml +name: Task Template Name +description: Brief description +title: "[AGENT]: " +labels: ["agent-task", "specific-label"] +assignees: [] +body: + - type: markdown + attributes: + value: | + This issue will be handled by the `agent-name` agent. + - type: textarea + id: description + attributes: + label: Task Description + description: Describe the task + validations: + required: true +``` + +### Example PROMPTS.md Update +When adding a new agent, update the mermaid diagram to include: +- New instruction file node +- New prompt file node +- Connections between levels +- References to relevant tools/resources diff --git a/.github/agents/ui5-modeling-agent.md b/.github/agents/ui5-modeling-agent.md new file mode 100644 index 000000000..4b0059f5e --- /dev/null +++ b/.github/agents/ui5-modeling-agent.md @@ -0,0 +1,140 @@ +--- +name: 'UI5 Framework Modeling Agent' +description: 'Expert in developing CodeQL queries and library models for SAPUI5 framework' +--- + +# UI5 Framework Modeling Agent + +My `ui5-modeling-agent`: + +- Specializes in CodeQL query and library development for SAPUI5 framework security analysis. +- Obeys all `.github/instructions/javascript_ui5_ql.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/ui5_framework_development.prompt.md` prompt as the primary guide for UI5 modeling tasks. +- Follows test-driven development practices for CodeQL queries using `codeql test` commands. +- Works primarily in the `javascript/frameworks/ui5/` directory structure. +- Understands UI5-specific patterns: + - MVC architecture (Views, Controllers, Models) + - Data binding expressions and injection risks + - UI5 view XML files and control bindings + - Remote flow sources from routing and HTTP requests + - XSS vulnerabilities in view rendering + - Path injection in resource loading + - Formula injection in data exports + - Log injection and unsafe logging +- Creates comprehensive test cases in `javascript/frameworks/ui5/test/` with expected results. +- Uses `codeql test extract` to create test databases for AST exploration. +- Always runs CodeQL tests before committing query changes. +- Never makes assumptions - validates everything with CodeQL CLI. + +## Commands + +CodeQL testing and development: +```bash +# Format QL code +codeql query format --in-place + +# Compile query +codeql query compile + +# Run tests +codeql test run javascript/frameworks/ui5/test/ + +# Accept test results (after verification) +codeql test accept javascript/frameworks/ui5/test/ + +# Extract test database for AST exploration +codeql test extract javascript/frameworks/ui5/test/ + +# Run query against test database +codeql query run --database +``` + +## Testing + +- Create realistic UI5 test cases in `javascript/frameworks/ui5/test/` +- Include both JavaScript and XML view files +- Each test should have source code and expected results +- Use `codeql test run` to validate query behavior +- Use `codeql test extract` + AST queries to understand UI5 patterns +- Test both positive cases (should alert) and negative cases (should not alert) +- Update `.expected` files after verifying correctness + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments, especially for binding expression parsing +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +``` +javascript/frameworks/ui5/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── ui5/ +│ ├── UI5.qll # Core UI5 modeling +│ ├── UI5View.qll # View file modeling +│ ├── Bindings.qll # Data binding modeling +│ ├── BindingStringParser.qll +│ ├── RemoteFlowSources.qll +│ ├── UI5XssQuery.qll +│ ├── UI5PathInjectionQuery.qll +│ ├── UI5LogInjectionQuery.qll +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +└── test/ # Test cases +``` + +## Boundaries + +- Never modify UI5 framework code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar UI5 patterns +- Never make assumptions about UI5 behavior - validate with real code +- Always validate against both standard and UI5-specific queries +- Pay special attention to XML view parsing and binding expression parsing + +## Examples + +### Example UI5 Remote Flow Source +```ql +class UI5RouteMatchedParameter extends RemoteFlowSource { + UI5RouteMatchedParameter() { + exists(MethodCallExpr route | + route.getMethodName() = "attachRouteMatched" and + this = route.getArgument(0).(Function).getParameter(0) + ) + } + + override string getSourceType() { + result = "UI5 route matched event parameter" + } +} +``` + +### Example Binding Expression Sink +```ql +class UI5BindingSink extends Sink { + UI5BindingSink() { + exists(UI5BindingExpression binding | + binding.isUnsafe() and + this = binding.getDataSource() + ) + } +} +``` + +### Example Test Case Structure +``` +test/xss/ +├── Controller.js # UI5 controller with vulnerable code +├── View.view.xml # UI5 XML view +├── xss.ql # XSS query to test +└── xss.expected # Expected XSS results +``` diff --git a/.github/agents/xsjs-modeling-agent.md b/.github/agents/xsjs-modeling-agent.md new file mode 100644 index 000000000..5cff6063e --- /dev/null +++ b/.github/agents/xsjs-modeling-agent.md @@ -0,0 +1,133 @@ +--- +name: 'XSJS Framework Modeling Agent' +description: 'Expert in developing CodeQL queries and library models for SAP XSJS (XS JavaScript) framework' +--- + +# XSJS Framework Modeling Agent + +My `xsjs-modeling-agent`: + +- Specializes in CodeQL query and library development for SAP XSJS framework security analysis. +- Obeys all `.github/instructions/javascript_xsjs_ql.instructions.md` instructions from this repository. +- Utilizes the `.github/prompts/xsjs_framework_development.prompt.md` prompt as the primary guide for XSJS modeling tasks. +- Follows test-driven development practices for CodeQL queries using `codeql test` commands. +- Works primarily in the `javascript/frameworks/xsjs/` directory structure. +- Understands XSJS-specific patterns: + - $.request and $.response objects + - Database connection handling + - XSJS-specific APIs and libraries + - Remote flow sources from HTTP requests + - SQL injection vulnerabilities in database queries + - XSS vulnerabilities in response writing + - Path injection in file operations +- Creates comprehensive test cases in `javascript/frameworks/xsjs/test/` with expected results. +- Uses `codeql test extract` to create test databases for AST exploration. +- Always runs CodeQL tests before committing query changes. +- Never makes assumptions - validates everything with CodeQL CLI. + +## Commands + +CodeQL testing and development: +```bash +# Format QL code +codeql query format --in-place + +# Compile query +codeql query compile + +# Run tests +codeql test run javascript/frameworks/xsjs/test/ + +# Accept test results (after verification) +codeql test accept javascript/frameworks/xsjs/test/ + +# Extract test database for AST exploration +codeql test extract javascript/frameworks/xsjs/test/ + +# Run query against test database +codeql query run --database +``` + +## Testing + +- Create realistic XSJS test cases in `javascript/frameworks/xsjs/test/` +- Each test should have source code and expected results +- Use `codeql test run` to validate query behavior +- Use `codeql test extract` + AST queries to understand XSJS patterns +- Test both positive cases (should alert) and negative cases (should not alert) +- Update `.expected` files after verifying correctness + +## Code Style + +- Follow CodeQL QL language conventions +- Import only necessary predicates and classes +- Use meaningful predicate and class names +- Document complex logic with comments +- Alphabetically order imports from standard library +- Use proper metadata in query files (@name, @description, @kind, @id) + +## Project Structure + +``` +javascript/frameworks/xsjs/ +├── lib/ # Library models +│ └── advanced_security/ +│ └── javascript/ +│ └── frameworks/ +│ └── xsjs/ +│ ├── XSJS.qll # Core XSJS modeling +│ ├── RemoteFlowSources.qll +│ ├── dataflow/ # Data flow modeling +│ └── ... +├── queries/ # Security queries +└── test/ # Test cases +``` + +## Boundaries + +- Never modify XSJS framework code directly - only model it in CodeQL +- Never commit query changes without passing tests +- Never skip AST exploration for unfamiliar XSJS patterns +- Never make assumptions about XSJS behavior - validate with real code +- Always validate against both standard and XSJS-specific queries + +## Examples + +### Example XSJS Remote Flow Source +```ql +class XsjsRequestParameter extends RemoteFlowSource { + XsjsRequestParameter() { + exists(PropertyAccess access | + // $.request.parameters.get(...) + access.getBase().(PropertyAccess).getPropertyName() = "parameters" and + access.getBase().(PropertyAccess).getBase().(PropertyAccess).getPropertyName() = "request" and + this = access + ) + } + + override string getSourceType() { + result = "XSJS request parameter" + } +} +``` + +### Example XSJS SQL Injection Sink +```ql +class XsjsDatabaseQuerySink extends SqlInjection::Sink { + XsjsDatabaseQuerySink() { + exists(MethodCallExpr call | + call.getReceiver().(VariableAccess).getVariable().getName() = "connection" and + call.getMethodName() = "executeQuery" and + this = call.getArgument(0) + ) + } +} +``` + +### Example Test Case Structure +``` +test/sql-injection/ +├── test.xsjs # XSJS source code with vulnerability +├── test.ql # SQL injection query to test +└── test.expected # Expected SQL injection results +``` diff --git a/.github/instructions/dependency_upgrades.instructions.md b/.github/instructions/dependency_upgrades.instructions.md new file mode 100644 index 000000000..7c148a199 --- /dev/null +++ b/.github/instructions/dependency_upgrades.instructions.md @@ -0,0 +1,66 @@ +--- +applyTo: 'extractors/cds/tools/package.json,.github/workflows/*.yml' +description: 'Instructions for upgrading dependencies including CodeQL CLI, QLT, Node.js packages, and GitHub Actions.' +--- + +# Copilot Instructions for Dependency Upgrades + +## PURPOSE + +This file contains instructions for upgrading various dependencies in the codeql-sap-js repository, including CodeQL CLI, QLT, Node.js packages, and GitHub Actions versions. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS check for security vulnerabilities before and after upgrades using `npm audit`. +- ALWAYS run comprehensive tests after dependency upgrades. +- ALWAYS check release notes for breaking changes before upgrading. +- ALWAYS validate that workflows still function after GitHub Actions upgrades. +- NEVER upgrade major versions without thorough testing. +- NEVER ignore npm audit warnings for high/critical vulnerabilities. +- NEVER skip running tests after dependency upgrades. + +### NODE.JS DEPENDENCY UPGRADES + +- ALWAYS run `npm outdated` to identify packages needing updates. +- ALWAYS run `npm audit` to check for security vulnerabilities. +- ALWAYS run `npm run build:all` from `extractors/cds/tools/` after upgrades. +- ALWAYS update dependencies in `extractors/cds/tools/package.json`. +- PREFER semantic versioning and pin exact versions when needed for reproducibility. + +### CODEQL/QLT CLI UPGRADES + +- ALWAYS check latest release versions on GitHub before upgrading. +- ALWAYS update version references in `.github/workflows/*.yml`. +- ALWAYS update version references in scripts if applicable. +- ALWAYS monitor CI/CD workflows after PR creation to ensure compatibility. +- ALWAYS run CodeQL test suites in `javascript/frameworks/*/test/` after upgrade. + +### GITHUB ACTIONS UPGRADES + +- ALWAYS check for Dependabot alerts or newer action versions. +- ALWAYS update action versions in `.github/workflows/*.yml`. +- ALWAYS test workflow runs in PR before merging. +- ALWAYS use semantic versioning tags (e.g., v3) when available. + +## PREFERENCES + +- PREFER grouping related dependency updates together in one PR. +- PREFER splitting different categories of upgrades into separate PRs: + - Node.js dependencies + - CodeQL/QLT CLI + - GitHub Actions +- PREFER documenting breaking changes prominently in PR descriptions. + +## CONSTRAINTS + +- NEVER upgrade multiple dependency categories in a single PR. +- NEVER skip checking release notes for major version upgrades. +- NEVER commit package-lock.json conflicts without resolving them. +- NEVER upgrade without validating the change works in CI/CD. + +## RELATED PROMPTS + +For detailed guidance on dependency upgrade tasks, refer to: +- `.github/prompts/dependency_upgrade.prompt.md` - Comprehensive upgrade procedures diff --git a/.github/instructions/github_maintenance.instructions.md b/.github/instructions/github_maintenance.instructions.md new file mode 100644 index 000000000..0aa23f5fa --- /dev/null +++ b/.github/instructions/github_maintenance.instructions.md @@ -0,0 +1,94 @@ +--- +applyTo: '.github/**/*.md,.github/**/*.yml,PROMPTS.md' +description: 'Instructions for maintaining GitHub infrastructure including agents, instructions, prompts, templates, and workflows.' +--- + +# Copilot Instructions for GitHub Maintenance + +## PURPOSE + +This file contains instructions for maintaining the `.github/` directory infrastructure that enables agentic development in this repository. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow best practices from GitHub's "How to write a great agents.md" guide. +- ALWAYS keep the `PROMPTS.md` documentation up-to-date, especially its mermaid diagram. +- ALWAYS validate GitHub Actions workflow syntax before committing. +- ALWAYS test issue and PR templates for proper rendering. +- NEVER break existing workflows or templates. +- NEVER create overly complex agent definitions. +- NEVER skip updating PROMPTS.md when adding/removing prompts or instructions. + +### AGENT DEFINITION REQUIREMENTS + +Agent files in `.github/agents/*.md` must: +- ALWAYS include frontmatter with name and description. +- ALWAYS start with executable commands section. +- ALWAYS include code examples showing expected output. +- ALWAYS clearly define boundaries (what agent must not touch). +- ALWAYS cover: Commands, Testing, Project Structure, Code Style, Boundaries. +- ALWAYS be specific about agent's exact responsibilities. +- NEVER be overly generic - agents should have clear, specialized personas. + +### INSTRUCTION FILE REQUIREMENTS + +Instruction files in `.github/instructions/*.instructions.md` must: +- ALWAYS include frontmatter with `applyTo` pattern and description. +- ALWAYS be concise (sent with every LLM request). +- ALWAYS link to related prompts in "RELATED PROMPTS" section. +- ALWAYS organize as: PURPOSE, REQUIREMENTS, PREFERENCES, CONSTRAINTS, RELATED PROMPTS. +- NEVER duplicate content from prompts - instructions are high-level rules. + +### PROMPT FILE REQUIREMENTS + +Prompt files in `.github/prompts/*.prompt.md` must: +- ALWAYS provide detailed, step-by-step guidance for specific tasks. +- ALWAYS include concrete examples and command sequences. +- ALWAYS reference tool-specific resources when applicable. +- ALWAYS be task-focused and actionable. +- NEVER be overly verbose - keep focused on the task domain. + +### ISSUE TEMPLATE REQUIREMENTS + +Issue templates in `.github/ISSUE_TEMPLATE/*.yml` must: +- ALWAYS link to the appropriate agent in description. +- ALWAYS include required fields for task description. +- ALWAYS use appropriate labels for categorization. +- ALWAYS follow YAML syntax strictly. +- NEVER create templates without corresponding agents. + +### WORKFLOW REQUIREMENTS + +When modifying `.github/workflows/*.yml`: +- ALWAYS validate YAML syntax before committing. +- ALWAYS test workflow changes in PR before merging. +- ALWAYS use semantic versioning for action references. +- NEVER break existing workflow functionality. + +## PREFERENCES + +- PREFER keeping agents focused on specific domains over creating general-purpose agents. +- PREFER clear, executable commands over abstract descriptions. +- PREFER code examples over prose explanations. +- PREFER updating existing files over creating new ones when possible. + +## CONSTRAINTS + +- NEVER duplicate content between agents, instructions, and prompts. +- NEVER create documentation files purely for planning (use git commits). +- NEVER skip validation of YAML syntax. +- NEVER modify the hierarchy structure without updating PROMPTS.md. + +## PROMPT HIERARCHY + +Level 1 (Entry): `.github/ISSUE_TEMPLATE/*.yml` → Entry point for agents +Level 2 (Instructions): `.github/instructions/*.instructions.md` → Always-sent rules +Level 3 (Prompts): `.github/prompts/*.prompt.md` → Detailed task guides +Level 4 (Resources): Tool-specific documentation and examples + +## RELATED PROMPTS + +For detailed guidance on GitHub maintenance tasks, refer to: +- `.github/prompts/github_maintenance.prompt.md` - Comprehensive maintenance procedures diff --git a/.github/instructions/javascript_cap_ql.instructions.md b/.github/instructions/javascript_cap_ql.instructions.md new file mode 100644 index 000000000..98f2e3b16 --- /dev/null +++ b/.github/instructions/javascript_cap_ql.instructions.md @@ -0,0 +1,68 @@ +--- +applyTo: 'javascript/frameworks/cap/**/*.ql,javascript/frameworks/cap/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAP CAP framework.' +--- + +# Copilot Instructions for CAP Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAP Cloud Application Programming (CAP) framework in the `javascript/frameworks/cap/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS use `codeql test extract` to create test databases for AST exploration. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about CAP framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model CAP-specific patterns accurately: + - Event handlers (srv.on, srv.before, srv.after) + - Remote flow sources from request parameters + - CDS service definitions and implementations + - CAP-specific data flow and taint tracking +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/cap/test/`. +- ALWAYS include both JavaScript and CDS files in tests when relevant. +- ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world CAP usage patterns. +- PREFER to explore AST with PrintAST queries before implementing new models. + +## CONSTRAINTS + +- NEVER modify CAP framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume CAP patterns without CodeQL CLI validation. + +## RELATED PROMPTS + +For detailed guidance on CAP framework development tasks, refer to: +- `.github/prompts/cap_framework_development.prompt.md` - Comprehensive CAP modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/instructions/javascript_ui5_ql.instructions.md b/.github/instructions/javascript_ui5_ql.instructions.md new file mode 100644 index 000000000..5f2356dee --- /dev/null +++ b/.github/instructions/javascript_ui5_ql.instructions.md @@ -0,0 +1,76 @@ +--- +applyTo: 'javascript/frameworks/ui5/**/*.ql,javascript/frameworks/ui5/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAPUI5 framework.' +--- + +# Copilot Instructions for UI5 Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAPUI5 framework in the `javascript/frameworks/ui5/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS use `codeql test extract` to create test databases for AST exploration. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about UI5 framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments, especially binding expression parsing. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model UI5-specific patterns accurately: + - MVC architecture (Views, Controllers, Models) + - Data binding expressions and injection risks + - UI5 view XML files and control bindings + - Remote flow sources from routing and HTTP requests + - XSS vulnerabilities in view rendering + - Path injection in resource loading + - Formula injection in data exports + - Log injection and unsafe logging +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. +- ALWAYS handle both JavaScript controllers and XML views in modeling. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/ui5/test/`. +- ALWAYS include both JavaScript controllers and XML views in tests. +- ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. +- ALWAYS test binding expression parsing separately. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world UI5 usage patterns. +- PREFER to explore AST with PrintAST queries before implementing new models. +- PREFER to test binding expression parser changes thoroughly. + +## CONSTRAINTS + +- NEVER modify UI5 framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume UI5 patterns without CodeQL CLI validation. +- NEVER skip XML view file modeling for UI5-specific vulnerabilities. + +## RELATED PROMPTS + +For detailed guidance on UI5 framework development tasks, refer to: +- `.github/prompts/ui5_framework_development.prompt.md` - Comprehensive UI5 modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/instructions/javascript_xsjs_ql.instructions.md b/.github/instructions/javascript_xsjs_ql.instructions.md new file mode 100644 index 000000000..d3e5e6536 --- /dev/null +++ b/.github/instructions/javascript_xsjs_ql.instructions.md @@ -0,0 +1,71 @@ +--- +applyTo: 'javascript/frameworks/xsjs/**/*.ql,javascript/frameworks/xsjs/**/*.qll' +description: 'Instructions for CodeQL queries and library modeling for SAP XSJS framework.' +--- + +# Copilot Instructions for XSJS Framework CodeQL Files + +## PURPOSE + +This file contains instructions for working with CodeQL query (`.ql`) and library (`.qll`) files for the SAP XSJS (XS JavaScript) framework in the `javascript/frameworks/xsjs/` directory. + +## REQUIREMENTS + +### COMMON REQUIREMENTS + +- ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS run `codeql query format --in-place ` before committing changes to QL files. +- ALWAYS use `codeql test run` to validate query changes before committing. +- ALWAYS use `codeql test extract` to create test databases for AST exploration. +- ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. +- NEVER make assumptions about XSJS framework behavior - validate with real code and CodeQL CLI. +- NEVER commit query changes without passing tests. + +### QUERY DEVELOPMENT REQUIREMENTS + +- ALWAYS include proper metadata (@name, @description, @kind, @id, @tags). +- ALWAYS import only necessary predicates and classes from the standard library. +- ALWAYS use meaningful predicate and class names that reflect their purpose. +- ALWAYS document complex logic with clear comments. +- ALWAYS alphabetically order imports from the CodeQL standard library. + +### LIBRARY DEVELOPMENT REQUIREMENTS + +- ALWAYS model XSJS-specific patterns accurately: + - $.request and $.response objects + - Database connection handling + - XSJS-specific APIs and libraries + - Remote flow sources from HTTP requests + - SQL injection vulnerabilities in database queries + - XSS vulnerabilities in response writing + - Path injection in file operations +- ALWAYS extend appropriate CodeQL standard library classes. +- ALWAYS provide source type information for remote flow sources. + +### TESTING REQUIREMENTS + +- ALWAYS create comprehensive test cases in `javascript/frameworks/xsjs/test/`. +- ALWAYS include realistic XSJS code patterns in tests. +- ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. + +## PREFERENCES + +- PREFER using CodeQL's standard library classes and predicates over custom implementations. +- PREFER precise modeling that minimizes false positives. +- PREFER test cases that represent real-world XSJS usage patterns. +- PREFER to explore AST with PrintAST queries before implementing new models. + +## CONSTRAINTS + +- NEVER modify XSJS framework source code - only model it in CodeQL. +- NEVER skip test validation. +- NEVER commit without formatting QL files. +- NEVER assume XSJS patterns without CodeQL CLI validation. + +## RELATED PROMPTS + +For detailed guidance on XSJS framework development tasks, refer to: +- `.github/prompts/xsjs_framework_development.prompt.md` - Comprehensive XSJS modeling guide +- `.github/prompts/test_driven_ql_development.prompt.md` - TDD best practices for QL +- `.github/prompts/cli_resources.prompt.md` - CodeQL CLI command reference diff --git a/.github/prompts/cap_framework_development.prompt.md b/.github/prompts/cap_framework_development.prompt.md new file mode 100644 index 000000000..06115ad56 --- /dev/null +++ b/.github/prompts/cap_framework_development.prompt.md @@ -0,0 +1,180 @@ +# CAP Framework Development Prompt + +This prompt provides comprehensive guidance for developing CodeQL queries and library models for the SAP Cloud Application Programming (CAP) framework. + +## Overview + +CAP is a framework for building enterprise-grade services and applications. This prompt helps you model CAP-specific patterns in CodeQL to detect security vulnerabilities. + +## CAP Framework Basics + +### Key Concepts + +1. **CDS (Core Data Services)**: Domain modeling language +2. **Event Handlers**: Functions that handle service events (CREATE, READ, UPDATE, DELETE) +3. **Service Definitions**: Define service interfaces in .cds files +4. **Service Implementations**: JavaScript/TypeScript code implementing service logic + +### Common Patterns + +```javascript +// Event handler registration +srv.on('READ', 'Books', async (req) => { + // req is a remote flow source + const query = req.data.query; // Potentially tainted +}); + +// Alternative handler registration +srv.before('CREATE', 'Books', async (req) => { + // Pre-processing logic +}); + +srv.after('READ', 'Books', async (data, req) => { + // Post-processing logic +}); +``` + +## CodeQL Modeling Workflow + +### 1. Understand the Pattern + +Before modeling, understand how the pattern works: + +```bash +# Extract a test database +codeql test extract javascript/frameworks/cap/test/example + +# Run PrintAST query to see structure +codeql query run PrintAST.ql --database javascript/frameworks/cap/test/example +``` + +### 2. Create Test Cases + +Always create tests first: + +```javascript +// javascript/frameworks/cap/test/sql-injection/test.js +const cds = require('@sap/cds'); + +module.exports = async (srv) => { + srv.on('READ', 'Books', async (req) => { + const userInput = req.data.title; // Source + await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); // Sink + }); +}; +``` + +Expected results: +``` +// javascript/frameworks/cap/test/sql-injection/sql-injection.expected +| test.js:5:12:5:59 | ... + ... | test.js:4:23:4:37 | req.data.title | This query depends on a $@. | test.js:4:23:4:37 | user-provided value | user-provided value | +``` + +### 3. Implement Remote Flow Sources + +Model sources in `lib/.../RemoteFlowSources.qll`: + +```ql +private import javascript + +class CapEventHandlerParameter extends RemoteFlowSource { + CapEventHandlerParameter() { + exists(CallExpr call, Function handler | + call.getCallee().(PropAccess).getPropertyName() = "on" and + handler = call.getArgument(2) and + this = handler.getParameter(0) + ) + } + + override string getSourceType() { + result = "CAP event handler request parameter" + } +} +``` + +### 4. Model Additional Taint Steps + +If needed, add taint steps in `lib/.../dataflow/FlowSteps.qll`: + +```ql +private class CapRequestDataStep extends TaintTracking::AdditionalTaintStep { + override predicate step(DataFlow::Node pred, DataFlow::Node succ) { + exists(PropAccess access | + access.getBase() = pred.asExpr() and + access.getPropertyName() = "data" and + succ.asExpr() = access + ) + } +} +``` + +### 5. Test the Model + +```bash +# Run tests +codeql test run javascript/frameworks/cap/test/sql-injection + +# If results differ from expected, review them +codeql bqrs decode javascript/frameworks/cap/test/sql-injection/.../results.bqrs --format=text + +# Accept if correct +codeql test accept javascript/frameworks/cap/test/sql-injection +``` + +## Common CAP Patterns to Model + +### Event Handler Registrations + +```ql +class CapServiceEventHandler extends CallExpr { + CapServiceEventHandler() { + this.getCallee().(PropAccess).getPropertyName() in ["on", "before", "after"] + } + + Function getHandler() { + result = this.getArgument(2) + } + + string getEventType() { + result = this.getArgument(0).getStringValue() + } +} +``` + +### CDS Service References + +```ql +class CdsServiceReference extends Expr { + CdsServiceReference() { + exists(CallExpr require | + require.getCallee().getName() = "require" and + require.getArgument(0).getStringValue() = "@sap/cds" and + this = require + ) + } +} +``` + +## Testing Best Practices + +1. **Realistic test cases**: Use actual CAP code patterns +2. **Both positive and negative cases**: Test what should and shouldn't alert +3. **Include CDS files**: When relevant to the pattern +4. **Document expected behavior**: Comment in test files why something should alert + +## Validation Checklist + +Before committing: + +- [ ] Tests created in `javascript/frameworks/cap/test/` +- [ ] Tests pass: `codeql test run` +- [ ] Query formatted: `codeql query format --in-place` +- [ ] Query compiles: `codeql query compile` +- [ ] Expected results verified and accepted +- [ ] Documentation updated if needed + +## Related Resources + +- SAP CAP documentation: https://cap.cloud.sap/docs/ +- CodeQL JavaScript library: https://codeql.github.com/codeql-standard-libraries/javascript/ +- CodeQL taint tracking: https://codeql.github.com/docs/writing-codeql-queries/about-data-flow-analysis/ diff --git a/.github/prompts/cds_extractor_development.prompt.md b/.github/prompts/cds_extractor_development.prompt.md new file mode 100644 index 000000000..3e9f98181 --- /dev/null +++ b/.github/prompts/cds_extractor_development.prompt.md @@ -0,0 +1,196 @@ +# CDS Extractor Development Prompt + +This prompt provides comprehensive guidance for developing and maintaining the CodeQL CDS (Core Data Services) extractor TypeScript implementation. + +## Overview + +The CDS extractor is a TypeScript-based tool that integrates with CodeQL's JavaScript extractor to analyze CDS files in SAP CAP projects. It compiles CDS files to JavaScript and ensures proper extraction for CodeQL analysis. + +## Development Workflow + +### 1. Understanding the Architecture + +The CDS extractor follows a modular architecture: + +``` +extractors/cds/tools/ +├── cds-extractor.ts # Main entry point (orchestration only) +├── src/ +│ ├── cds/ +│ │ ├── compiler/ # CDS compilation logic +│ │ └── parser/ # CDS parsing logic +│ ├── logging/ # Unified logging and diagnostics +│ ├── packageManager/ # npm dependency management +│ ├── codeql.ts # CodeQL extractor integration +│ ├── diagnostics.ts # Error reporting +│ └── environment.ts # Environment setup +├── test/ +│ └── src/ # Unit tests mirroring src/ structure +└── dist/ # Compiled outputs (auto-generated) +``` + +### 2. Test-Driven Development + +Always follow TDD: + +1. **Write test first**: Create test in `test/src/**/*.test.ts` +2. **Run test (should fail)**: `npm test` +3. **Implement feature**: Update code in `src/` +4. **Run test (should pass)**: `npm test` +5. **Refactor**: Clean up code while keeping tests passing +6. **Build**: Run `npm run build:all` to ensure everything passes + +### 3. Common Tasks + +#### Adding New Functionality + +```bash +# 1. Create test file +touch extractors/cds/tools/test/src/newfeature/newfeature.test.ts + +# 2. Write failing test +# (edit test file) + +# 3. Run tests to confirm failure +cd extractors/cds/tools +npm test + +# 4. Implement feature +touch extractors/cds/tools/src/newfeature.ts + +# 5. Run tests until passing +npm test + +# 6. Build and validate +npm run build:all +``` + +#### Fixing a Bug + +```bash +# 1. Write test that reproduces bug +# (edit or create test file) + +# 2. Confirm test fails +npm test + +# 3. Fix bug in source +# (edit source file) + +# 4. Confirm test passes +npm test + +# 5. Build and validate +npm run build:all +``` + +#### Upgrading Dependencies + +```bash +cd extractors/cds/tools + +# 1. Check for outdated packages +npm outdated + +# 2. Check for security issues +npm audit + +# 3. Update specific package +npm install @latest + +# 4. Run all tests +npm run build:all + +# 5. Commit if successful +git add package.json package-lock.json +git commit -m "Upgrade to " +``` + +### 4. Error Handling Best Practices + +The CDS extractor should never exit with non-zero code during extraction. Instead: + +```typescript +try { + await processCdsFile(file); +} catch (error) { + // Report diagnostic instead of throwing + diagnostics.reportError( + getRelativePath(sourceRoot, file), + `Failed to compile: ${error.message}` + ); + // Continue processing other files + return; +} +``` + +### 5. Logging Best Practices + +Always use the unified logging system: + +```typescript +import { logger } from './logging'; + +// Performance tracking +const timer = logger.startTimer('operationName'); +// ... perform operation ... +timer.end(); + +// Logging messages +logger.info('Processing file', { file: filename }); +logger.warn('Unexpected condition', { details }); +logger.error('Operation failed', { error: error.message }); +``` + +### 6. Testing Best Practices + +Use Jest with TypeScript: + +```typescript +import { mockFs } from 'mock-fs'; +import { MyClass } from '../../src/mymodule'; + +describe('MyClass', () => { + beforeEach(() => { + // Setup + mockFs({ + 'test.txt': 'content' + }); + }); + + afterEach(() => { + // Cleanup + mockFs.restore(); + }); + + it('should perform expected operation', async () => { + // Arrange + const instance = new MyClass(); + + // Act + const result = await instance.operation(); + + // Assert + expect(result).toBe(expectedValue); + }); +}); +``` + +## Validation Checklist + +Before committing any changes: + +- [ ] All tests pass: `npm test` +- [ ] Linting passes: `npm run lint:fix` +- [ ] Build succeeds: `npm run build` +- [ ] Bundle validation passes: `npm run build:all` +- [ ] No trailing whitespace in any file +- [ ] Test coverage maintained or improved +- [ ] Documentation updated if needed + +## Related Resources + +- Jest documentation: https://jestjs.io/ +- TypeScript documentation: https://www.typescriptlang.org/ +- SAP CDS documentation: https://cap.cloud.sap/docs/cds/ +- CodeQL extractor documentation: https://codeql.github.com/docs/codeql-cli/extractor-options/ diff --git a/.github/prompts/dependency_upgrade.prompt.md b/.github/prompts/dependency_upgrade.prompt.md new file mode 100644 index 000000000..cf48b0249 --- /dev/null +++ b/.github/prompts/dependency_upgrade.prompt.md @@ -0,0 +1,195 @@ +# Dependency Upgrade Prompt + +This prompt provides comprehensive guidance for upgrading dependencies in the codeql-sap-js repository. + +## Overview + +Regular dependency upgrades are essential for security, bug fixes, and new features. This guide covers upgrading: + +1. Node.js dependencies in the CDS extractor +2. CodeQL CLI versions +3. QLT (CodeQL Testing) CLI versions +4. GitHub Actions versions + +## Node.js Dependency Upgrades + +### Workflow + +```bash +cd extractors/cds/tools + +# 1. Check for outdated packages +npm outdated + +# 2. Check for security vulnerabilities +npm audit + +# 3. Review each outdated package +# - Check release notes for breaking changes +# - Determine if upgrade is needed + +# 4. Upgrade specific package +npm install @ + +# 5. Run all tests +npm run build:all + +# 6. If tests pass, commit +git add package.json package-lock.json +git commit -m "Upgrade from to " +``` + +### Security Vulnerabilities + +If `npm audit` shows vulnerabilities: + +```bash +# Review audit report +npm audit + +# Try automatic fix +npm audit fix + +# For breaking changes that can't be auto-fixed +npm audit fix --force # Use with caution + +# Run tests +npm run build:all + +# If tests fail, investigate and fix +``` + +### Major Version Upgrades + +For major version upgrades: + +1. Read migration guides carefully +2. Check for breaking changes +3. Update code if needed +4. Run comprehensive tests +5. Consider creating a separate PR for large upgrades + +## CodeQL/QLT CLI Upgrades + +### Check Latest Versions + +```bash +# Check current versions +codeql version +qlt --version + +# Check latest releases on GitHub +# CodeQL: https://github.com/github/codeql-cli-binaries/releases +# QLT: Check internal sources +``` + +### Update Workflows + +Update version in `.github/workflows/*.yml`: + +```yaml +# Before +- uses: github/codeql-action/init@v2 + with: + codeql-version: 2.14.6 + +# After +- uses: github/codeql-action/init@v3 + with: + codeql-version: 2.15.4 +``` + +### Validation + +1. Create PR with version update +2. Monitor all workflow runs +3. Ensure CodeQL tests pass +4. Ensure CDS extractor tests pass +5. Merge if all checks pass + +## GitHub Actions Upgrades + +### Check for Updates + +```bash +# Check Dependabot alerts +# Navigate to repository → Security → Dependabot alerts + +# Or manually check action repositories +# Example: https://github.com/actions/checkout/releases +``` + +### Update Actions + +```yaml +# Before +- uses: actions/checkout@v3 + +# After +- uses: actions/checkout@v4 +``` + +### Common Actions to Monitor + +- `actions/checkout` +- `actions/setup-node` +- `github/codeql-action/*` +- `actions/upload-artifact` +- `actions/download-artifact` + +## PR Best Practices + +### PR Title Format + +``` +Upgrade : from to +``` + +Examples: +- `Upgrade Node.js dependencies: @sap/cds from 7.4.0 to 7.5.0` +- `Upgrade CodeQL CLI from 2.14.6 to 2.15.4` +- `Upgrade GitHub Actions: checkout from v3 to v4` + +### PR Description Template + +```markdown +## Dependency Upgrade + +### Category +[Node.js / CodeQL / QLT / GitHub Actions] + +### Changes +- Package: `` +- Old version: `` +- New version: `` + +### Reason for Upgrade +[Security fix / Bug fix / New features / Routine maintenance] + +### Breaking Changes +[None / List breaking changes and migration steps] + +### Testing +- [ ] All tests pass +- [ ] No new security vulnerabilities +- [ ] Workflows function correctly (for CLI/Actions upgrades) + +### Release Notes +[Link to release notes or changelog] +``` + +## Validation Checklist + +Before committing: + +- [ ] Tests pass locally +- [ ] No new security vulnerabilities +- [ ] Breaking changes documented +- [ ] PR description complete +- [ ] Appropriate labels applied + +## Related Resources + +- npm documentation: https://docs.npmjs.com/ +- CodeQL releases: https://github.com/github/codeql-cli-binaries/releases +- GitHub Actions marketplace: https://github.com/marketplace?type=actions diff --git a/.github/prompts/github_maintenance.prompt.md b/.github/prompts/github_maintenance.prompt.md new file mode 100644 index 000000000..3c86d0a20 --- /dev/null +++ b/.github/prompts/github_maintenance.prompt.md @@ -0,0 +1,223 @@ +# GitHub Maintenance Prompt + +This prompt provides guidance for maintaining the `.github/` infrastructure that enables agentic development. + +## Overview + +The `.github/` directory contains the infrastructure for agentic maintenance: +- Agents: Specialized AI assistants for different domains +- Instructions: High-level rules sent with every request +- Prompts: Detailed task-specific guidance +- Issue Templates: Entry points for agent workflows +- PR Templates: Standardized pull request descriptions +- Workflows: CI/CD automation + +## Maintenance Workflow + +### Adding a New Agent + +1. **Create agent file**: `.github/agents/.md` + +```markdown +--- +name: 'Agent Display Name' +description: 'One-line description of agent expertise' +--- + +# Agent Display Name + +My `agent-name`: + +- Specializes in [domain] +- Obeys [instructions] +- Utilizes [prompts] +- [Key responsibilities] + +## Commands + +[Executable commands here] + +## Testing + +[Testing approach] + +## Code Style + +[Style guidelines] + +## Boundaries + +[What agent must not do] + +## Examples + +[Code examples] +``` + +2. **Create instruction file**: `.github/instructions/.instructions.md` + +```markdown +--- +applyTo: '' +description: 'Brief description' +--- + +# Instructions Title + +## PURPOSE +## REQUIREMENTS +## PREFERENCES +## CONSTRAINTS +## RELATED PROMPTS +``` + +3. **Create prompt file**: `.github/prompts/.prompt.md` + +```markdown +# Task Prompt + +## Overview +## Workflow +## Examples +## Validation Checklist +## Related Resources +``` + +4. **Create issue template**: `.github/ISSUE_TEMPLATE/.yml` + +```yaml +name: Task Name +description: Task description +title: "[AGENT]: " +labels: ["agent-task", "domain-label"] +body: + - type: markdown + attributes: + value: | + This issue will be handled by the `agent-name` agent. + - type: textarea + id: description + attributes: + label: Task Description + validations: + required: true +``` + +5. **Update PROMPTS.md**: Add nodes and connections to mermaid diagram + +### Updating an Existing Agent + +1. Update the agent file with new capabilities +2. Update related instruction files if needed +3. Update related prompt files if needed +4. Update PROMPTS.md diagram if structure changed +5. Test agent behavior with sample task + +### Modifying Workflows + +When updating `.github/workflows/*.yml`: + +```bash +# Validate syntax +actionlint .github/workflows/*.yml + +# Or use yamllint +yamllint .github/workflows/*.yml + +# Test in PR before merging +``` + +### Maintaining PROMPTS.md + +The mermaid diagram shows the hierarchy: + +```mermaid +graph TD; + ISSUE[Issue Template] + INSTRUCTION[Instruction File] + PROMPT[Prompt File] + RESOURCE[Resource/Tool Doc] + + ISSUE --> INSTRUCTION + INSTRUCTION --> PROMPT + PROMPT --> RESOURCE +``` + +When adding files, update the diagram to include: +- New nodes for files +- Connections showing dependencies +- Grouping for related components + +## Best Practices + +### Agent Definitions + +1. **Be Specific**: Clear, narrow scope +2. **Commands First**: Executable commands at top +3. **Show Examples**: Real code, not prose +4. **Define Boundaries**: What NOT to do +5. **Keep Focused**: One domain per agent + +### Instruction Files + +1. **Concise**: Sent with every request +2. **High-level**: Rules, not procedures +3. **Link to Prompts**: Reference detailed guides +4. **Organized**: Consistent structure + +### Prompt Files + +1. **Detailed**: Step-by-step procedures +2. **Actionable**: Focus on tasks +3. **Examples**: Include concrete examples +4. **Resources**: Link to external docs + +### Issue Templates + +1. **YAML Format**: Use .yml extension +2. **Clear Purpose**: What agent handles it +3. **Required Fields**: Ensure necessary info +4. **Labels**: For organization + +## Validation Checklist + +Before committing: + +- [ ] Agent file has proper frontmatter +- [ ] Instruction file has applyTo pattern +- [ ] Prompt file is task-focused +- [ ] Issue template YAML is valid +- [ ] PROMPTS.md diagram updated +- [ ] No broken internal links +- [ ] No duplicate content across files + +## Testing + +### Test Agent Files + +1. Read through as AI would +2. Check commands are executable +3. Verify examples are realistic +4. Ensure boundaries are clear + +### Test Issue Templates + +```bash +# Create issue from template (requires gh CLI) +gh issue create --web + +# Select template and verify rendering +``` + +### Test Workflows + +1. Create PR with changes +2. Monitor workflow runs +3. Check for failures +4. Verify expected behavior + +## Related Resources + +- Agents.md guide: https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/ +- GitHub Actions syntax: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions +- Mermaid diagram syntax: https://mermaid-js.github.io/mermaid/ diff --git a/.github/prompts/ui5_framework_development.prompt.md b/.github/prompts/ui5_framework_development.prompt.md new file mode 100644 index 000000000..d0f8ac701 --- /dev/null +++ b/.github/prompts/ui5_framework_development.prompt.md @@ -0,0 +1,92 @@ +# UI5 Framework Development Prompt + +This prompt provides guidance for developing CodeQL queries and library models for the SAPUI5 framework. + +## Overview + +SAPUI5 is a JavaScript framework for building enterprise web applications with MVC architecture. This prompt helps model UI5-specific security patterns. + +## UI5 Framework Basics + +### Key Concepts + +1. **MVC Architecture**: Views (XML/JS/HTML), Controllers (JS), Models (JSON/OData/XML) +2. **Data Binding**: Declarative binding in XML views using `{...}` syntax +3. **Controls**: UI components like sap.m.Button, sap.m.Input +4. **Routing**: Navigation between views with parameters + +### Common Vulnerability Patterns + +1. **XSS in View Rendering**: Unsafe HTML content in controls +2. **Path Injection**: User-controlled resource paths +3. **Formula Injection**: Unsafe data in Excel exports +4. **Log Injection**: Unvalidated data in logging + +## CodeQL Modeling Workflow + +### 1. Model Remote Flow Sources + +```ql +class UI5RouteParameter extends RemoteFlowSource { + UI5RouteParameter() { + exists(MethodCallExpr route | + route.getMethodName() = "attachRouteMatched" and + this = route.getArgument(0).(Function).getParameter(0) + ) + } + + override string getSourceType() { + result = "UI5 route parameter" + } +} +``` + +### 2. Model View Bindings + +UI5 XML views contain binding expressions that need special parsing: + +```xml + + +``` + +The `BindingStringParser.qll` parses these expressions. + +### 3. Model Sinks + +```ql +class UI5HtmlContentSink extends Sink { + UI5HtmlContentSink() { + exists(MethodCallExpr call | + call.getReceiver().getType().hasQualifiedName("sap.ui.core.HTML") and + call.getMethodName() = "setContent" and + this = call.getArgument(0) + ) + } +} +``` + +## Testing Best Practices + +Include both Controller.js and View.xml files in tests: + +``` +test/xss/ +├── Controller.controller.js +├── View.view.xml +├── xss.ql +└── xss.expected +``` + +## Validation Checklist + +- [ ] Tests include both JS and XML files +- [ ] Binding expression parsing tested +- [ ] Tests pass: `codeql test run` +- [ ] Query formatted and compiled +- [ ] Expected results verified + +## Related Resources + +- SAPUI5 SDK: https://sapui5.hana.ondemand.com/ +- UI5 Data Binding: https://sapui5.hana.ondemand.com/#/topic/68b9644a253741e8a4b9e4279a35c247 diff --git a/.github/prompts/xsjs_framework_development.prompt.md b/.github/prompts/xsjs_framework_development.prompt.md new file mode 100644 index 000000000..c67afcf35 --- /dev/null +++ b/.github/prompts/xsjs_framework_development.prompt.md @@ -0,0 +1,78 @@ +# XSJS Framework Development Prompt + +This prompt provides guidance for developing CodeQL queries and library models for the SAP XSJS (XS JavaScript) framework. + +## Overview + +XSJS is SAP's server-side JavaScript runtime for HANA. This prompt helps model XSJS-specific security patterns. + +## XSJS Framework Basics + +### Key Concepts + +1. **$.request**: HTTP request object with parameters, body, entities +2. **$.response**: HTTP response object for writing output +3. **$.db**: Database connection API +4. **$.session**: Session management + +### Common Vulnerability Patterns + +1. **SQL Injection**: Unsafe queries via $.db.getConnection() +2. **XSS**: Unvalidated output to $.response +3. **Path Injection**: User-controlled file paths + +## CodeQL Modeling Workflow + +### 1. Model Remote Flow Sources + +```ql +class XsjsRequestParameter extends RemoteFlowSource { + XsjsRequestParameter() { + exists(PropAccess access | + access.getBase().(PropAccess).getBase().(GlobalVarAccess).getName() = "$" and + access.getBase().(PropAccess).getPropertyName() = "request" and + access.getPropertyName() in ["parameters", "body", "entities"] + ) + } + + override string getSourceType() { + result = "XSJS request parameter" + } +} +``` + +### 2. Model Sinks + +```ql +class XsjsSqlInjectionSink extends SqlInjection::Sink { + XsjsSqlInjectionSink() { + exists(MethodCallExpr call | + call.getMethodName() = "executeQuery" and + this = call.getArgument(0) + ) + } +} +``` + +## Testing Best Practices + +Create .xsjs files for tests: + +```javascript +// test.xsjs +var userInput = $.request.parameters.get("id"); +var query = "SELECT * FROM users WHERE id = '" + userInput + "'"; +var conn = $.db.getConnection(); +conn.executeQuery(query); +``` + +## Validation Checklist + +- [ ] Tests use .xsjs extension +- [ ] XSJS-specific APIs modeled +- [ ] Tests pass: `codeql test run` +- [ ] Query formatted and compiled + +## Related Resources + +- XSJS API Reference: SAP HANA Developer Guide diff --git a/PROMPTS.md b/PROMPTS.md new file mode 100644 index 000000000..6cddccd7e --- /dev/null +++ b/PROMPTS.md @@ -0,0 +1,253 @@ +# PROMPTS + +This document outlines the hierarchy of instructions, prompts, and other file resources intended for use by AI agents assigned with maintenance and development tasks in the codeql-sap-js repository. + +## Prompts Hierarchy Description + +In order to keep the prompt system organized and efficient, the following hierarchy is established: + +### 1. `.github/ISSUE_TEMPLATE/*.yml` + +- Entry point for AI agent workflows via GitHub Copilot. +- Pre-structured for different development and maintenance scenarios. +- Links to relevant agents, instructions, and prompts. +- Categories: + - CDS Extractor Development + - CAP Framework Modeling + - UI5 Framework Modeling + - XSJS Framework Modeling + - Dependency Upgrades + - GitHub Infrastructure Maintenance + +### 2. `.github/agents/*.md` + +- Specialized AI agent definitions for specific domains. +- Define agent persona, responsibilities, and boundaries. +- Include executable commands and code examples. +- Follow best practices from GitHub's agents.md guide. +- One agent per domain for focused expertise. + +### 3. `.github/instructions/*.instructions.md` + +- Highest level of abstraction in the prompt hierarchy. +- Sent with every request to the AI, so must be concise and clear. +- Define rules, requirements, preferences, and constraints. +- Link to detailed prompts in "RELATED PROMPTS" section. +- Use `applyTo` pattern in frontmatter to target specific file types. + +### 4. `.github/prompts/*.prompt.md` + +- High-level prompts for multi-step development tasks. +- Provides detailed, step-by-step guidance for specific tasks. +- Includes concrete examples and command sequences. +- Task-focused and actionable. +- References external documentation and resources. + +## Prompts Hierarchy Visualization + +The following diagram shows the relationships between agents, instructions, prompts, and issue templates. + +```mermaid +graph TD; + %% Level 1: GitHub Issue Templates + ISSUE_CDS[".github/ISSUE_TEMPLATE/cds-extractor-task.yml"] + ISSUE_CAP[".github/ISSUE_TEMPLATE/cap-modeling-task.yml"] + ISSUE_UI5[".github/ISSUE_TEMPLATE/ui5-modeling-task.yml"] + ISSUE_XSJS[".github/ISSUE_TEMPLATE/xsjs-modeling-task.yml"] + ISSUE_DEPS[".github/ISSUE_TEMPLATE/dependency-upgrade.yml"] + ISSUE_GITHUB[".github/ISSUE_TEMPLATE/github-maintenance.yml"] + + %% Level 1.5: Agent Definitions + AGENT_CDS[".github/agents/cds-extractor-agent.md"] + AGENT_CAP[".github/agents/cap-modeling-agent.md"] + AGENT_UI5[".github/agents/ui5-modeling-agent.md"] + AGENT_XSJS[".github/agents/xsjs-modeling-agent.md"] + AGENT_DEPS[".github/agents/dependency-upgrade-agent.md"] + AGENT_GITHUB[".github/agents/github-maintenance-agent.md"] + + %% Level 2: Instructions + INSTR_CDS[".github/instructions/extractors_cds_tools_ts.instructions.md"] + INSTR_CAP[".github/instructions/javascript_cap_ql.instructions.md"] + INSTR_UI5[".github/instructions/javascript_ui5_ql.instructions.md"] + INSTR_XSJS[".github/instructions/javascript_xsjs_ql.instructions.md"] + INSTR_DEPS[".github/instructions/dependency_upgrades.instructions.md"] + INSTR_GITHUB[".github/instructions/github_maintenance.instructions.md"] + + %% Level 3: Prompts + PROMPT_CDS[".github/prompts/cds_extractor_development.prompt.md"] + PROMPT_CAP[".github/prompts/cap_framework_development.prompt.md"] + PROMPT_UI5[".github/prompts/ui5_framework_development.prompt.md"] + PROMPT_XSJS[".github/prompts/xsjs_framework_development.prompt.md"] + PROMPT_DEPS[".github/prompts/dependency_upgrade.prompt.md"] + PROMPT_GITHUB[".github/prompts/github_maintenance.prompt.md"] + + %% Issue Templates to Agents + ISSUE_CDS --> AGENT_CDS + ISSUE_CAP --> AGENT_CAP + ISSUE_UI5 --> AGENT_UI5 + ISSUE_XSJS --> AGENT_XSJS + ISSUE_DEPS --> AGENT_DEPS + ISSUE_GITHUB --> AGENT_GITHUB + + %% Agents to Instructions + AGENT_CDS --> INSTR_CDS + AGENT_CAP --> INSTR_CAP + AGENT_UI5 --> INSTR_UI5 + AGENT_XSJS --> INSTR_XSJS + AGENT_DEPS --> INSTR_DEPS + AGENT_GITHUB --> INSTR_GITHUB + + %% Instructions to Prompts + INSTR_CDS --> PROMPT_CDS + INSTR_CAP --> PROMPT_CAP + INSTR_UI5 --> PROMPT_UI5 + INSTR_XSJS --> PROMPT_XSJS + INSTR_DEPS --> PROMPT_DEPS + INSTR_GITHUB --> PROMPT_GITHUB + + %% Cross-references for related prompts + INSTR_CAP --> PROMPT_GITHUB + INSTR_UI5 --> PROMPT_GITHUB + INSTR_XSJS --> PROMPT_GITHUB +``` + +## Agent Descriptions + +### CDS Extractor Development Agent + +**File**: `.github/agents/cds-extractor-agent.md` + +**Responsibilities**: +- TypeScript development for CodeQL CDS extractor +- Test-driven development with Jest +- Dependency management for extractor +- Build and bundle validation + +**Key Commands**: +- `npm run build:all` - MANDATORY before commit +- `npm run lint:fix` - Fix linting issues +- `npm test` - Run Jest tests + +### CAP Framework Modeling Agent + +**File**: `.github/agents/cap-modeling-agent.md` + +**Responsibilities**: +- CodeQL query development for SAP CAP framework +- Library modeling for CAP-specific patterns +- Event handler source modeling +- Test case creation and validation + +**Key Commands**: +- `codeql test run` - Run CodeQL tests +- `codeql test extract` - Create test databases +- `codeql query format` - Format queries + +### UI5 Framework Modeling Agent + +**File**: `.github/agents/ui5-modeling-agent.md` + +**Responsibilities**: +- CodeQL query development for SAPUI5 framework +- XML view and binding expression modeling +- XSS, path injection, formula injection detection +- Binding expression parser maintenance + +**Key Commands**: +- `codeql test run` - Run CodeQL tests +- `codeql test extract` - Create test databases +- `codeql query format` - Format queries + +### XSJS Framework Modeling Agent + +**File**: `.github/agents/xsjs-modeling-agent.md` + +**Responsibilities**: +- CodeQL query development for SAP XSJS framework +- $.request and $.response modeling +- SQL injection and XSS detection +- XSJS-specific API modeling + +**Key Commands**: +- `codeql test run` - Run CodeQL tests +- `codeql test extract` - Create test databases +- `codeql query format` - Format queries + +### Dependency Upgrade Agent + +**File**: `.github/agents/dependency-upgrade-agent.md` + +**Responsibilities**: +- Node.js dependency upgrades +- CodeQL/QLT CLI version updates +- GitHub Actions version updates +- Security vulnerability remediation + +**Key Commands**: +- `npm outdated` - Check outdated packages +- `npm audit` - Security audit +- `npm run build:all` - Validate after upgrade + +### GitHub Maintenance Agent + +**File**: `.github/agents/github-maintenance-agent.md` + +**Responsibilities**: +- Maintain agent definitions +- Update instruction and prompt files +- Create and update issue templates +- Update PROMPTS.md (this file) +- Validate workflow syntax + +**Key Commands**: +- `actionlint` - Validate workflow YAML +- `yamllint` - Validate YAML syntax + +## Best Practices + +### For Agent Definitions + +1. **Be Specific**: Clear, narrow scope per agent +2. **Commands First**: Executable commands at the top +3. **Show Examples**: Real code, not just prose +4. **Define Boundaries**: Explicitly state what NOT to do +5. **Keep Focused**: One domain per agent + +### For Instruction Files + +1. **Concise**: Sent with every request, keep brief +2. **High-level**: Rules and requirements, not detailed procedures +3. **Link to Prompts**: Reference detailed guidance +4. **Structured**: Use consistent sections (PURPOSE, REQUIREMENTS, PREFERENCES, CONSTRAINTS, RELATED PROMPTS) + +### For Prompt Files + +1. **Detailed**: Step-by-step procedures +2. **Actionable**: Task-focused, not theoretical +3. **Examples**: Include concrete code examples +4. **Resources**: Link to external documentation + +### For Issue Templates + +1. **YAML Format**: Use .yml extension for structured templates +2. **Link to Agent**: Clearly indicate which agent handles it +3. **Required Fields**: Ensure necessary information is collected +4. **Labels**: Use consistent labeling for organization + +## Updating This Documentation + +When adding or modifying agents, instructions, or prompts: + +1. **Update the mermaid diagram** to reflect new files and relationships +2. **Add agent description** if creating a new agent +3. **Update cross-references** if changing relationships +4. **Validate mermaid syntax** before committing +5. **Ensure consistency** across all levels of the hierarchy + +## Related Resources + +- GitHub Agents.md Guide: https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/ +- CodeQL Documentation: https://codeql.github.com/docs/ +- SAP CAP Documentation: https://cap.cloud.sap/docs/ +- SAPUI5 Documentation: https://sapui5.hana.ondemand.com/ +- Mermaid Diagram Syntax: https://mermaid-js.github.io/mermaid/ From 8de96a0552c261a19bd774b8c2cd2ba152d4e882 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Thu, 20 Nov 2025 18:20:23 +0000 Subject: [PATCH 03/19] Add README for instructions directory Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/instructions/README.md | 147 +++++++++++++++++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 .github/instructions/README.md diff --git a/.github/instructions/README.md b/.github/instructions/README.md new file mode 100644 index 000000000..b5c732bcf --- /dev/null +++ b/.github/instructions/README.md @@ -0,0 +1,147 @@ +# Copilot Instructions + +This directory contains instruction files for GitHub Copilot and other AI coding agents. These instructions are automatically applied when working with specific file types in this repository. + +## Purpose + +Instruction files provide high-level guidance, requirements, preferences, and constraints that AI agents should follow when modifying code. They are: + +- **Concise**: Sent with every AI request, so kept brief and focused +- **High-level**: Define rules and requirements, not detailed procedures +- **File-scoped**: Applied based on `applyTo` pattern in frontmatter +- **Linked**: Reference detailed prompts in `.github/prompts/` for task-specific guidance + +## Structure + +Each instruction file follows this template: + +```markdown +--- +applyTo: '' +description: 'Brief description of what this instruction covers' +--- + +# Copilot Instructions for [Area] + +## PURPOSE +What this instruction file is for + +## REQUIREMENTS +### COMMON REQUIREMENTS +- ALWAYS/NEVER statements for critical rules + +### SPECIFIC REQUIREMENTS +- Domain-specific requirements + +## PREFERENCES +- PREFER statements for recommended approaches + +## CONSTRAINTS +- NEVER statements for forbidden actions + +## RELATED PROMPTS +Links to detailed `.github/prompts/*.prompt.md` files +``` + +## Instruction Files + +### `extractors_cds_tools_ts.instructions.md` + +**Applies to**: `extractors/cds/tools/**/*.ts` + +Provides guidance for TypeScript development in the CodeQL CDS extractor, including build requirements, testing practices, and architectural patterns. + +**Related Agent**: `.github/agents/cds-extractor-agent.md` + +**Related Prompt**: `.github/prompts/cds_extractor_development.prompt.md` + +### `javascript_cap_ql.instructions.md` + +**Applies to**: `javascript/frameworks/cap/**/*.ql`, `javascript/frameworks/cap/**/*.qll` + +Provides guidance for developing CodeQL queries and library models for the SAP CAP framework. + +**Related Agent**: `.github/agents/cap-modeling-agent.md` + +**Related Prompt**: `.github/prompts/cap_framework_development.prompt.md` + +### `javascript_ui5_ql.instructions.md` + +**Applies to**: `javascript/frameworks/ui5/**/*.ql`, `javascript/frameworks/ui5/**/*.qll` + +Provides guidance for developing CodeQL queries and library models for the SAPUI5 framework. + +**Related Agent**: `.github/agents/ui5-modeling-agent.md` + +**Related Prompt**: `.github/prompts/ui5_framework_development.prompt.md` + +### `javascript_xsjs_ql.instructions.md` + +**Applies to**: `javascript/frameworks/xsjs/**/*.ql`, `javascript/frameworks/xsjs/**/*.qll` + +Provides guidance for developing CodeQL queries and library models for the SAP XSJS framework. + +**Related Agent**: `.github/agents/xsjs-modeling-agent.md` + +**Related Prompt**: `.github/prompts/xsjs_framework_development.prompt.md` + +### `dependency_upgrades.instructions.md` + +**Applies to**: `extractors/cds/tools/package.json`, `.github/workflows/*.yml` + +Provides guidance for upgrading dependencies including Node.js packages, CodeQL/QLT CLI versions, and GitHub Actions. + +**Related Agent**: `.github/agents/dependency-upgrade-agent.md` + +**Related Prompt**: `.github/prompts/dependency_upgrade.prompt.md` + +### `github_maintenance.instructions.md` + +**Applies to**: `.github/**/*.md`, `.github/**/*.yml`, `PROMPTS.md` + +Provides guidance for maintaining the GitHub infrastructure including agents, instructions, prompts, templates, and workflows. + +**Related Agent**: `.github/agents/github-maintenance-agent.md` + +**Related Prompt**: `.github/prompts/github_maintenance.prompt.md` + +## How Instructions Are Used + +1. **Automatic Application**: When an AI agent works on a file matching the `applyTo` pattern, the corresponding instruction file is automatically included in the context. + +2. **Hierarchy**: Instructions are part of a larger hierarchy documented in `PROMPTS.md`: + - Level 1: `.github/ISSUE_TEMPLATE/*.yml` - Entry points + - Level 2: `.github/agents/*.md` - Agent definitions + - Level 3: `.github/instructions/*.instructions.md` - High-level rules (this directory) + - Level 4: `.github/prompts/*.prompt.md` - Detailed task guides + +3. **Enforcement**: Instructions define "ALWAYS" and "NEVER" rules that agents should strictly follow. + +4. **Guidance**: Instructions provide "PREFER" recommendations for best practices. + +## Best Practices + +When creating or modifying instruction files: + +1. **Keep them concise** - They're sent with every request +2. **Use clear ALWAYS/NEVER statements** - Make requirements unambiguous +3. **Link to prompts** - Reference detailed guidance in the RELATED PROMPTS section +4. **Maintain structure** - Use the standard template format +5. **Update the hierarchy** - Reflect changes in `PROMPTS.md` + +## Related Documentation + +- **Prompts Hierarchy**: `PROMPTS.md` - Overview of the entire agentic maintenance system +- **Agents**: `.github/agents/` - Specialized AI agent definitions +- **Prompts**: `.github/prompts/` - Detailed task-specific guidance +- **Issue Templates**: `.github/ISSUE_TEMPLATE/` - Entry points for agent workflows + +## Contributing + +When adding new instruction files: + +1. Follow the template structure +2. Define a clear `applyTo` pattern +3. Link to related agents and prompts +4. Update `PROMPTS.md` with the new instruction +5. Test that the `applyTo` pattern matches the intended files From 1b29566c261deeddd02a258b8a2b08e380c8d695 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 17:58:48 +0000 Subject: [PATCH 04/19] Address PR feedback: Add Copilot PR template, improve agent markdown links, enhance framework docs, fix commands, simplify README Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .../PULL_REQUEST_TEMPLATE/copilot-template.md | 34 +++++++ .github/agents/cap-modeling-agent.md | 48 +++++---- .github/agents/cds-extractor-agent.md | 71 +++++--------- .github/agents/dependency-upgrade-agent.md | 7 +- .github/agents/github-maintenance-agent.md | 7 +- .github/agents/ui5-modeling-agent.md | 48 +++++---- .github/agents/xsjs-modeling-agent.md | 48 +++++---- .github/instructions/README.md | 97 ------------------- .../cap_framework_development.prompt.md | 67 +++++++++++-- .../cds_extractor_development.prompt.md | 78 +++++++++++++++ .../prompts/codeql_test_commands.prompt.md | 90 +++++++++++++++++ .../ui5_framework_development.prompt.md | 45 +++++++++ .../xsjs_framework_development.prompt.md | 34 +++++++ 13 files changed, 442 insertions(+), 232 deletions(-) create mode 100644 .github/PULL_REQUEST_TEMPLATE/copilot-template.md create mode 100644 .github/prompts/codeql_test_commands.prompt.md diff --git a/.github/PULL_REQUEST_TEMPLATE/copilot-template.md b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md new file mode 100644 index 000000000..69fbcc7b0 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md @@ -0,0 +1,34 @@ +# Pull Request + +## Agent Information + +**Generated by**: `@copilot` using the [agent-name] agent +**Agent Definition**: [../.github/agents/agent-name.md](../.github/agents/agent-name.md) + +## Changes Summary + + + +## Checklist + +### Agent Compliance +- [ ] **Agent Boundaries Respected**: Changes align with agent's defined scope and constraints +- [ ] **Instructions Followed**: All applicable `.github/instructions/*.instructions.md` requirements met +- [ ] **Related Prompts Used**: Guidance from `.github/prompts/*.prompt.md` applied where relevant + +### Testing & Validation +- [ ] **Tests Passing**: All unit/integration tests pass successfully + - If tests are not passing, explain status and remediation plan below + +### Code Quality +- [ ] **Linting**: Code passes all linting checks +- [ ] **Build**: Code builds successfully (if applicable) +- [ ] **No Regressions**: Existing functionality not broken by changes + +## Test Status + + + +## Additional Notes + + diff --git a/.github/agents/cap-modeling-agent.md b/.github/agents/cap-modeling-agent.md index fba53536c..7b76b891c 100644 --- a/.github/agents/cap-modeling-agent.md +++ b/.github/agents/cap-modeling-agent.md @@ -7,50 +7,46 @@ description: 'Expert in developing CodeQL queries and library models for SAP Clo My `cap-modeling-agent`: -- Specializes in CodeQL query and library development for SAP CAP framework security analysis. -- Obeys all `.github/instructions/javascript_cap_ql.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/cap_framework_development.prompt.md` prompt as the primary guide for CAP modeling tasks. -- Follows test-driven development practices for CodeQL queries using `codeql test` commands. -- Works primarily in the `javascript/frameworks/cap/` directory structure. -- Understands CAP-specific patterns: +- Specializes in CodeQL query and library development for SAP CAP framework security analysis +- Obeys all [CAP framework instructions](../instructions/javascript_cap_ql.instructions.md) +- Utilizes the [CAP framework development prompt](../prompts/cap_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows +- Follows test-driven development practices for CodeQL queries +- Works primarily in the `javascript/frameworks/cap/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands CAP-specific patterns (see [CAP development prompt](../prompts/cap_framework_development.prompt.md) for details): - Event handlers (srv.on, srv.before, srv.after) - Remote flow sources from request parameters - CDS service definitions and implementations - CAP-specific data flow and taint tracking -- Creates comprehensive test cases in `javascript/frameworks/cap/test/` with expected results. -- Uses `codeql test extract` to create test databases for AST exploration. -- Always runs CodeQL tests before committing query changes. -- Never makes assumptions - validates everything with CodeQL CLI. +- Creates comprehensive test cases in `javascript/frameworks/cap/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI ## Commands -CodeQL testing and development: -```bash -# Format QL code -codeql query format --in-place +See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. -# Compile query -codeql query compile - -# Run tests +**Primary workflow:** +```bash +# Run tests (extracts DB and runs query) codeql test run javascript/frameworks/cap/test/ -# Accept test results (after verification) +# Accept results after verification codeql test accept javascript/frameworks/cap/test/ -# Extract test database for AST exploration -codeql test extract javascript/frameworks/cap/test/ - -# Run query against test database -codeql query run --database +# Format query files +codeql query format --in-place ``` +Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference and important notes. + ## Testing +Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. + - Create realistic test cases in `javascript/frameworks/cap/test/` - Each test should have source code and expected results -- Use `codeql test run` to validate query behavior -- Use `codeql test extract` + AST queries to understand code structure +- Use `codeql test run` to validate query behavior (see commands reference) - Test both positive cases (should alert) and negative cases (should not alert) - Update `.expected` files after verifying correctness diff --git a/.github/agents/cds-extractor-agent.md b/.github/agents/cds-extractor-agent.md index ef8b08fe1..bd270ed70 100644 --- a/.github/agents/cds-extractor-agent.md +++ b/.github/agents/cds-extractor-agent.md @@ -7,31 +7,29 @@ description: 'Expert in developing, testing, and maintaining the CodeQL CDS extr My `cds-extractor-agent`: -- Specializes in TypeScript development for CodeQL extractors with deep knowledge of the CDS (Core Data Services) language and SAP CAP framework. -- Obeys all `.github/instructions/extractors_cds_tools_ts.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/cds_extractor_development.prompt.md` prompt as the primary guide for CDS extractor development tasks. -- Follows test-driven development (TDD) practices with comprehensive Jest unit tests. -- Knows the CDS extractor structure: +- Specializes in TypeScript development for CodeQL extractors with deep knowledge of CDS (Core Data Services) language and SAP CAP framework +- Obeys all [CDS extractor instructions](../instructions/extractors_cds_tools_ts.instructions.md) +- Utilizes the [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) as primary guide +- Follows test-driven development (TDD) practices with comprehensive Jest unit tests +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Knows the CDS extractor structure (see [CDS development prompt](../prompts/cds_extractor_development.prompt.md) for details): - `extractors/cds/tools/cds-extractor.ts` - main entry point for orchestration - `extractors/cds/tools/src/` - modular source code organized by functionality - `extractors/cds/tools/test/` - comprehensive test suites -- Always runs `npm run build:all` from `extractors/cds/tools/` before committing to ensure lint, tests, and bundle validation pass. -- Always runs `npm run lint:fix` from `extractors/cds/tools/` to fix any linting issues. -- Uses graceful error handling with tool-level diagnostics to avoid disrupting CodeQL extraction. -- Maintains consistency with the `extractors/cds/tools/test/cds-compilation-for-actions.test.sh` script. -- Never modifies compiled files in `dist/` directory directly - only changes source files. -- Never leaves trailing whitespace on any line. -- Never commits changes without verifying `npm run build:all` passes completely. -- Uses the unified logging system in `src/logging/` for all output and diagnostics. -- Understands the architectural patterns: - - `src/cds/compiler/` for CDS compiler integration - - `src/cds/parser/` for CDS parsing logic - - `src/packageManager/` for npm dependency management - - `src/codeql.ts` for CodeQL JavaScript extractor integration - - `src/environment.ts` for environment validation +- Understands how the extractor is built and tested (see `.github/workflows/cds-extractor-dist-bundle.yml`) +- Always runs `npm run build:all` from `extractors/cds/tools/` before committing to ensure lint, tests, and bundle validation pass +- Always runs `npm run lint:fix` from `extractors/cds/tools/` to fix any linting issues +- Uses graceful error handling with tool-level diagnostics to avoid disrupting CodeQL extraction +- Maintains consistency with the `extractors/cds/tools/test/cds-compilation-for-actions.test.sh` script +- Never modifies compiled files in `dist/` directory directly - only changes source files +- Never leaves trailing whitespace on any line +- Never commits changes without verifying `npm run build:all` passes completely +- Uses the unified logging system in `src/logging/` for all output and diagnostics ## Commands +Refer to [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for complete build and test workflows. + Build and test: ```bash cd extractors/cds/tools @@ -41,8 +39,12 @@ npm test # Run Jest tests npm run test:coverage # Run tests with coverage report ``` +See the [CDS development prompt](../prompts/cds_extractor_development.prompt.md) and `.github/workflows/cds-extractor-dist-bundle.yml` for how the extractor is actually used and tested. + ## Testing +Refer to [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for complete testing approach. + - Write unit tests in `test/src/**/*.test.ts` mirroring the `src/` structure - Follow AAA pattern (Arrange, Act, Assert) - Mock filesystem operations using `mock-fs` @@ -71,29 +73,8 @@ npm run test:coverage # Run tests with coverage report ## Examples -### Example Test Structure -```typescript -describe('CdsCompiler', () => { - it('should compile valid CDS files', async () => { - // Arrange - const mockFs = { 'test.cds': 'service MyService {}' }; - mock(mockFs); - - // Act - const result = await compiler.compile('test.cds'); - - // Assert - expect(result.success).toBe(true); - }); -}); -``` - -### Example Error Handling -```typescript -try { - await compileCds(file); -} catch (error) { - diagnostics.reportError(getRelativePath(file), error.message); - // Continue processing instead of exiting -} -``` +See [CDS extractor development prompt](../prompts/cds_extractor_development.prompt.md) for comprehensive examples of: +- Test structure with Jest and mock-fs +- Error handling with diagnostics +- Logging best practices +- Build and test workflows diff --git a/.github/agents/dependency-upgrade-agent.md b/.github/agents/dependency-upgrade-agent.md index dd7b1cef1..534031683 100644 --- a/.github/agents/dependency-upgrade-agent.md +++ b/.github/agents/dependency-upgrade-agent.md @@ -7,9 +7,10 @@ description: 'Expert in upgrading CodeQL CLI, QLT, Node.js dependencies, and Git My `dependency-upgrade-agent`: -- Specializes in maintaining up-to-date dependencies across the codeql-sap-js repository. -- Obeys all `.github/instructions/dependency_upgrades.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/dependency_upgrade.prompt.md` prompt as the primary guide for upgrade tasks. +- Specializes in maintaining up-to-date dependencies across the codeql-sap-js repository +- Obeys all [dependency upgrade instructions](../instructions/dependency_upgrades.instructions.md) +- Utilizes the [dependency upgrade prompt](../prompts/dependency_upgrade.prompt.md) as primary guide +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests - Manages several categories of dependencies: - CodeQL CLI versions in workflows and scripts - QLT (CodeQL Testing) CLI versions diff --git a/.github/agents/github-maintenance-agent.md b/.github/agents/github-maintenance-agent.md index ad2c1e2a7..b72bbad0e 100644 --- a/.github/agents/github-maintenance-agent.md +++ b/.github/agents/github-maintenance-agent.md @@ -7,9 +7,10 @@ description: 'Expert in maintaining .github infrastructure including workflows, My `github-maintenance-agent`: -- Specializes in maintaining the `.github/` directory infrastructure for agentic development. -- Obeys all `.github/instructions/github_maintenance.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/github_maintenance.prompt.md` prompt as the primary guide. +- Specializes in maintaining the `.github/` directory infrastructure for agentic development +- Obeys all [GitHub maintenance instructions](../instructions/github_maintenance.instructions.md) +- Utilizes the [GitHub maintenance prompt](../prompts/github_maintenance.prompt.md) as primary guide +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests - Maintains the hierarchy of prompts as documented in `PROMPTS.md`. - Updates and creates: - Agent definitions in `.github/agents/*.md` diff --git a/.github/agents/ui5-modeling-agent.md b/.github/agents/ui5-modeling-agent.md index 4b0059f5e..f4a2c1401 100644 --- a/.github/agents/ui5-modeling-agent.md +++ b/.github/agents/ui5-modeling-agent.md @@ -7,12 +7,14 @@ description: 'Expert in developing CodeQL queries and library models for SAPUI5 My `ui5-modeling-agent`: -- Specializes in CodeQL query and library development for SAPUI5 framework security analysis. -- Obeys all `.github/instructions/javascript_ui5_ql.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/ui5_framework_development.prompt.md` prompt as the primary guide for UI5 modeling tasks. -- Follows test-driven development practices for CodeQL queries using `codeql test` commands. -- Works primarily in the `javascript/frameworks/ui5/` directory structure. -- Understands UI5-specific patterns: +- Specializes in CodeQL query and library development for SAPUI5 framework security analysis +- Obeys all [UI5 framework instructions](../instructions/javascript_ui5_ql.instructions.md) +- Utilizes the [UI5 framework development prompt](../prompts/ui5_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows +- Follows test-driven development practices for CodeQL queries +- Works primarily in the `javascript/frameworks/ui5/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands UI5-specific patterns (see [UI5 development prompt](../prompts/ui5_framework_development.prompt.md) for details): - MVC architecture (Views, Controllers, Models) - Data binding expressions and injection risks - UI5 view XML files and control bindings @@ -21,41 +23,35 @@ My `ui5-modeling-agent`: - Path injection in resource loading - Formula injection in data exports - Log injection and unsafe logging -- Creates comprehensive test cases in `javascript/frameworks/ui5/test/` with expected results. -- Uses `codeql test extract` to create test databases for AST exploration. -- Always runs CodeQL tests before committing query changes. -- Never makes assumptions - validates everything with CodeQL CLI. +- Creates comprehensive test cases in `javascript/frameworks/ui5/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI ## Commands -CodeQL testing and development: -```bash -# Format QL code -codeql query format --in-place +See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. -# Compile query -codeql query compile - -# Run tests +**Primary workflow:** +```bash +# Run tests (extracts DB and runs query) codeql test run javascript/frameworks/ui5/test/ -# Accept test results (after verification) +# Accept results after verification codeql test accept javascript/frameworks/ui5/test/ -# Extract test database for AST exploration -codeql test extract javascript/frameworks/ui5/test/ - -# Run query against test database -codeql query run --database +# Format query files +codeql query format --in-place ``` +Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. + ## Testing +Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. + - Create realistic UI5 test cases in `javascript/frameworks/ui5/test/` - Include both JavaScript and XML view files - Each test should have source code and expected results -- Use `codeql test run` to validate query behavior -- Use `codeql test extract` + AST queries to understand UI5 patterns +- Use `codeql test run` to validate query behavior (see commands reference) - Test both positive cases (should alert) and negative cases (should not alert) - Update `.expected` files after verifying correctness diff --git a/.github/agents/xsjs-modeling-agent.md b/.github/agents/xsjs-modeling-agent.md index 5cff6063e..32deb295d 100644 --- a/.github/agents/xsjs-modeling-agent.md +++ b/.github/agents/xsjs-modeling-agent.md @@ -7,12 +7,14 @@ description: 'Expert in developing CodeQL queries and library models for SAP XSJ My `xsjs-modeling-agent`: -- Specializes in CodeQL query and library development for SAP XSJS framework security analysis. -- Obeys all `.github/instructions/javascript_xsjs_ql.instructions.md` instructions from this repository. -- Utilizes the `.github/prompts/xsjs_framework_development.prompt.md` prompt as the primary guide for XSJS modeling tasks. -- Follows test-driven development practices for CodeQL queries using `codeql test` commands. -- Works primarily in the `javascript/frameworks/xsjs/` directory structure. -- Understands XSJS-specific patterns: +- Specializes in CodeQL query and library development for SAP XSJS framework security analysis +- Obeys all [XSJS framework instructions](../instructions/javascript_xsjs_ql.instructions.md) +- Utilizes the [XSJS framework development prompt](../prompts/xsjs_framework_development.prompt.md) as primary guide +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows +- Follows test-driven development practices for CodeQL queries +- Works primarily in the `javascript/frameworks/xsjs/` directory structure +- Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests +- Understands XSJS-specific patterns (see [XSJS development prompt](../prompts/xsjs_framework_development.prompt.md) for details): - $.request and $.response objects - Database connection handling - XSJS-specific APIs and libraries @@ -20,40 +22,34 @@ My `xsjs-modeling-agent`: - SQL injection vulnerabilities in database queries - XSS vulnerabilities in response writing - Path injection in file operations -- Creates comprehensive test cases in `javascript/frameworks/xsjs/test/` with expected results. -- Uses `codeql test extract` to create test databases for AST exploration. -- Always runs CodeQL tests before committing query changes. -- Never makes assumptions - validates everything with CodeQL CLI. +- Creates comprehensive test cases in `javascript/frameworks/xsjs/test/` with expected results +- Never makes assumptions - validates everything with CodeQL CLI ## Commands -CodeQL testing and development: -```bash -# Format QL code -codeql query format --in-place +See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. -# Compile query -codeql query compile - -# Run tests +**Primary workflow:** +```bash +# Run tests (extracts DB and runs query) codeql test run javascript/frameworks/xsjs/test/ -# Accept test results (after verification) +# Accept results after verification codeql test accept javascript/frameworks/xsjs/test/ -# Extract test database for AST exploration -codeql test extract javascript/frameworks/xsjs/test/ - -# Run query against test database -codeql query run --database +# Format query files +codeql query format --in-place ``` +Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. + ## Testing +Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. + - Create realistic XSJS test cases in `javascript/frameworks/xsjs/test/` - Each test should have source code and expected results -- Use `codeql test run` to validate query behavior -- Use `codeql test extract` + AST queries to understand XSJS patterns +- Use `codeql test run` to validate query behavior (see commands reference) - Test both positive cases (should alert) and negative cases (should not alert) - Update `.expected` files after verifying correctness diff --git a/.github/instructions/README.md b/.github/instructions/README.md index b5c732bcf..d76450fc9 100644 --- a/.github/instructions/README.md +++ b/.github/instructions/README.md @@ -24,87 +24,12 @@ description: 'Brief description of what this instruction covers' # Copilot Instructions for [Area] ## PURPOSE -What this instruction file is for - ## REQUIREMENTS -### COMMON REQUIREMENTS -- ALWAYS/NEVER statements for critical rules - -### SPECIFIC REQUIREMENTS -- Domain-specific requirements - ## PREFERENCES -- PREFER statements for recommended approaches - ## CONSTRAINTS -- NEVER statements for forbidden actions - ## RELATED PROMPTS -Links to detailed `.github/prompts/*.prompt.md` files ``` -## Instruction Files - -### `extractors_cds_tools_ts.instructions.md` - -**Applies to**: `extractors/cds/tools/**/*.ts` - -Provides guidance for TypeScript development in the CodeQL CDS extractor, including build requirements, testing practices, and architectural patterns. - -**Related Agent**: `.github/agents/cds-extractor-agent.md` - -**Related Prompt**: `.github/prompts/cds_extractor_development.prompt.md` - -### `javascript_cap_ql.instructions.md` - -**Applies to**: `javascript/frameworks/cap/**/*.ql`, `javascript/frameworks/cap/**/*.qll` - -Provides guidance for developing CodeQL queries and library models for the SAP CAP framework. - -**Related Agent**: `.github/agents/cap-modeling-agent.md` - -**Related Prompt**: `.github/prompts/cap_framework_development.prompt.md` - -### `javascript_ui5_ql.instructions.md` - -**Applies to**: `javascript/frameworks/ui5/**/*.ql`, `javascript/frameworks/ui5/**/*.qll` - -Provides guidance for developing CodeQL queries and library models for the SAPUI5 framework. - -**Related Agent**: `.github/agents/ui5-modeling-agent.md` - -**Related Prompt**: `.github/prompts/ui5_framework_development.prompt.md` - -### `javascript_xsjs_ql.instructions.md` - -**Applies to**: `javascript/frameworks/xsjs/**/*.ql`, `javascript/frameworks/xsjs/**/*.qll` - -Provides guidance for developing CodeQL queries and library models for the SAP XSJS framework. - -**Related Agent**: `.github/agents/xsjs-modeling-agent.md` - -**Related Prompt**: `.github/prompts/xsjs_framework_development.prompt.md` - -### `dependency_upgrades.instructions.md` - -**Applies to**: `extractors/cds/tools/package.json`, `.github/workflows/*.yml` - -Provides guidance for upgrading dependencies including Node.js packages, CodeQL/QLT CLI versions, and GitHub Actions. - -**Related Agent**: `.github/agents/dependency-upgrade-agent.md` - -**Related Prompt**: `.github/prompts/dependency_upgrade.prompt.md` - -### `github_maintenance.instructions.md` - -**Applies to**: `.github/**/*.md`, `.github/**/*.yml`, `PROMPTS.md` - -Provides guidance for maintaining the GitHub infrastructure including agents, instructions, prompts, templates, and workflows. - -**Related Agent**: `.github/agents/github-maintenance-agent.md` - -**Related Prompt**: `.github/prompts/github_maintenance.prompt.md` - ## How Instructions Are Used 1. **Automatic Application**: When an AI agent works on a file matching the `applyTo` pattern, the corresponding instruction file is automatically included in the context. @@ -117,31 +42,9 @@ Provides guidance for maintaining the GitHub infrastructure including agents, in 3. **Enforcement**: Instructions define "ALWAYS" and "NEVER" rules that agents should strictly follow. -4. **Guidance**: Instructions provide "PREFER" recommendations for best practices. - -## Best Practices - -When creating or modifying instruction files: - -1. **Keep them concise** - They're sent with every request -2. **Use clear ALWAYS/NEVER statements** - Make requirements unambiguous -3. **Link to prompts** - Reference detailed guidance in the RELATED PROMPTS section -4. **Maintain structure** - Use the standard template format -5. **Update the hierarchy** - Reflect changes in `PROMPTS.md` - ## Related Documentation - **Prompts Hierarchy**: `PROMPTS.md` - Overview of the entire agentic maintenance system - **Agents**: `.github/agents/` - Specialized AI agent definitions - **Prompts**: `.github/prompts/` - Detailed task-specific guidance - **Issue Templates**: `.github/ISSUE_TEMPLATE/` - Entry points for agent workflows - -## Contributing - -When adding new instruction files: - -1. Follow the template structure -2. Define a clear `applyTo` pattern -3. Link to related agents and prompts -4. Update `PROMPTS.md` with the new instruction -5. Test that the `applyTo` pattern matches the intended files diff --git a/.github/prompts/cap_framework_development.prompt.md b/.github/prompts/cap_framework_development.prompt.md index 06115ad56..81b7bebfb 100644 --- a/.github/prompts/cap_framework_development.prompt.md +++ b/.github/prompts/cap_framework_development.prompt.md @@ -6,6 +6,61 @@ This prompt provides comprehensive guidance for developing CodeQL queries and li CAP is a framework for building enterprise-grade services and applications. This prompt helps you model CAP-specific patterns in CodeQL to detect security vulnerabilities. +## CAP Framework Documentation + +When working with CAP framework modeling, reference these official documentation resources: + +### Core Concepts +- [CAP Best Practices](https://cap.cloud.sap/docs/about/best-practices) - Recommended patterns and approaches +- [CAP Bad Practices](https://cap.cloud.sap/docs/about/bad-practices) - Anti-patterns to avoid +- [Conceptual Definition Language (CDL)](https://cap.cloud.sap/docs/cds/cdl) - Domain modeling language +- [Principles of CDS Models](https://cap.cloud.sap/docs/cds/models) - Model structure and organization + +### CDS Language & Schema +- [CDS Core Schema Notation (CSN)](https://cap.cloud.sap/docs/cds/csn) - Core schema representation +- [CDS Expression Notation (CXN)](https://cap.cloud.sap/docs/cds/cxn) - Expression syntax +- [CDS Core / Built-in Types](https://cap.cloud.sap/docs/cds/types) - Type system +- [CDS Common Reuse Types and Aspects](https://cap.cloud.sap/docs/cds/common) - Reusable components +- [CDS Common Annotations](https://cap.cloud.sap/docs/cds/annotations) - Metadata annotations +- [CDS Compiler Messages](https://cap.cloud.sap/docs/cds/compiler/messages) - Compiler diagnostics +- [CDS Aspect Oriented Modeling](https://cap.cloud.sap/docs/cds/aspects) - Cross-cutting concerns + +### Node.js Runtime +- [CAP Node.js `cds` facade object](https://cap.cloud.sap/docs/node.js/cds-facade) - Main API interface +- [CAP Node.js Best Practices](https://cap.cloud.sap/docs/node.js/best-practices) - Runtime patterns +- [CAP Node.js Authentication](https://cap.cloud.sap/docs/node.js/authentication) - Security guide +- [CAP Node.js Transaction Management](https://cap.cloud.sap/docs/node.js/cds-tx) - Database transactions + +Use these resources to understand CAP/CDS patterns when modeling security vulnerabilities. + +## Agent Goals for CAP Framework Modeling + +When working with CAP framework CodeQL queries and libraries, focus on these primary objectives: + +### 1. Identify and Fix Modeling Gaps +- Review existing CodeQL library modeling in `javascript/frameworks/cap/lib/` +- Identify missing or incomplete models for CAP/CDS APIs, annotations, and patterns +- Extend models to cover additional CAP framework components +- Ensure accurate modeling of data flow through CAP event handlers and services + +### 2. Create and Improve Test Cases +- Develop **COMPLIANT** test cases showing correct/safe CAP usage patterns +- Develop **NON_COMPLIANT** test cases demonstrating security vulnerabilities +- Ensure tests cover realistic CAP application scenarios +- Include both JavaScript service implementations and CDS schema definitions + +### 3. Improve Existing Queries +- Enhance query performance (runtime efficiency) +- Reduce false positives while maintaining detection coverage +- Improve result precision and accuracy +- Only modify queries when specifically requested + +### 4. Write New Queries +- Create queries for distinct problematic CAP/CDS patterns not covered by existing queries +- Focus on security vulnerabilities specific to CAP framework usage +- Ensure queries leverage CAP-specific library models +- Include comprehensive test coverage for new queries + ## CAP Framework Basics ### Key Concepts @@ -41,11 +96,11 @@ srv.after('READ', 'Books', async (data, req) => { Before modeling, understand how the pattern works: ```bash -# Extract a test database -codeql test extract javascript/frameworks/cap/test/example +# Create test database and run query in one command +codeql test run javascript/frameworks/cap/test/example -# Run PrintAST query to see structure -codeql query run PrintAST.ql --database javascript/frameworks/cap/test/example +# View test results +cat javascript/frameworks/cap/test/example/*.expected ``` ### 2. Create Test Cases @@ -111,11 +166,11 @@ private class CapRequestDataStep extends TaintTracking::AdditionalTaintStep { ### 5. Test the Model ```bash -# Run tests +# Run tests (extracts database and runs query) codeql test run javascript/frameworks/cap/test/sql-injection # If results differ from expected, review them -codeql bqrs decode javascript/frameworks/cap/test/sql-injection/.../results.bqrs --format=text +cat javascript/frameworks/cap/test/sql-injection/*.actual # Accept if correct codeql test accept javascript/frameworks/cap/test/sql-injection diff --git a/.github/prompts/cds_extractor_development.prompt.md b/.github/prompts/cds_extractor_development.prompt.md index 3e9f98181..28f29d47a 100644 --- a/.github/prompts/cds_extractor_development.prompt.md +++ b/.github/prompts/cds_extractor_development.prompt.md @@ -6,6 +6,84 @@ This prompt provides comprehensive guidance for developing and maintaining the C The CDS extractor is a TypeScript-based tool that integrates with CodeQL's JavaScript extractor to analyze CDS files in SAP CAP projects. It compiles CDS files to JavaScript and ensures proper extraction for CodeQL analysis. +## CDS Documentation Resources + +When working with the CDS extractor, reference these official documentation resources: + +### Core CDS Concepts +- [CDS Overview](https://cap.cloud.sap/docs/cds/) - Introduction to CDS +- [Conceptual Definition Language (CDL)](https://cap.cloud.sap/docs/cds/cdl) - CDS syntax and grammar +- [CDS Compiler](https://cap.cloud.sap/docs/cds/compiler) - Compiler behavior and options +- [CDS Core Schema Notation (CSN)](https://cap.cloud.sap/docs/cds/csn) - Compiler output format + +### CDS Language Features +- [CDS Types](https://cap.cloud.sap/docs/cds/types) - Built-in types +- [CDS Annotations](https://cap.cloud.sap/docs/cds/annotations) - Metadata annotations +- [CDS Aspects](https://cap.cloud.sap/docs/cds/aspects) - Aspect-oriented modeling +- [CDS Models](https://cap.cloud.sap/docs/cds/models) - Model structure + +Use these resources to understand how CDS files should be parsed and compiled. + +## CDS Extractor Build and Test + +The CDS extractor is built and tested using the workflow defined in `.github/workflows/cds-extractor-dist-bundle.yml`. + +### Build Process + +```bash +cd extractors/cds/tools + +# Install dependencies +npm install + +# Run linting +npm run lint:fix + +# Run tests +npm test + +# Build distribution bundle +npm run build + +# Complete validation (lint + test + build) +npm run build:all +``` + +### Testing Approach + +The extractor uses Jest for unit testing with the following structure: + +``` +extractors/cds/tools/ +├── src/ # Source code +│ ├── cds/ +│ │ ├── compiler/ # CDS compilation logic +│ │ └── parser/ # CDS parsing logic +│ ├── logging/ # Logging utilities +│ ├── packageManager/ # npm dependency management +│ ├── codeql.ts # CodeQL integration +│ ├── diagnostics.ts # Error reporting +│ └── environment.ts # Environment setup +└── test/ + └── src/ # Unit tests (mirrors src/ structure) + ├── cds/ + ├── logging/ + └── ... +``` + +### Test Execution + +```bash +# Run all tests +npm test + +# Run tests with coverage +npm run test:coverage + +# Run specific test file +npm test -- path/to/test.test.ts +``` + ## Development Workflow ### 1. Understanding the Architecture diff --git a/.github/prompts/codeql_test_commands.prompt.md b/.github/prompts/codeql_test_commands.prompt.md new file mode 100644 index 000000000..030232231 --- /dev/null +++ b/.github/prompts/codeql_test_commands.prompt.md @@ -0,0 +1,90 @@ +# CodeQL Test Commands Reference + +This file provides common CodeQL test commands used across framework modeling agents. + +## Running Tests + +The `codeql test run` command is the primary way to test CodeQL queries. It: +1. Extracts a test database from source code in the test directory +2. Runs the query against the extracted database +3. Compares results to `.expected` files + +```bash +# Run tests for a specific test directory +codeql test run + +# Examples: +codeql test run javascript/frameworks/cap/test/sql-injection +codeql test run javascript/frameworks/ui5/test/xss +codeql test run javascript/frameworks/xsjs/test/sql-injection +``` + +## Accepting Test Results + +After reviewing test results and confirming they are correct: + +```bash +# Accept test results (updates .expected files) +codeql test accept + +# Example: +codeql test accept javascript/frameworks/cap/test/sql-injection +``` + +## Formatting Queries + +Always format QL files before committing: + +```bash +# Format a single query file +codeql query format --in-place + +# Format a library file +codeql query format --in-place +``` + +## Compiling Queries + +Verify query syntax: + +```bash +# Compile query to check for errors +codeql query compile +``` + +## Viewing Test Results + +```bash +# View actual test results +cat /*.actual + +# View expected results +cat /*.expected + +# Compare differences +diff /*.expected /*.actual +``` + +## Common Workflow + +```bash +# 1. Create test case files in test directory +# 2. Run tests +codeql test run + +# 3. Review results +cat /*.actual + +# 4. If correct, accept +codeql test accept + +# 5. Format query files +codeql query format --in-place +``` + +## Important Notes + +- **Do NOT use** `codeql test extract` in normal workflow - `codeql test run` handles extraction +- **Do NOT use** `codeql query run` for testing - use `codeql test run` instead +- Tests automatically handle database extraction, query execution, and result comparison +- Always review `.actual` files before accepting with `codeql test accept` diff --git a/.github/prompts/ui5_framework_development.prompt.md b/.github/prompts/ui5_framework_development.prompt.md index d0f8ac701..22c500a4c 100644 --- a/.github/prompts/ui5_framework_development.prompt.md +++ b/.github/prompts/ui5_framework_development.prompt.md @@ -6,6 +6,51 @@ This prompt provides guidance for developing CodeQL queries and library models f SAPUI5 is a JavaScript framework for building enterprise web applications with MVC architecture. This prompt helps model UI5-specific security patterns. +## UI5 Framework Documentation + +When working with UI5 framework modeling, reference these official documentation resources: + +### Core References +- [SAPUI5 SDK API Reference](https://sapui5.hana.ondemand.com/sdk/#/api) - Complete API documentation +- [SAPUI5 Documentation](https://sapui5.hana.ondemand.com/) - Main documentation portal +- [UI5 Developer Guide](https://sapui5.hana.ondemand.com/sdk/#/topic) - Development topics and guides + +### Key Topics for Security Modeling +- [Data Binding](https://sapui5.hana.ondemand.com/#/topic/68b9644a253741e8a4b9e4279a35c247) - Data binding concepts and syntax +- [XML Views](https://sapui5.hana.ondemand.com/#/topic/2d3eb2f322ea4a82983c1c62a33ec4ae) - Declarative view definitions +- [Controllers](https://sapui5.hana.ondemand.com/#/topic/121b8e6337d147af9819129e428f1f75) - Controller implementation +- [Models](https://sapui5.hana.ondemand.com/#/topic/e1b625940c104b558e52f47afe5ddb4f) - Data models (JSON, OData, XML) +- [Routing and Navigation](https://sapui5.hana.ondemand.com/#/topic/3d18f20bd2294228acb6910d8e8a5fb5) - App navigation patterns +- [Security Guidelines](https://sapui5.hana.ondemand.com/#/topic/91f3768f6f4d1014b6dd926db0e91070) - Security best practices + +### UI5 Controls Documentation +- [sap.m Controls](https://sapui5.hana.ondemand.com/sdk/#/api/sap.m) - Mobile controls library +- [sap.ui.core](https://sapui5.hana.ondemand.com/sdk/#/api/sap.ui.core) - Core UI5 functionality +- [sap.ui.table](https://sapui5.hana.ondemand.com/sdk/#/api/sap.ui.table) - Table controls + +Use these resources to understand UI5 patterns when modeling security vulnerabilities. + +## Agent Goals for UI5 Framework Modeling + +Focus on security vulnerabilities specific to UI5: + +### 1. XSS Vulnerabilities +- Model unsafe HTML content rendering in controls +- Track tainted data through data binding expressions +- Identify unsafe use of `sap.ui.core.HTML` and similar controls + +### 2. Path Injection +- Model user-controlled resource loading paths +- Track tainted paths in component and view loading + +### 3. Formula Injection +- Identify unsafe data in Excel/CSV exports +- Model data export control usage patterns + +### 4. Data Binding Expression Injection +- Parse and analyze binding expressions in XML views +- Identify injection risks in binding syntax + ## UI5 Framework Basics ### Key Concepts diff --git a/.github/prompts/xsjs_framework_development.prompt.md b/.github/prompts/xsjs_framework_development.prompt.md index c67afcf35..5150bccc2 100644 --- a/.github/prompts/xsjs_framework_development.prompt.md +++ b/.github/prompts/xsjs_framework_development.prompt.md @@ -6,6 +6,40 @@ This prompt provides guidance for developing CodeQL queries and library models f XSJS is SAP's server-side JavaScript runtime for HANA. This prompt helps model XSJS-specific security patterns. +## XSJS Framework Documentation + +When working with XSJS framework modeling, reference these official documentation resources: + +### Core References +- [SAP HANA XS JavaScript Reference](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/b907648a90cd49caabb30dc2d5a7de05.html) - Main API reference +- [XS JavaScript API Documentation](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/2b62a71581794f7ebc782da49d2c8d38.html) - Complete API docs + +### Key APIs for Security Modeling +- [$.request Object](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/cd7dcc4a7e3d4f06b7e8f1f6f3e0f34f.html) - HTTP request handling +- [$.response Object](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/cd7dcc4a7e3d4f06b7e8f1f6f3e0f34f.html) - HTTP response writing +- [$.db Connection](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/cd7dcc4a7e3d4f06b7e8f1f6f3e0f34f.html) - Database access +- [$.session Object](https://help.sap.com/docs/SAP_HANA_PLATFORM/d89d4595fae647eabc14002c0340a999/cd7dcc4a7e3d4f06b7e8f1f6f3e0f34f.html) - Session management + +Use these resources to understand XSJS patterns when modeling security vulnerabilities. + +## Agent Goals for XSJS Framework Modeling + +Focus on security vulnerabilities specific to XSJS: + +### 1. SQL Injection +- Model unsafe database query construction +- Track tainted data from $.request to $.db operations +- Identify parameterized vs concatenated queries + +### 2. XSS Vulnerabilities +- Model unsafe $.response output +- Track tainted data written to HTTP response +- Identify missing output encoding + +### 3. Path Injection +- Model file system operations with user-controlled paths +- Track tainted paths in XSJS library access + ## XSJS Framework Basics ### Key Concepts From b5eb0bf7f7a47ef00debb977926d4f176b32a1b1 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 18:56:34 +0000 Subject: [PATCH 05/19] Fix incorrect codeql test extract references and PR template path Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/PULL_REQUEST_TEMPLATE/copilot-template.md | 2 +- .github/instructions/javascript_cap_ql.instructions.md | 3 +-- .github/instructions/javascript_ui5_ql.instructions.md | 3 +-- .github/instructions/javascript_xsjs_ql.instructions.md | 3 +-- 4 files changed, 4 insertions(+), 7 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE/copilot-template.md b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md index 69fbcc7b0..871363b7e 100644 --- a/.github/PULL_REQUEST_TEMPLATE/copilot-template.md +++ b/.github/PULL_REQUEST_TEMPLATE/copilot-template.md @@ -3,7 +3,7 @@ ## Agent Information **Generated by**: `@copilot` using the [agent-name] agent -**Agent Definition**: [../.github/agents/agent-name.md](../.github/agents/agent-name.md) +**Agent Definition**: [agent-name.md](../agents/agent-name.md) ## Changes Summary diff --git a/.github/instructions/javascript_cap_ql.instructions.md b/.github/instructions/javascript_cap_ql.instructions.md index 98f2e3b16..206acf9d5 100644 --- a/.github/instructions/javascript_cap_ql.instructions.md +++ b/.github/instructions/javascript_cap_ql.instructions.md @@ -16,7 +16,6 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. - ALWAYS run `codeql query format --in-place ` before committing changes to QL files. - ALWAYS use `codeql test run` to validate query changes before committing. -- ALWAYS use `codeql test extract` to create test databases for AST exploration. - ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. - NEVER make assumptions about CAP framework behavior - validate with real code and CodeQL CLI. - NEVER commit query changes without passing tests. @@ -44,7 +43,7 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/cap/test/`. - ALWAYS include both JavaScript and CDS files in tests when relevant. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. +- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. ## PREFERENCES diff --git a/.github/instructions/javascript_ui5_ql.instructions.md b/.github/instructions/javascript_ui5_ql.instructions.md index 5f2356dee..fde223a5f 100644 --- a/.github/instructions/javascript_ui5_ql.instructions.md +++ b/.github/instructions/javascript_ui5_ql.instructions.md @@ -16,7 +16,6 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. - ALWAYS run `codeql query format --in-place ` before committing changes to QL files. - ALWAYS use `codeql test run` to validate query changes before committing. -- ALWAYS use `codeql test extract` to create test databases for AST exploration. - ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. - NEVER make assumptions about UI5 framework behavior - validate with real code and CodeQL CLI. - NEVER commit query changes without passing tests. @@ -49,7 +48,7 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/ui5/test/`. - ALWAYS include both JavaScript controllers and XML views in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. +- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. - ALWAYS test binding expression parsing separately. ## PREFERENCES diff --git a/.github/instructions/javascript_xsjs_ql.instructions.md b/.github/instructions/javascript_xsjs_ql.instructions.md index d3e5e6536..ae360cea3 100644 --- a/.github/instructions/javascript_xsjs_ql.instructions.md +++ b/.github/instructions/javascript_xsjs_ql.instructions.md @@ -16,7 +16,6 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. - ALWAYS run `codeql query format --in-place ` before committing changes to QL files. - ALWAYS use `codeql test run` to validate query changes before committing. -- ALWAYS use `codeql test extract` to create test databases for AST exploration. - ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. - NEVER make assumptions about XSJS framework behavior - validate with real code and CodeQL CLI. - NEVER commit query changes without passing tests. @@ -47,7 +46,7 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/xsjs/test/`. - ALWAYS include realistic XSJS code patterns in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test extract` + PrintAST query to understand unfamiliar patterns. +- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. ## PREFERENCES From a255f05f03a1f3aa3f2cbf87265e482912fe346d Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 18:59:19 +0000 Subject: [PATCH 06/19] Remove all PrintAST and codeql test extract references per new requirement Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/instructions/javascript_cap_ql.instructions.md | 2 -- .github/instructions/javascript_ui5_ql.instructions.md | 2 -- .github/instructions/javascript_xsjs_ql.instructions.md | 2 -- .github/prompts/codeql_test_commands.prompt.md | 1 - 4 files changed, 7 deletions(-) diff --git a/.github/instructions/javascript_cap_ql.instructions.md b/.github/instructions/javascript_cap_ql.instructions.md index 206acf9d5..7ba4a559f 100644 --- a/.github/instructions/javascript_cap_ql.instructions.md +++ b/.github/instructions/javascript_cap_ql.instructions.md @@ -43,14 +43,12 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/cap/test/`. - ALWAYS include both JavaScript and CDS files in tests when relevant. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. ## PREFERENCES - PREFER using CodeQL's standard library classes and predicates over custom implementations. - PREFER precise modeling that minimizes false positives. - PREFER test cases that represent real-world CAP usage patterns. -- PREFER to explore AST with PrintAST queries before implementing new models. ## CONSTRAINTS diff --git a/.github/instructions/javascript_ui5_ql.instructions.md b/.github/instructions/javascript_ui5_ql.instructions.md index fde223a5f..62c40c31e 100644 --- a/.github/instructions/javascript_ui5_ql.instructions.md +++ b/.github/instructions/javascript_ui5_ql.instructions.md @@ -48,7 +48,6 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/ui5/test/`. - ALWAYS include both JavaScript controllers and XML views in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. - ALWAYS test binding expression parsing separately. ## PREFERENCES @@ -56,7 +55,6 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - PREFER using CodeQL's standard library classes and predicates over custom implementations. - PREFER precise modeling that minimizes false positives. - PREFER test cases that represent real-world UI5 usage patterns. -- PREFER to explore AST with PrintAST queries before implementing new models. - PREFER to test binding expression parser changes thoroughly. ## CONSTRAINTS diff --git a/.github/instructions/javascript_xsjs_ql.instructions.md b/.github/instructions/javascript_xsjs_ql.instructions.md index ae360cea3..f43e08ece 100644 --- a/.github/instructions/javascript_xsjs_ql.instructions.md +++ b/.github/instructions/javascript_xsjs_ql.instructions.md @@ -46,14 +46,12 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/xsjs/test/`. - ALWAYS include realistic XSJS code patterns in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. -- ALWAYS use `codeql test run` with a PrintAST query to understand unfamiliar patterns. ## PREFERENCES - PREFER using CodeQL's standard library classes and predicates over custom implementations. - PREFER precise modeling that minimizes false positives. - PREFER test cases that represent real-world XSJS usage patterns. -- PREFER to explore AST with PrintAST queries before implementing new models. ## CONSTRAINTS diff --git a/.github/prompts/codeql_test_commands.prompt.md b/.github/prompts/codeql_test_commands.prompt.md index 030232231..8b3a8f9b7 100644 --- a/.github/prompts/codeql_test_commands.prompt.md +++ b/.github/prompts/codeql_test_commands.prompt.md @@ -84,7 +84,6 @@ codeql query format --in-place ## Important Notes -- **Do NOT use** `codeql test extract` in normal workflow - `codeql test run` handles extraction - **Do NOT use** `codeql query run` for testing - use `codeql test run` instead - Tests automatically handle database extraction, query execution, and result comparison - Always review `.actual` files before accepting with `codeql test accept` From 147ea77a21a95686ad846f8c49c1c20be54696ac Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 19:13:47 +0000 Subject: [PATCH 07/19] Add copilot-setup-steps.yml workflow for agentic maintenance infrastructure Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 96 +++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 .github/workflows/copilot-setup-steps.yml diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml new file mode 100644 index 000000000..59e7b7b84 --- /dev/null +++ b/.github/workflows/copilot-setup-steps.yml @@ -0,0 +1,96 @@ +name: "Copilot Setup Steps" + +on: + # Allow manual testing through the repository's "Actions" tab + workflow_dispatch: {} + # Automatically run the setup steps when an associated workflow is changed. + push: + branches: + - main + paths: + - .github/workflows/copilot-setup-steps.yml + - .github/actions/setup-codeql-environment/action.yml + - qlt.conf.json + pull_request: + branches: + - main + paths: + - .github/workflows/copilot-setup-steps.yml + - .github/actions/setup-codeql-environment/action.yml + - qlt.conf.json + +jobs: + # The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot. + copilot-setup-steps: + permissions: + contents: read + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v5 + + - name: Install QLT + id: install-qlt + uses: advanced-security/codeql-development-toolkit/.github/actions/install-qlt@main + with: + qlt-version: 'latest' + add-to-path: true + + - name: Install CodeQL + id: install-codeql + shell: bash + run: | + echo "Installing CodeQL" + qlt codeql run install + echo "-----------------------------" + echo "CodeQL Home: $QLT_CODEQL_HOME" + echo "CodeQL Binary: $QLT_CODEQL_PATH" + + - name: Verify Versions of Tooling + shell: bash + run: | + echo -e "Checking CodeQL Version:" + $QLT_CODEQL_PATH --version + + echo -e "Checking QLT Version:" + echo "QLT Home: ${{ steps.install-qlt.outputs.qlt-home }}" + qlt version + + - name: Install QL Packs + shell: bash + run: | + echo "Installing QL pack dependencies" + qlt codeql run pack install --no-strict-mode javascript/frameworks/cap + qlt codeql run pack install --no-strict-mode javascript/frameworks/ui5 + qlt codeql run pack install --no-strict-mode javascript/frameworks/xsjs + + - name: Setup Node.js for CDS Extractor + uses: actions/setup-node@v4 + with: + node-version: '20' + cache: 'npm' + cache-dependency-path: extractors/cds/tools/package-lock.json + + - name: Install CDS Extractor Dependencies + shell: bash + working-directory: extractors/cds/tools + run: | + echo "Installing CDS extractor npm dependencies" + npm ci + + - name: Build CDS Extractor + shell: bash + working-directory: extractors/cds/tools + run: | + echo "Building CDS extractor" + npm run build + + - name: Verify Setup Complete + shell: bash + run: | + echo "✅ Setup complete and verified:" + echo " - QLT installed: $(qlt version)" + echo " - CodeQL installed: $($QLT_CODEQL_PATH --version)" + echo " - Node.js: $(node --version)" + echo " - npm: $(npm --version)" + echo " - CDS extractor built successfully" From ba0fbfa0c7a13bf8ec9e34e78bafd978b8339b21 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 19:38:35 +0000 Subject: [PATCH 08/19] Fix copilot-setup-steps workflow to add codeql and qlt to PATH Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 42 +++++++++++++++++++---- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 59e7b7b84..21decd09b 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -45,14 +45,40 @@ jobs: echo "-----------------------------" echo "CodeQL Home: $QLT_CODEQL_HOME" echo "CodeQL Binary: $QLT_CODEQL_PATH" + + # Add CodeQL to PATH for subsequent steps + if [[ -n "$QLT_CODEQL_HOME" ]]; then + echo "$QLT_CODEQL_HOME" >> "$GITHUB_PATH" + echo "✅ Added CodeQL to PATH: $QLT_CODEQL_HOME" + else + echo "❌ QLT_CODEQL_HOME not set" + exit 1 + fi + + - name: Verify CLI Tools in PATH + shell: bash + run: | + echo "Verifying CLI tools are available in PATH:" + + if ! command -v qlt >/dev/null 2>&1; then + echo "❌ qlt not found in PATH" + exit 1 + fi + echo "✅ qlt is available: $(which qlt)" + + if ! command -v codeql >/dev/null 2>&1; then + echo "❌ codeql not found in PATH" + exit 1 + fi + echo "✅ codeql is available: $(which codeql)" - name: Verify Versions of Tooling shell: bash run: | - echo -e "Checking CodeQL Version:" - $QLT_CODEQL_PATH --version + echo "Checking CodeQL Version:" + codeql --version - echo -e "Checking QLT Version:" + echo "Checking QLT Version:" echo "QLT Home: ${{ steps.install-qlt.outputs.qlt-home }}" qlt version @@ -60,9 +86,9 @@ jobs: shell: bash run: | echo "Installing QL pack dependencies" - qlt codeql run pack install --no-strict-mode javascript/frameworks/cap - qlt codeql run pack install --no-strict-mode javascript/frameworks/ui5 - qlt codeql run pack install --no-strict-mode javascript/frameworks/xsjs + codeql pack install --no-strict-mode javascript/frameworks/cap + codeql pack install --no-strict-mode javascript/frameworks/ui5 + codeql pack install --no-strict-mode javascript/frameworks/xsjs - name: Setup Node.js for CDS Extractor uses: actions/setup-node@v4 @@ -90,7 +116,9 @@ jobs: run: | echo "✅ Setup complete and verified:" echo " - QLT installed: $(qlt version)" - echo " - CodeQL installed: $($QLT_CODEQL_PATH --version)" + echo " - QLT in PATH: $(which qlt)" + echo " - CodeQL installed: $(codeql --version | head -1)" + echo " - CodeQL in PATH: $(which codeql)" echo " - Node.js: $(node --version)" echo " - npm: $(npm --version)" echo " - CDS extractor built successfully" From dcfef4fea89d93e226577c20f30e799076a74b7e Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 19:50:42 +0000 Subject: [PATCH 09/19] Fix copilot-setup-steps workflow trigger paths to use existing files Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 21decd09b..7d2906d90 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -3,21 +3,25 @@ name: "Copilot Setup Steps" on: # Allow manual testing through the repository's "Actions" tab workflow_dispatch: {} - # Automatically run the setup steps when an associated workflow is changed. + # Automatically run the setup steps when an associated file is changed. push: branches: - main paths: - .github/workflows/copilot-setup-steps.yml - - .github/actions/setup-codeql-environment/action.yml - qlt.conf.json + - extractors/cds/tools/package.json + - extractors/cds/tools/package-lock.json + - javascript/frameworks/*/qlpack.yml pull_request: branches: - main paths: - .github/workflows/copilot-setup-steps.yml - - .github/actions/setup-codeql-environment/action.yml - qlt.conf.json + - extractors/cds/tools/package.json + - extractors/cds/tools/package-lock.json + - javascript/frameworks/*/qlpack.yml jobs: # The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot. From 5f20a4b00c10c593f841c05fa11dce3cabdfebef Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 19:55:43 +0000 Subject: [PATCH 10/19] Fix copilot-setup-steps workflow to use correct qlt command syntax Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 7d2906d90..856fa4e90 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -44,8 +44,8 @@ jobs: id: install-codeql shell: bash run: | - echo "Installing CodeQL" - qlt codeql run install + echo "Installing CodeQL using configuration from qlt.conf.json" + qlt codeql install echo "-----------------------------" echo "CodeQL Home: $QLT_CODEQL_HOME" echo "CodeQL Binary: $QLT_CODEQL_PATH" From 5a56e5302ea971855207151a9055274175df0e55 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 20:02:35 +0000 Subject: [PATCH 11/19] Fix copilot-setup-steps workflow to use correct qlt codeql run install command Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 856fa4e90..0c010dbb8 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -45,7 +45,7 @@ jobs: shell: bash run: | echo "Installing CodeQL using configuration from qlt.conf.json" - qlt codeql install + qlt codeql run install echo "-----------------------------" echo "CodeQL Home: $QLT_CODEQL_HOME" echo "CodeQL Binary: $QLT_CODEQL_PATH" From 85479aa6f7c6b73914c871d229968328d783cb88 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Fri, 21 Nov 2025 20:54:14 +0000 Subject: [PATCH 12/19] Fix copilot-setup-steps workflow to retrieve CodeQL paths using qlt commands Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com> --- .github/workflows/copilot-setup-steps.yml | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 0c010dbb8..6261d4295 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -47,15 +47,20 @@ jobs: echo "Installing CodeQL using configuration from qlt.conf.json" qlt codeql run install echo "-----------------------------" - echo "CodeQL Home: $QLT_CODEQL_HOME" - echo "CodeQL Binary: $QLT_CODEQL_PATH" + + # Get CodeQL paths from QLT + CODEQL_HOME=$(qlt codeql get home) + CODEQL_PATH=$(qlt codeql get path) + + echo "CodeQL Home: $CODEQL_HOME" + echo "CodeQL Binary: $CODEQL_PATH" # Add CodeQL to PATH for subsequent steps - if [[ -n "$QLT_CODEQL_HOME" ]]; then - echo "$QLT_CODEQL_HOME" >> "$GITHUB_PATH" - echo "✅ Added CodeQL to PATH: $QLT_CODEQL_HOME" + if [[ -n "$CODEQL_HOME" ]]; then + echo "$CODEQL_HOME" >> "$GITHUB_PATH" + echo "✅ Added CodeQL to PATH: $CODEQL_HOME" else - echo "❌ QLT_CODEQL_HOME not set" + echo "❌ CodeQL home path not found" exit 1 fi From efc36129e1d138550e054dfb2551e7c9037f2b43 Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Fri, 21 Nov 2025 14:03:40 -0700 Subject: [PATCH 13/19] Fixes for .github/workflows/** --- .../workflows/cds-extractor-dist-bundle.yml | 16 ++-- .github/workflows/copilot-setup-steps.yml | 78 ++++--------------- 2 files changed, 22 insertions(+), 72 deletions(-) diff --git a/.github/workflows/cds-extractor-dist-bundle.yml b/.github/workflows/cds-extractor-dist-bundle.yml index d71bfbc3e..f5797a10f 100644 --- a/.github/workflows/cds-extractor-dist-bundle.yml +++ b/.github/workflows/cds-extractor-dist-bundle.yml @@ -3,7 +3,7 @@ name: CDS Extractor Distribution Bundle on: push: branches: [ main ] - paths: + paths: - 'extractors/cds/**' pull_request: branches: [ main ] @@ -16,30 +16,30 @@ jobs: bundle-validation: name: CDS extractor bundle validation runs-on: ubuntu-latest - + steps: - name: Checkout repository uses: actions/checkout@v5 - + - name: Setup Node.js uses: actions/setup-node@v6 with: - node-version: '20' cache: 'npm' cache-dependency-path: 'extractors/cds/tools/package-lock.json' - + node-version-file: 'extractors/cds/tools/package.json' + - name: Install node dependencies working-directory: extractors/cds/tools run: npm ci - + - name: Run TS code linter working-directory: extractors/cds/tools run: npm run lint - + - name: Run TS code unit tests with coverage report working-directory: extractors/cds/tools run: npm run test:coverage - + - name: Build and validate the CDS extractor bundle working-directory: extractors/cds/tools run: npm run build:validate diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 6261d4295..c7f812cfc 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -33,65 +33,15 @@ jobs: - name: Checkout repository uses: actions/checkout@v5 - - name: Install QLT - id: install-qlt - uses: advanced-security/codeql-development-toolkit/.github/actions/install-qlt@main + - name: Copilit Setup Steps - Install codeql and qlt + id: setup-codeql-environment + uses: advanced-security/codeql-development-template/.github/actions/setup-codeql-environment@main with: - qlt-version: 'latest' - add-to-path: true + install-codeql: true + install-language-runtimes: false + install-ql-packs: false - - name: Install CodeQL - id: install-codeql - shell: bash - run: | - echo "Installing CodeQL using configuration from qlt.conf.json" - qlt codeql run install - echo "-----------------------------" - - # Get CodeQL paths from QLT - CODEQL_HOME=$(qlt codeql get home) - CODEQL_PATH=$(qlt codeql get path) - - echo "CodeQL Home: $CODEQL_HOME" - echo "CodeQL Binary: $CODEQL_PATH" - - # Add CodeQL to PATH for subsequent steps - if [[ -n "$CODEQL_HOME" ]]; then - echo "$CODEQL_HOME" >> "$GITHUB_PATH" - echo "✅ Added CodeQL to PATH: $CODEQL_HOME" - else - echo "❌ CodeQL home path not found" - exit 1 - fi - - - name: Verify CLI Tools in PATH - shell: bash - run: | - echo "Verifying CLI tools are available in PATH:" - - if ! command -v qlt >/dev/null 2>&1; then - echo "❌ qlt not found in PATH" - exit 1 - fi - echo "✅ qlt is available: $(which qlt)" - - if ! command -v codeql >/dev/null 2>&1; then - echo "❌ codeql not found in PATH" - exit 1 - fi - echo "✅ codeql is available: $(which codeql)" - - - name: Verify Versions of Tooling - shell: bash - run: | - echo "Checking CodeQL Version:" - codeql --version - - echo "Checking QLT Version:" - echo "QLT Home: ${{ steps.install-qlt.outputs.qlt-home }}" - qlt version - - - name: Install QL Packs + - name: Copilot Setup Steps - Install QL packs shell: bash run: | echo "Installing QL pack dependencies" @@ -99,28 +49,28 @@ jobs: codeql pack install --no-strict-mode javascript/frameworks/ui5 codeql pack install --no-strict-mode javascript/frameworks/xsjs - - name: Setup Node.js for CDS Extractor - uses: actions/setup-node@v4 + - name: Copilot Setup Steps - Setup NodeJS for CDS extractor + uses: actions/setup-node@v6 with: - node-version: '20' - cache: 'npm' + cache: npm cache-dependency-path: extractors/cds/tools/package-lock.json + node-version-file: extractors/cds/tools/package.json - - name: Install CDS Extractor Dependencies + - name: Copilot Setup Steps - Install CDS Extractor Dependencies shell: bash working-directory: extractors/cds/tools run: | echo "Installing CDS extractor npm dependencies" npm ci - - name: Build CDS Extractor + - name: Copilot Setup Steps - Build CDS extractor shell: bash working-directory: extractors/cds/tools run: | echo "Building CDS extractor" npm run build - - name: Verify Setup Complete + - name: Copilot Setup Steps - Verify setup complete shell: bash run: | echo "✅ Setup complete and verified:" From 1f4103a31dec46f656c3a63ae4bb025e0867febf Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Fri, 21 Nov 2025 14:15:20 -0700 Subject: [PATCH 14/19] Update copilot-setup-steps install of qlpacks --- .github/workflows/copilot-setup-steps.yml | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index c7f812cfc..86a7ae824 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -45,9 +45,20 @@ jobs: shell: bash run: | echo "Installing QL pack dependencies" - codeql pack install --no-strict-mode javascript/frameworks/cap - codeql pack install --no-strict-mode javascript/frameworks/ui5 - codeql pack install --no-strict-mode javascript/frameworks/xsjs + codeql pack install --no-strict-mode javascript/frameworks/cap/ext + codeql pack install --no-strict-mode javascript/frameworks/cap/lib + codeql pack install --no-strict-mode javascript/frameworks/cap/src + codeql pack install --no-strict-mode javascript/frameworks/cap/test + codeql pack install --no-strict-mode javascript/frameworks/ui5/ext + codeql pack install --no-strict-mode javascript/frameworks/ui5/lib + codeql pack install --no-strict-mode javascript/frameworks/ui5/src + codeql pack install --no-strict-mode javascript/frameworks/ui5/test + codeql pack install --no-strict-mode javascript/frameworks/xsjs/ext + codeql pack install --no-strict-mode javascript/frameworks/xsjs/lib + codeql pack install --no-strict-mode javascript/frameworks/xsjs/src + codeql pack install --no-strict-mode javascript/frameworks/xsjs/test + codeql pack install --no-strict-mode javascript/heuristic-models/ext + codeql pack install --no-strict-mode javascript/heuristic-models/tests - name: Copilot Setup Steps - Setup NodeJS for CDS extractor uses: actions/setup-node@v6 From c1ab338da9176864ce8268284fd3b200ff921fcd Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Fri, 21 Nov 2025 14:20:08 -0700 Subject: [PATCH 15/19] Sync package-lock.json update for CDS extractor Fixes an error where the `package.json` and `package-lock.json` files for the CDS extractor were out of sync, which had prevented running `npm ci` without error. --- extractors/cds/tools/package-lock.json | 305 +++++++++++++++++++++++++ 1 file changed, 305 insertions(+) diff --git a/extractors/cds/tools/package-lock.json b/extractors/cds/tools/package-lock.json index d15e0c220..1cd668b14 100644 --- a/extractors/cds/tools/package-lock.json +++ b/extractors/cds/tools/package-lock.json @@ -90,6 +90,7 @@ "integrity": "sha512-bXYxrXFubeYdvB0NhD/NBB3Qi6aZeV20GOWVI47t2dkecCEoneR4NPVcb7abpXDEvejgrUfFtG6vG/zxAKmg+g==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@ampproject/remapping": "^2.2.0", "@babel/code-frame": "^7.27.1", @@ -575,6 +576,40 @@ "dev": true, "license": "MIT" }, + "node_modules/@emnapi/core": { + "version": "1.7.1", + "resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.7.1.tgz", + "integrity": "sha512-o1uhUASyo921r2XtHYOHy7gdkGLge8ghBEQHMWmyJFoXlpU58kIrhhN3w26lpQb6dspetweapMn2CSNwQ8I4wg==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/wasi-threads": "1.1.0", + "tslib": "^2.4.0" + } + }, + "node_modules/@emnapi/runtime": { + "version": "1.7.1", + "resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.7.1.tgz", + "integrity": "sha512-PVtJr5CmLwYAU9PZDMITZoR5iAOShYREoR45EyyLrbntV50mdePTgUn4AmOw90Ifcj+x2kRjdzr1HP3RrNiHGA==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@emnapi/wasi-threads": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.1.0.tgz", + "integrity": "sha512-WI0DdZ8xFSbgMjR1sFsKABJ/C5OnRrjT06JXbZKexJGrDuPTzZdDYfFlsgcCXCyf+suG5QU2e/y1Wo2V/OapLQ==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, "node_modules/@esbuild/aix-ppc64": { "version": "0.25.10", "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.10.tgz", @@ -1974,6 +2009,19 @@ "@jridgewell/sourcemap-codec": "^1.4.14" } }, + "node_modules/@napi-rs/wasm-runtime": { + "version": "0.2.12", + "resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-0.2.12.tgz", + "integrity": "sha512-ZVWUcfwY4E/yPitQJl481FjFo3K22D6qF0DuFH6Y/nbnE11GY5uguDxZMGXPQ8WQ0128MXQD7TnfHyK4oWoIJQ==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/core": "^1.4.3", + "@emnapi/runtime": "^1.4.3", + "@tybys/wasm-util": "^0.10.0" + } + }, "node_modules/@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -2070,6 +2118,17 @@ "@sinonjs/commons": "^3.0.1" } }, + "node_modules/@tybys/wasm-util": { + "version": "0.10.1", + "resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.1.tgz", + "integrity": "sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, "node_modules/@types/babel__core": { "version": "7.20.5", "resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz", @@ -2282,6 +2341,7 @@ "integrity": "sha512-EHrrEsyhOhxYt8MTg4zTF+DJMuNBzWwgvvOYNj/zm1vnaD/IC5zCXFehZv94Piqa2cRFfXrTFxIvO95L7Qc/cw==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@typescript-eslint/scope-manager": "8.44.1", "@typescript-eslint/types": "8.44.1", @@ -2528,12 +2588,240 @@ "darwin" ] }, + "node_modules/@unrs/resolver-binding-darwin-x64": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-darwin-x64/-/resolver-binding-darwin-x64-1.8.1.tgz", + "integrity": "sha512-inaphBsOqqzauNvx6kSHrgqDLShicPg3+fInBcEdD7Ut8sUUbm2z19LL+S9ccGpHnYoNiJ+Qrf7/B8hRsCUvBw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@unrs/resolver-binding-freebsd-x64": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-freebsd-x64/-/resolver-binding-freebsd-x64-1.8.1.tgz", + "integrity": "sha512-LkGw7jDoLKEZO6yYwTKUlrboD6Qmy9Jkq7ZDPlJReq/FnCnNh0k1Z1hjtevpqPCMLz9hGW0ITMb04jdDZ796Cg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ] + }, + "node_modules/@unrs/resolver-binding-linux-arm-gnueabihf": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-arm-gnueabihf/-/resolver-binding-linux-arm-gnueabihf-1.8.1.tgz", + "integrity": "sha512-6vhu22scv64dynXTVmeClenn3OPI8cwdhtydLFDkoW4UJzNwcgJ5mVtzbtikDGM9PmIQa+ekpH6tdvKt0ToK3A==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-arm-musleabihf": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-arm-musleabihf/-/resolver-binding-linux-arm-musleabihf-1.8.1.tgz", + "integrity": "sha512-SrQ286JVFWlnZSm1/TJwulTgJVOdb1x8BWW2ecOK0Sx+acdRpoMf4WSxH+/+R4LyE/YYyekcEtUrPhSEgJ748g==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-arm64-gnu": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-arm64-gnu/-/resolver-binding-linux-arm64-gnu-1.8.1.tgz", + "integrity": "sha512-I2s4L27V+2kAee43x/qAkFjTZJgmDvSd9vtnyINOdBEdz5+QqiG6ccd5pgOw06MsUwygkrhB4jOe4ZN4SA6IwA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-arm64-musl": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-arm64-musl/-/resolver-binding-linux-arm64-musl-1.8.1.tgz", + "integrity": "sha512-Drq80e/EQbdSVyJpheF65qVmfYy8OaDdQqoWV+09tZHz/P1SdSulvVtgtYrk216D++9hbx3c1bwVXwR5PZ2TzA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-ppc64-gnu": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-ppc64-gnu/-/resolver-binding-linux-ppc64-gnu-1.8.1.tgz", + "integrity": "sha512-EninHQHw8Zkq8K5qB6KWNDqjCtUzTDsCRQ6LzAtQWIxic/VQxR5Kl36V/GCXNvQaR7W0AB5gvJLyQtJwkf+AJA==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-riscv64-gnu": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-riscv64-gnu/-/resolver-binding-linux-riscv64-gnu-1.8.1.tgz", + "integrity": "sha512-s7Xu5PS4vWhsb5ZFAi+UBguTn0g8qDhN+BbB1t9APX23AdAI7TS4DRrJV5dBVdQ6a8MiergGr1Cjb0Q1V/sW8w==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-riscv64-musl": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-riscv64-musl/-/resolver-binding-linux-riscv64-musl-1.8.1.tgz", + "integrity": "sha512-Ca+bVzOJtgQ3OrMkRSeDLYWJIjRmEylDHSZuSKqqPmZI2vgX6yZgzrKY28I6hjjG9idlW4DcJzLv/TjFXev+4Q==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-s390x-gnu": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-s390x-gnu/-/resolver-binding-linux-s390x-gnu-1.8.1.tgz", + "integrity": "sha512-ut1vBBFs6AC5EcerH8HorcmS/9wAy6iI1tfpzT7jy+SKnMgmPth/psc3W5V04njble7cyLPjFHwYJTlxmozQ/g==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-x64-gnu": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-x64-gnu/-/resolver-binding-linux-x64-gnu-1.8.1.tgz", + "integrity": "sha512-w5agLxesvrYKrCOlAsUkwRDogjnyRBi4/vEaujZRkXbeRCupJ9dFD0qUhLXZyIed+GSzJJIsJocUZIVzcTHYXQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-linux-x64-musl": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-linux-x64-musl/-/resolver-binding-linux-x64-musl-1.8.1.tgz", + "integrity": "sha512-vk5htmWYCLRpfjn2wmCUne6pLvlcYUFDAAut4g02/2iWeGeZO/3GmSLmiZ9fcn9oH0FUzgetg0/zSo8oZ7liIg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@unrs/resolver-binding-wasm32-wasi": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-wasm32-wasi/-/resolver-binding-wasm32-wasi-1.8.1.tgz", + "integrity": "sha512-RcsLTcrqDT5XW/TnhhIeM7lVLgUv/gvPEC4WaH+OhkLCkRfH6EEuhprwrcp1WhdlrtL/U5FkHh4NtFLnMXoeXA==", + "cpu": [ + "wasm32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@napi-rs/wasm-runtime": "^0.2.11" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/@unrs/resolver-binding-win32-arm64-msvc": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-win32-arm64-msvc/-/resolver-binding-win32-arm64-msvc-1.8.1.tgz", + "integrity": "sha512-XbSRLZY/gEi5weYv/aCkiUiSWvrNKkvec3m6/bDypDI+ZACwMllPH7smeOW/fdnIGhf9YtPATNliJHAS2GyMUA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@unrs/resolver-binding-win32-ia32-msvc": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-win32-ia32-msvc/-/resolver-binding-win32-ia32-msvc-1.8.1.tgz", + "integrity": "sha512-SbCJMKOmqOsIBCklT5c+t0DjVbOkseE7ZN0OtMxRnraLKdj1AAv7d3cjJMYkPd9ZGKosHoMXo66gBs02YM8KeA==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@unrs/resolver-binding-win32-x64-msvc": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/@unrs/resolver-binding-win32-x64-msvc/-/resolver-binding-win32-x64-msvc-1.8.1.tgz", + "integrity": "sha512-DdHqo7XbeUa/ZOcxq+q5iuO4sSxhwX9HR1JPL0JMOKEzgkIO4OKF2TPjqmo6UCCGZUXIMwrAycFXj/40sICagw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, "node_modules/acorn": { "version": "8.15.0", "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.15.0.tgz", "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==", "dev": true, "license": "MIT", + "peer": true, "bin": { "acorn": "bin/acorn" }, @@ -2926,6 +3214,7 @@ } ], "license": "MIT", + "peer": true, "dependencies": { "caniuse-lite": "^1.0.30001718", "electron-to-chromium": "^1.5.160", @@ -3635,6 +3924,7 @@ "integrity": "sha512-hB4FIzXovouYzwzECDcUkJ4OcfOEkXTv2zRY6B9bkwjx/cprAq0uvm1nl7zvQ0/TsUk0zQiN4uPfJpB9m+rPMQ==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@eslint-community/eslint-utils": "^4.8.0", "@eslint-community/regexpp": "^4.12.1", @@ -3696,6 +3986,7 @@ "integrity": "sha512-82GZUjRS0p/jganf6q1rEO25VSoHH0hKPCTrgillPjdI/3bgBhAE1QzHrHTizjpRvy6pGAvKjDJtk2pF9NDq8w==", "dev": true, "license": "MIT", + "peer": true, "bin": { "eslint-config-prettier": "bin/cli.js" }, @@ -3832,6 +4123,7 @@ "integrity": "sha512-whOE1HFo/qJDyX4SnXzP4N6zOWn79WhnCUY/iDR0mPfQZO8wcYE4JClzI2oZrhBnnMUCBCHZhO6VQyoBU95mZA==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@rtsao/scc": "^1.1.0", "array-includes": "^3.1.9", @@ -5380,6 +5672,7 @@ "integrity": "sha512-Ry+p2+NLk6u8Agh5yVqELfUJvRfV51hhVBRIB5yZPY7mU0DGBmOuFG5GebZbMbm86cdQNK0fhJuDX8/1YorISQ==", "dev": true, "license": "MIT", + "peer": true, "dependencies": { "@jest/core": "30.1.3", "@jest/types": "30.0.5", @@ -6937,6 +7230,7 @@ "integrity": "sha512-I7AIg5boAr5R0FFtJ6rCfD+LFsWHp81dolrFD8S79U9tb8Az2nGrJncnMSnys+bpQJfRUzqs9hnA81OAA3hCuQ==", "dev": true, "license": "MIT", + "peer": true, "bin": { "prettier": "bin/prettier.cjs" }, @@ -7788,6 +8082,7 @@ "integrity": "sha512-M7BAV6Rlcy5u+m6oPhAPFgJTzAioX/6B0DxyvDlo9l8+T3nLKbrczg2WLUyzd45L8RqfUMyGPzekbMvX2Ldkwg==", "dev": true, "license": "MIT", + "peer": true, "engines": { "node": ">=12" }, @@ -7939,6 +8234,14 @@ "node": ">=4" } }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "dev": true, + "license": "0BSD", + "optional": true + }, "node_modules/type-check": { "version": "0.4.0", "resolved": "https://registry.npmjs.org/type-check/-/type-check-0.4.0.tgz", @@ -8059,6 +8362,7 @@ "integrity": "sha512-CWBzXQrc/qOkhidw1OzBTQuYRbfyxDXJMVJ1XNwUHGROVmuaeiEm3OslpZ1RV96d7SKKjZKrSJu3+t/xlw3R9A==", "dev": true, "license": "Apache-2.0", + "peer": true, "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" @@ -8114,6 +8418,7 @@ "dev": true, "hasInstallScript": true, "license": "MIT", + "peer": true, "dependencies": { "napi-postinstall": "^0.2.2" }, From 0eb39a4cc24d0a0626e22a14ac5abdc92fcfc1ca Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Mon, 24 Nov 2025 08:55:07 -0700 Subject: [PATCH 16/19] Remove PROMPTS.md --- PROMPTS.md | 253 ----------------------------------------------------- 1 file changed, 253 deletions(-) delete mode 100644 PROMPTS.md diff --git a/PROMPTS.md b/PROMPTS.md deleted file mode 100644 index 6cddccd7e..000000000 --- a/PROMPTS.md +++ /dev/null @@ -1,253 +0,0 @@ -# PROMPTS - -This document outlines the hierarchy of instructions, prompts, and other file resources intended for use by AI agents assigned with maintenance and development tasks in the codeql-sap-js repository. - -## Prompts Hierarchy Description - -In order to keep the prompt system organized and efficient, the following hierarchy is established: - -### 1. `.github/ISSUE_TEMPLATE/*.yml` - -- Entry point for AI agent workflows via GitHub Copilot. -- Pre-structured for different development and maintenance scenarios. -- Links to relevant agents, instructions, and prompts. -- Categories: - - CDS Extractor Development - - CAP Framework Modeling - - UI5 Framework Modeling - - XSJS Framework Modeling - - Dependency Upgrades - - GitHub Infrastructure Maintenance - -### 2. `.github/agents/*.md` - -- Specialized AI agent definitions for specific domains. -- Define agent persona, responsibilities, and boundaries. -- Include executable commands and code examples. -- Follow best practices from GitHub's agents.md guide. -- One agent per domain for focused expertise. - -### 3. `.github/instructions/*.instructions.md` - -- Highest level of abstraction in the prompt hierarchy. -- Sent with every request to the AI, so must be concise and clear. -- Define rules, requirements, preferences, and constraints. -- Link to detailed prompts in "RELATED PROMPTS" section. -- Use `applyTo` pattern in frontmatter to target specific file types. - -### 4. `.github/prompts/*.prompt.md` - -- High-level prompts for multi-step development tasks. -- Provides detailed, step-by-step guidance for specific tasks. -- Includes concrete examples and command sequences. -- Task-focused and actionable. -- References external documentation and resources. - -## Prompts Hierarchy Visualization - -The following diagram shows the relationships between agents, instructions, prompts, and issue templates. - -```mermaid -graph TD; - %% Level 1: GitHub Issue Templates - ISSUE_CDS[".github/ISSUE_TEMPLATE/cds-extractor-task.yml"] - ISSUE_CAP[".github/ISSUE_TEMPLATE/cap-modeling-task.yml"] - ISSUE_UI5[".github/ISSUE_TEMPLATE/ui5-modeling-task.yml"] - ISSUE_XSJS[".github/ISSUE_TEMPLATE/xsjs-modeling-task.yml"] - ISSUE_DEPS[".github/ISSUE_TEMPLATE/dependency-upgrade.yml"] - ISSUE_GITHUB[".github/ISSUE_TEMPLATE/github-maintenance.yml"] - - %% Level 1.5: Agent Definitions - AGENT_CDS[".github/agents/cds-extractor-agent.md"] - AGENT_CAP[".github/agents/cap-modeling-agent.md"] - AGENT_UI5[".github/agents/ui5-modeling-agent.md"] - AGENT_XSJS[".github/agents/xsjs-modeling-agent.md"] - AGENT_DEPS[".github/agents/dependency-upgrade-agent.md"] - AGENT_GITHUB[".github/agents/github-maintenance-agent.md"] - - %% Level 2: Instructions - INSTR_CDS[".github/instructions/extractors_cds_tools_ts.instructions.md"] - INSTR_CAP[".github/instructions/javascript_cap_ql.instructions.md"] - INSTR_UI5[".github/instructions/javascript_ui5_ql.instructions.md"] - INSTR_XSJS[".github/instructions/javascript_xsjs_ql.instructions.md"] - INSTR_DEPS[".github/instructions/dependency_upgrades.instructions.md"] - INSTR_GITHUB[".github/instructions/github_maintenance.instructions.md"] - - %% Level 3: Prompts - PROMPT_CDS[".github/prompts/cds_extractor_development.prompt.md"] - PROMPT_CAP[".github/prompts/cap_framework_development.prompt.md"] - PROMPT_UI5[".github/prompts/ui5_framework_development.prompt.md"] - PROMPT_XSJS[".github/prompts/xsjs_framework_development.prompt.md"] - PROMPT_DEPS[".github/prompts/dependency_upgrade.prompt.md"] - PROMPT_GITHUB[".github/prompts/github_maintenance.prompt.md"] - - %% Issue Templates to Agents - ISSUE_CDS --> AGENT_CDS - ISSUE_CAP --> AGENT_CAP - ISSUE_UI5 --> AGENT_UI5 - ISSUE_XSJS --> AGENT_XSJS - ISSUE_DEPS --> AGENT_DEPS - ISSUE_GITHUB --> AGENT_GITHUB - - %% Agents to Instructions - AGENT_CDS --> INSTR_CDS - AGENT_CAP --> INSTR_CAP - AGENT_UI5 --> INSTR_UI5 - AGENT_XSJS --> INSTR_XSJS - AGENT_DEPS --> INSTR_DEPS - AGENT_GITHUB --> INSTR_GITHUB - - %% Instructions to Prompts - INSTR_CDS --> PROMPT_CDS - INSTR_CAP --> PROMPT_CAP - INSTR_UI5 --> PROMPT_UI5 - INSTR_XSJS --> PROMPT_XSJS - INSTR_DEPS --> PROMPT_DEPS - INSTR_GITHUB --> PROMPT_GITHUB - - %% Cross-references for related prompts - INSTR_CAP --> PROMPT_GITHUB - INSTR_UI5 --> PROMPT_GITHUB - INSTR_XSJS --> PROMPT_GITHUB -``` - -## Agent Descriptions - -### CDS Extractor Development Agent - -**File**: `.github/agents/cds-extractor-agent.md` - -**Responsibilities**: -- TypeScript development for CodeQL CDS extractor -- Test-driven development with Jest -- Dependency management for extractor -- Build and bundle validation - -**Key Commands**: -- `npm run build:all` - MANDATORY before commit -- `npm run lint:fix` - Fix linting issues -- `npm test` - Run Jest tests - -### CAP Framework Modeling Agent - -**File**: `.github/agents/cap-modeling-agent.md` - -**Responsibilities**: -- CodeQL query development for SAP CAP framework -- Library modeling for CAP-specific patterns -- Event handler source modeling -- Test case creation and validation - -**Key Commands**: -- `codeql test run` - Run CodeQL tests -- `codeql test extract` - Create test databases -- `codeql query format` - Format queries - -### UI5 Framework Modeling Agent - -**File**: `.github/agents/ui5-modeling-agent.md` - -**Responsibilities**: -- CodeQL query development for SAPUI5 framework -- XML view and binding expression modeling -- XSS, path injection, formula injection detection -- Binding expression parser maintenance - -**Key Commands**: -- `codeql test run` - Run CodeQL tests -- `codeql test extract` - Create test databases -- `codeql query format` - Format queries - -### XSJS Framework Modeling Agent - -**File**: `.github/agents/xsjs-modeling-agent.md` - -**Responsibilities**: -- CodeQL query development for SAP XSJS framework -- $.request and $.response modeling -- SQL injection and XSS detection -- XSJS-specific API modeling - -**Key Commands**: -- `codeql test run` - Run CodeQL tests -- `codeql test extract` - Create test databases -- `codeql query format` - Format queries - -### Dependency Upgrade Agent - -**File**: `.github/agents/dependency-upgrade-agent.md` - -**Responsibilities**: -- Node.js dependency upgrades -- CodeQL/QLT CLI version updates -- GitHub Actions version updates -- Security vulnerability remediation - -**Key Commands**: -- `npm outdated` - Check outdated packages -- `npm audit` - Security audit -- `npm run build:all` - Validate after upgrade - -### GitHub Maintenance Agent - -**File**: `.github/agents/github-maintenance-agent.md` - -**Responsibilities**: -- Maintain agent definitions -- Update instruction and prompt files -- Create and update issue templates -- Update PROMPTS.md (this file) -- Validate workflow syntax - -**Key Commands**: -- `actionlint` - Validate workflow YAML -- `yamllint` - Validate YAML syntax - -## Best Practices - -### For Agent Definitions - -1. **Be Specific**: Clear, narrow scope per agent -2. **Commands First**: Executable commands at the top -3. **Show Examples**: Real code, not just prose -4. **Define Boundaries**: Explicitly state what NOT to do -5. **Keep Focused**: One domain per agent - -### For Instruction Files - -1. **Concise**: Sent with every request, keep brief -2. **High-level**: Rules and requirements, not detailed procedures -3. **Link to Prompts**: Reference detailed guidance -4. **Structured**: Use consistent sections (PURPOSE, REQUIREMENTS, PREFERENCES, CONSTRAINTS, RELATED PROMPTS) - -### For Prompt Files - -1. **Detailed**: Step-by-step procedures -2. **Actionable**: Task-focused, not theoretical -3. **Examples**: Include concrete code examples -4. **Resources**: Link to external documentation - -### For Issue Templates - -1. **YAML Format**: Use .yml extension for structured templates -2. **Link to Agent**: Clearly indicate which agent handles it -3. **Required Fields**: Ensure necessary information is collected -4. **Labels**: Use consistent labeling for organization - -## Updating This Documentation - -When adding or modifying agents, instructions, or prompts: - -1. **Update the mermaid diagram** to reflect new files and relationships -2. **Add agent description** if creating a new agent -3. **Update cross-references** if changing relationships -4. **Validate mermaid syntax** before committing -5. **Ensure consistency** across all levels of the hierarchy - -## Related Resources - -- GitHub Agents.md Guide: https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/ -- CodeQL Documentation: https://codeql.github.com/docs/ -- SAP CAP Documentation: https://cap.cloud.sap/docs/ -- SAPUI5 Documentation: https://sapui5.hana.ondemand.com/ -- Mermaid Diagram Syntax: https://mermaid-js.github.io/mermaid/ From fabf915f546f293c377820da23380279051c86ea Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Mon, 24 Nov 2025 09:02:42 -0700 Subject: [PATCH 17/19] Refine on.{pull_request,push}.paths for copilot-setup-steps --- .github/workflows/copilot-setup-steps.yml | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 86a7ae824..6f8f38be5 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -9,19 +9,25 @@ on: - main paths: - .github/workflows/copilot-setup-steps.yml - - qlt.conf.json - extractors/cds/tools/package.json - extractors/cds/tools/package-lock.json - - javascript/frameworks/*/qlpack.yml + - javascript/frameworks/*/*/codeql-pack.lock.yml + - javascript/frameworks/*/*/qlpack.yml + - javascript/heuristic-models/*/codeql-pack.lock.yml + - javascript/heuristic-models/*/qlpack.yml + - qlt.conf.json pull_request: branches: - main paths: - .github/workflows/copilot-setup-steps.yml - - qlt.conf.json - extractors/cds/tools/package.json - extractors/cds/tools/package-lock.json - - javascript/frameworks/*/qlpack.yml + - javascript/frameworks/*/*/codeql-pack.lock.yml + - javascript/frameworks/*/*/qlpack.yml + - javascript/heuristic-models/*/codeql-pack.lock.yml + - javascript/heuristic-models/*/qlpack.yml + - qlt.conf.json jobs: # The job MUST be called `copilot-setup-steps` or it will not be picked up by Copilot. From ad8139b105a1f2eae97b15ee016c818327cb2649 Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Mon, 24 Nov 2025 14:19:11 -0700 Subject: [PATCH 18/19] Cleanup https links used in .github/** --- .github/agents/cds-extractor-agent.md | 6 +++--- .github/agents/dependency-upgrade-agent.md | 2 +- .github/agents/github-maintenance-agent.md | 9 +-------- .github/agents/ui5-modeling-agent.md | 4 ++-- .github/agents/xsjs-modeling-agent.md | 4 ++-- .github/prompts/cds_extractor_development.prompt.md | 2 -- .github/prompts/dependency_upgrade.prompt.md | 5 ----- .github/prompts/github_maintenance.prompt.md | 1 - .github/workflows/copilot-setup-steps.yml | 7 +++++++ 9 files changed, 16 insertions(+), 24 deletions(-) diff --git a/.github/agents/cds-extractor-agent.md b/.github/agents/cds-extractor-agent.md index bd270ed70..d678bf72b 100644 --- a/.github/agents/cds-extractor-agent.md +++ b/.github/agents/cds-extractor-agent.md @@ -1,11 +1,11 @@ --- -name: 'CDS Extractor Development Agent' -description: 'Expert in developing, testing, and maintaining the CodeQL CDS extractor TypeScript implementation' +name: 'codeql-cds-extractor-agent' +description: 'Expert in developing, testing, troubleshooting and fixing the CodeQL CDS extractor TypeScript implementation' --- # CDS Extractor Development Agent -My `cds-extractor-agent`: +My `codeql-cds-extractor-agent`: - Specializes in TypeScript development for CodeQL extractors with deep knowledge of CDS (Core Data Services) language and SAP CAP framework - Obeys all [CDS extractor instructions](../instructions/extractors_cds_tools_ts.instructions.md) diff --git a/.github/agents/dependency-upgrade-agent.md b/.github/agents/dependency-upgrade-agent.md index 534031683..c9c93b0cf 100644 --- a/.github/agents/dependency-upgrade-agent.md +++ b/.github/agents/dependency-upgrade-agent.md @@ -1,5 +1,5 @@ --- -name: 'Dependency Upgrade Agent' +name: 'dependency-upgrade-agent' description: 'Expert in upgrading CodeQL CLI, QLT, Node.js dependencies, and GitHub Actions versions' --- diff --git a/.github/agents/github-maintenance-agent.md b/.github/agents/github-maintenance-agent.md index b72bbad0e..8ccfd7ccd 100644 --- a/.github/agents/github-maintenance-agent.md +++ b/.github/agents/github-maintenance-agent.md @@ -1,5 +1,5 @@ --- -name: 'GitHub Maintenance Agent' +name: 'github-maintenance-agent' description: 'Expert in maintaining .github infrastructure including workflows, templates, agents, instructions, and prompts' --- @@ -27,13 +27,6 @@ My `github-maintenance-agent`: Validate GitHub Actions workflows: ```bash -# Install actionlint if not available -# brew install actionlint # macOS -# or download from https://github.com/rhysd/actionlint - -# Validate all workflows -actionlint .github/workflows/*.yml - # Check YAML syntax yamllint .github/workflows/*.yml ``` diff --git a/.github/agents/ui5-modeling-agent.md b/.github/agents/ui5-modeling-agent.md index f4a2c1401..afc0d2cb0 100644 --- a/.github/agents/ui5-modeling-agent.md +++ b/.github/agents/ui5-modeling-agent.md @@ -1,11 +1,11 @@ --- -name: 'UI5 Framework Modeling Agent' +name: 'javascript-ui5-modeling-agent' description: 'Expert in developing CodeQL queries and library models for SAPUI5 framework' --- # UI5 Framework Modeling Agent -My `ui5-modeling-agent`: +My `javascript-ui5-modeling-agent`: - Specializes in CodeQL query and library development for SAPUI5 framework security analysis - Obeys all [UI5 framework instructions](../instructions/javascript_ui5_ql.instructions.md) diff --git a/.github/agents/xsjs-modeling-agent.md b/.github/agents/xsjs-modeling-agent.md index 32deb295d..e81c67dd9 100644 --- a/.github/agents/xsjs-modeling-agent.md +++ b/.github/agents/xsjs-modeling-agent.md @@ -1,11 +1,11 @@ --- -name: 'XSJS Framework Modeling Agent' +name: 'javascript-xsjs-modeling-agent' description: 'Expert in developing CodeQL queries and library models for SAP XSJS (XS JavaScript) framework' --- # XSJS Framework Modeling Agent -My `xsjs-modeling-agent`: +My `javascript-xsjs-modeling-agent`: - Specializes in CodeQL query and library development for SAP XSJS framework security analysis - Obeys all [XSJS framework instructions](../instructions/javascript_xsjs_ql.instructions.md) diff --git a/.github/prompts/cds_extractor_development.prompt.md b/.github/prompts/cds_extractor_development.prompt.md index 28f29d47a..eb6d9911d 100644 --- a/.github/prompts/cds_extractor_development.prompt.md +++ b/.github/prompts/cds_extractor_development.prompt.md @@ -268,7 +268,5 @@ Before committing any changes: ## Related Resources -- Jest documentation: https://jestjs.io/ -- TypeScript documentation: https://www.typescriptlang.org/ - SAP CDS documentation: https://cap.cloud.sap/docs/cds/ - CodeQL extractor documentation: https://codeql.github.com/docs/codeql-cli/extractor-options/ diff --git a/.github/prompts/dependency_upgrade.prompt.md b/.github/prompts/dependency_upgrade.prompt.md index cf48b0249..ee046b7aa 100644 --- a/.github/prompts/dependency_upgrade.prompt.md +++ b/.github/prompts/dependency_upgrade.prompt.md @@ -114,9 +114,6 @@ Update version in `.github/workflows/*.yml`: ```bash # Check Dependabot alerts # Navigate to repository → Security → Dependabot alerts - -# Or manually check action repositories -# Example: https://github.com/actions/checkout/releases ``` ### Update Actions @@ -190,6 +187,4 @@ Before committing: ## Related Resources -- npm documentation: https://docs.npmjs.com/ - CodeQL releases: https://github.com/github/codeql-cli-binaries/releases -- GitHub Actions marketplace: https://github.com/marketplace?type=actions diff --git a/.github/prompts/github_maintenance.prompt.md b/.github/prompts/github_maintenance.prompt.md index 3c86d0a20..3391e006c 100644 --- a/.github/prompts/github_maintenance.prompt.md +++ b/.github/prompts/github_maintenance.prompt.md @@ -220,4 +220,3 @@ gh issue create --web - Agents.md guide: https://github.blog/ai-and-ml/github-copilot/how-to-write-a-great-agents-md-lessons-from-over-2500-repositories/ - GitHub Actions syntax: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions -- Mermaid diagram syntax: https://mermaid-js.github.io/mermaid/ diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml index 6f8f38be5..24acc4d38 100644 --- a/.github/workflows/copilot-setup-steps.yml +++ b/.github/workflows/copilot-setup-steps.yml @@ -47,6 +47,13 @@ jobs: install-language-runtimes: false install-ql-packs: false + - name: Copilot Setup Steps - Install apt packages + shell: bash + run: | + echo "Installing apt packages" + sudo apt update + sudo apt install -y jq yamllint + - name: Copilot Setup Steps - Install QL packs shell: bash run: | From a18ac54ef9a599ce81e52d0b6147e8f7db05bca2 Mon Sep 17 00:00:00 2001 From: Nathan Randall Date: Mon, 24 Nov 2025 15:47:48 -0700 Subject: [PATCH 19/19] Cleanup yaml lint and TDD agents --- .github/ISSUE_TEMPLATE/dependency-upgrade.yml | 4 +- .github/ISSUE_TEMPLATE/ui5-modeling-task.yml | 2 +- ...agent.md => codeql-cds-extractor-agent.md} | 0 ...nt.md => javascript-cap-modeling-agent.md} | 48 ++--- ...nt.md => javascript-ui5-modeling-agent.md} | 31 ++- ...t.md => javascript-xsjs-modeling-agent.md} | 28 ++- .../javascript_cap_ql.instructions.md | 16 +- .../javascript_ui5_ql.instructions.md | 6 + .../javascript_xsjs_ql.instructions.md | 6 + .../cap_framework_development.prompt.md | 122 ++++++++++-- .../prompts/codeql_test_commands.prompt.md | 187 +++++++++++++++++- .../prompts/test_driven_development.prompt.md | 184 +++++++++++++++++ .github/workflows/code_scanning.yml | 4 +- .../run-codeql-unit-tests-javascript.yml | 3 - 14 files changed, 563 insertions(+), 78 deletions(-) rename .github/agents/{cds-extractor-agent.md => codeql-cds-extractor-agent.md} (100%) rename .github/agents/{cap-modeling-agent.md => javascript-cap-modeling-agent.md} (71%) rename .github/agents/{ui5-modeling-agent.md => javascript-ui5-modeling-agent.md} (81%) rename .github/agents/{xsjs-modeling-agent.md => javascript-xsjs-modeling-agent.md} (81%) create mode 100644 .github/prompts/test_driven_development.prompt.md diff --git a/.github/ISSUE_TEMPLATE/dependency-upgrade.yml b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml index 9b0fd593a..dd6a3aff9 100644 --- a/.github/ISSUE_TEMPLATE/dependency-upgrade.yml +++ b/.github/ISSUE_TEMPLATE/dependency-upgrade.yml @@ -37,10 +37,10 @@ body: placeholder: | - @sap/cds: 7.4.0 → 7.5.0 - axios: 1.5.0 → 1.6.2 - + Or for CLI: - CodeQL CLI: 2.14.6 → 2.15.4 - + Or for Actions: - actions/checkout: v3 → v4 validations: diff --git a/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml index 032cd676e..55dbc5ca5 100644 --- a/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml +++ b/.github/ISSUE_TEMPLATE/ui5-modeling-task.yml @@ -86,7 +86,7 @@ body: ``` - + ```javascript // Controller.controller.js onInit: function() { diff --git a/.github/agents/cds-extractor-agent.md b/.github/agents/codeql-cds-extractor-agent.md similarity index 100% rename from .github/agents/cds-extractor-agent.md rename to .github/agents/codeql-cds-extractor-agent.md diff --git a/.github/agents/cap-modeling-agent.md b/.github/agents/javascript-cap-modeling-agent.md similarity index 71% rename from .github/agents/cap-modeling-agent.md rename to .github/agents/javascript-cap-modeling-agent.md index 7b76b891c..3d638703e 100644 --- a/.github/agents/cap-modeling-agent.md +++ b/.github/agents/javascript-cap-modeling-agent.md @@ -1,17 +1,17 @@ --- -name: 'CAP Framework Modeling Agent' -description: 'Expert in developing CodeQL queries and library models for SAP Cloud Application Programming (CAP) framework' +name: 'javascript-cap-modeling-agent' +description: 'Expert in developing CodeQL queries and library models for SAP Cloud Application Programming (CAP) framework for Node.js applications.' --- # CAP Framework Modeling Agent -My `cap-modeling-agent`: +My `javascript-cap-modeling-agent`: - Specializes in CodeQL query and library development for SAP CAP framework security analysis - Obeys all [CAP framework instructions](../instructions/javascript_cap_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Utilizes the [CAP framework development prompt](../prompts/cap_framework_development.prompt.md) as primary guide -- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows -- Follows test-driven development practices for CodeQL queries +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats - Works primarily in the `javascript/frameworks/cap/` directory structure - Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests - Understands CAP-specific patterns (see [CAP development prompt](../prompts/cap_framework_development.prompt.md) for details): @@ -22,34 +22,26 @@ My `cap-modeling-agent`: - Creates comprehensive test cases in `javascript/frameworks/cap/test/` with expected results - Never makes assumptions - validates everything with CodeQL CLI -## Commands +## Testing Workflow -See [CodeQL Test Commands Reference](../prompts/codeql_test_commands.prompt.md) for detailed command usage. +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation -**Primary workflow:** +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**Common Commands:** ```bash -# Run tests (extracts DB and runs query) -codeql test run javascript/frameworks/cap/test/ +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/cap/test/ -# Accept results after verification -codeql test accept javascript/frameworks/cap/test/ +# Review and accept results +codeql test accept javascript/frameworks/cap/test/ -# Format query files -codeql query format --in-place +# Format queries +codeql query format --in-place ``` -Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference and important notes. - -## Testing - -Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. - -- Create realistic test cases in `javascript/frameworks/cap/test/` -- Each test should have source code and expected results -- Use `codeql test run` to validate query behavior (see commands reference) -- Test both positive cases (should alert) and negative cases (should not alert) -- Update `.expected` files after verifying correctness - ## Code Style - Follow CodeQL QL language conventions @@ -61,7 +53,7 @@ Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md ## Project Structure -``` +```text javascript/frameworks/cap/ ├── lib/ # Library models │ └── advanced_security/ @@ -108,7 +100,7 @@ class CapEventHandlerParameter extends RemoteFlowSource { ``` ### Example Test Case Structure -``` +```text test/ ├── sql-injection/ │ ├── test.js # Test source code diff --git a/.github/agents/ui5-modeling-agent.md b/.github/agents/javascript-ui5-modeling-agent.md similarity index 81% rename from .github/agents/ui5-modeling-agent.md rename to .github/agents/javascript-ui5-modeling-agent.md index afc0d2cb0..271b4830a 100644 --- a/.github/agents/ui5-modeling-agent.md +++ b/.github/agents/javascript-ui5-modeling-agent.md @@ -9,9 +9,9 @@ My `javascript-ui5-modeling-agent`: - Specializes in CodeQL query and library development for SAPUI5 framework security analysis - Obeys all [UI5 framework instructions](../instructions/javascript_ui5_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Utilizes the [UI5 framework development prompt](../prompts/ui5_framework_development.prompt.md) as primary guide -- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows -- Follows test-driven development practices for CodeQL queries +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats - Works primarily in the `javascript/frameworks/ui5/` directory structure - Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests - Understands UI5-specific patterns (see [UI5 development prompt](../prompts/ui5_framework_development.prompt.md) for details): @@ -44,16 +44,27 @@ codeql query format --in-place Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. -## Testing +## Testing Workflow -Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation -- Create realistic UI5 test cases in `javascript/frameworks/ui5/test/` -- Include both JavaScript and XML view files -- Each test should have source code and expected results -- Use `codeql test run` to validate query behavior (see commands reference) -- Test both positive cases (should alert) and negative cases (should not alert) -- Update `.expected` files after verifying correctness +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**UI5-Specific:** Include both JavaScript controllers and XML view files in tests. + +**Common Commands:** +```bash +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/ui5/test/ + +# Review and accept results +codeql test accept javascript/frameworks/ui5/test/ + +# Format queries +codeql query format --in-place +``` ## Code Style diff --git a/.github/agents/xsjs-modeling-agent.md b/.github/agents/javascript-xsjs-modeling-agent.md similarity index 81% rename from .github/agents/xsjs-modeling-agent.md rename to .github/agents/javascript-xsjs-modeling-agent.md index e81c67dd9..88ea67e8f 100644 --- a/.github/agents/xsjs-modeling-agent.md +++ b/.github/agents/javascript-xsjs-modeling-agent.md @@ -9,9 +9,9 @@ My `javascript-xsjs-modeling-agent`: - Specializes in CodeQL query and library development for SAP XSJS framework security analysis - Obeys all [XSJS framework instructions](../instructions/javascript_xsjs_ql.instructions.md) +- Follows [test-driven development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Utilizes the [XSJS framework development prompt](../prompts/xsjs_framework_development.prompt.md) as primary guide -- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for testing workflows -- Follows test-driven development practices for CodeQL queries +- References [CodeQL test commands](../prompts/codeql_test_commands.prompt.md) for command syntax and `.expected` file formats - Works primarily in the `javascript/frameworks/xsjs/` directory structure - Uses [Copilot PR template](../PULL_REQUEST_TEMPLATE/copilot-template.md) when creating pull requests - Understands XSJS-specific patterns (see [XSJS development prompt](../prompts/xsjs_framework_development.prompt.md) for details): @@ -43,15 +43,25 @@ codeql query format --in-place Refer to the [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete command reference. -## Testing +## Testing Workflow -Refer to [CodeQL test commands prompt](../prompts/codeql_test_commands.prompt.md) for complete testing workflow. +**Primary Resources:** +- [Test-Driven Development (TDD) methodology](../prompts/test_driven_development.prompt.md) - Complete TDD workflow for new and existing queries/models +- [CodeQL test commands reference](../prompts/codeql_test_commands.prompt.md) - Command syntax, `.expected` file formats, and interpretation -- Create realistic XSJS test cases in `javascript/frameworks/xsjs/test/` -- Each test should have source code and expected results -- Use `codeql test run` to validate query behavior (see commands reference) -- Test both positive cases (should alert) and negative cases (should not alert) -- Update `.expected` files after verifying correctness +**Key TDD Principle:** For new queries/models, generate `.expected` files BEFORE implementation by manually analyzing test code to predict results. + +**Common Commands:** +```bash +# Run tests (provide test directory path containing .qlref) +codeql test run javascript/frameworks/xsjs/test/ + +# Review and accept results +codeql test accept javascript/frameworks/xsjs/test/ + +# Format queries +codeql query format --in-place +``` ## Code Style diff --git a/.github/instructions/javascript_cap_ql.instructions.md b/.github/instructions/javascript_cap_ql.instructions.md index 7ba4a559f..3f186946a 100644 --- a/.github/instructions/javascript_cap_ql.instructions.md +++ b/.github/instructions/javascript_cap_ql.instructions.md @@ -14,6 +14,8 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar ### COMMON REQUIREMENTS - ALWAYS follow test-driven development (TDD) practices using CodeQL test commands. +- ALWAYS generate `.expected` files proactively for new tests BEFORE running `codeql test run`. +- ALWAYS analyze test code to predict expected results rather than only accepting actual results. - ALWAYS run `codeql query format --in-place ` before committing changes to QL files. - ALWAYS use `codeql test run` to validate query changes before committing. - ALWAYS validate query behavior with both positive (should alert) and negative (should not alert) test cases. @@ -42,7 +44,19 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/cap/test/`. - ALWAYS include both JavaScript and CDS files in tests when relevant. -- ALWAYS verify expected results before accepting with `codeql test accept`. +- FOR NEW TESTS: Generate `.expected` file BEFORE implementing the query/model: + 1. Document what pattern should be detected + 2. Create test code demonstrating the pattern + 3. Manually create `.expected` file with predicted results based on analysis + 4. Implement the query/model + 5. Run `codeql test run` - ideally test passes immediately +- FOR EXISTING TESTS: Verify expected results before accepting with `codeql test accept`. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/cap/ -type f -name "*.expected"` to locate test files. ## PREFERENCES diff --git a/.github/instructions/javascript_ui5_ql.instructions.md b/.github/instructions/javascript_ui5_ql.instructions.md index 62c40c31e..21b8b8f38 100644 --- a/.github/instructions/javascript_ui5_ql.instructions.md +++ b/.github/instructions/javascript_ui5_ql.instructions.md @@ -49,6 +49,12 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS include both JavaScript controllers and XML views in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. - ALWAYS test binding expression parsing separately. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/ui5/ -type f -name "*.expected"` to locate test files. ## PREFERENCES diff --git a/.github/instructions/javascript_xsjs_ql.instructions.md b/.github/instructions/javascript_xsjs_ql.instructions.md index f43e08ece..cc73f06f6 100644 --- a/.github/instructions/javascript_xsjs_ql.instructions.md +++ b/.github/instructions/javascript_xsjs_ql.instructions.md @@ -46,6 +46,12 @@ This file contains instructions for working with CodeQL query (`.ql`) and librar - ALWAYS create comprehensive test cases in `javascript/frameworks/xsjs/test/`. - ALWAYS include realistic XSJS code patterns in tests. - ALWAYS verify expected results before accepting with `codeql test accept`. +- ALWAYS understand the format of `.expected` files: + - Model tests: Each line = one matched instance of the modeled API/pattern + - Query tests: Multiple sections (edges, nodes, #select) showing data flow and alerts +- ALWAYS validate that `.expected` files contain the correct number of results. +- ALWAYS check that `#select` section in query tests shows only legitimate security alerts. +- ALWAYS use `find javascript/frameworks/xsjs/ -type f -name "*.expected"` to locate test files. ## PREFERENCES diff --git a/.github/prompts/cap_framework_development.prompt.md b/.github/prompts/cap_framework_development.prompt.md index 81b7bebfb..073074ac5 100644 --- a/.github/prompts/cap_framework_development.prompt.md +++ b/.github/prompts/cap_framework_development.prompt.md @@ -91,43 +91,111 @@ srv.after('READ', 'Books', async (data, req) => { ## CodeQL Modeling Workflow -### 1. Understand the Pattern +### TDD Approach for New Queries/Models -Before modeling, understand how the pattern works: +Follow this test-driven development approach: -```bash -# Create test database and run query in one command -codeql test run javascript/frameworks/cap/test/example +### 1. Document Detection Goal + +Clearly specify what the query/model should detect: -# View test results -cat javascript/frameworks/cap/test/example/*.expected +```markdown +## Detection Goal +Pattern: SQL injection via CAP srv.run() with user input +Source: req.data properties in event handlers +Sink: String argument to srv.run() +Expected Results: 2 alerts (positive cases), 0 alerts for sanitized input (negative case) ``` -### 2. Create Test Cases +### 2. Create Test Code -Always create tests first: +Write test cases demonstrating both vulnerable and safe patterns: ```javascript -// javascript/frameworks/cap/test/sql-injection/test.js +// javascript/frameworks/cap/test/queries/sql-injection/test.js const cds = require('@sap/cds'); module.exports = async (srv) => { + // POSITIVE CASE 1: Direct injection srv.on('READ', 'Books', async (req) => { - const userInput = req.data.title; // Source - await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); // Sink + const userInput = req.data.title; // Source at line 6 + await srv.run(`SELECT * FROM Books WHERE title = '${userInput}'`); // Sink at line 7 + }); + + // POSITIVE CASE 2: Via variable + srv.on('UPDATE', 'Books', async (req) => { + const id = req.data.id; // Source at line 12 + const query = `DELETE FROM Books WHERE id = ${id}`; // Flow at line 13 + await srv.run(query); // Sink at line 14 + }); + + // NEGATIVE CASE: Parameterized (safe) + srv.on('DELETE', 'Books', async (req) => { + await srv.run('SELECT * FROM Books WHERE id = ?', [req.data.id]); // Safe }); }; ``` -Expected results: +### 3. Generate `.expected` File BEFORE Implementation + +Manually create the expected results based on your analysis: + +**For Model Tests** (validating library modeling): ``` -// javascript/frameworks/cap/test/sql-injection/sql-injection.expected -| test.js:5:12:5:59 | ... + ... | test.js:4:23:4:37 | req.data.title | This query depends on a $@. | test.js:4:23:4:37 | user-provided value | user-provided value | +// javascript/frameworks/cap/test/models/cql/insert/insert.expected +| insert.js:2:14:5:2 | insert.js:2 | insert.js:2:14:5:2 | INSERT( ... " },\\n]) | +| insert.js:6:14:9:2 | insert.js:6 | insert.js:6:14:9:2 | INSERT( ... " },\\n]) | ``` +Each line = one matched instance of the modeled API/pattern. + +**For Query Tests** (validating security queries): + +Analyze the test code to predict data flow: +- Line 6: `req.data.title` is the source +- Line 7: Template literal flows tainted data to `srv.run()` sink +- Line 12: `req.data.id` is another source +- Line 13: Assignment creates intermediate flow node +- Line 14: `srv.run(query)` is the sink +- Line 19: Safe case - should have NO alert + +Create `.expected` file with predicted results: + +```bash +cat > javascript/frameworks/cap/test/queries/sql-injection/sql-injection.expected << 'EOF' +edges +| test.js:6:18:6:32 | req.data.title | test.js:6:11:6:19 | userInput | provenance | | +| test.js:6:11:6:19 | userInput | test.js:7:56:7:65 | userInput | provenance | | +| test.js:12:15:12:23 | req.data.id | test.js:12:11:12:12 | id | provenance | | +| test.js:12:11:12:12 | id | test.js:13:47:13:48 | id | provenance | | +| test.js:13:11:13:15 | query | test.js:14:18:14:22 | query | provenance | | + +nodes +| test.js:6:18:6:32 | req.data.title | semmle.label | req.data.title | +| test.js:6:11:6:19 | userInput | semmle.label | userInput | +| test.js:7:56:7:65 | userInput | semmle.label | userInput | +| test.js:12:15:12:23 | req.data.id | semmle.label | req.data.id | +| test.js:12:11:12:12 | id | semmle.label | id | +| test.js:13:47:13:48 | id | semmle.label | id | +| test.js:13:11:13:15 | query | semmle.label | query | +| test.js:14:18:14:22 | query | semmle.label | query | + +#select +| test.js:7:11:7:67 | srv.run(...) | test.js:6:18:6:32 | req.data.title | test.js:7:56:7:65 | userInput | This query depends on a $@. | test.js:6:18:6:32 | req.data.title | user-provided value | +| test.js:14:11:14:23 | srv.run(query) | test.js:12:15:12:23 | req.data.id | test.js:14:18:14:22 | query | This query depends on a $@. | test.js:12:15:12:23 | req.data.id | user-provided value | +EOF +``` + +**Key Analysis Points:** +- Count expected alerts: 2 (lines 6-7 and 12-14) +- Identify all flow steps for `edges` section +- Include all nodes in data flow for `nodes` section +- Format `#select` with proper message template + +### 4. Implement the Query/Model -### 3. Implement Remote Flow Sources +Now implement the CodeQL code to detect the pattern: -Model sources in `lib/.../RemoteFlowSources.qll`: +**For Remote Flow Sources** (`lib/.../RemoteFlowSources.qll`): ```ql private import javascript @@ -216,6 +284,26 @@ class CdsServiceReference extends Expr { 2. **Both positive and negative cases**: Test what should and shouldn't alert 3. **Include CDS files**: When relevant to the pattern 4. **Document expected behavior**: Comment in test files why something should alert +5. **Understand .expected files**: + - Model tests: Count lines to verify all expected matches are found + - Query tests: Focus on `#select` section for actual alerts + - Validate data flow paths in `edges` section make logical sense +6. **Find existing tests**: Use `find javascript/frameworks/cap/ -type f -name "*.expected"` to locate similar tests + +### Understanding Test Results + +**Model Test Results** (`test/models/`): +- Simple output: one line per matched API usage +- Validates that library correctly identifies CAP/CDS patterns +- Example: Testing `CqlInsert` class finds all `INSERT.into()` calls + +**Query Test Results** (`test/queries/`): +- Complex output with multiple sections: + - `edges`: Shows data flow from source → sink + - `nodes`: All intermediate taint tracking steps + - `#select`: **Final alerts** (this is what users see) +- Validates end-to-end security vulnerability detection +- Count lines in `#select` to know how many alerts are expected ## Validation Checklist diff --git a/.github/prompts/codeql_test_commands.prompt.md b/.github/prompts/codeql_test_commands.prompt.md index 8b3a8f9b7..25f5ac84c 100644 --- a/.github/prompts/codeql_test_commands.prompt.md +++ b/.github/prompts/codeql_test_commands.prompt.md @@ -65,25 +65,202 @@ cat /*.expected diff /*.expected /*.actual ``` -## Common Workflow +## Test-Driven Development (TDD) Workflow +For **new** queries or models, follow this TDD approach to generate `.expected` files proactively: + +### Step 1: Define Detection Target +Document what the query/model should detect: +- Specific API calls, patterns, or code constructs +- Source and sink locations for data flow +- Expected number of results + +### Step 2: Create Test Code +Write test cases demonstrating the pattern: +```javascript +// test.js - Example showing vulnerable pattern +const userInput = req.data.id; // Source at line 1 +db.run(`SELECT * FROM table WHERE id = ${userInput}`); // Sink at line 2 +``` + +### Step 3: Generate .expected File +**BEFORE running tests**, create the `.expected` file based on your analysis: + +**For model tests:** +```bash +# Create .expected with predicted matches +# Format: | location | context | location | code | +cat > test.expected << 'EOF' +| test.js:2:1:2:50 | test.js:2 | test.js:2:1:2:50 | db.run(...) | +EOF +``` + +**For query tests:** +```bash +# Create .expected with predicted data flow and alerts +cat > test.expected << 'EOF' +edges +| test.js:1:15:1:25 | req.data.id | test.js:2:40:2:49 | userInput | provenance | | + +nodes +| test.js:1:15:1:25 | req.data.id | semmle.label | req.data.id | +| test.js:2:40:2:49 | userInput | semmle.label | userInput | + +#select +| test.js:2:1:2:50 | db.run(...) | test.js:1:15:1:25 | req.data.id | test.js:2:40:2:49 | userInput | This query depends on a $@. | test.js:1:15:1:25 | req.data.id | user-provided value | +EOF +``` + +### Step 4: Implement Query/Model +Write the CodeQL code to detect the pattern. + +### Step 5: Run Tests +```bash +codeql test run +``` + +### Step 6: Validate Results +**Ideal outcome:** Test passes immediately (actual matches expected) ```bash -# 1. Create test case files in test directory +# If test passes - no action needed! + +# If test fails - analyze differences +diff /*.expected /*.actual +``` + +### Step 7: Iterate if Needed +- If actual has extra results → false positives, refine query +- If actual missing results → false negatives, expand query +- Update `.expected` only if your initial analysis was incorrect + +### Step 8: Format and Commit +```bash +codeql query format --in-place +``` + +## Traditional Workflow (Updating Existing Tests) + +When modifying existing queries/models: + +```bash +# 1. Format query files +codeql query format --in-place + # 2. Run tests codeql test run # 3. Review results cat /*.actual -# 4. If correct, accept +# 4. ONLY accept new results (i.e. update .expected results) if +# you are certain that all lines (i.e. results) in the .actual +# file are correct and, thus, to be expected. codeql test accept +``` -# 5. Format query files -codeql query format --in-place +## Understanding .expected Files + +The `.expected` file format varies depending on the query type: + +### Model Tests (Library .qll files) + +Model tests validate CodeQL library modeling. Each line in the `.expected` file represents a single result tuple: + +``` +| :::: | : | :::: | | +``` + +**Example from `javascript/frameworks/cap/test/models/cql/insert/insert.expected`:** +``` +| insert.js:2:14:5:2 | insert.js:2 | insert.js:2:14:5:2 | INSERT( ... " },\\n]) | +| insert.js:6:14:9:2 | insert.js:6 | insert.js:6:14:9:2 | INSERT( ... " },\\n]) | +``` + +Each line represents: +- Column 1: Location of the matched AST node (file:start_line:start_col:end_line:end_col) +- Column 2: Context location (file:line) +- Column 3: Full location range +- Column 4: String representation of the code + +### Query Tests (Security .ql files) + +Query tests for security vulnerabilities include data flow information. The `.expected` file has multiple sections: + +**Section 1: `edges` - Data flow edges showing taint propagation** +``` +edges +| | | | | provenance | | +``` + +**Section 2: `nodes` - All data flow nodes involved** +``` +nodes +| | | semmle.label |