Skip to content

Commit 84faf53

Browse files
committed
Add profile_codeql_query_from_logs tool & other
- Add `list_mrva_run_results` tool to discover Multi-Repository Variant Analysis run results from `CODEQL_MRVA_RUN_RESULTS_DIRS` directories - Add `profile_codeql_query_from_logs` tool to parse evaluator logs into performance profiles (JSON + Mermaid) without re-running queries - Add reusable evaluator log parser supporting both raw and summary formats - Extend `codeql_database_analyze` with logDir, evaluator-log, and tuple-counting parameters, matching `codeql_query_run` logging support - Generate evaluator log summaries post-execution for query run and database analyze commands - Add `getMrvaRunResultsDirs` to discovery config for MRVA env var - Add unit tests for all new tools and libraries
1 parent 9ea4b14 commit 84faf53

34 files changed

+4978
-399
lines changed
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# `profile_codeql_query_from_logs` - multi_query_raw_log
2+
3+
## Purpose
4+
5+
Tests the `profile_codeql_query_from_logs` tool with a multi-query raw
6+
evaluator log (`evaluator-log.jsonl`) produced by `codeql database analyze`.
7+
8+
This validates that the parser correctly groups predicates by their
9+
`queryCausingWork` event ID reference, producing separate per-query profiles.
10+
11+
## Inputs
12+
13+
- `before/evaluator-log.jsonl` — Raw evaluator log containing two queries
14+
(QueryA.ql and QueryB.ql) with distinct predicates grouped by
15+
`queryCausingWork` event ID references.
16+
- `test-config.json` — Specifies `evaluatorLog`, `outputDir`, and `topN=10`.
17+
18+
## Outputs
19+
20+
- `after/query-evaluation-profile.json` — Structured profile with two queries:
21+
QueryA (2 predicates, 1 cache hit, 200ms) and QueryB (3 predicates, 0 cache
22+
hits, 300ms).
23+
- `after/query-evaluation-profile.md` — Mermaid diagram with multi-query
24+
layout showing a root node branching to Q0 (QueryA) and Q1 (QueryB) with
25+
their respective predicate nodes.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{
2+
"sessions": [
3+
{
4+
"id": "integration_test_session",
5+
"calls": [
6+
{
7+
"tool": "profile_codeql_query_from_logs",
8+
"timestamp": "2025-10-08T21:00:00.000Z",
9+
"status": "success",
10+
"arguments": {
11+
"evaluatorLog": "client/integration-tests/primitives/tools/profile_codeql_query_from_logs/multi_query_raw_log/before/evaluator-log.jsonl",
12+
"outputDir": "client/integration-tests/primitives/tools/profile_codeql_query_from_logs/multi_query_raw_log/after",
13+
"topN": 10
14+
},
15+
"response": {
16+
"message": "Query log profiling completed successfully",
17+
"filesGenerated": [
18+
"query-evaluation-profile.json",
19+
"query-evaluation-profile.md"
20+
]
21+
}
22+
}
23+
]
24+
}
25+
]
26+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
{
2+
"codeqlVersion": "2.23.1",
3+
"logFormat": "raw",
4+
"queries": [
5+
{
6+
"queryName": "/workspace/QueryA.ql",
7+
"totalDurationMs": 200,
8+
"predicateCount": 2,
9+
"predicates": [
10+
{
11+
"predicateName": "QueryA::source/0#abc123",
12+
"position": "/workspace/QueryA.ql:10,1-12,5",
13+
"durationMs": 65,
14+
"resultSize": 5,
15+
"pipelineCount": 1,
16+
"evaluationStrategy": "SIMPLE_INTENSIONAL",
17+
"dependencies": []
18+
},
19+
{
20+
"predicateName": "QueryA::sink/0#def456",
21+
"position": "/workspace/QueryA.ql:14,1-16,5",
22+
"durationMs": 45,
23+
"resultSize": 3,
24+
"pipelineCount": 1,
25+
"evaluationStrategy": "SIMPLE_INTENSIONAL",
26+
"dependencies": [
27+
"QueryA::source/0#abc123"
28+
]
29+
}
30+
],
31+
"cacheHits": 1
32+
},
33+
{
34+
"queryName": "/workspace/QueryB.ql",
35+
"totalDurationMs": 300,
36+
"predicateCount": 3,
37+
"predicates": [
38+
{
39+
"predicateName": "QueryB::entryPoint/0#jkl012",
40+
"position": "/workspace/QueryB.ql:8,1-10,3",
41+
"durationMs": 95,
42+
"resultSize": 8,
43+
"pipelineCount": 1,
44+
"evaluationStrategy": "EXTENSIONAL",
45+
"dependencies": []
46+
},
47+
{
48+
"predicateName": "QueryB::flowStep/2#mno345",
49+
"position": "/workspace/QueryB.ql:12,1-18,5",
50+
"durationMs": 95,
51+
"resultSize": 12,
52+
"pipelineCount": 1,
53+
"evaluationStrategy": "SIMPLE_INTENSIONAL",
54+
"dependencies": [
55+
"QueryB::entryPoint/0#jkl012"
56+
]
57+
},
58+
{
59+
"predicateName": "QueryB::result/3#pqr678",
60+
"position": "/workspace/QueryB.ql:20,1-22,5",
61+
"durationMs": 45,
62+
"resultSize": 2,
63+
"pipelineCount": 1,
64+
"evaluationStrategy": "SIMPLE_INTENSIONAL",
65+
"dependencies": [
66+
"QueryB::flowStep/2#mno345",
67+
"QueryB::entryPoint/0#jkl012"
68+
]
69+
}
70+
],
71+
"cacheHits": 0
72+
}
73+
],
74+
"totalEvents": 27
75+
}
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
```mermaid
2+
graph TD
3+
4+
ROOT["Evaluation Log<br/>2 queries"]
5+
6+
Q0["QueryA.ql<br/>200.00ms<br/>Predicates: 2"]
7+
ROOT --> Q0
8+
Q0P0["QueryA::source/0#abc123<br/>65.00ms | 5 results"]
9+
Q0 --> Q0P0
10+
Q0P1["QueryA::sink/0#def456<br/>45.00ms | 3 results"]
11+
Q0 --> Q0P1
12+
13+
Q1["QueryB.ql<br/>300.00ms<br/>Predicates: 3"]
14+
ROOT --> Q1
15+
Q1P0["QueryB::entryPoint/0#jkl012<br/>95.00ms | 8 results"]
16+
Q1 --> Q1P0
17+
Q1P1["QueryB::flowStep/2#mno345<br/>95.00ms | 12 results"]
18+
Q1 --> Q1P1
19+
Q1P2["QueryB::result/3#pqr678<br/>45.00ms | 2 results"]
20+
Q1 --> Q1P2
21+
22+
23+
classDef default fill:#e1f5ff,stroke:#333,stroke-width:2px
24+
classDef query fill:#ffe1e1,stroke:#333,stroke-width:3px
25+
class QUERY query
26+
```

0 commit comments

Comments
 (0)