fix(inference): ungate Measured Energy axes and give them a Pareto direction / 开放实测能耗坐标轴并补齐 Pareto 方向 - #720
Conversation
…rection The Measured Energy group was hidden behind the konami feature gate, but the gate only ever hid the dropdown row: InferenceContext reads i_metric straight from the URL with no allowlist, so ?i_metric=y_measuredJPerOutputToken already renders these charts in production today, on official data. Make the group selectable. Three of the six axes (y_measuredAvgPower, y_measuredPrefillAvgPower, y_measuredDecodeAvgPower) had no _roofline direction. Without one, InferenceTable sorts a lower-is-better metric descending, and ScatterGraph falls back to lower_right, which is backwards on the e2e block. Declare the direction in both chart blocks, matching the J/token axes. 中文：实测能耗坐标轴组此前被 konami 特性开关隐藏，但该开关只隐藏了下拉选项本身—— InferenceContext 直接从 URL 读取 i_metric 且不做白名单校验，因此 ?i_metric=y_measuredJPerOutputToken 今天就能在生产环境渲染这些图表，用的是正式数据。 本次改动让该分组可以正常选择。六个坐标轴中有三个缺少 _roofline 方向声明，导致表格视图 对“越低越好”的指标按降序排列，且 ScatterGraph 回退到 lower_right——在 e2e 图表块上方向 是反的。现为这三个坐标轴在两个图表块中补齐方向声明，与 J/token 系列保持一致。
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @edwingao28's task in 3m 1s —— View job 🟢 LGTM — no blocking issues foundReviewed the diff and verified the changed behavior at runtime against the real read-only DB. What the PR does (all sound):
Runtime verification (localhost, real data):
CI: typecheck/lint/component + E2E shards (incl. the touched Minor, non-blocking: after this change no metric group in The four out-of-scope items called out in the PR description (partial-coverage legend pruning, empty-state copy, Historical Trends reuse, disagg J/token vintage) are pre-existing and reachable today via |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b129bb8f50
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Claude finished @edwingao28's task in 2m 51s —— View job Re-review: 🟢 LGTM with one non-blocking test-coverage noteThe fix itself is correct and I confirmed it at runtime. Since the prior review (LGTM), the only new commit is a
Correctness — verified. Dumped every CI: typecheck/unit, lint, component, Chrome E2E shards (incl. the touched 🟡 Non-blocking — regression coverage gap (agrees with Codex P1 #2): the fix lives entirely in JSON config values, but no touched test asserts them — reversing On Codex P1 #1 ( No blocking issues. Good to merge; adding the config-direction regression test before merge would be the ideal close-out. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
…ches Ungating the Measured Energy axes made a destructive flow part of the normal UI. Those axes only exist for configs whose run recorded power telemetry; useChartData drops the rest, and the legend's active set was reconciled against that metric-filtered set. reconcileActiveSet never re-widens, so switching back to throughput left the pruned configs deselected until the user reset them. Split the two sets instead of tracking and restoring: activeHwTypes now lives in a metric-independent universe (selectableHwTypes, built from the scope filters only), while hwTypesWithData stays metric-aware and drives what the legend renders and the chart draws. Nothing is pruned, so nothing has to be restored, and a config the user removed by hand stays off. Toggle, select-all, preset, URL-restore, reset, and i_active serialization all move to the universe so their set-size comparisons stay consistent. Also add the regression coverage the new _roofline values were missing: the measured-power frontier picks the lower-right corner on the interactivity block and the lower-left corner on e2e, dominated points stay out of the Optimal Only set, and the table sorts ascending. The table sort moved to a pure helper so it can be asserted directly. Flipping or deleting the three _roofline values makes these tests fail. 中文：解除 Measured Energy 坐标轴的隐藏，等于把一条破坏性流程带进了常规交互。这些坐标轴 只对记录了功耗遥测的配置存在，useChartData 会过滤掉其余配置，而图例的选中集合此前正是与 这个按指标过滤后的集合做交集。reconcileActiveSet 只收不放，因此切回吞吐量坐标轴时，被裁掉 的配置仍处于未选中状态，只能靠重置或刷新恢复。 本次改为拆分两个集合，而不是记录后再补回：activeHwTypes 现在处于与指标无关的全集 （selectableHwTypes，仅由作用域过滤器构成），hwTypesWithData 仍与指标相关，负责决定图例 渲染和图表绘制的内容。既然不再裁剪，就无需恢复；用户手动移除的配置也会保持关闭。图例切换、 全选、预设、URL 还原、重置以及 i_active 序列化都改用该全集，以保证各处的集合规模比较一致。 同时补上新增 _roofline 值缺失的回归测试：实测功耗的 Pareto 前沿在 interactivity 图表块取 右下角、在 e2e 图表块取左下角，被支配的点不会进入 Optimal Only 集合，表格按升序排列。表格 排序逻辑抽成纯函数以便直接断言。翻转或删除这三个 _roofline 值都会让这些测试失败。
The selection universe is no longer metric-filtered, so activeHwTypes and hwTypesWithData can now come from different sets and a size comparison is not a valid "something is deselected" test — on a partially-covered axis the active set is the larger one, which would hide the reset action exactly when it is needed. 中文：选择范围不再按指标过滤，activeHwTypes 与 hwTypesWithData 可能来自不同集合， 因此用集合大小比较判断"是否有取消勾选"已不成立——在覆盖不完整的坐标轴上，活动集合反而更大， 会在最需要重置操作时把它隐藏掉。改为按成员判断。
Resolves the InferenceContext conflict between the metric-independent selection universe added here and the bestPerSku / bestHwTypes mode that landed on master meanwhile. bestPerSku keeps deriving from the current axis by design; every automatic full-set write falls back to selectableHwTypes instead of the metric-filtered set. 中文：解决 InferenceContext 的冲突——本分支引入的"与指标无关的可选范围"与期间合入 master 的 bestPerSku / bestHwTypes 模式相遇。bestPerSku 按设计仍随当前坐标轴推导；其余所有自动写入完整 集合的位置改用 selectableHwTypes，而不是按指标过滤后的集合。
|
Claude finished @edwingao28's task in 7m 52s —— View job Re-review: 🔴 One blocking issue — new E2E spec fails in CIFocused on the new commits since the last LGTM (
🔴 BLOCKING — the new spec Root cause: the legend-reset effect ( The rest of the follow-up is sound:
CI: Lint & Format, Tests (Unit), Code Quality all green; Tests (E2E) failing (shard 2, both browsers) — the blocking item above. Once the reset-key change lands and shard 2 goes green, this is good to merge. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit b4d3e17. Configure here.
This comment has been minimized.
This comment has been minimized.
It guarded nothing: the passing half also passes with the InferenceContext fix reverted, and the failing half asserted that a hand-removed config survives a Y-metric switch, which hwResetKey has always reset by design. useChartData.test.ts covers the fix.
|
Claude finished @edwingao28's task in 3m 34s —— View job Re-review: 🟢 LGTM — prior blocking issue resolved
The one new commit since the last review is
Regression coverage for the fix that actually shipped is retained at the unit level, and it targets the real mechanism rather than the incidental reset:
So the deleted spec guarded nothing the remaining tests don't, and its removal turns shard 2 green. CI: Lint & Format, Tests (Unit), Tests (E2E), Code Quality — all green. Runtime (localhost, real read-only DB): The four out-of-scope items in the PR description remain pre-existing and reachable today via |
Pulls in SemiAnalysisAI#720 (Measured Energy axes). No conflicts: that change is confined to the inference chart components and shares no files with the overview work on this branch. Full unit suite, typecheck, lint and format are clean on the merged tree. Co-authored-by: Cursor <cursoragent@cursor.com>
1 participant
The Measured Energy group was hidden behind the konami feature gate. But the gate only ever hid the dropdown row —
InferenceContextreadsi_metricstraight from the URL with no allowlist, so?i_metric=y_measuredJPerOutputTokenalready renders these charts in production today, on official data, for anyone who types the URL. This makes the group selectable from the dropdown, and fixes the two bugs that URL path already exposes.Three of the six axes had no
_rooflinedirection:y_measuredAvgPower,y_measuredPrefillAvgPower,y_measuredDecodeAvgPower(the three J/token axes already had one). Without a direction:InferenceTablesorts a lower-is-better metric descending — the highest-wattage config lands at the top of the list.ScatterGraphfalls back tolower_right, which is correct on the interactivity block but backwards on the e2e block, where every lower-is-better metric islower_left. The published "optimal" line traces the slowest config at each power level.Both are reachable today via
?i_metric=.The gate mechanism itself is kept —
useFeatureGatestill drives the hidden tab popover intab-nav.tsx.Knowingly out of scope, each pre-existing and reachable today:
useChartContext.tsreconcileActiveSet). Needs integration coverage; separate PR.ScatterGraph.tsx,GPUGraph.tsx).ChartControls, so Measured Energy appears there too, keeping its own generic empty-state message.joules_per_output_tokenhas two vintages on disaggregated rows: per-stage for rows produced before 2026-08-05, cluster-wide for rows fromaggregate_power_multinode.py(merged in InferenceX on 2026-08-05). Nothing on a row distinguishes them. Tracked separately; it does not bite today because multi-node runs are not yet instrumented in production.Verified:
bun run typecheck,bun run lint,bun run fmtclean;bun run test:unit3730 tests pass across all four workspaces. Cypress not run locally (needs a server + DB); the only spec touched ismeasured-power-overlay.cy.ts.中文说明
实测能耗（Measured Energy）分组此前被 konami 特性开关隐藏。但该开关只隐藏了下拉菜单中的选项行——
InferenceContext直接从 URL 读取i_metric且不做白名单校验，因此任何人手动输入?i_metric=y_measuredJPerOutputToken，今天就能在生产环境看到这些图表，用的是正式数据。本次改动让该分组可以正常从下拉菜单选择，并修复该 URL 路径已经暴露出来的两个缺陷。六个坐标轴中有三个缺少
_roofline方向声明：y_measuredAvgPower、y_measuredPrefillAvgPower、y_measuredDecodeAvgPower（三个 J/token 坐标轴已经有了）。缺少方向声明会导致：InferenceTable对“越低越好”的指标按降序排列，功耗最高的配置排在列表最前。ScatterGraph回退到lower_right。这在 interactivity 图表块上恰好正确，但在 e2e 图表块上是反的——该块中所有“越低越好”的指标都是lower_left，于是发布出去的“最优”曲线画的是每个功耗水平下最慢的配置。这两个问题今天通过
?i_metric=就能触发。特性开关机制本身予以保留——
useFeatureGate仍在tab-nav.tsx中驱动隐藏标签页的弹层。以下为明确不在本次范围内的已知问题，均为既有问题且今天即可触发：
useChartContext.ts的reconcileActiveSet）。需要集成测试覆盖，另开 PR。ScatterGraph.tsx、GPUGraph.tsx）。ChartControls，因此实测能耗也会出现在该标签页，其空状态沿用原有通用文案。joules_per_output_token存在两种口径：2026-08-05 之前产生的行为分阶段口径，之后由aggregate_power_multinode.py产生的行为全集群口径（该聚合器于 2026-08-05 合入 InferenceX），而行上没有任何字段可以区分。已单独跟踪；由于生产环境的多节点运行尚未接入功耗采集，目前不会实际影响图表。验证：
bun run typecheck、bun run lint、bun run fmt均通过；bun run test:unit四个 workspace 共 3730 项测试全部通过。Cypress 未在本地运行（需要服务端与数据库）；本次改动的 spec 仅measured-power-overlay.cy.ts一个。Note
Medium Risk
Touches core inference chart selection, URL sharing, and legend reconciliation; behavior change is intentional but affects many code paths in
InferenceContext.Overview
Measured Energy is no longer hidden behind the konami feature gate in the Y-axis dropdown, and Cypress no longer sets
inferencex-feature-gatefor overlay tests.The three watt-based measured metrics get
_rooflineentries in chart config (lower_righton interactivity,lower_lefton e2e) so table sort, Pareto frontiers, and Optimal Only use lower-is-better behavior instead of the previouslower_rightfallback. Table sorting moves into sharedsortRowsByYMetric, with regression tests on frontier corners and row order.Legend hardware selection is decoupled from the active Y-metric:
useChartDataexposesselectionPointsviaapplyScopeFilters(GPU/quick-filter/compare only), andInferenceContextdrives toggles, presets, URLi_active, and resets fromselectableHwTypeswhilehwTypesWithDatastays metric-aware for what the chart draws. That stops Measured Energy axes from permanently pruning configs without telemetry when users switch metrics. Historical Trends’ “reset filter” action now treats a partial legend selection as filtered even when sizes match.Reviewed by Cursor Bugbot for commit f021de6. Bugbot is set up for automated code reviews on this repo. Configure here.