Skip to content

Fix multiclass macro F-score (mean of per-class F, not F of the means)#448

Open
gaoflow wants to merge 1 commit into
rust-ml:masterfrom
gaoflow:fix-multiclass-macro-f-score
Open

Fix multiclass macro F-score (mean of per-class F, not F of the means)#448
gaoflow wants to merge 1 commit into
rust-ml:masterfrom
gaoflow:fix-multiclass-macro-f-score

Conversation

@gaoflow

@gaoflow gaoflow commented Jul 21, 2026

Copy link
Copy Markdown

Multiclass f_score / f1_score returned the F-beta of the macro-averaged precision and recall, but the macro F-score is the mean of the per-class F-beta scores. Those agree only when every class shares the same precision and recall, so multiclass f1_score was wrong for most inputs.

Repro (y_true=[0,0,1,1,2,2], y_pred=[0,1,1,1,2,2]): f1_score() returned 0.860215; the macro F1 is 0.822222 (per-class F1 = [2/3, 4/5, 1], matching sklearn f1_score(average="macro")). Macro precision and recall were already correct, which isolates the error to the F combination.

The fix mirrors the existing precision/recall macro averaging: average the per-class one-vs-all F-beta. A class the model never gets right has precision = recall = 0 (a 0/0 F); it now counts as 0 (matching sklearn's zero_division=0) instead of turning the whole average into NaN. Binary matrices are unchanged.

Two regression tests added; full cargo test --workspace, cargo fmt and cargo clippy pass.

One caveat: for beta != 1 the aggregation is now correct, but linfa's per-class precision and recall are oriented opposite to sklearn's, so f_score(beta) lines up with sklearn's fbeta_score(beta=1/beta). That orientation is pre-existing (it affects the binary path too) and left out of scope here; f1 is symmetric so it is unaffected.

f_score/f1_score computed the F-beta of the macro-averaged precision and
recall for multilabel matrices, but macro F is the mean of the per-class
F-beta scores; the two differ whenever classes have unequal precision or
recall. Average the per-class one-vs-all F-beta instead (mirroring
precision/recall), counting an all-wrong class (0/0) as 0. Binary
matrices are unchanged.
@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.92%. Comparing base (7fe5c86) to head (26d20db).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #448      +/-   ##
==========================================
+ Coverage   77.53%   77.92%   +0.39%     
==========================================
  Files         106      104       -2     
  Lines        7585     7534      -51     
==========================================
- Hits         5881     5871      -10     
+ Misses       1704     1663      -41     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant