Skip to content

feat(scores): support text scores in v3#1661

Merged
wochinge merged 5 commits into
v3-stablefrom
backport-text-scores-v3
May 21, 2026
Merged

feat(scores): support text scores in v3#1661
wochinge merged 5 commits into
v3-stablefrom
backport-text-scores-v3

Conversation

@wochinge
Copy link
Copy Markdown
Contributor

@wochinge wochinge commented May 20, 2026

Summary

  • Backports TEXT score support to the v3 SDK score creation APIs and public score typing.
  • Adds v3 generated API-style TEXT score enums, models, response unions, and exports so score and score-config responses can deserialize TEXT values.
  • Keeps experiment evaluator score typing limited to numeric, categorical, and boolean scores.
  • Adds live coverage for creating a TEXT score and creating/listing a TEXT score config.
  • Backport from: feat(api): update API spec from langfuse/langfuse 1999706 #1615

Linear

Major Decisions

  • Applied the TEXT score API changes as a targeted v3 generated-style patch instead of regenerating from the current Fern spec, because the v3 generated API layout has drifted from current main and a full regeneration would introduce unrelated churn.

Review Focus

  • TEXT variants in the generated API-style unions and exports under langfuse/api/resources.
  • SDK overload/type behavior for string score values, especially preserving categorical defaults while allowing explicit TEXT.
  • ExperimentScoreType usage to avoid unintentionally enabling TEXT scores in experiment evaluator typing.
  • Live score-config test behavior, including archiving the created TEXT config after assertions.

Backport TEXT score typing and generated API models to v3-stable.
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: adec90aa51

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread langfuse/api/resources/commons/types/score_config_data_type.py
Comment thread langfuse/api/resources/commons/types/score_data_type.py
Comment thread langfuse/api/__init__.py Outdated
wochinge added 3 commits May 20, 2026 10:42
Backport dataset archived-item assertions and update flaky live test expectations.
Update generated export ordering and use the requested audio model.
Comment thread .github/workflows/ci.yml
Comment thread tests/test_datasets.py
Allow the TEXT score response to omit null numeric values while asserting stringValue.
@wochinge wochinge requested a review from hassiebp May 20, 2026 09:45
@wochinge wochinge merged commit 321b40a into v3-stable May 21, 2026
10 of 11 checks passed
@wochinge wochinge deleted the backport-text-scores-v3 branch May 21, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants