fix: handle JSON-string finish arguments in ToolEnvironment._extract_llm_answer

[howl-anderson]

Summary

ToolEnvironment._extract_llm_answer raises AttributeError: 'str' object has no attribute 'get' when the model ends an episode with an explicit finish tool call (e.g. Qwen-format <tool_call>{"name": "finish", "arguments": {"response": "..."}}</tool_call>) instead of plain text.

Root cause

The two code paths in ToolAgent.update_from_model produce inconsistent types for function.arguments:

• Fallback path (no tool call parsed → synthetic finish): arguments is a dict ({"response": response}).
• Parser path (tool call parsed from model text): arguments is json.dumps-stringified:

# tunix/rl/agentic/agents/tool_agent.py
args = tool_call.arguments
if isinstance(args, dict):
    args = json.dumps(args)

_extract_llm_answer only handles the dict shape:

args = call["function"].get("arguments", {})
return args.get("response", "")   # AttributeError when args is a JSON string

So any model that learned to emit finish through the documented tool-call format crashes the rollout during reward computation. (The episode-termination check just above, name == "finish", matches fine — only the answer extraction breaks.)

Fix

Make _extract_llm_answer tolerant of the stringified form: parse JSON-string arguments, and fall back to returning the raw string when it isn't valid JSON. Changing ToolAgent to stop stringifying was considered, but other consumers may rely on the string form, so this is the narrower fix.

Testing

Added tests/rl/agentic/environments/tool_environment_test.py covering:

• dict arguments (existing behavior, still passes)
• JSON-string arguments (regression: crashed before this fix)
• non-JSON string arguments
• an end-to-end env.step() with a JSON-string finish call producing the correct reward

5 passed in 1.20s

Fixes #2050