feat(smallestai): update TTS plugin for Lightning v3.1 Pro and WebSocket streaming#5799
Conversation
…ified endpoint
- Add lightning_v3.1_pro model (premium 44.1 kHz pool, curated voice catalog)
- Rename lightning-v3.1 → lightning_v3.1 to match updated API model ID notation
- Remove lightning-v2 (deprecated) and its exclusive params (consistency, similarity, enhancement)
- Switch TTS endpoint from /{model}/get_speech to unified /tts; model now sent as body param
- Update TTSEncoding: mulaw → ulaw to match new API spec
…ophia for standard
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…cy TTS Switch the agent pipeline path from HTTP POST (/tts) to persistent WebSocket streaming (/tts/live) backed by a ConnectionPool. - Add SynthesizeStream using wss://api.smallest.ai/waves/v1/tts/live - ConnectionPool keeps the WebSocket alive across turns (conn_reused=True) - prewarm() pre-establishes the connection before the first user turn - Keep ChunkedStream (HTTP POST) for direct synthesize() calls - Change capabilities to streaming=True so the agent pipeline uses stream()
| ) from None | ||
| except APIStatusError: | ||
| raise | ||
| except Exception as e: |
There was a problem hiding this comment.
🟡 SynthesizeStream._run catches APIStatusError but not APIConnectionError, causing error message loss
The exception handler at lines 289-292 catches APIStatusError and re-raises it, but does not catch APIConnectionError (or its parent APIError). When _run_ws raises APIConnectionError at livekit-plugins/livekit-plugins-smallestai/livekit/plugins/smallestai/tts.py:343-345 (e.g., "SmallestAI TTS error: {message}"), this falls through to the generic except Exception as e: raise APIConnectionError() from e handler, which wraps it in a new APIConnectionError("Connection error.") — discarding the specific error message. The established pattern (used by Deepgram at livekit-plugins/livekit-plugins-deepgram/livekit/plugins/deepgram/tts.py:295) is to catch APIError (the common base class) rather than just APIStatusError.
| ) from None | |
| except APIStatusError: | |
| raise | |
| except Exception as e: | |
| except APIStatusError: | |
| raise | |
| except APIConnectionError: | |
| raise |
Was this helpful? React with 👍 or 👎 to provide feedback.
harshitajain165
changed the title
harshitajain165
changed the title
|
Hey @tinalenguyen |
tinalenguyen
left a comment
tinalenguyen
left a comment
There was a problem hiding this comment.
tested it and it lgtm, thank you!
2 participants
Summary
lightning_v3.1_promodel (premium 44.1 kHz pool, curated American, British, and Indian voices)meherfor Pro,sophiafor standardlightning-v3.1→lightning_v3.1to match updated API model ID notationlightning-v2(deprecated) and its exclusive params (consistency,similarity,enhancement)/{model}/get_speechto unified/tts; model now sent as body paramTTSEncoding:mulaw→ulawto match new API specwss://api.smallest.ai/waves/v1/tts/live) backed byConnectionPoolfor significantly lower latencyTest plan
conn_reused=True, audio delivered correctly