fix(voice): honor pcm16 audio sample rate#1909
Conversation
Add a sampleRate option to VoiceAgentOptions and include it in the audio_config message sent when a call starts. This lets servers declare the native rate of raw pcm16 output while preserving the existing 16 kHz default. Teach VoiceClient to read sampleRate from audio_config and use it when constructing AudioBuffer instances for pcm16 playback instead of always assuming 16 kHz. Encoded formats continue to rely on browser decoding. Add protocol and playback tests for configured 24 kHz pcm16 output and the default 16 kHz fallback, including a dedicated test Durable Object binding.
🦋 Changeset detectedLatest commit: 6bacf49 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if (typeof msg.sampleRate === "number" && msg.sampleRate > 0) { | ||
| this.#sampleRate = msg.sampleRate; | ||
| } |
There was a problem hiding this comment.
🟡 Sample rate from a previous call leaks into the next call when the server omits it
The sample rate is only updated when the server sends a positive number (voice-client.ts:680-682) but is never reset to the default 16000 when the field is absent, so a stale value from a prior call persists.
Impact: Audio from a second call could play at the wrong speed (e.g. 24 kHz instead of 16 kHz) if the server omits the sample rate.
Trigger condition and mechanism
#sampleRate is initialized to 16000 at voice-client.ts:273 and conditionally set inside the audio_config handler at voice-client.ts:680-682. Unlike #audioFormat (which is unconditionally assigned at voice-client.ts:679), #sampleRate is only written when msg.sampleRate is a positive number. Neither endCall() (voice-client.ts:538-548), startCall() (voice-client.ts:499-536), nor #stopLocalCall() (voice-client.ts:560-571) resets it.
The VoiceServerMessage type at types.ts:47 declares sampleRate as optional (sampleRate?: number), so a server that sends { type: "audio_config", format: "pcm16" } without sampleRate is valid. If a prior call set #sampleRate to 24000, the second call's pcm16 audio will be decoded at 24 kHz instead of the default 16 kHz, producing distorted playback.
| if (typeof msg.sampleRate === "number" && msg.sampleRate > 0) { | |
| this.#sampleRate = msg.sampleRate; | |
| } | |
| if (typeof msg.sampleRate === "number" && msg.sampleRate > 0) { | |
| this.#sampleRate = msg.sampleRate; | |
| } else { | |
| this.#sampleRate = 16000; | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
| get sampleRate(): number { | ||
| return this.#sampleRate; | ||
| } |
There was a problem hiding this comment.
🟡 Missing changeset for public API additions to a published package
New public API surface (a sampleRate getter on VoiceClient at voice-client.ts:653-655 and a sampleRate option on VoiceAgentOptions at voice.ts:131) is added to @cloudflare/voice without a corresponding changeset entry.
Impact: The feature will not appear in the changelog and the package version will not be bumped on release.
Rule and affected locations
AGENTS.md states: "Changes to packages/ that affect the public API or fix bugs need a changeset." The existing .changeset/voice-transcriber-readiness.md covers a different feature (transcriber readiness). This PR introduces a new public getter VoiceClient.sampleRate and a new configuration option VoiceAgentOptions.sampleRate, both of which are public API additions requiring their own changeset.
Was this helpful? React with 👍 or 👎 to provide feedback.
agents
@cloudflare/ai-chat
@cloudflare/codemode
create-think
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
…s it; add changeset Co-Authored-By: Beta-Devin AI <248786709+beta-devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Beta-Devin AI <248786709+beta-devin-ai-integration[bot]@users.noreply.github.com>
1 participant
Summary
sampleRateoption toVoiceAgentOptionsfor raw PCM output.sampleRatein the serveraudio_configmessage so clients can playpcm16at the provider-native rate.VoiceClientto readsampleRatefromaudio_configand use it when constructingAudioBufferinstances for rawpcm16payloads.sampleRate.Validation
pnpm run checkDiff scope
This replacement PR is based directly on
origin/mainand contains only the voice sample-rate commit.