fix(audio): keep built-in mic capture alive on Apple Silicon - #912
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b79d729a95
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Greptile SummaryThe PR keeps direct Core Audio capture alive for the internal microphone on Apple Silicon by starting a built-in output device for the capture lifetime.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02aa44437f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| isInternalMicrophone && | ||
| selectedInputUID != nil && | ||
| selectedInputUID == defaultInputUID |
There was a problem hiding this comment.
When the user prioritizes the built-in microphone in FluidVoice while macOS has an external input as its default, this equality makes the compatibility policy return false, so startConfiguredAudioCapture falls through to DirectCoreAudioLifecycleController for the built-in device. That is precisely the backend this change identifies as becoming silent after a few seconds on Apple Silicon, so dictation can still be truncated in this supported independent-routing configuration. The new UID-equality guard avoids the previously reported misrouting, but the fresh behavior is that it routes the selected built-in microphone back through the known-silent backend; the compatibility solution needs to preserve the selected endpoint rather than opting out.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
That's intended. Otherwise the app would ignore the selected user choice.
|
Hi, what is this PR supposed to do again? You have an issue? Microphone is a very sensitive area and the issues that come up are very large. So I'd be very careful before merging this. But if you want to have a discussion on this, we are happy to take a look at it. @grohith327 for viz |
|
@altic-dev yes, FluidVoice is not usable for me since 1.6.7. The issue I had created: #852 This PR fixes that built-in microphone stops capturing input after 3 seconds of dictation on Apple M1. It switches built-in microphone capture to AVAudioEngine instead of using direct Core Audio. Only switches if both selected and default microphone are built-in and it's Apple Silicon. |
We're trying to move away from AvAudio and w e need to find a bette way for this. @grohith327 wdyt? Not sure how to fix this in a bette way |
|
@Defake The AVAudioEngine flow that exists right now will simply be removed in future commits as we have fully committed to the CoreAudio flow. The equivalent fix for this in CoreAudio flow is there in this commit - 6fe6143 Can you try building this and let us know if it works. Thank you for trying really hard to get FV work! |
|
@grohith327 thanks, I tried it 2 weeks ago, it didn't fix the issue: #852 (comment) The problem is not zero-PCM. Capture starts normally and works for 3.5 seconds. Then corespeechd mutes the built-in microphone. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 95b08c30a4
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| status = fv_start_output_keep_alive(capture); | ||
| if (status != noErr) { | ||
| (void) fv_stop_and_destroy_output_keep_alive(capture); | ||
| return status; |
There was a problem hiding this comment.
When the built-in output is alive but cannot start this IOProc—for example, while held in exclusive/hog mode or when its first callback exceeds the fixed one-second timeout—this path aborts startup even though the selected microphone remains independently usable. startConfiguredAudioCapture propagates direct-capture failures, and the direct backend is mandatory, so after retrying or switching inputs the recording ultimately fails; treat a keep-alive failure as a degraded capture mode or fall back to another backend instead of making output availability a prerequisite for dictation.
Useful? React with 👍 / 👎.
|
I reworked this PR so it works with Core Audio now. |
|
Could you try a simpler Core Audio approach first? Start the built-in output with The keep alive should also be best effort. If it fails, it shouldn’t delay or stop microphone capture. The current version you have could cause dictation fails in a lot of devices if it fails. |
95b08c3 to
5aef500
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5aef50044f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| private func stopOutputKeepAlive() { | ||
| guard let deviceID = self.outputKeepAliveDeviceID else { return } | ||
| self.outputKeepAliveDeviceID = nil | ||
| _ = AudioDeviceStop(deviceID, nil) |
There was a problem hiding this comment.
When another application is playing through the built-in output during dictation, AudioDeviceStop(deviceID, nil) stops the device as a whole rather than a keep-alive operation owned by FluidVoice, so ending direct capture can interrupt or strand that application's playback. Register and retain a dedicated no-op output IOProc and pass its ID to both AudioDeviceStart and AudioDeviceStop instead of using the device-wide nil form.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
This suggests to reintroduce the larger IOProc solution we decided to remove
|
@altic-dev thanks, it worked! I reworked the PR to use the simpler solution, please take a look |
3 participants
Description
On Apple Silicon, direct Core Audio capture from the built-in microphone can go silent after dictation starts and truncate the transcription.
This change starts the built-in output device with
AudioDeviceStart(deviceID, nil)during built-in microphone capture, then stops it withAudioDeviceStop. It does not play audio or change the selected devices. If the output fails to start, microphone capture continues normally.Type of Change
Related Issue or Discussion
Closes #852
Testing
swiftlint --strict --config .swiftlint.yml Sourcesswiftformat --config .swiftformat SourcesDirectAudioReliabilityTestspassedDebug builds pass for arm64 and x86_64. The full test run did not finish because the test host repeatedly tried to download a Whisper model and set up Accessibility. The changed files pass SwiftLint and SwiftFormat; repository-wide checks still report existing issues outside this PR.
Screenshots / Video
Notes
The workaround only runs for direct Core Audio capture with the internal microphone on Apple Silicon. Intel and external microphone paths are unchanged.