Skip to content

fix: explicitly select CUDAExecutionProvider to avoid silent CPU fallback when TensorRT is absent (closes #5860)#5877

Open
botbikamordehai2-sketch wants to merge 1 commit into
livekit:mainfrom
botbikamordehai2-sketch:fix/issue-5860-1779959407
Open

fix: explicitly select CUDAExecutionProvider to avoid silent CPU fallback when TensorRT is absent (closes #5860)#5877
botbikamordehai2-sketch wants to merge 1 commit into
livekit:mainfrom
botbikamordehai2-sketch:fix/issue-5860-1779959407

Conversation

@botbikamordehai2-sketch
Copy link
Copy Markdown

What

When force_cpu=False and a CUDA GPU is available but TensorRT is not installed, ONNX Runtime silently falls back to CPUExecutionProvider instead of using CUDAExecutionProvider. This happens because ORT's default provider priority list places TensorrtExecutionProvider before CUDAExecutionProvider, and when TRT fails to load it skips CUDA entirely.

Fix

Explicitly build the providers list in new_inference_session() by checking onnxruntime.get_available_providers() and preferring CUDAExecutionProvider when it is available and force_cpu=False. This ensures CUDA is used whenever possible, regardless of whether TensorRT is installed.

Closes #5860

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


botbikamordehai2-sketch seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

Open in Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

silero VAD silently falls back to CPU when TensorRT is absent, despite CUDA being available

2 participants