Research / Benchmark Notes
Building a Local CUDA TTS Testing Bench
A practical CUDA TTS testing bench for comparing local voice cloning models, first-run issues, and RTX 5070 behavior.
Published on
Focus
Testing local voice cloning models with repeatable Docker and PowerShell workflows.
Notes
Includes first-run setup problems, speed checks, model setup, and practical RTX 5070 test observations.
Testing Stack
Docker, PowerShell, CUDA, voice samples, model notes, and a local GPU workflow designed for quick comparison.
I built a small local project for testing text to speech and voice cloning models on a CUDA GPU. The repo is called TTS-testers-local-cuda. The idea is simple. Pick one model from a PowerShell menu, start only that Docker container, generate audio locally, then unload the model when I quit.
This is not a hosted TTS product or a production API. It is a local bench for testing model load time, synthesis speed, output quality, Docker isolation, and basic voice clone behavior on the same machine.
Why I Built It
Local TTS testing gets messy fast. Each model wants its own Python packages, CUDA stack, cache folder, codec setup, and runtime defaults. I did not want one broken dependency to wreck every other model, so I split the runners into separate Docker containers.
The PowerShell launcher is the frontend. I run .\run.ps1, choose a model, wait for the ready prompt, type text, and get WAV and JSON output under the local output folder.
Included Models
The current public version includes XTTS-v2, Qwen3-TTS 12Hz 0.6B Base, MioTTS 0.1B, and LuxTTS 100M.
The launcher runs one model at a time. That keeps GPU memory easier to reason about and makes it clearer which model caused a problem during testing.
RTX 5070 Target
The main test target was an NVIDIA RTX 5070. The default build targets RTX 50 series and Blackwell GPUs with PyTorch CUDA 13 wheels.
The default stack uses torch==2.11.0, torchaudio==2.11.0, and the CUDA 13 PyTorch wheel index. There is also a CUDA 12.8 option for cases where that stack is needed.
The launcher checks for the RTX 5070 with nvidia-smi, sets the Docker GPU mask, and exposes only the selected GPU to the active container.
First Run Notes
The first run can take a while because models and caches need to download. The important part is to wait for the model specific ready line before typing synthesis text.
The ready prompt names are xttsv2, qwen0.6btts, miotts0.1b, and luxtts100m. Typing early can make it look like the runner is stuck when it is still downloading or loading.
Voice references stay local. The normal filenames are reference.wav and reference.txt. The WAV is the voice sample and the TXT is the exact transcript. Those files are ignored by git along with generated audio, model caches, Docker exports, and local environment files.
Informal Test Notes
These are rough local measurements from short prompts on the RTX 5070. They are not lab benchmarks, but they were useful for comparing local inference behavior.
LuxTTS 100M used 87 characters. Load and voice prompt prep took about 18.2 seconds. It generated 7.55 seconds of audio in 3.94 seconds with a 0.52x realtime factor. It was fast after load, but I slowed it down to reduce skipped words.
MioTTS 0.1B used 84 characters. Load, codec setup, and voice embedding took about 10.9 seconds. It generated 5.28 seconds of audio in 12.54 seconds with a 2.37x realtime factor. A token cap helped reduce drawn out vowel loops.
Qwen3-TTS 0.6B Base used 64 characters. Load and voice prompt prep took about 35.6 seconds. It generated 4.88 seconds of audio in 19.61 seconds with a 4.02x realtime factor. It runs on SDPA by default because flash_attention_3 failed on the RTX 5070 target with a CUDA kernel image error.
For realtime factor, below 1.0x is faster than realtime. Above 1.0x is slower than realtime.
What Needed Tuning
LuxTTS initially spoke too fast and skipped words. Slowing it down and letting it use the full reference prompt helped.
MioTTS initially stretched vowels and syllables. I added a dynamic token cap plus repetition controls to keep short generations from running away.
Qwen3-TTS loaded and ran on the GPU, but it was still much slower than I would want for live conversation. It is useful for quality testing, but the early numbers make it a poor fit for low latency voice loops on this machine.
Public Repo Hygiene
Before making the repo public, I kept it to source and build instructions only. No voice samples, transcripts, generated WAV files, model weights, Docker image exports, API keys, private machine defaults, or local clone settings are included.
The repo is source available for local testing and evaluation. It is not a hosted service and it is not meant to redistribute built containers, model caches, or third party model weights.