Building a Private Local Transcription App with Whisper

Why I Built It

Transcription is a good example of a task that local AI can handle without becoming a complicated assistant or agent. The input is clear, the output is measurable, and the privacy advantage is easy to understand. I wanted a straightforward way to take a recording from my computer, turn it into text, and keep the entire working process local.

The model was only one part of the problem. Running Whisper from a terminal is useful for testing, but it is not the workflow I wanted to use repeatedly. I wanted a browser interface where I could drop in a file, see whether the local service was ready, follow the job state, copy the result, and download the finished transcript.

I also wanted the app to start predictably on Windows. Local AI projects often work after a series of manual commands and then become inconvenient to use later. This project packages the model, media tools, server, and browser interface into one Docker-based application with simple start and stop shortcuts.

The Basic Workflow

The user opens the local page and selects or drops in one audio or video file. The interface supports common recording formats including MOV, MP4, M4A, MP3, WAV, WebM, MKV, FLAC, and OGG. The selected file stays visible with its name, size, and format before transcription begins.

After submission, the app creates a local job and moves through upload, queue, decoding, transcription, and saving states. The processing view shows the filename, configured model, selected device, elapsed time, and a plain explanation of what the service is doing.

When the job finishes, the complete transcript appears in the browser. It can be copied immediately or downloaded as a plain-text file. A second copy is already saved in the local transcripts folder, and the user can reset the interface to process another recording.

Local Whisper Inference

The application uses the open-source Whisper model locally rather than sending recordings to a transcription API. The model files are downloaded during first use and then kept in a persistent local cache. Once the software, container image, and model are available on the machine, transcription does not depend on a hosted transcription service.

The current configuration uses the English-focused small.en model. The model is configurable, so the runtime can trade speed, memory use, and accuracy according to the machine and recording. Larger models can be selected when the additional processing cost makes sense.

Spoken content is treated only as media to transcribe. It is never inserted into an instruction prompt or interpreted as a command for another system. That boundary keeps the application focused on one task and avoids turning arbitrary recording content into application control.

GPU Acceleration with CPU Fallback

The supplied startup process supports separate NVIDIA and CPU runtime images. In automatic mode, the launcher checks whether Docker can actually access CUDA before selecting the accelerated service. If the GPU build or compatibility check fails, the app falls back to the portable CPU service instead of becoming unusable.

Device selection continues inside the transcription runtime. It checks available GPU memory before loading the chosen model and can select the safest available NVIDIA device. If a CUDA model load or transcription attempt fails, the app clears that state and retries on the CPU.

During this release review, the running GPU service was healthy and detected an NVIDIA GeForce RTX 5070. That confirms the current container can see the card. I did not run a formal transcription speed or accuracy benchmark for this post, so I am not presenting performance numbers yet.

Preparing Media for the Model

Audio and video uploads cannot be passed blindly into the model. The app first inspects the file, confirms that it contains an audio stream, checks the container type, and rejects unsupported or unreadable media. It also applies configured file-size and recording-duration limits before model work begins.

Accepted media is converted into a consistent mono WAV format before Whisper sees it. This gives the model a predictable input and keeps video tracks, subtitles, and unrelated data out of the transcription stage. The decoder is restricted to local file input rather than network playlists or remote media locations.

Uploaded media and the temporary WAV file are deleted after the job succeeds or fails. Startup cleanup also removes old temporary files owned by the app while leaving unrelated files alone. The original recording on the user's computer is never modified.

A Bounded Local Queue

Whisper can use a large amount of memory, especially with larger models and long recordings. The app therefore processes one transcription at a time through a bounded local queue. Status requests remain responsive while the model-heavy work is serialized.

This design favors predictable resource use over trying to process several large recordings simultaneously. Each job has a clear queued, running, completed, or failed state. The current configuration accepts a small number of waiting jobs and gives the user a direct message if the queue is full.

Job state is intentionally simple and process-local. This is a single-user desktop utility, not a distributed processing service. Restarting the app preserves completed transcripts and cached model files, while active in-memory jobs are not treated as durable work after a restart.

Safe and Predictable Output

Completed transcripts are written into a host-visible folder as UTF-8 text. The app derives a safe output name from the original recording and adds a number when a transcript with the same name already exists. It does not silently overwrite an earlier result.

Each transcript is first written to a temporary file and then moved into place as a completed output. That keeps an interrupted write from appearing as a valid final transcript. Model output and transcript content are also kept out of application logs and generic error messages.

The result format is intentionally plain. This version is designed to produce complete text that can be copied, searched, summarized, or moved into another workflow. It is not trying to become a full audio editor or collaborative transcript platform.

Local Privacy and Security Boundaries

The default service listens only on the local computer. The supplied configuration is not intended to be exposed directly to a local network or the public internet. The browser and server also verify that requests come through the expected local origin and reject unsupported cross-origin submissions.

Uploaded filenames are sanitized, supported extensions are allowlisted, and media files are inspected before decoding. The Docker service runs without administrator privileges, uses a read-only container filesystem, drops unnecessary operating-system capabilities, and writes only to the dedicated work, model, and transcript locations.

The repository excludes recordings, transcripts, local configuration, model weights, caches, logs, private keys, and common personal-document formats. These controls do not make every modified deployment automatically secure, but they establish a careful default for a single-user local tool.

Windows Startup as Part of the Product

The startup shortcut checks whether Docker Desktop is installed and ready, determines whether a prior transcriber is already healthy, finds an available local port when necessary, selects GPU or CPU mode, waits for the health check, and opens the browser only after the service is ready.

This removes much of the friction that normally surrounds local model software. The user does not need to remember a sequence of container commands or wonder whether the page opened before the backend finished loading. The stop shortcut shuts down both possible runtime profiles while preserving transcripts and downloaded models.

Packaging matters here. A model that works in a development shell is a technical demonstration. A tool that can be started, used, stopped, and reopened reliably is much closer to a real application.

Current Test Notes

I ran the focused backend suite during this review and all 15 tests passed in 0.42 seconds. The tests cover the complete upload-to-download path with a controlled transcription fixture, local host and request protections, upload limits, supported extensions, filename cleanup, restricted media decoding, temporary-file cleanup, GPU selection, and CPU fallback behavior.

I also validated the combined CPU and GPU Docker configuration. The live GPU container reported healthy status with an empty queue, the configured small.en model, automatic device selection, and successful CUDA access to the RTX 5070.

These checks prove that the application structure, guarded media path, queue workflow, and current GPU runtime are working as tested. They are not a substitute for a larger accuracy study across different microphones, speakers, accents, background noise, or recording lengths.

What This Project Showed Me

Local AI becomes useful when the model is surrounded by a dependable workflow. For transcription, that means accepting the files people already have, normalizing media safely, managing scarce compute, preserving outputs, cleaning temporary data, and making startup understandable.

It also showed the value of keeping a project narrow. The app has one job and a clear success condition. That made it possible to spend time on privacy, fallback behavior, container restrictions, output integrity, and the small interface details that determine whether I will actually use the tool again.

Publishing the source keeps these implementation notes connected to the code they describe. Future work can now be documented alongside the project as the workflow develops.

GitHub Repository

TheSuperDenis/Transcription-Automation on GitHub