Applied AI · Released v0.12.0 · Open source (AGPL-3.0)
OpenEar
Live captions from a soundboard feed to every phone on the local Wi-Fi, served by one ordinary Windows PC: no cloud, no app, no graphics card. Translation is an opt-in add-on with an NVIDIA card, on a model that covers about 200 languages.
Status
Released. It has run in live services, including one outside installation from the public repo.
Version
0.12.0, tagged v0.12.0 on 25 Sep 2026
Licence
AGPL-3.0-or-later. Model weights are downloaded at setup and never bundled: Parakeet is CC BY 4.0; NLLB-200 is CC BY-NC 4.0, non-commercial.
Runs on
One Windows PC. Captions run on the CPU; translation needs an NVIDIA GPU.
~200: the translation model is NLLB-200 (3.3B parameters, int8), named in server.py; NLLB-200 is the model family published to cover 200 languages. The app’s own language menu is a curated subset of them, chosen for need.
124.8 min: the durations of the five public-domain LibriVox recordings in the benchmark set, summed with afinfo (7,489 seconds).
960: the scored items across the 24 judge worksheets and their answer keys, 768 real items and 192 hidden controls. Both language panels list every judge chunk as accepted and none as rejected.
21/21: score_translation.py --self-test, run on a scratch copy so nothing was written to the project: 21 pass, 0 fail.
Fig. 1The same passage in English, Korean and Spanish, captured in iPhone simulators on 29 Jul 2026 from synthetic speech fed through a virtual audio cable into the real pipeline, served by a machine on the local network.Fig. 227 seconds, trimmed only at the start, no audio track, recorded on 29 Jul 2026 from synthetic speech on the development server, not in a live service. Spanish arrives a whole sentence at a time by design, because translation waits for the end of a sentence; nothing is shown provisionally or revised.
The short version
Kept the speech model that lost on word error rate, because it punctuates: on the benchmark’s 21-minute recording, 96% of its chunks ended a sentence against 48% for Whisper, and translation waits for the end of a sentence. It also runs at 36× real time on the CPU, which leaves the graphics card to translation.
Built a translation-quality instrument that tests its own judges: 960 blind LLM-judge judgments, every judge checked by hidden controls, none rejected. It kept NLLB-200 and exposed a 14-point quality gap between Spanish and Korean.
Split captions from translation after the first outside install, so captions need no graphics card: 0 MB of video memory in captions-only mode, against about 4.6 GB with translation recorded 25 Sep 2026.
Case 1 of 5
Choosing the recogniser that lost on accuracy, then re-checking the arithmetic
Whisper large-v3 made fewer word errors than NVIDIA’s Parakeet, and Parakeet was still the right choice.
I built a harness for the job (test_pipeline.py) and ran 124.8 minutes of public-domain LibriVox audio with known text through three recognisers. Chunk length mattered more than the model. On the same 21-minute recording, 3-second slices produced a 14.4% word error rate and 10-second slices 3.0%. Cutting on trailing silence between 5 and 10 seconds brought it to 1.9%, so the server cuts on pauses, with a 10-second cap.
Across all five recordings, Whisper large-v3 averaged 2.9% per recording against Parakeet’s 3.3% (pooled, 644 against 725 errors in 21,801 words: about one extra error every 270 words). Punctuation separated them. OpenEar translates a sentence only once its text ends in . ! or ?, and a verb-final language like Korean can’t be translated from half a sentence. On the 21-minute recording, Parakeet ended 96% of its chunks that way and left 1% bare. Whisper large-v3 ended 48%, and left 49% with no punctuation at all.
Parakeet also ran at 36× real time on the CPU, against 19× for Whisper large-v3 on a GPU. Whisper large-v3-turbo was faster still, at 51× on a GPU, but only 35% of its chunks ended a sentence. Keeping recognition on the CPU leaves the whole graphics card free for a 3.3-billion-parameter translation model.
Word error rate by chunk length: Parakeet, one 21‑minute recording. Lower is better.
3 s chunks
14.4%
5 s chunks
4.1%
10 s chunks
3.0%
Cut on pauses, 5–10 s
1.9%
Word error rate, mean of all five recordings, on the same scale. Lower is better: Parakeet lost here.
Whisper large-v3
2.9%
Parakeet
3.3%
Chunks ending in . ! or ? on the 21-minute recording. Higher is better.
Parakeet
96%
Whisper large-v3
48%
Whisper large‑v3‑turbo
35%
Speed on the same recording, in multiples of real time: Parakeet on the CPU, both Whisper models on a GPU. Higher is better.
Parakeet
36×
Whisper large-v3
19×
Whisper large‑v3‑turbo
51×
A sentence buffer is added: translation now waits for . ! or ?
Parakeet replaces Whisper as the recogniser.
A five-phase benchmark tests the switch against Whisper large-v3 and large‑v3‑turbo, and confirms it.
Every figure is re-derived from the raw outputs for this page.
How we know
Word error rate is substitutions, deletions and insertions over reference words, parsed from the benchmark’s raw output for every recording and re-derived on 12 Oct 2026; the averages above are means across recordings. Punctuation shares come from a regular expression over each model’s chunk lines on the same 21-minute recording (176 of 184, 90 of 186 and 66 of 186). Speeds come from each run’s own timing line: 1,264 s of audio in 35.0 s for Parakeet, whose log shows the GPU provider failing to load, 65.9 s for Whisper large-v3 and 24.8 s for large-v3-turbo, both in CUDA float16. Turbo is left out of the five-recording error-rate comparison because its run on one recording stopped before reporting an error rate. The harness is in the public repo; the raw outputs and the March report are not. Dates come from the project’s pre-release git history and the report’s own date.
Case 2 of 5
A translation-quality instrument that tests its own judges
Translation has no single right answer, so the instrument scores meaning blind and throws out any judge that fails a hidden control.
String metrics scored against one reference, such as BLEU or chrF, penalise valid paraphrase, and the earlier accuracy figures came from one person reading parallel text once. I replaced them with score_translation.py, built in layers:
Deterministic checks, standard library only: delivery rate, truncation by length ratio, and text left in the source script. On a real service log they caught a clause that never reached Korean readers.
Blind adequacy judging. The tool writes shuffled, unlabelled worksheets, and the judge stays outside it. Today that judge is a capable LLM.
Hidden controls made from the data. A source paired with another segment’s translation must score 30 or less; one paired with the first 40% of its own translation must average more than 15 points below the real items. A judge chunk that fails is dropped and named, never averaged in.
The validation is tested too: its 21 self-test controls include a rubber-stamp judge and a judge blind to truncation, and both must be rejected. The first full run put a 192-segment corpus through 24 LLM judge passes, 960 judgments in all, and no chunk was rejected.
Blind adequacy (0–100, higher is better) on the same 192 segments per language: NLLB-200 3.3B against MADLAD-400 3B.
Language
NLLB-200 3.3B
MADLAD-400 3B
Reading
Spanish
87.3
87.4
A tie. NLLB took 0.33 s per segment against 0.49 s. recorded
Korean
72.9
70.7
NLLB ahead, and 2.6× faster: 0.28 s per segment against 0.73 s. recorded
The finding that matters more than the model choice is the 14-point gap between languages. Korean readers get materially worse translation than Spanish readers, and the Korean errors include outright reversals of meaning.
How we know
Counted on 12 Oct 2026 from the 24 worksheets and their answer keys: 768 real items, 96 mismatched-pair controls and 96 truncated controls, every one scored. Both panel results list 12 accepted judge chunks and none rejected. The self-test ran on a scratch copy of the scorer. Per-segment speeds were recorded on the CUDA machine and appear in the public model registry (MODELS.md); they were not re-measured. The scorer is in the public repo; the worksheets and panel files are not.
Case 3 of 5
Pairing by construction, not by guesswork
The scorer kept mis-pairing source and translation. The cause was upstream, so the fix went where the data is written.
The server logged every recognition fragment on the English side but only completed sentences on the translation side. The two logs were never one-to-one, and nothing said so. The scorer had to guess pairs by character length, which can’t see meaning and guessed wrong on real data. That also explained an earlier alarm, when a March service log seemed to turn 10 English segments into 8 Korean ones.
Release 0.9.0 made both sides log the same unit, the completed sentence, stamped with a shared monotonic segment ID, so pairs are exact by construction. A replay mode re-buffers old fragment-level logs so historical runs stay comparable; replayed, the March log read 100% delivery with one damaged segment. Before trusting any new number, I checked the offline path against production: re-translating the March service’s source reproduced its Korean log byte for byte.
That release also found a guard that the check’s own docstring described but the code never had: skip the untranslated-text check when both languages share a script. Every earlier test had been English to Korean, so the gap never showed. The first English-to-Spanish run reported 0% structural integrity for a translation judged 95% adequate.
How we know
In the public code, broadcast() in server.py writes both logs with the shared segment ID, and score_translation.py pairs by ID, re-buffers legacy logs with --rebuffer-source, and now skips same-script pairs instead of reporting them as a pass or a failure. The public release notes for 0.7.0 and 0.9.0 record the replay and reproduction results.
Case 4 of 5
The first outside install, and the split that took captions off the graphics card
An outside install took down the video software on the same PC, so captions and translation became separate modes, and captions now need no graphics card.
In August 2026 a congregation outside my circle installed OpenEar from the public repo without help, ran it in a live service and reported a defect: starting OpenEar took down the video-production software on the same PC, while OpenEar’s own log showed a clean start. Its whole GPU footprint, about 4.6 GB recorded, was the translation model; speech recognition used none. Under Windows’ WDDM driver model, the driver can evict another process’s video memory rather than refuse a request. That is the leading explanation, and the site’s confirmation of the fix is still pending.
Release 0.12.0 split the product. Setup asks one question, captions only (the default) or captions plus translation, and downloads about 2.5 GB of models for the first or about 15.9 GB for the second. Translation on a PC without an NVIDIA card is refused before anything installs. Four more gaps closed:
The CPU provider is pinned. Parakeet had been on the CPU only because both GPU providers failed to load; a machine with those libraries would quietly have moved speech onto the GPU.
No download at startup. Fetching the 13.4 GB translation model there had kept captions down for the whole download.
A pinned byte size reports a truncated model file plainly, instead of an obscure failure inside CTranslate2.
Phones are offered only the languages the server will actually translate.
video memory, captions only recorded
0 MB
video memory with translation recorded
≈4.6 GB
How we know
In the public code, openear_config.py resolves the mode and holds the pinned model size, server.py pins the CPU execution provider and explains a failed translation start, and setup.bat carries the captions-only and translation paths. The 13.4 GB is the translation model’s size on disk (13,387,884,764 bytes), which matches the pinned constant. Download sizes come from the model folder on disk and the speech model’s file listing on Hugging Face; Python packages are extra. The video-memory figures were measured per process on Windows on the CUDA machine for the 0.12.0 release (tagged 25 Sep 2026), as its release notes record, and can’t be re-measured from here. The outside site is deliberately not named.
Case 5 of 5
Real-time fan-out on one box, where cost scales with languages, not listeners
One process captures, recognises, translates and serves every phone, without a model ever blocking a connection.
A PortAudio callback thread appends 100 ms blocks to a lock-protected buffer. An async loop checks four times a second for a silence-bounded cut. Recognition and translation both run in an executor, so the WebSockets never stall.
Since 0.12.0, each chunk goes to English readers first, before any translation starts. Text then accumulates until a sentence ends and is translated once per language in use, so forty Korean readers cost one inference.
Edge cases are handled explicitly. A reader who leaves English mid-sentence is tracked by character offset, so coming back shows only the unread part. A phone that switches to English while a send or a translation is in flight still gets that text, once. Start and stop requests from two open admin pages are serialised by a lock, so no transcription loop is orphaned.
One API bug had made every error invisible. Returning a tuple from FastAPI produced a JSON array with HTTP 200, so the admin page never showed a single error. Errors now carry real status codes.
Captions: always on, CPU only
Translation: opt-in, NVIDIA GPU
Fig. 3One process, two paths: English goes out chunk by chunk, translations sentence by sentence.
Fig. 4Arrival timing, from the 27-second recording. The faint first frame is the “Waiting for captions…” placeholder fading out while the first sentence fades in, not provisional text. From 1.5 s to 5 s nothing changes; the next whole sentence lands by 8 s, and later ones push older text up.How we know
In the public code: audio_callback(), transcription_loop(), broadcast() and websocket_captions() in server.py, and appendText() in the phone page, whose 300 ms fade is a CSS animation. The per-language cost is stated in the broadcast code’s own documentation; it was recorded on the Windows machine and not re-measured. The eight frames were extracted from the recording with headless Chrome via Playwright on 12 Oct 2026.
