by wbnns
Streaming speech to text, speaker labels and endpointing, from open weight models. No GPU, no API key, no gated downloads. A 1GB VPS is enough.
The demo below is a recording. Every number on this page was measured on an Apple M4 with no accelerator, and each one is reproducible with a command in the repository.
Meta published Muse Voice Transcribe: one model that streams a transcript, labels who is speaking and marks where an utterance ended, all through a single shared vocabulary. This project asks a narrower question. Can the same capability set be assembled from open weight parts, run on a laptop with no GPU, and cost nothing to operate.
The parts exist and are permissively licensed. What does not exist is the agreement between them. One model with a shared vocabulary gets that for free, and four separate models do not. Deciding when a word is final, which speaker it belongs to, and whether a pause ended a thought is the whole of the work here, and it is ordinary Python rather than a training run.
Read the reference figures as a ceiling and not a target: 3.1% word error rate, 17.5% diarization error, and about 0.16s from speech to a final word. What follows is what this measures against them, including where it falls short and the one case where it does not work at all.
Apple M4, CPU only, no accelerator. The delay figure is how long after a word is spoken before it is final and will never change.
Neither dominates, which is why both ship. Nemotron commits sooner and
punctuates natively; the zipformer costs a fifth of the CPU. Pick with
--model.
| Model | Delay p50 | WER | Real time factor |
|---|---|---|---|
| nemotron-3.5-160ms | 0.36s | 6.81% | 0.26× |
| zipformer-en | 0.52s | 4.40% | 0.05× |
The zipformer was trained on LibriSpeech, so its word error rate there is flattered: its wins concentrate in the proper nouns of the books the corpus is read from. Read the two as "both in the 4 to 7% range on easy read speech", not as a ranking.
Nothing tells HearWrite how many people are talking.
| Speakers | Confusion |
|---|---|
| 2 | 3.1% |
| 4 | 6.1% |
| 8 | 6.7% |
| 16 | 7.2% |
| 24 | 7.6% |
Published because a benchmark without its failure case is marketing.
A real three person conversation, recorded through a laptop, came back as two speakers, with turn boundaries falling mid sentence. The transcript itself was the most accurate measured anywhere in this project.
The cause is measurable rather than mysterious. Two of the three voices sat at 0.54 cross similarity against 0.55 within, so no threshold exists between them. Four window and threshold combinations were tried and all returned two speakers. Every voice arrives through the same room, the same distance and the same microphone, and that shared channel signature swamps the individual one.
The flat line above is per speaker recordings, where that confound is absent. It is what the same code does with clean capture, and it is why the fix is per speaker streams rather than a better threshold.
Honestly: it cannot be placed on the published leaderboards, and pretending otherwise would be the easiest thing on this page to get wrong.
The streaming word error index those charts use is a private corpus, and word error rate moves more between corpora than it does between good systems. Running our own audio and calling the result comparable would be a category error.
The diarization figures come from AMI and VoxConverse, which are public. We have not run them. That is the single most valuable measurement still missing here, and it is on the roadmap rather than in a chart.
What this does instead is publish what it measures, with the command that reproduces it and the bias attached. On the axis that actually differentiates it, the comparison is easier: it runs on a laptop, with no API key, no account and no gated weights.
pip install 'hearwrite[onnx,turn,server]' hearwrite serve --open
That downloads the models on first use, starts the service and opens this page. The base install pulls exactly one package with zero dependencies; the models are fetched from their publishers against pinned checksums, and nothing gated is accepted.