by wbnns
Streaming speech to text, speaker labels and endpointing, from open weight models. No GPU, no API key and no gated downloads. A 1GB VPS is enough with the light recogniser; the default one wants 4GB.
The demo below is a recording. Every number on this page was measured on an Apple M4 with no accelerator, and each one is reproducible with a command in the repository.
Meta published Muse Voice Transcribe: one model that streams a transcript, labels who is speaking and marks where an utterance ended, all through a single shared vocabulary. This project asks a narrower question. Can the same capability set be assembled from open weight parts, run on a laptop with no GPU, and cost nothing to operate.
Microsoft has since published MAI-Transcribe-2, cheaper again and top of its own benchmark, but batch: it wants the recording to be finished before it starts. That is a different job, and an easier one, because a system that has the whole file can look ahead and change its mind. The shape being chased here is still Muse's.
The parts exist and are permissively licensed. What does not exist is the agreement between them. One model with a shared vocabulary gets that for free, and four separate models do not. Deciding when a word is final, which speaker it belongs to, and whether a pause ended a thought is the whole of the work here, and it is ordinary Python rather than a training run.
OpenAI shows what that costs when nobody does the assembling. GPT-Transcribe turns a file into text for $0.27 an hour. Live audio is a different model at $1.02. Who spoke is a third model, and it does not stream. Word timestamps are a fourth. Every one of them is good, and there is no combination of them that gives you the whole capability set at once, because the agreement between the signals is exactly the part nobody sells you.
Read Muse's figures as a ceiling and not a target: 3.1% word error rate, 17.5% diarization error, and about 0.16s from speech to a final word. What follows is what this measures against them, including where it falls short, the one case where it does not work at all, and what the three hosted alternatives charge by the hour.
Apple M4, CPU only, no accelerator, all four figures for the default recogniser. The delay figure is how long after a word is spoken before it is final and will never change. The light recogniser trades word error rate for a fifth of the CPU and a quarter of the memory; the charts below put the two side by side.
Neither dominates, which is why both ship. Nemotron commits sooner and
punctuates natively; the zipformer costs a fifth of the CPU. Pick with
--model.
| Model | Delay p50 | WER | Real time factor |
|---|---|---|---|
| nemotron-3.5-160ms | 0.36s | 6.81% | 0.26× |
| zipformer-en | 0.52s | 4.40% | 0.05× |
The zipformer was trained on LibriSpeech, so its word error rate there is flattered: its wins concentrate in the proper nouns of the books the corpus is read from. Read the two as "both in the 4 to 7% range on easy read speech", not as a ranking.
Nothing tells HearWrite how many people are talking.
| Speakers | Confusion |
|---|---|
| 2 | 3.1% |
| 4 | 6.1% |
| 8 | 6.7% |
| 16 | 7.2% |
| 24 | 7.6% |
Published because a benchmark without its failure case is marketing.
A real three person conversation, recorded through a laptop, came back as two speakers, with turn boundaries falling mid sentence. The transcript itself was the most accurate measured anywhere in this project.
The cause is measurable rather than mysterious. Two of the three voices sat at 0.54 cross similarity against 0.55 within, so no threshold exists between them. Four window and threshold combinations were tried and all returned two speakers. Every voice arrives through the same room, the same distance and the same microphone, and that shared channel signature swamps the individual one.
The flat line above is per speaker recordings, where that confound is absent. It is what the same code does with clean capture, and it is why the fix is per speaker streams rather than a better threshold.
Honestly: it cannot be placed on the published leaderboards, and pretending otherwise would be the easiest thing on this page to get wrong.
The streaming word error index those charts use is a private corpus, and word error rate moves more between corpora than it does between good systems. Running our own audio and calling the result comparable would be a category error.
The diarization figures come from AMI and VoxConverse, which are public. We have not run them. That is the single most valuable measurement still missing here, and it is on the roadmap rather than in a chart.
What this does instead is publish what it measures, with the command that reproduces it and the bias attached. On the axis that actually differentiates it, the comparison is easier: it runs on a laptop, with no API key, no account and no gated weights.
Three systems are worth naming by price rather than by leaderboard. Muse Voice Transcribe is the streaming ceiling this project was built against, and it bills $0.18 an hour. GPT-Transcribe is the one most people have already used, at $0.27, for files up to 25MB. MAI-Transcribe-2 arrived calling itself the fastest, most accurate and cheapest speech recognition model in the world, at $0.10. None of those claims is in dispute on its own benchmark, and none of them can be read across to the last two rows, for the reason the note under the table gives. What they have in common is the part a table shows better than a sentence: the audio goes somewhere else, and the meter runs by the hour.
| Shape | Price per hour | Delay to a final word | Speakers | Word error rate | |
|---|---|---|---|---|---|
| Muse Voice Transcribe | Streaming, hosted. | $0.18 | About 0.16s. | 20 and more, live. | 3.1%, Artificial Analysis streaming. |
| GPT-Transcribe | Files to 25MB, hosted. | $0.27 | Not at this price. Live audio is a different model, at $1.02. | A different model again, which does not stream. | None published. |
| MAI-Transcribe-2 | Batch to 300MB, hosted. | $0.10 | Needs the recording first. An hour of audio in about ten seconds. | Over the finished file. | 5.2% average, FLEURS, 60 languages. |
| HearWrite, default | Streaming, local. | None | 0.36s | Discovered live, abstaining rather than guessing. | 6.81%, LibriSpeech dev-clean. |
| HearWrite, light | Streaming, local. | None | 0.52s | Discovered live, abstaining rather than guessing. | 4.40%, LibriSpeech dev-clean. |
The row the table cannot hold is the shortest one: the
first three send your audio somewhere else, and the last one does not. The rest
of the detail sits behind that. Muse runs on the Meta Model API, eight
concurrent streams per tenant and sessions capped at 60 minutes. GPT-Transcribe
splits the capability set across models, so a transcript, who spoke, low latency
and word timestamps are four products rather than one. MAI-Transcribe-2 is
Microsoft Foundry, in public preview. None of the three ships weights. This runs
on a laptop or a 1GB VPS, CPU only, on open weights with pinned checksums, and
admission defaults to the core count capped at 16. The light recogniser is
--model zipformer-en, and it is the one a 1GB box holds; the
default wants 4GB. On languages it is not close: 25 and more, a long ISO list,
and 60 with code switching, against English.
The last column is five cells from four corpora, one of them empty, so it is not a ranking. In particular: 4.40% sits under 5.2% and beats nothing. The light recogniser was trained on LibriSpeech, which is the set it is scored on, and a model measured on its own training distribution is flattered; MAI's figure is an average over sixty languages including hard ones. Those are not the same measurement and the smaller number is not the better system. Everything in the first three rows is what Meta, OpenAI and Microsoft publish about themselves. None of it was measured here, which is the opposite of the standard the rest of this page holds itself to. The arithmetic is theirs too, and it is only a rate times hours: one always on stream is about 730 hours a month, so $131 at Muse's rate, $197 at OpenAI's and $73 at Microsoft's launch rate, and eight streams, which is where Muse's default ceiling sits, is a little over $1,000. The concession that goes with it is that for a finished recording in one of sixty languages, or for accuracy on hard audio, those rows win and it is not close. What this has is the last two: live, local, and not metered.
pip install 'hearwrite[onnx,turn,server]' hearwrite serve --open
That downloads the models on first use, starts the service and opens this page. The base install pulls exactly one package with zero dependencies; the models are fetched from their publishers against pinned checksums, and nothing gated is accepted.