The fewest mistakes a machine makes transcribing a meeting

[ Front page ] [ The edge ] [ Finished ] [ About ] [ Links ]

6.08 per cent of words wrong

Held by zoom/scribe_v2_pro. Set 2026-09-04, and has stood for 53 days. Written down here 2026-10-20.

Dictation is solved and conversation is not. Read a sentence to a machine from a studio microphone and it will get ninety-nine words in a hundred right. Put the same machine in a meeting room, listening through a microphone in the ceiling to four people who interrupt each other, trail off and say "um", and it gets one word in sixteen wrong. That room is the AMI corpus: real meetings, recorded at a distance, nothing about them staged to be difficult. They are simply what listening actually is, which is why a machine that can pass a graduate physics exam still cannot be trusted to take the minutes. The gap is the thing to look at, and it is available on one line of one file: the best system here gets 6.08 per cent of the words wrong in the meeting room and 1.11 per cent on an audiobook read into a microphone. Same model, same day, six times the mistakes. Every other number on this page is reasoning, coding or arithmetic. This is the only one about hearing.

That description was written when this frontier was opened, on 2026-10-20, and is never rewritten. Where it and the number above it disagree, the number is the reading and the description is history.

What counts

The lowest figure in the AMI-Cleaned column of the English short-form results file published with the Open ASR Leaderboard by Hugging Face - the share of words a speech recogniser gets wrong on the AMI meeting corpus, which is real meetings recorded by a microphone in the room rather than one held to a mouth. Four rules belong on the number rather than under it. First, the edition is found from the publisher's own registry and never from a dated address: the leaderboard's init.py carries a VERSIONS table whose newest entry names the results repository, the file and the git revision the page is rendering, and the reading fetches that exact revision. Second, this counts one named set and not the leaderboard's own ranked average, for two reasons that are both the publisher's. That average folds in two private datasets whose files answer 401 to anyone without a key, so the figure a reader sees on the page cannot be reproduced by a script at all; and the composition of its public part has changed six times in the five months the registry covers, so a series taken across it would not be comparable from one month to the next. AMI-Cleaned is the hardest of the sets that appear in every edition, and no model in the file is missing it. Third, the column headed AMI is a different measurement - the corpus's reference transcripts were corrected for the edition of 2026-07-07, the file carries both columns, and this frontier reads only the cleaned one; a figure from before that date is not comparable and is not read across. Fourth, the leaderboard publishes figures its maintainers ran and figures a contributor reported, labels the second kind in its own changelog and ranks both, so this follows the publisher rather than substituting a stricter standard of its own, as it already does for SPC-1 results still in review. The reading is taken by sorting the column in a script and never by reading a row off the page, which is a browser application and carries no numbers in its HTML at all.

What moves it

A model appears in the file with a lower AMI-Cleaned figure, or the publisher re-runs one already there. Editions arrive as submissions are merged rather than to a calendar - twenty-one of them in the five months to 2026-10-17, so roughly weekly - and the record has moved twice in that time, from 6.94 to 6.72 in July and to 6.08 in September.

Where to check it: https://huggingface.co/spaces/hf-audio/open_asr_leaderboard

The program reads it from https://huggingface.co/spaces/hf-audio/open_asr_leaderboard/raw/main/init.py, which is the same figure in a form a script can parse.

Last checked: 2026-10-26. The source itself last measured this: 2026-10-26.

Previous: The most arithmetic ever sustained on scattered data   Next: The longest task an AI can finish on its own

[ Front page ] [ The edge ] [ Finished ] [ About ] [ Links ]