The least time ever taken to train a four-hundred-billion-parameter model

[ Front page ] [ The edge ] [ Finished ] [ About ] [ Links ]

7.07 minutes

Held by Azure GB200 (128x NVIDIA GB200 NVL72). Set 2026-06-16, and has stood for 133 days. Written down here 2026-10-05.

Everything else on this page is a rate: so much arithmetic a second, so many questions an hour, so much data read while you wait. This one is a stopwatch, and the thing being timed is the most expensive single act in modern computing - teaching a very large model to predict text. The task is fixed down to the last detail by whoever publishes the benchmark rather than by whoever enters it: this model, four hundred and five billion parameters of it, on this slice of a crawl of the web, trained until it predicts held-out text to a stated standard, and the clock stops when it gets there. Nobody may substitute an easier model or settle for a worse answer, which is what makes two entries a year apart mean something against each other. What is not fixed is the machine, and that is where the whole spread of the list comes from: the same benchmark, finished in seven minutes by eight thousand one hundred and ninety-two rented accelerators across a hundred and twenty-eight racks, and in very nearly eight hours by seventy-two of them in one rack. So the record is not quite "how quickly can this be done" - it is "how quickly has anyone shown it can be done, with as much hardware as they were willing to point at it", which is a different and more honest question. The reason it is worth a page anyway is the sentence it licenses. The thing being trained here is not a toy; it is the same class of object as the models that write and reason in public, and a decade ago training one would have been a research programme with a name and a paper. It now takes seven minutes, on hardware that can be rented by the hour, and the number goes down every six months. The institution counting is worth a line too: MLCommons is an industry consortium whose members are the companies being ranked, which sounds like a conflict and is in fact the oldest working answer to one - the same arrangement as the councils behind the transaction and storage records elsewhere here, and the same as the committee that has ranked the fastest computers on earth since the year this page is pretending to be from. Every entrant runs the same code to the same target, publishes the configuration and the logs, and gets to see everyone else's.

That description was written when this frontier was opened, on 2026-10-05, and is never rewritten. Where it and the number above it disagree, the number is the reading and the description is history.

What counts

The smallest figure in the Llama 3.1 405B time-to-train column of the current MLPerf Training results sheet, published by MLCommons. The sheet is the raw file the benchmark's own page links, and the read is two steps rather than one, deliberately: fetch https://mlcommons.org/benchmarks/training/, take the Google Sheets address it gives under "Results sheet" for the newest round, and export that sheet as CSV. A round's sheet has its own identifier and the page always links the current one, so the evergreen page is what this frontier reads and a sheet address is never written down here - an opaque per-round identifier is the dated-edition trap without a date in it, and it would go on serving one round's numbers for ever with nothing about it looking broken. Three rules belong on the number rather than under it. The Closed division only: MLCommons runs an Open division that permits a different model, and comparing the two is comparing different work - every 405B result published so far is Closed, so this costs nothing today and settles it before it does. All three availability categories count, cloud, on-premise and otherwise, because the publisher ranks them in one list and this site follows the publisher rather than substituting a stricter standard of its own. And the figure counts only as long as MLCommons keeps running this benchmark: the suite is revised every round, the publisher states that figures from different reference implementations are not comparable, and if Llama 3.1 405B is retired from the suite this frontier needs a decision before it needs a reading - the successor is a new frontier rather than a re-pointing of this one.

What moves it

MLCommons publishes a round carrying a smaller figure. Rounds arrive on a calendar rather than as submissions clear: v5.0 in June 2025, v5.1 in November 2025, v6.0 in June 2026, so the next one is due around November and this number can only move twice a year.

Where to check it: https://mlcommons.org/benchmarks/training/

The program reads it from https://mlcommons.org/benchmarks/training/, which is the same figure in a form a script can parse.

Last checked: 2026-10-22. The source itself last measured this: 2026-06-16.

Previous: The most questions ever answered against a hundred terabytes of data   Next: The most tokens a seventy-billion-parameter model can produce in a second

[ Front page ] [ The edge ] [ Finished ] [ About ] [ Links ]