From the publisher · ITANU

ITANU publishes this magazine and makes the products this article is about. It is the company’s own account, not news reporting.

No engine tops KolWrite's Oracle Max in five languages on a public test

On the FLEURS test, no engine on the Open ASR Leaderboard has a lower word error rate than Oracle Max in five languages. In Italian it got 0.72% of the words wrong.

Numbers in brackets link to the sources listed below.

On 21 and 22 November 2026 we ran Oracle Max, the top cloud tier of KolWrite, our transcription app, on the FLEURS test in five languages: Italian, German, Spanish, Portuguese as spoken in Brazil, and French[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. That is 4,230 recorded sentences, about 15 hours of speech[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. The Open ASR Leaderboard publishes results for 46 to 48 other engines on the same clips[3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]. In four of the languages Oracle Max has the lowest word error rate of them all, and in Spanish it shares the lowest with one other engine[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]].

The five results

A word error rate counts the words an engine got wrong, left out or added, as a share of the words that were spoken[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. These are ours, each with the lowest other figure on the leaderboard for that language[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]].

  • Italian: Oracle Max 0.72%, ElevenLabs Scribe v2 0.90%[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]
  • German: Oracle Max 1.58%, Microsoft Azure Speech 1.62%[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]
  • Spanish: Oracle Max 1.76%, Microsoft Azure Speech 1.76%[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]
  • Portuguese: Oracle Max 2.35%, Microsoft Azure Speech 2.45%[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]
  • French: Oracle Max 2.54%, Microsoft Azure Speech 2.64%[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1], 3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]]

In Italian, Oracle Max wrote 753 of the 865 sentences with no error at all, and made 152 word errors in 21,223 words[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. The leaderboard lists OpenAI's gpt-transcribe at 1.35% in Italian and 3.45% in French, and AssemblyAI's Universal-3.5 Pro at 1.03% and 2.82%[3Hugging FaceOpen ASR LeaderboardOpen sourceSources [3]].

Same clips, same scoring

FLEURS is a set of sentences read aloud and recorded in 102 languages[2arXivFLEURS: Few-shot Learning Evaluation of Universal Representations of SpeechOpen sourceSources [2]]. We used the test clips and the reference texts that the leaderboard uses, and scored Oracle Max with the leaderboard's own published code[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. Before comparing anything, we rescored the leaderboard's own files with our copy of that code, and all 169 figures came out within 0.13 percentage points of its table[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

We sent the clips through the public KolWrite API, the same service a customer uses, with the language named[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. Every clip is counted, and none came back empty[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

Clip by clip

For 33 or 34 engines in each language the leaderboard publishes the transcript of every clip, so two engines can be compared on exactly the same sentences[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. We drew 10,000 random resamples of the clips and scored both sides on each one[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. In Italian, German and French, Oracle Max was lower than every one of those engines, with a 95% interval that leaves out zero each time[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

In Spanish and Portuguese the same holds against 33 of the 34[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. Against the remaining one, Mistral's Voxtral Small, the gap is about a tenth of a point and too small to call[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. No engine with a per-clip file was lower than Oracle Max in any of the five languages[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

Engines without transcripts

Microsoft Azure Speech, ElevenLabs, AssemblyAI and OpenAI appear on the leaderboard as one total for each language, with no per-clip file[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. Against them we can set one figure beside another, and we cannot test the gap[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. Azure's figure sits inside Oracle Max's 95% interval in German, Spanish, Portuguese and French, so those four are close[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. In Italian the interval ends at 0.86%, below the figure of every other engine[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

The leaderboard has further tests for these languages, and we ran FLEURS only[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. FLEURS is clean read speech, and meetings and phone calls are harder for every engine[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. These five are not every language we have measured: they are the ones in which no engine on this test has a lower figure[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

Check it yourself

Every transcript and its score are public[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]]. The Italian file is at itanu.org/data/oracle-max-italian-fleurs-865-2026-11-22.csv, and the note beside it, oracle-max-five-languages-fleurs-2026-11-22.md, links the other four files and gives the method[1ITANU Studios LtdKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Open sourceSources [1]].

Sources

  1. 1
    Material supplied by ITANU Studios Ltd

    SuppliedKolWrite Oracle Max on the FLEURS test in Italian, German, Spanish, Portuguese and French, measured 21 and 22 November 2026Accessed

  2. 2
  3. 3
    Open ASR Leaderboard

    IndependentHugging FaceAccessed

ITANU’s newsroom system produced this article with AI and checked it against the sources listed above. How we work

ITANU Magazine

AI, computing and science news