What is word error rate (WER)?

The formula

WER compares two transcripts of the same audio. The reference is a careful human transcript. The hypothesis is what the speech recognizer produced.

Line the two up word by word and count three kinds of error. Then divide by the number of words in the reference (N):

Substitutions (S): a reference word transcribed as a different word.
Deletions (D): a reference word missing from the hypothesis.
Insertions (I): a word in the hypothesis that nobody said.
WER = (S + D + I) / N, usually shown as a percentage.

Worked example

Here is an illustrative example, not output from any particular product. The reference sentence has 9 words: "please send the final budget to Priya by Friday".

The recognizer returns: "please send the budget to Maria by Friday too". Aligning the two gives three errors:

Deletion: "final" is missing (D = 1).
Substitution: "Priya" became "Maria" (S = 1).
Insertion: "too" was added (I = 1).
WER = (1 + 1 + 1) / 9 = 3 / 9, or about 33%.

How to read the number

Microsoft's Azure Speech documentation treats a WER of 5–10% as good quality. It calls 20% acceptable but worth more training, and 30% or more poor.

WER can go above 100%, because insertions are counted against the reference length. A recognizer that invents many words can make more errors than there were words spoken.

The mix of errors also tells you something. Microsoft links many deletions to weak audio, insertions to noise and crosstalk, and substitutions to unfamiliar domain terms.

What WER doesn't tell you

WER treats every word as equally important and ignores most of what makes a meeting transcript readable. Watch for these gaps:

Formatting. Google's Cloud Speech-to-Text removes punctuation and ignores capitalization before scoring, so "Friday." and "friday" match. Microsoft adds a token error rate to score punctuation, casing and formats like "$900".
Who said it. WER scores words, not speaker labels. A transcript can have a low WER and still attribute a commitment to the wrong person.
Words that carry the meaning. In the example, a wrong name counts the same as a missing "final". One wrong name, number or date can matter more than ten missed filler words.

Why vendor WER figures don't compare

A WER figure only describes the audio it was measured on. Microsoft's own scenario table shows the spread: voice assistants and dictation can stay under 10%, call centers under 30%, and video captioning under 50%.

Scoring rules differ too. One test may strip punctuation and casing while another counts them. Google also notes that the reference has to follow the same conventions as the model, for example on whether "umm" and "yep" are written down.

So a vendor's 5% on clean read speech and another's 12% on noisy calls tell you little about each other. Ask what audio, language and normalization produced a number before you compare it.

How to test WER on your own meetings

The most useful WER is the one measured on your own calls. Google recommends at least 30 minutes of audio for a meaningful result; Microsoft suggests 30 minutes to 5 hours of representative audio.

Pick meetings that look like your real ones: the same rooms, microphones, accents and jargon.
Have a person write a careful reference transcript of each one.
Export each tool's transcript as text, then normalize both sides the same way, such as lowercasing and removing punctuation.
Score them with a standard tool such as sclite from NIST's Scoring Toolkit (SCTK), which Microsoft points to for local WER checks.
Then read the errors. Check names, numbers, decisions and speaker labels by hand, since WER weighs them like any other word.

Checking Bridge transcripts

You can run the same test on Bridge. Every completed meeting has a speaker-labeled transcript, and Export Transcript copies it or downloads it as PDF, Word or Markdown, ready to score against your reference.

Speaker names improve when Bridge knows who is in the meeting, from calendar guests, workspace members and people in Memory.

Sources & further reading

Keep readingWhat is speaker diarization? →