How accurate are AI meeting notes

“How accurate are AI meeting notes” is a question that comes up early in any tool evaluation, and the honest answer is “it depends”. The accuracy depends on the audio, the speakers, the subject matter, the language, the model, and what the user is going to do with the document afterwards. Marketing material that gives a single number (“99% accurate”, “95% accurate”) usually does so by picking a benchmark that flatters the model. The real question is how the tool behaves on the kinds of meeting the buyer actually runs.

This piece is for buyers comparing meeting tools, and for anyone who has been told the AI is the answer to the meeting documentation problem and wants to understand what they are signing up for. It covers what affects the result, where the failure modes are, and how to test a tool on a real meeting before committing.

The two different “accuracy” questions

AI meeting notes have two stages, and accuracy means something different at each stage.

Transcription accuracy. The speech-to-text part. How close is the written transcript to what was actually said. Measured as word error rate (WER): the proportion of words in the transcript that are wrong, missing or inserted compared to what was spoken. Modern transcription on clean meeting audio in English typically gets WER below 10%; on noisy audio it is much higher.

Document accuracy. The summarisation/document-generation part. How well does the document the AI wrote represent what the meeting actually decided. This is harder to measure. A document with no transcription errors can still be inaccurate if the model summarised something incorrectly, missed an important decision, or imported a tangential mention as a main point.

Different tools fail in different ways at each stage. Some tools have very strong transcription and weaker document generation; some have the opposite. The buying decision needs to consider both, and the relative importance depends on what the user does with the output.

What affects transcription accuracy

Six factors, in roughly decreasing order of impact:

1. Audio quality. A meeting recorded with a single laptop microphone at the centre of a six-person table does not produce the same audio as a meeting recorded with a dedicated USB conference mic. Transcription quality drops sharply once the signal-to-noise ratio falls below a certain threshold.

2. Speaker proximity. A speaker far from the microphone is harder to transcribe than one close to it. Hybrid meetings where remote speakers are crystal-clear (their audio comes through system output) and in person speakers are muffled (they’re across the room from the mic) produce uneven transcripts.

3. Number of overlapping speakers. Two people talking over each other is harder than two people speaking in turn. Three or more overlapping speakers approach the limits of any current transcription model.

4. Language and accent. English (especially North American English) is what the models are trained on most, and it transcribes best. Non-English transcription quality varies. Strong regional accents within a language can also push WER up.

5. Specialist vocabulary. Medical, legal, financial and technical jargon is where transcription models commonly fail. Whistle Enterprise’s transcription model handles common business and legal vocabulary well; uncommon terms (drug names, case names, technical compounds) sometimes mistranscribe.

6. Background noise. Office hum, fans, phones, the person typing in the back row. Each of these adds error.

The first two factors dwarf the others. A meeting with good audio and well-spaced speakers transcribes accurately almost regardless of the model; a meeting with bad audio and overlapping speakers will fail in any tool.

What affects document accuracy

Document accuracy is harder to evaluate because there is no objective ground truth. Two readers can disagree about whether a particular sentence in the document represents the meeting correctly. There are three failure modes worth knowing about:

Lost points. A topic was raised in the meeting but did not make it into the document. Usually this happens when the topic was raised briefly and the model judged it unimportant.

Promoted tangents. A topic was mentioned in passing but the document treats it as a decision or an action. Usually this happens when the model misreads the conversational register (someone said “we could do X” hypothetically and the model writes “the team agreed to do X”).

Compressed nuance. A discussion that included caveats, conditions or hedging gets reported in the document as a clean decision. The decision is correct but the document does not preserve the nuance the participants would have remembered.

All three failure modes are visible to a reader who attended the meeting. The document does not look wrong on its own; it looks wrong when compared to memory or to the transcript. The fix is the same in each case: the reviewer reads the document with the transcript open beside it and corrects anywhere the document drifted.

This is one of the reasons source traceability matters in practice. Whistle Enterprise’s bi-directional link between the document and the transcript lets the reviewer click any sentence in the document and see the part of the transcript it was generated from. The check that takes ten minutes with traceability would take an hour without.

How to evaluate accuracy on real meetings

The standard ways tool vendors report accuracy (WER on a benchmark dataset, blind review of summaries) do not match what a buyer actually cares about. The buyer cares about how the tool behaves on the meetings they run. The way to find out is to run it on those meetings.

A reasonable evaluation:

1. Pick three meetings that represent the work. One easy (good audio, two speakers, familiar topic). One medium (some background, three or four speakers, mixed register). One hard (noisy, multiple speakers, specialist vocabulary).
2. Run each through the candidate tool.
3. For each meeting, read the transcript and document while remembering the meeting. Note how many transcript words look wrong (estimate WER). Note any document failures (lost points, promoted tangents, compressed nuance).
4. Score each meeting against your tolerance: a transcript that needs more than 5% manual correction is more work than no transcript at all. A document that needs more than three substantive corrections per meeting is more work than typing the document yourself.
5. The right tool for the work is one that fails gracefully on the hard meeting and works well on the medium one.

For a more detailed walkthrough of the buyer-side comparison process, on-premise alternatives to cloud meeting tools covers it. For what Whistle Enterprise’s output specifically looks like, the audio-to-document walkthrough shows the structure of the document.

What to do with the accuracy answer

Once you have evaluated accuracy on real meetings, the question is what you do with the result. There are three reasonable answers:

• The accuracy is good enough that the document is the record. The user reads, spot-checks against the transcript and uses the document as-is. This works for internal meetings, hybrid meetings, most board work.
• The accuracy is good enough that the document is a draft. The user uses the document as a starting point and revises it before it becomes the official record. This is the common pattern for legal file notes, clinical records and witness statements.
• The accuracy is not good enough. This is the answer when the meeting type is hard and the regulatory regime is strict. Some meeting types (criminal investigations, complex multilingual work) still call for a human transcriber.

The free 30 day trial is the right way to find out which of these three answers applies to your work. Run the evaluation on three of your real meetings; the result will tell you whether the trade-off is the right one for the documentation you need.

Back to all articles