OCR Confidence Score: What the Number Does and Does Not Say

An OCR confidence score is the software's own estimate of how likely it is that a character, word or page was read correctly, often shown as a number from 0 to 100. The score is a guess made without the right answer. It can help you pick pages to check, and it proves nothing about any single word.

Not every OCR tool produces such a number, and those that do may not show it to you. This post explains the idea in general terms, so that when a tool does report confidence you know what you are looking at. It makes no claim about what any particular product reports.

What is an OCR confidence score?

An OCR confidence score is a number the recognition software attaches to its own output to say how sure it was. A high score means the software found the text easy to decide. A low score means it hesitated. The score describes the software's state, not the page's truth.

That distinction is the whole subject. Compare two statements:

• "This word was read correctly." That is a fact about the page, and only a comparison with the page can establish it.
• "The software scored this word at 96." That is a fact about the software. It was produced before anyone checked anything.

The first kind of statement is what accuracy testing measures, by comparing output with a transcript a person has verified. The post on how OCR accuracy is measured covers that. Confidence is different in kind. It is available instantly, for every page, with no human work, which is why it is attractive. It is also unverified, which is why it has to be handled with care.

A few properties are worth knowing before you rely on a score:

• There is no standard scale. One tool may use 0 to 100, another 0 to 1, another a handful of bands such as high, medium and low. A 90 from one tool and a 90 from another are not the same claim.
• It is not a percentage of anything you can count. A page scored at 95 does not have 95 percent of its words right.
• It may be hidden. A tool can calculate confidence internally and never expose it, expose it only in a data export, or show it as colored highlighting on a review screen.
• It may be absent. A tool may produce no per-word score at all.

How does an engine arrive at a confidence figure?

An engine arrives at a confidence figure from the same calculation that produces the text. For each mark on the page it weighs several candidate readings and scores each one. The winning candidate becomes the output, and the strength of its win becomes the confidence.

In outline, for a single letter shape:

1. The engine isolates a shape, or a narrow slice of a text line.
2. It compares what it sees with what it knows about each character it can output.
3. Each candidate gets a score. A clear printed "e" might score very high for "e" and very low for everything else. A blurred one might score moderately for "e", "c" and "o" all at once.
4. The top candidate is chosen. Confidence reflects how high it scored, or how far ahead of the runner-up it finished, or both.

Then context gets a say. An engine may check candidate words against a vocabulary or a statistical model of the language. If the letter shapes suggest "tbe" and the vocabulary knows "the", the engine may output "the" and raise its confidence, because the result is a real word. The engine is the component that does this work, and the post on what an OCR engine is explains how it sits inside the tool you use.

Two consequences follow from that design.

First, the score depends on what the engine was built from. An engine that has seen a great deal of clean office print will be confident on clean office print. Faced with a typewriter face it has rarely met, its scores may be lower even where its reading is right, or misleadingly high where its reading is wrong.

Second, the score is only as honest as its calibration. A score is well calibrated if, among all the words scored at 90, about nine in ten are in fact correct. Nothing guarantees that. Calibration is checked on test documents, and it can drift a long way on documents unlike the test set. The number on your screen is a ranking of the software's doubt more reliably than it is a probability.

Character, word and page confidence are different numbers

Character, word and page confidence are separate figures, each built from the level below it. A character score describes one letter. A word score combines its letters. A page score combines its words. Combining means averaging or taking a minimum, and the two methods give very different pictures.

Take a dollar amount in a fictional settlement statement. The printed figure is $4,150.00, and a crease runs through the fourth character.

Nine characters, one of them doubtful. The engine was unsure whether the creased digit is a 1 or a 7.

• Word score by average: the nine scores sum to 826, which gives about 92. That looks safe.
• Word score by minimum: 41. That looks alarming, and it is the better warning, because one wrong digit makes the whole amount wrong.

Now put that amount on a page with 350 other words, all scored in the high nineties. The page average barely moves. A page-level figure of 97 is entirely compatible with the one number you care about being a coin toss.

Two practical points come out of this.

Use each level for its own job. Page scores sort a batch into "probably fine" and "probably poor". Word scores direct your eye within a page. Neither substitutes for the other.

A high score still leaves you with a page to read. Open one medical bill and check the amount, when the injury occurred, and the party the claim lists. A confident misread of that injury date, or of the damages line, stays unmarked. The number records that the engine liked its own guess.

And when a tool reports a single figure, find out how it was combined. An average over a long page is the least informative number of all. A minimum, or a count of words below some level, tells you more.

Why can a wrong word carry a high confidence score?

A wrong word can carry a high confidence score because the score measures how easily the software decided, not whether it decided correctly. When a wrong reading looks clean and plausible to the engine, there is no hesitation to record. The software is confidently mistaken.

The situations in which that happens are predictable:

• Characters that share a shape. A lowercase "l", a capital "I" and the digit 1 are close to identical in many typefaces. So are "O" and 0. In a word, context settles it. In an account number, a parcel number or a citation, there is no context, and the engine may pick the wrong one without doubt.
• Sharp but wrong digits. A 3 with a filled-in loop looks like a well-formed 8. The shape is crisp, so the score is high.
• Vocabulary pull. If the engine checks words against a vocabulary, an unusual surname or drug name may be replaced by a common word that looks similar. The replacement is a real word, which raises confidence. The correct but unfamiliar name would have scored lower.
• Fluent guesses. Model-based readers can produce a smooth, plausible phrase where the page is damaged. The post on AI OCR lists this among the new failure modes to watch for.
• Clean print the engine was not built for. An unfamiliar typeface can be misread consistently and without hesitation.

Some errors carry no score at all, because the score attaches only to text the software produced:

• Skipped text. If a footer, a margin note or a whole block was classed as a picture and never read, there is no word to score. Nothing is flagged, because nothing is there.
• Wrong order. Two columns read as one, or table cells shifted by a row, can consist entirely of high-scoring words in the wrong place.
• Wrong page structure. A header spliced into a sentence is made of correctly read words.

The reverse case happens too. A correct word can carry a low score: a rare surname, a Latin phrase, a word under a light stamp that the engine in fact read properly. Low confidence means "look here", not "this is wrong".

The table summarizes how to read the four combinations.

Confidence earns its keep in the bottom right cell. It does nothing about the top right one, and that is where the errors that hurt a legal document tend to sit.

Using confidence flags to decide which pages a person reads

Where a tool reports confidence, use it to set the order and depth of human review, never to excuse a page from review that the task requires. A confidence flag is a marker on text scored below some level. Flags tell a reviewer where to start. They do not tell anyone where to stop.

This section applies only if your tool exposes scores or flags. Not every tool does, and nothing here describes a specific product.

Calibrate before you rely on a threshold

A threshold is the score below which text is flagged. Do not accept a default. Test it on your own documents:

1. Take twenty to thirty pages that reflect your real intake.
2. Check every word, or every critical value, against the page images, and mark the errors.
3. Group the checked items by score band.
4. Count the errors in each band.

Suppose, as an invented illustration, a sample of 2,000 checked words came out like this:

Read it two ways. The score does track error: lower bands are much worse. But 9 of the 56 errors, about one in six, sat in the top band, where no flag would fire. If you reviewed only words under 95, you would read 300 words, catch 47 errors and miss 9. Whether that is acceptable depends on the task.

Rules that keep triage honest

• Critical values are checked whatever their score. Amounts, dates, names, identifying numbers and citations are read against the image every time.
• Review order follows the score; review scope follows the task. For a filing, every page is read. Confidence decides which you read first and most slowly.
• Run a completeness check separately. Confidence cannot flag text that was never read, so missing pages and skipped blocks need their own check.
• Sample the unflagged pages. Read a fixed share of high-scoring pages, such as one in ten. If you find errors, widen the sample.
• Record the rule you used. Note the threshold, the sample rate and who reviewed what.

For the review itself, the post on OCR errors gives a fixed routine for proofreading OCR output against the original. Confidence changes where that routine begins. It does not shorten the list of things the routine checks.

Where triage by confidence fits

Asking a vendor whether its OCR reports confidence

Ask a vendor directly whether its OCR reports confidence, at what level, on what scale and where you would see it. Then test the answer on your own pages. People searching for OCR with confidence flags, or OCR that flags unreadable text, are really asking whether the tool will point at its own doubtful output.

Do not assume the answer for any tool. Rely on what a product's documentation states and on what you can see in its output. Schema Law's home page, for example, says in its questions section that nothing marks a passage as uncertain, and tells you to check names, dates and amounts against the original.

Questions worth putting in writing:

The question about unreadable text matters more than it looks. A tool that leaves a visible gap or a marker where it could not read is easier to check than one that always outputs something. Ask to see an example on a damaged page.

A test you can run in an hour

1. Choose ten pages: three clean, three faxed or photocopied, two with stamps or handwriting, two with tables of figures.
2. Run them and export whatever the tool offers that includes scores or flags.
3. Without looking at the scores, check twenty critical values per page against the images and list every error.
4. Now look at the scores for the errors you found. Count how many were flagged or scored low.
5. Count the flags on values that were in fact correct.
6. Look for anything on the page that is missing from the output, and note whether the tool told you.

The result is two figures: the share of real errors the flags caught, and the share of flags that were false alarms. A tool whose flags catch most errors with a tolerable number of false alarms will save review time. A tool whose errors mostly sit in unflagged text will not, whatever its documentation says about confidence.

If a tool reports no confidence at all, nothing is lost that a careful process depended on. The check that counts is the comparison of output with the page image, and it works the same way with or without a score. You can try that comparison on Schema Law's free plan, which reads one document a day of up to three pages. The pricing page lists the plans.