OCR Preprocessing: What Software Does to a Scan First

OCR preprocessing is the cleanup software performs on a scanned image before it tries to read any letters. It turns shades into pure ink and paper, removes specks, and wipes dark edges, so the reader sees clean shapes. The same cleanup can also delete small real marks, such as a decimal point.

That last sentence is why a lawyer should care about a step that sounds like plumbing. Preprocessing throws parts of the picture away on purpose.

What is OCR preprocessing?

OCR preprocessing is every change made to the page image between the scan arriving and recognition starting. Its job is to give the reader an image in which text is dark, background is light, lines are level and nothing else is on the page. It runs first, so its mistakes reach every later stage.

The operations that fall under the name:

Deskewing has its own explanation in the section on how skew is corrected before recognition. What the reader does with the cleaned image is the subject of the post on how OCR works.

Who does the cleanup

Cleanup can happen in three places, and a page may pass through all of them.

• The scanner. A device set to black and white, or with "background removal" switched on, has already processed the image before any file exists.
• The recognition core. The Tesseract project's guide to improving the quality of the output says: "Tesseract does various image processing operations internally (using the Leptonica library) before doing the actual OCR."
• The application around the core. A PDF tool may offer further steps as options. The open-source tool OCRmyPDF, for example, lists page rotation, background removal, deskewing and cleaning in its cookbook of common tasks.

You control the first. You may or may not control the other two, depending on the product.

Binarization: turning a gray page into ink and paper

Binarization converts a page with shades of gray or color into two values, ink and paper. The software picks a brightness cutoff, called a threshold, and sorts every pixel to one side of it. A single cutoff for the whole page is global thresholding. A cutoff recalculated for each neighborhood is adaptive thresholding.

The Tesseract guide describes the step and its weak point in two sentences: "This is converting an image to black and white. Tesseract does this internally (Otsu algorithm), but the result can be suboptimal, particularly if the page background is of uneven darkness."

Global thresholding

A global method looks at all the pixels on the page and chooses one cutoff. Otsu's method is a well-known way of choosing it. The scikit-image project's guide to thresholding says the method calculates an optimal threshold "by maximizing the variance between two classes of pixels, which are separated by the threshold." In plain terms, it looks for the cutoff that best divides the page into a dark group and a light group.

That works when the page really has two groups. A laser-printed letter on white paper does.

Adaptive thresholding

The same scikit-image guide states when one cutoff is not enough: "if there is large variation in the background intensity, adaptive thresholding (a.k.a. local or dynamic thresholding) may produce better results." Its companion page on Niblack and Sauvola thresholding explains the idea: "Instead of calculating a single global threshold for the entire image, several thresholds are calculated for every pixel", using the brightness of the pixels around each one.

The Tesseract guide records that version 5.0.0 "added two new Leptonica based binarization methods: Adaptive Otsu and Sauvola." So one engine can binarize the same page in more than one way, depending on how the product built on it was set up.

A worked example

Picture a grayscale scan of a recorded deed from the fictional Tillery County, photocopied from a bound book. The left third of the page is in the shadow of the binding. Brightness runs from 0 for black to 255 for white.

With one cutoff, the shadowed strip becomes a solid black band and the grantor's name inside it is gone. With a local cutoff, the software notices that in this neighborhood "paper" means about 115, sets the line under that, and the name survives.

Adaptive methods have their own failure. In a large blank area, the local method has no real ink to anchor on and may promote faint paper texture to ink, which shows up as a rash of specks in the margins. The despeckling step then has more to remove.

The scanner may have got there first

Software binarization only has shades to work with if the file still contains shades. A page scanned in black and white mode was reduced to two values inside the scanner, with the scanner's cutoff. Nothing the OCR software does afterwards can bring back the pencil note that fell on the wrong side. The post on scanning modes explains what black and white scanning does to each pixel.

Despeckling, border removal and punch-hole cleanup

Despeckling, border removal and hole cleanup all delete marks that are not text: dust and toner specks, the dark band a scanner leaves around a sheet, and the black discs left by punched holes. Each works by a rule about size, darkness or position, and has no idea what the mark means.

Despeckling

The Tesseract guide defines the target: "Noise is random variation of brightness or colour in an image, that can make the text of the image more difficult to read." It adds that some noise survives binarization and can lower accuracy.

A despeckle filter removes marks under a size limit. The open-source cleanup program unpaper, which OCRmyPDF uses for its cleaning options, documents one in its description of image processing features: "The noisefilter removes small clusters of pixels ("noise") from the sheet." A second filter in the same document removes what it calls "lonely" clusters, meaning small marks with very few other dark pixels near them.

Size and loneliness are the whole test. A speck of toner between two lines passes it. So does a period at the end of a short line.

After despeckle, open a pleading and search for a citation, a statute number, and a short notice in the margin. Those marks are small facts on the page. If a filter removes a period or a line number, the search misses them while the surrounding paragraph still looks clean.

Border removal

The Tesseract guide explains why edges matter: "Scanned pages often have dark borders around them. These can be erroneously picked up as extra characters, especially if they vary in shape and gradation." A black band down the side of a fax can come back as a column of l, I and | characters.

unpaper's guide to its basic concepts describes border detection as starting at the outer edges of the sheet and scanning toward the middle "until some content-pixels are reached". Then "the area outside the detected border on the sheet will be wiped out". Anything the scan decides is outside the content goes white.

On a legal page, the edge is not empty. Bates numbers, fax header lines, page numbers, line numbers on pleading paper and marginal initials all live there.

Large dark areas and punch holes

unpaper documents a third filter for big black regions: "The blackfilter can help to find large areas of black and wipe them out automatically." The same passage warns the user to be careful with diagrams, since some "intentionally contain large areas of dark color".

Punch holes are a harder question to answer from documentation. None of the tool documentation cited in this post describes a step aimed at punch holes by name. A hole scanned against a dark lid is a round black disc near the edge, so it may be caught by a border wipe, by a filter for dark areas, or by nothing. Scanner drivers may offer their own hole-fill setting. The only way to know what your tools do is to scan a punched page and look at both the page image and the text that comes back. If the holes survive, expect an O, a 0 or a bullet in the margin of the output.

Can image cleanup erase a period, a decimal point or a faint initial?

Yes. Image cleanup removes marks by size, darkness and position, and a period, a decimal point or a faint set of initials can meet the same test as a speck of dust. The makers of cleanup tools say so in their own documentation and tell users to review the pages afterwards.

unpaper's documentation is specific about the noise filter's size setting: "This value must not be chosen too high in order not to remove relevant elements of page content, e.g. normal text-points ("."). As a consequence, this option might have to be adjusted on images with a low scan resolution."

That last sentence matters for faxes and low-resolution scans. The fewer pixels a period is made of, the closer it is to the size of noise.

OCRmyPDF's cookbook gives a general warning about its image processing options: "It is suggested that the user review files after image processing as these commands might remove desirable content, especially from poor quality scans."

Marks at risk on a legal page

One more filter deserves a mention. unpaper's gray filter wipes areas judged to be gray only, and its documentation cautions that on color scans "any bright color might be considered as gray and be wiped out." Highlighter and colored ink are bright colors.

How to check

1. Pick three pages from the file with the smallest and faintest content: a page of figures, a page with handwritten initials, a page with a stamp.
2. Run them through the tool.
3. Search the text result for a figure you can see on the page, typed with its decimal point. No hit means the point, or a digit, was lost.
4. If the tool returns a page image, open it beside the original at 300 percent or more and compare the initials, the stamp and the margins.
5. Count the periods in one citation-heavy paragraph on the page and in the output.

A missing mark in the text does not prove cleanup was at fault, since the reader can drop small marks on its own. A missing mark in the returned image does.

Does preprocessing change the page image you get back?

Whether preprocessing changes the page image you get back depends on the tool and its settings. Some tools clean a working copy, read it, discard it and return your original picture with text added. Others put the cleaned image into the output file. The first affects only the words. The second changes what the page looks like.

OCRmyPDF documents both behaviors as two separate options, which makes it a clear illustration of the difference:

The same page adds a note that applies to more than cleaning: "In many cases image processing will rasterize PDF pages as images, potentially losing quality."

Tools also differ in whether they let you see the working copy. The Tesseract guide says a configuration setting makes the engine save the image as processed, and advises: "If the resulting tessinput.tif file looks problematic, try some of these image processing operations before passing the image to Tesseract." The browser library Tesseract.js lists among the changes in its fourth version, in its project README, that "Processed images (rotated, grayscale, binary) can now be retrieved". A hosted product may expose none of this.

Why the answer matters for a file you keep

If a scan is the firm's copy of an executed document, or a page that will be produced or filed, the picture is the record. A returned image with the margins wiped and the specks gone looks tidier and is no longer what was received. Three habits cover the risk with any tool:

• Keep the scan as it arrived, under its own name, before any conversion.
• Find out which image the tool's searchable PDF contains by comparing one page with the original at high zoom. Look at a speck you can identify. If the speck is gone, the image was cleaned.
• Treat exports that contain only text, such as Word or plain text, as working documents and not as copies of the page.

Schema Law's homepage describes its searchable PDF export as "Your original file with a hidden text layer, so search finds the page", in the list of export formats. The site makes no statement about what cleanup happens before reading, so apply the same checks there as anywhere else.

Should you clean a scan yourself before sending it for OCR?

As a starting rule, no. Send the scan as it is first and look at the result. Clean it yourself only when you can see a specific fault that is hurting the output, you cannot rescan the paper, and you can work on a copy. Manual cleanup stacked on automatic cleanup is where real marks get lost.

Rescan before you retouch

If the paper is available, a better scan beats any repair. Grayscale in place of black and white, a higher resolution, a straight feed and a closed lid remove the causes that cleanup can only patch. The post on the best DPI for OCR covers the resolution side.

When your own cleanup can help

The Tesseract guide lists situations where preparing the image is worth trying. Among them:

• A dark scanning border. Cropping the black band off the edge removes a source of invented characters.
• A crop that is too tight. The guide notes that text with no margin at all can cause problems and suggests adding a small white border.
• Light text on a dark background. The guide tells users of the 4.x versions to supply dark text on a light background.
• A page too tilted for line finding. The guide says to rotate the page image so that the text lines are horizontal.

Each of these is a single, visible defect with a single fix.

When it harms

The fourth row is not a technical point. If the page is evidence or a record, editing its image raises questions that have nothing to do with OCR. Do any cleanup on a duplicate made only for recognition, and keep it apart from the file copy.

A short decision routine

1. Upload the scan untouched and read the result against the page. In Schema Law the parsed pages appear on screen for that comparison, as the how it works section describes.
2. If the text is good enough for your purpose, stop.
3. If it is not, name the fault you can see on the image: border, shadow, tilt, faint print.
4. If the paper exists, rescan with a setting that addresses that fault.
5. If it does not, fix that one fault on a copy, upload the copy, and compare the two results on the names, dates and amounts that matter.