Lossy vs Lossless Compression in a Scanned PDF

Lossless compression stores a scanned page so that every pixel comes back exactly as the scanner captured it. Lossy compression throws some of the picture away to make a smaller file, and what comes back is an approximation. One lossy method for black and white pages can go further and replace one character shape with another.

Which of the two a scan uses is decided by a setting on the copier, or by whatever program last saved the file. The setting may not use the word "lossy" at all. Look for "quality", "file size" or "compact", and it is worth knowing what sits behind those words before a scan becomes the only copy of a signed page.

What is the difference between lossy and lossless compression in a scan?

Lossless compression in a scan describes the same pixels in fewer bytes, so opening the file gives back the identical image. Lossy compression saves a simplified picture that looks close to the page and can never be turned back into what the scanner saw. The first costs storage. The second costs detail.

The PDF specification draws the line in one note. In the published copy of ISO 32000-1, the PDF 1.7 standard, a note to clause 7.4.1 says: "Some types of compression are lossy, meaning that some data is lost during the encoding, resulting in a loss of quality when the data is decompressed. Compression in which no loss of data occurs is called lossless."

Three consequences follow for a paper file.

• Lossy is one-way. Saving a damaged image again at a higher quality setting makes a bigger file with the same damage in it.
• Lossy stacks. Each lossy save approximates the previous approximation. A page that was scanned, emailed through a system that shrinks attachments, and then "optimized" by a PDF program has been simplified three times.
• Lossless is not the same as faithful. A lossless method preserves whatever the scanner handed it. If the scanner was set to black and white and dropped a pencil note before compression began, lossless compression keeps the page without the note, perfectly.

That third point is where compression meets color mode. Black and white, grayscale and color decide how much the scanner records for each dot, and compression then acts on that record. The comparison of how color mode changes the size of a scanned file covers the first of those two decisions.

The compression methods inside a scanned PDF: Group 4, JPEG, JBIG2 and Flate

A scanned PDF stores each page image through a filter, which is the PDF standard's word for a compression method. Group 4 and Flate are lossless. JPEG is lossy. JBIG2 can be either, depending on how the device that wrote the file was set.

The quotations in the last column come from clause 7.4 of the PDF 1.7 standard. The Library of Congress adds, in its format description for Group 4 fax compression, that the method offers "lossless compression of bitonal" images, meaning pages that are pure black and white.

Group 4

Group 4 was built for fax machines and records a black and white page as runs of white and black along each line. It cannot hold gray. A Group 4 page is therefore exactly as good as the black and white decision the scanner made, and no worse.

JPEG

JPEG is the lossy method a PDF uses for grayscale and color page images. It divides the image into small blocks and keeps a simplified version of each. On a photograph the loss is hard to see. On text it shows as gray smudging and faint checkerboard patterns around the letters, strongest at low quality settings. The JPEG family does include lossless modes, as the Library of Congress notes in its description of the JPEG image encoding family, but the baseline form a PDF's DCTDecode filter reads is the lossy one.

JBIG2

JBIG2 was designed for black and white pages. The PDF 1.7 standard says that, in general, it "provides considerably better compression than the existing CCITT standard", meaning Group 4 and its predecessor. Its international standard, ITU-T Recommendation T.88, carries the title "Lossy/lossless coding of bi-level images", which tells you both modes are part of the design. Nothing visible in a PDF reader says which mode a given file used.

Flate

Flate is the general-purpose method also used in ZIP files. It is lossless and works on any data, so it can be applied to page images, to fonts and to the hidden text layer alike. A scan saved as a PNG and then placed in a PDF is one route by which a page image ends up Flate-compressed.

The same methods appear inside TIFF files, where the choice of compression also decides which viewers can open the file. The post on what a TIFF file is explains that container.

Can lossy compression swap one digit for another on a scanned page?

Yes. One family of lossy compression for black and white pages stores a single picture of each character shape and reuses it wherever it judges another shape to be the same. When that judgment is wrong, a 6 can be drawn where the paper shows an 8, and the result looks crisp and correct.

How the method works

The PDF 1.7 standard describes the technique in its JBIG2 clause, 7.4.7: "The JBIG2 encoder shall build a table of unique symbol bitmaps found in the image, and other symbols found later in the image shall be matched against the table. Matching symbols shall be replaced by an index into the table". A page of contract text contains thousands of letter shapes and only a few dozen distinct ones, so storing each distinct shape once saves a great deal of space.

Everything turns on the word "matching". In lossless mode the page is rebuilt exactly. In lossy mode the encoder may accept a close match in place of an exact one. If the encoder decides that a small, slightly blurred 8 is close enough to a 6 it has already stored, the file records "draw the 6 here".

The published account

In August 2013 Xerox published a statement about its own devices. The statement, by a Xerox principal engineer, is preserved in an archived copy of Xerox's August 2013 clarification on the scanning issue. It opens by acknowledging "articles about Xerox devices randomly altering numbers in scanned documents" and gives the cause in one sentence: "The problem stems from a combination of compression level and resolution setting."

The statement said the design "utilizes the recognized industry standard JBIG2 compressor" and quoted the notice shown beside the quality setting on the devices: "The normal quality option produces small file sizes by using advanced compression techniques. Image quality is generally acceptable, however, text quality degradation and character substitution errors may occur with some originals." It said Xerox did "not normally see a character substitution issue with the factory default settings" and that "the defect may be seen at lower quality and resolution settings". It recommended the factory defaults with quality set to "higher", and an editor's note on the same page says Xerox later published further information.

That statement describes particular devices and settings in 2013. It is not evidence about the copier in your office. It does establish the general point, in the manufacturer's own words: a compression and resolution setting can substitute a character on a page image, and the manufacturer traced the fault to compression, not to a recognition step.

Why this failure is different

A blurred digit warns you. A substituted digit does not, because it is a clean copy of a real character taken from elsewhere on the same page. Proofreading the scan against itself cannot find it. Only a comparison with the paper, or with a second scan made at different settings, can.

Consider a fictional example. A paralegal scans a rent ledger for Tolliver Storage at the copier's smallest file setting. In the paper ledger the March payment is $8,650.00. In the scan it reads $6,650.00, in sharp type. The spreadsheet built from the scan carries the wrong figure, the running balance stops agreeing, and an hour goes to checking the arithmetic before anyone looks at the paper.

Keep the paper nearby when a compressed exhibit is your only digital copy. Check a damages figure, a statute section, and a citation against that paper. Heavy compression can swap a digit on the damages line or crack a character inside the citation. Search then returns the wrong amount, and the picture agrees with it.

Three ways a wrong character reaches your text

Substitution by compression is one of three different mechanisms, and each calls for a different check.

When the page image is right, you can catch the error by reading text against image. When the image itself is wrong, you need the paper. The post on why OCR gets numbers wrong covers the second row, and the post on OCR hallucination covers the third.

Mixed raster content: one page split into text and background layers

Mixed raster content, or MRC, compresses a page by splitting it into layers: a black and white mask that holds the shapes of the letters, and color layers that hold the ink color and the paper behind it. Each layer is then compressed with the method that suits it.

The model has its own international standard, ITU-T Recommendation T.44, titled "Mixed Raster Content (MRC)". The Library of Congress describes it in its format description for the JPEG 2000 compound image file format, which stores a page as a set of layout objects, each consisting of "an MRC mask-image object pair".

The attraction is size. A color scan of a contract on cream paper is mostly paper. Under MRC the paper texture goes into a background layer saved at low resolution and heavy compression, while the letter shapes stay sharp in the mask. If your scanner or PDF program offers an output option with a name like "compact PDF" or "high compression PDF", look in its documentation for whether the option uses MRC and which method it applies to each layer.

What to keep in mind about an MRC file:

• The layers are compressed separately. The mask is a black and white image, so it may be stored with JBIG2. If that JBIG2 is lossy, the risk of a substituted character applies to an MRC file that looks like an ordinary color scan.
• Faint marks can land in the wrong layer. Software has to decide which pixels are text. A pale stamp, a pencil initial or a light signature that it classes as background is saved at the background's low quality.
• The page stops being one picture. Zoom in on an MRC page and you may see sharp letters floating on a soft, blotchy ground. That look is a sign of layered compression, not of a poor scanner.

MRC is a reasonable choice for a working copy that has to travel. It is a harder choice to defend for the only copy of an original, because two separate lossy decisions have been made about what on the page mattered.

How does heavy compression affect the text OCR recognizes?

Heavy lossy compression changes the shapes that recognition software has to read. Smudged edges, filled-in loops and broken thin strokes all make letters look like other letters, so the recognized text carries more errors. Lossless compression has no effect on recognition, because the software receives the same pixels the scanner captured.

No published figure tells you how much compression your documents will tolerate, and none is offered here. What can be said is where the damage falls first:

• Small type. Footnotes, fax headers and the fine print of a form have the fewest pixels per letter, so each lost pixel matters more. The post on the best dpi for OCR explains why small print needs more dots per inch.
• Look-alike characters. The difference between 3 and 8, or between c and e, is a small gap. Compression noise fills small gaps.
• Punctuation. A decimal point is a handful of pixels.
• Text on a tinted or textured background. The compression spends its effort on the texture.

A developer of model-based readers gives a similar warning about its own input. Anthropic's documentation for image input, cited here only as a published example, says in its guidance on image quality that lossy compression "can introduce artifacts that are detrimental to model performance, especially when multiple compression passes are applied", and that "heavy JPEG compression can make text difficult to read."

Two practical rules follow. First, run recognition on the least compressed copy you hold, and shrink afterwards if you need to. Second, when a scan must be made smaller to fit an upload or filing limit, change one thing at a time and compare the recognized text of a hard page before and after. The post on converting large scans has a section on shrinking a scanned PDF without hurting recognition.

A side-by-side test settles the question for your own equipment. Scan one page of small print twice, at the copier's highest quality setting and at its smallest file setting. Run both through your OCR tool, export plain text, and compare the two results line by line. Schema Law accepts a PDF, PNG or JPEG of up to 20 MB, and plain text is among its export formats, so the test needs nothing special. The free plan reads one document a day of up to three pages, which means either combining the two scans into one two-page PDF or running them on consecutive days.

Which compression to accept for a scan kept as the firm's record

For a scan that will stand in for the paper, accept lossless compression, or lossy JPEG at a high quality setting on a grayscale or color page. Do not accept lossy pattern-matching compression. For a copy you can remake from the paper or from a master file, smaller and lossier is fine.

Finding out what your device does

1. Open the copier's scan settings and write down every option that mentions quality, compression, file size or compact output. Note the default.
2. Look in the device's administrator guide for the compression method behind each option. If the guide names JBIG2, look for whether it says lossy or lossless.
3. If the documentation does not say, ask the supplier in writing, and keep the answer with your scanning procedure.
4. Scan a test sheet of small digits, such as a page of 6-point numbers in rows, at each setting. Compare every digit with the printed sheet.
5. Record the setting you chose in the firm's scanning standard, so that the choice is not remade by whoever stands at the copier next.

Whether a scanned copy can stand in for an original at all is a legal question that depends on your jurisdiction, the kind of document and any retention rules that apply to the matter. Compression is only the technical half of that decision. Settle the legal half before paper is destroyed.

One more habit protects you from other people's settings. When a scan of figures arrives from outside and something does not add up, ask for the page to be rescanned at a higher quality setting before you assume a typing or arithmetic error. Schema Law reads the pages you upload into text and tables as they are, and nothing in that process can restore a character that was replaced before the file reached you. The steps are set out under how it works.