Make Document Production Searchable After Receipt

To make a document production searchable, lock away the files exactly as they arrived, then run copies through OCR and export searchable PDFs or text. First check whether text files came with the production and were never loaded, and whether the documents were electronic to begin with, because then you may be able to ask for a searchable form.

This post explains why productions turn up as pictures, what the Federal Rules of Civil Procedure say about the form of a production, how to do the conversion yourself, and how to raise unreadable pages with the other side. It describes document handling and is not legal advice. Discovery rules differ between federal and state courts and from one judge's standing order to the next, so read the ones that govern your case.

Why do productions arrive as unsearchable images?

Productions arrive as images because of how they were made. The documents were paper, or they were electronic files that someone printed, stamped and scanned, or they were converted to page images for numbering and redaction. In each case the result is a picture of every page with no text behind it.

The cause matters, because it decides what you can do about it.

• The originals were paper. Handwritten logs, signed contracts from a filing cabinet, old personnel files. A scan is the only electronic form these ever had. Nobody removed anything.
• Electronic files were printed and scanned. Emails and spreadsheets were printed, Bates-stamped by hand or by a copier, and fed through a scanner. The originals were searchable and the production is not.
• Files were converted to images for processing. Litigation software commonly turns each page into a TIFF or image PDF so that numbers and redactions can be burned in. The text is normally delivered alongside, as separate text files tied to the images by a load file.
• Redaction flattened the page. A properly redacted page is often saved as an image so that nothing can be recovered from under the black box. The text went with it.
• The documents came from third parties. Medical providers, banks and agencies send faxes and copier scans, and a party passes them on as received.

Before you conclude that a production is unsearchable, look at what else is on the drive or in the download. A folder named TEXT or OCR, and files ending in .DAT or .OPT, mean the producing party did supply text and expected you to load it into a review platform. Opening the image folder alone in a PDF reader will make the set look dead when it is not. The post on the Concordance load file format explains what those files are and how they connect text to images.

Mixed sets are common too. A volume can hold born-electronic PDFs with real text beside scanned exhibits. Test several files from different parts of the range. The quick ways to tell are in the post on why you can't copy text from a PDF.

Can you require a searchable production?

Sometimes. In federal court, a request for electronically stored information may specify the form of production, and the rules protect searchability that already existed. Paper that was never electronic is different: none of the provisions quoted below tells a party to add text to it. State rules vary, so check your own.

The provisions worth reading in full are short, and all sit in Rule 34 of the Federal Rules of Civil Procedure:

• You can name the form. Under Rule 34(b)(1)(C), a request "may specify the form or forms in which electronically stored information is to be produced."
• The other side can object, but must say what it will use. Rule 34(b)(2)(D) allows an objection to the requested form, and a party that objects, or that received no specification, must state the form or forms it intends to use.
• There is a default when nobody specified. Rule 34(b)(2)(E)(ii) says that if a request does not specify a form, a party must produce electronically stored information "in a form or forms in which it is ordinarily maintained or in a reasonably usable form or forms."
• You get one form. Rule 34(b)(2)(E)(iii) says a party "need not produce the same electronically stored information in more than one form."

The advisory committee's note to the 2006 amendment, printed on the same page as the rule, speaks to searchability directly: "If the responding party ordinarily maintains the information it is producing in a way that makes it searchable by electronic means, the information should not be produced in a form that removes or significantly degrades this feature."

Read those together and three practical points follow.

1. Ask at the start. Because of the one-form rule, the time to specify a searchable form is in the request itself. Earlier still, the parties' discovery plan under Rule 26(f)(3)(C) must state their views on issues about electronically stored information, "including the form or forms in which it should be produced." An agreed protocol that says "searchable PDF or images with extracted text and a load file" settles the question before any documents move.
2. The protection is for electronic information. Every provision quoted above is about electronically stored information. For documents generally, Rule 34(b)(2)(E)(i) requires a party to produce them as kept in the usual course of business or to organize and label them to match the request. It says nothing about adding text to paper.
3. Printed-and-scanned email is the arguable case. If the originals were searchable and the production is not, the committee note is the passage to raise. Whether a court would order a new production depends on your facts, your requests and your judge.

If your requests did not specify a form and the documents were paper, plan to do the OCR yourself. That is the usual position, and it calls for conversion work, not a motion.

Look for a text folder when a production arrives as images, before you run your own recognition. If the producing party already sent text, load that text with the images so review searches the electronically stored information they produced. If the folder is empty, OCR the images and spot-check names and dates. Write down which text the review team should trust.

Making a received production searchable yourself

Work on copies, convert in production order, and check the output against page counts. The outline below takes a production from a download link to a set a reviewer can search. Splitting large files, naming and failure tracking each have their own traps, covered in the guide to batch OCR for discovery documents.

1. Secure the original. Copy the production to a read-only location before opening anything, so an untouched set always exists.
2. Inventory it. List every file with its Bates range, page count and size. Note which files already have text.
3. Set aside files that need no OCR. A PDF with a good text layer is already searchable. Running it through OCR again spends pages and can replace exact text with recognized text that is worse.
4. Test the worst pages first. Pick five or six of the ugliest pages in the set, such as faxes, tiny print and stamped pages, and convert those. If the text is unusable, you want to know now.
5. Convert the rest. Upload each image-only file. In Schema Law you sign in, upload the PDF in your browser and wait for the pages to be read. Uploads can be up to 20 MB, and the page allowance per document depends on the plan.
6. Export two formats. Take a searchable PDF for reading and citing, since it keeps the page image with its Bates stamp, and plain text for search tools and comparison. The export formats can be downloaded again later from the same upload, so you do not have to decide everything at once.
7. Check page counts. Each export should have the same number of pages as its source file.
8. File the results beside the originals, not over them. Keep a parallel folder with the same file names.

A note on what you get from Schema Law's searchable PDF. A search takes you to the right page, and the highlight may not sit on the matching words, because the hidden text is added page by page and not word by word. The post on searchable PDFs for court filing explains what kind of text layer that is, so test a file in your own PDF reader before converting the whole set.

Preserving the production as received

Keep one copy of the production that nobody ever edits, and be able to show it is unchanged. Everything you do afterwards, such as OCR, renaming, splitting and annotating, happens on working copies. When a dispute arises about what was produced, the untouched copy is the answer.

A routine that takes ten minutes on the day the production arrives:

• Save the transmittal. Keep the cover letter or email, the download link details and the date received with the files.
• Copy to a read-only folder. Name it with the producing party, volume and date. Restrict who can write to it.
• Record hash values. A hash is a fingerprint computed from a file's contents. If the file changes by a single byte, the hash changes. Your operating system can compute one with a built-in command, and many productions arrive with a hash list you can check against. Store your list with the transmittal.
• Never rename the originals. Sorting, prefixing and splitting belong in the working folder.
• Note anything wrong on arrival. Files that will not open, gaps in the Bates sequence, a volume that is shorter than the cover letter says.

Two points about the OCR output itself.

The text is your reading of the page, not the page. Recognition makes mistakes, and those mistakes are yours, not the producing party's. When you quote or cite a produced document, work from the image and its Bates number. Use the text to find the page. The guide to quoting text from a scanned PDF covers checking a passage before it goes into a brief.

Know where every copy is. Under Rule 26(b)(5)(B), if a producing party notifies you that privileged material was produced, you "must promptly return, sequester, or destroy the specified information and any copies it has." A copy uploaded to a conversion tool, and each export made from it, is a copy. Keep a note of which files went where, so that you can act on a notice quickly. One of the lines Schema Law lists for every plan is "Delete any document, at any time", and deleting a document removes the original file and everything parsed from it. Nothing is deleted automatically, so that step is yours to take. The security commitments list the others. What a clawback notice requires of you in a given case is a legal question for the lawyers on the matter.

What to search for first

Search first for things you already know are in the set. Known names, a known date, a document you have seen before. Those test searches tell you how far to trust the text before you rely on a search that comes back empty.

A sensible first hour:

1. A document you can find by hand. Pick a page, read a distinctive phrase from the image and search for it. If it is not found, look at the text behind that page and see what the recognition did to it.
2. The key names, in every form. Surname alone, initials, nicknames, email addresses and the spellings people actually use. Scanned text often turns "rn" into "m" and "l" into "1", so try a damaged version of an important name as well.
3. Dates in several formats. "March 3, 2021", "3/3/21", "03-03-2021" and "3 March" are four different strings to a search box.
4. Amounts and account numbers. Search with and without commas and currency signs. Digits are where recognition errors do the most harm, so treat a missing number with suspicion.
5. Terms of art from the case. Product names, project code names, contract section numbers.
6. The Bates number of a page you care about. If the stamp was read, searching for it is the fastest way to jump to a page. The post on how to OCR Bates numbered documents explains when that works.

Keep a list of what you searched and what came back. It becomes the record of your review, and it shows which terms need a second pass.

The mistake to avoid is treating no hits as proof of absence. A search of OCR text can only find what the software managed to read. Handwriting, faint faxes, text under stamps and tables with tight columns are the places where a responsive document can sit unseen. Where the stakes justify it, page through the worst-quality files by eye. The wider picture of how recognition quality affects search terms is in the overview of OCR in e-discovery.

Raising OCR problems with opposing counsel

Raise a problem with the other side when the documents themselves are deficient: pages too poor to read, text files that were promised and never arrived, or electronic records delivered as scans of print-outs. Be specific, cite Bates ranges, and ask for a named fix. Your own OCR errors are not their problem.

Sort what you found into two piles first.

A letter or email that gets results usually contains:

• The production volume and the date you received it.
• A table of the affected documents by Bates range, with a few words on each: "illegible, appears to be a fax of a copy", "pages 3 to 9 cut off at right margin".
• What you are asking for: a rescan from the original, the native file, or the extracted text and load file.
• The provision of the requests or the agreed protocol you rely on, if there is one.
• A date for a response and an offer to talk.

Asking is also a required step. In federal court, a motion to compel under Rule 37(a)(1) "must include a certification that the movant has in good faith conferred or attempted to confer" with the other party to get the discovery without court action. Your court or judge may add conference requirements of their own in local rules or a standing order, so check before drafting anything.

The log of failed and unreadable files from your conversion is the evidence for this conversation. Keep it by Bates number from the first day, and the request to the other side nearly writes itself.