How to Index Discovery Documents From a Scanned Set

To index discovery documents from a scanned set, give every document one row in a spreadsheet and record what its face shows: date, author, recipient, document type and Bates range. Read those fields from the OCR text, confirm them against the page image, and sample the finished index for coding errors before the team relies on it.

What is a discovery document index, and who uses it?

A discovery document index is a table with one row per document and columns for the facts that identify it, so that anyone on the team can find a document by date, author, type or number without opening the files.

The Sedona Conference Glossary gives this as one meaning of indexing: "Objective coding of documents to create a list similar to a table of contents."

Full-text search and an index answer different questions. Search finds the pages where a word appears. An index answers "show me every letter Ruth Amaya wrote in the second half of 2021", which no keyword can do if the letters never use her name the same way twice, or if the scan of her signature was read as noise.

The people who lean on it, and what each wants from it:

An index is a neutral list. It is not a privilege log, which covers only withheld documents and carries the basis for withholding each one. That log has its own requirements, set out in the post on privilege log columns a scanned letter or memo can fill. Later in the case, the same index rows are the raw material for the trial list, and the guide on how to create an exhibit list picks up from there.

Which index fields can be read off a scanned document?

The fields you can read off a scanned document are the ones printed or written on it: its date, who wrote it, who received it, who was copied, its title or subject line, and what kind of document it is. Everything else in the index comes from your scanning records or from a reviewer's judgment.

The Sedona glossary calls the first group bibliographic coding: "Manually recording objective information from documents such as date, authors, recipients, carbon copies, and blind copies, and associating the information with a specific document." It sets that apart from subjective coding, "Recording the judgments of a reviewer as to a document's relevancy, privilege, or importance with regard to factual or legal issues in a legal matter."

Sorting the columns by where each value comes from tells you who can fill it and how far to trust it.

Three things a scan cannot give you, however good the OCR: the time an email was sent if it was printed without a header, a file path or file name, and anyone blind-copied on paper correspondence. Leave those columns out of a paper index instead of carrying them empty.

Keep the judgment columns physically apart from the objective ones, at the right-hand end of the sheet. The objective columns should come out the same no matter who codes them. The judgment columns will not, and mixing the two makes it impossible to check either.

Tie each finished row to the Bates range and the production volume. A reviewer who filters on author and date is gathering documents for a deposition or a motion. If you withheld the document, the privilege log carries the basis, and the index row should say only that it was withheld. Copy the date and the names from the page in front of you.

Objective coding: date, author, recipient and document type from the page

Objective coding means reading four facts from each document and typing them the same way every time: the date on its face, the person who wrote it, the person it was sent to, and a document type from a short fixed list. Write the conventions down first, then code from the OCR text with the page image open beside it.

Before any coding starts, each file must be exactly one document. If the scan came off the machine as one PDF per folder, split it first. The post on document unitization explains how to find where each document starts and ends.

Write the coding rules on one page

The glossary's entry for author is a useful anchor for the hardest of these: "In the case of a document in the form of a letter, the author or originator is usually indicated on the letterhead or by signature." When the two disagree, such as a letter on company letterhead signed by an assistant "for" a manager, code the person who signed and note the other in a comments column.

For type, the glossary defines the field as one "that captures the general classification of a document, i.e., whether the document is correspondence, memo, report, article, and others." Keep your list to a dozen values or fewer. A starting list for a commercial dispute: Letter, Email printout, Memo, Agreement, Invoice, Purchase order, Report, Notes, Form, Spreadsheet printout, Photograph, Other.

Add a names list as you go. The first time "R. Amaya", "Ruth Amaya" and "Ruth A." appear, decide they are one person, record the standard form, and use it from then on. Without that list, a filter on author returns only the documents coded under one spelling.

Code from the text, confirm on the image

1. Convert each unitized file. In Schema Law you upload the PDF in the browser and export once the pages are read. The export formats include plain text, which puts the first lines of page 1 in front of you without opening the PDF. For a document with no tables, the CSV export lists the text line by line with a page number beside each line.
2. Read the top of the first page for the dateline, the addressee and the subject line. Read the foot of the last page for the signature block and the copy line.
3. Paste or type the values into the row, applying the rules.
4. Open the image and compare each value. Dates and names are where a misread character does the most damage: a 1 for a 7, "Osel" for "Osei".
5. Move to the next file. Do not stop to read the document for content. That is review, and it is a separate pass.

Schema Law reads the pages into text and tables. It does not fill these columns. The coder does, and the coder is the check.

A coded example

Three rows from the fictional case Harlan Ridge Dairy v. Coastal Refrigeration:

The third row shows the limits. Nothing on the page names the writer, so the author is coded Unknown. Guessing from the handwriting is a judgment, and it belongs in a comments column with the coder's initials, if anywhere.

How do you index discovery documents that have no date or author?

Index a document with no date or author by entering a fixed word, such as Undated or Unknown, never a blank and never a guess in the objective column. If context supports an estimate, put it in a separate column with its basis, so a reader can see what the page shows and what someone inferred.

A blank cell is ambiguous. It can mean the document has no date, the coder skipped it, or the text was unreadable. Three distinct values remove the doubt:

Choose one way of writing partial dates and keep to it. The 00 day shown here sorts to the start of the month in a text column. If your spreadsheet insists on real dates, use the first of the month and say so in the basis column.

On which date counts, the Sedona glossary describes document date as "Generally, the term used to describe the date the document was last modified or put in final form; applies equally to paper and electronic files." For a letter that is the dateline. A fax header line, a "received" stamp and a handwritten "filed" note are later events. Record them in the basis or comments column, because they can matter a great deal, but do not let them replace the date the document bears.

Authors follow the same pattern:

• Unsigned. Code Unknown. If the letterhead names an organization, fill the organization column and leave the person as Unknown.
• Signature only, unreadable. Code Illegible, and note any typed title beneath it.
• Signed on behalf of someone. Code the signer, and name the principal in comments.
• Forms. The author of a completed form is the person who filled it in, if the form says who that is. The company that printed the blank form is not its author.
• Drafts. Code what the draft shows, add "Draft" to the title, and resist merging it with the final. They are two documents.

Undated documents are worth a second look once the index is complete. Sort the whole sheet by box and folder, and an undated memo sitting between two letters from the same week has a likely home. Record that as an estimate with its basis. The objective column still says Undated.

Tying each index row to a Bates range and a request number

Tie each row to the production by recording the document's first and last Bates number and its page count, and tie it to the requests by recording the number of each request it answers. Those two cross-references let you move from a cite to a row and from a request to a list of documents.

The Bates columns

Use three columns: Beg Bates, End Bates and Pages. The Sedona glossary lists the first under the name Beginning Document Number: "A unique number identifying the first page of a document or a number assigned to identify a native file." The three check each other, because the last number must equal the first number plus the page count, minus one. A formula column that tests this on every row catches typing errors the moment they are made.

If a document came with attachments, add Beg Attach and End Attach columns holding the range of the whole family. Every member of the family carries the same two values, so filtering on Beg Attach pulls a letter and its enclosures together.

Sort the index by Beg Bates and compare neighbours. Each row's first number should be one more than the previous row's last. A break means a document is missing from the index or a range was typed wrong.

The request column

For documents your side is producing, a request-number column records why each one is in the set. It also supports one of the two ways a party may organize a production under Federal Rule of Civil Procedure 34(b)(2)(E)(i): "A party must produce documents as they are kept in the usual course of business or must organize and label them to correspond to the categories in the request". State rules have their own wording, so read the one that applies to you.

Practical conventions:

• Write request numbers with the set they belong to, such as RFP1-14 for request 14 of the first set, since numbering restarts with each set of requests.
• A document that answers several requests lists them all, separated by semicolons. Do not pick one.
• Fill the column when the document is collected, while the person who pulled it remembers why. Reconstructing it afterwards means rereading everything.
• For documents you received, the column can instead record which of your requests the other side said the document answers, if their response or cover letter says so.

With both cross-references in place, two lookups become mechanical. A brief cites CRS000119: filter for the row whose range contains it. Opposing counsel asks what was produced for request 14: filter the request column and read off the ranges.

Sampling an index built from OCR text for coding errors

Check an index by drawing a sample of rows, comparing every coded field in those rows with the page image, and counting errors by field. The count tells you whether a column can be trusted as it stands, needs a rule fixed, or has to be recoded.

OCR text gives no warning of its own mistakes. Schema Law's site says as much about its own output: nothing marks a passage as uncertain, so names, dates and amounts need checking against the original. An index typed from that text inherits whatever was misread, plus the coder's own slips.

Draw the sample

1. Add a column of random numbers to the index. In a spreadsheet, =RAND() in each row does it. Copy the column and paste it back as values so the numbers stop changing.
2. Sort by that column and take the first rows as your sample. The supervising lawyer sets how many. The worked tally in this section uses fifty.
3. Add every row coded Undated, Unknown or Illegible to the review, separately from the random sample. Those rows are where a coder gave up, and a second reader may make out what the first could not.
4. If more than one person coded, make sure the sample includes rows from each of them.

Check and tally

For each sampled row, open the document and compare field by field. Record the result in a small tally, one mark per wrong field:

Act on the pattern

Read the last column before the numbers. Errors come in two kinds, and they need different fixes.

• Rule errors. The four type errors in the tally are one misunderstanding repeated. Clarify the rule, filter the whole index for the affected value, and recode those rows. Sampling more will not help.
• Reading errors. Misread digits are scattered and have no pattern. If the sample finds more than the lawyer will accept in a column that matters, such as date, that column needs a full second pass against the images.

Fix the sampled rows, record the date and the result of the check in a note with the index, and sample again after any recoding. An index with a documented check is one the team can sort and filter with some confidence. The exports that feed it, CSV and Excel among them, are on every Schema Law plan, since each one includes every export format, and the pricing page shows the page allowance on each.