OCR Exhibits With Tables: Keeping Rows and Columns

To OCR exhibits with tables, use a tool that outputs real rows and columns, export to a spreadsheet, and then reconcile the result with the exhibit: count the rows, re-add the totals, and compare key cells with the page image. The image stays the evidence. The spreadsheet is your working copy.

Why do exhibit tables scramble under OCR?

Exhibit tables scramble because a table's meaning lives in position, and position is the first thing a scan degrades. A figure belongs to a row and a column only because of where it sits. Reading the characters correctly is not enough. The software must also decide which cell each one belongs to.

If one empty cell in a table is skipped, every figure to its right moves one column left, and a payment appears under the heading for a charge. The digits are all right and the table is wrong.

That shift is how a contract schedule becomes a different document. You are holding documentary evidence, and someone may turn the same grid into demonstrative evidence. A moved cell changes relevance and probative value even when every digit is right. If the parties reached a stipulation on a total, re-add that total before you cite a row.

Table reading is a separate problem from text reading, and engine makers say so. The documentation for the open-source Tesseract engine, on improving the quality of its output, states that it "has a problem to recognize text/data from tables" without custom segmentation or layout analysis. Tools built for tables add that analysis, and the post on getting a scanned PDF into Excel explains how it works.

Exhibits make the job harder than a fresh scan of the same table would be:

• They have been copied many times. An exhibit to a deposition becomes an exhibit to a motion and then to a trial binder. Each copy thins the ruling lines that mark the cells.
• They carry extra marks. Exhibit stickers, Bates numbers, fax headers and filing stamps often land on the header row or the totals row, which are the two rows you most need.
• They are shrunk or turned. A wide spreadsheet printed to fit a letter page has tiny type. A wide table bound into an upright set sits sideways.
• They are excerpts. Pages 3 to 5 of a twelve-page ledger arrive with no header row, so nothing on the page says what each column means.

Which exhibit layouts are hardest?

The hardest layouts are those with no ruled lines, headers that span several columns, cells that wrap onto extra lines, and pages turned sideways. Each one removes a clue that software uses to place a value in its cell.

Two smaller features cause trouble in any layout:

• Negative amounts. Accountants write a negative as (1,250.00), with a trailing minus, or in red. Parentheses can be dropped, and red disappears in a black-and-white copy. A lost sign changes a total by twice the amount.
• Ditto marks and blanks. A column that says "same as above" by leaving a blank or a ditto mark is clear to a person and empty to a spreadsheet.

Prose set in columns is a different problem from a table. If your exhibit is two columns of text, see the post on whether OCR can handle two-column documents.

Converting a tabular exhibit and keeping its grid

Convert the exhibit page by page, export it in a format that has real cells, and label the output with the exhibit and page it came from before you change anything.

1. Set the original aside. Keep the exhibit PDF as you received or marked it. Work on a copy.
2. Write down what you are converting. Note the exhibit number, the page range that holds the table, and any printed totals. You will need all three later.
3. Fix what you can before upload. If a wide page sits sideways, rotate it in your PDF reader and save the copy. If only pages 40 to 46 of a long exhibit are tabular, extract those pages into their own file.
4. Upload the file. Schema Law takes a PDF, PNG or JPEG of up to 20 MB. Page limits for one document depend on the plan and are listed on the pricing page.
5. Look at the parsed pages first. Before exporting, check that the table appears as a table and not as lines of loose text. If it came out as text, no export format will turn it back into a grid.
6. Export to Excel. Every plan, the free one included, exports Excel, CSV and JSON as well as Word, searchable PDF and plain text. If you want the table inside a document instead, the Word export carries it as a Word table.
7. Label the result. Rename the file and each sheet for the exhibit, and add a column for the source page.

It helps to know what the workbook will contain. In Schema Law's Excel export, each table found on a page becomes its own worksheet, named with the page it came from, and everything that is not a table goes to a final sheet called "Text". The guide to scanned tables in Excel describes the exported workbook in full. Two points matter more for an exhibit than for other scans.

The page number is the page of your upload. If you extracted pages 40 to 46 into a new file, "p1" is page 40 of the exhibit. Keep the offset in your notes, or add a column with the true exhibit page or Bates number.

A table that runs over several pages arrives as several sheets. You join them yourself, and the same guide has a section on tables that span several pages.

The CSV export puts every table in one file, separated by blank lines. The post on converting a scanned PDF to CSV covers when that suits you better. Whichever you pick, check how figures arrived before you total them. Amounts copied from a page as text will not add up in a formula until the cells are converted to numbers, and the same guide to Excel explains numbers, dates and currency in exported cells.

Ledgers, invoices and schedules as exhibits

Ledgers, invoices and schedules are the three table types that turn up most as exhibits, and each carries its own built-in check. A ledger has a running balance. An invoice has arithmetic on every line. A schedule has a fixed number of rows. Use the check the document gives you.

Ledgers and account statements

A ledger lists transactions in date order with a balance after each one. That running balance is a gift: each row's balance should equal the previous balance plus or minus the row's amount. Put that formula in a spare column and every misread amount shows itself on its own row.

Watch for separate debit and credit columns where most cells in one of them are blank. That is the layout most likely to shift.

Invoices and bills

An invoice is two things on one page: a block of header fields (number, date, parties, terms) and a table of line items. Only the line items belong in a grid. Expect the header block to arrive as text.

Each line normally shows a quantity, a unit price and an extended amount. Multiply the first two and compare with the third. Then add the extended amounts and compare with the subtotal, and add tax and shipping to reach the invoice total. For medical billing in particular, see the guide to converting medical records to a spreadsheet.

Schedules attached to agreements

Rent schedules, amortization tables, asset lists and payment schedules are usually clean, ruled and typed, so they convert well.

Count is your check. A sixty-month rent schedule has sixty rows. An amortization table has one row for each payment period. If the count is off, a row was split, merged or lost at a page break.

Reconciling an extracted table with the exhibit

Reconciling means proving that the spreadsheet says what the exhibit says. Do it in a fixed sequence: rows, then totals, then individual cells. Stop relying on the table the moment a check fails, find the cause, and record the correction.

1. Count rows. Count the entries on each page of the exhibit and compare with the sheet.
2. Check the first and last row of every page. Page breaks are where rows go missing or get doubled.
3. Re-add the printed totals. Sum each amount column and compare with the total printed on the exhibit. The Excel guide covers checking totals after extracting a table in more depth.
4. Use the document's own arithmetic. Run the running-balance or line-arithmetic check described for that document type.
5. Compare key cells by eye. For the rows you will cite, read each cell against the page image, digit by digit.
6. Check signs and dates. Confirm that credits, refunds and reversals are negative, and that dates did not swap day and month.

A worked example

Exhibit 14 is a three-page vendor ledger. The last page prints a total of 18,450.00 across 42 entries.

Two failures, two causes. On page 2, a long description wrapped onto a second line and became a row of its own with no amount. Joining it to the row above restores 15 rows and does not change the total.

The total is still 90.00 too high. A difference that divides evenly by nine often means two neighbouring digits were swapped, so look for an amount where that would produce 90.00. The running-balance column finds it on page 3: an entry of 1,340.00 on the exhibit was read as 1,430.00. Correct the cell, and the total agrees.

Record both fixes. A simple note is enough: "Ex. 14, p. 2, row 9: description rejoined. Ex. 14, p. 3, row 4: amount corrected from 1,430.00 to 1,340.00 after comparison with the image."

A table that passes rows and totals can still hold errors that cancel out, or errors in columns that are never summed, such as dates, cheque numbers and names. That is why the cells you intend to cite get read one by one. The post on why OCR output is full of errors describes the general method for checking output against a page.

Presenting extracted figures without misstating the exhibit

Treat the extracted table as your working paper and the exhibit as the source. Cite the exhibit page for every figure, mark anything you calculated or corrected, and keep the reconciliation notes. Nothing here is legal advice, and the rules on summaries and demonstratives differ between courts, so check your own.

Some habits keep a spreadsheet from drifting away from its source:

• Cite the page, not the sheet. "Ex. 14 at 3" can be checked by anyone. "Row 37 of my workbook" cannot.
• Separate what the exhibit says from what you worked out. Put sums, differences and percentages in clearly labelled columns or a separate sheet. A reader should never mistake your subtotal for a figure printed on the exhibit.
• Do not improve the exhibit. If the exhibit's own total is wrong, your spreadsheet should show the printed total and your recomputed one side by side.
• Keep illegible cells empty and marked. A cell nobody can read gets "[illegible]", never a likely value.
• Carry redactions through. A redacted cell is not zero. Mark it as redacted so totals are not read as complete.

If a chart or calculation built from the table may be offered in court, read the rule that governs summaries there. In federal court that is Federal Rule of Evidence 1006. It allows a court to admit "a summary, chart, or calculation offered to prove the content of voluminous admissible writings, recordings, or photographs that cannot be conveniently examined in court". It also requires the proponent to make "the underlying originals or duplicates available for examination or copying, or both, by other parties at a reasonable time and place". The same rule says that a summary, chart or calculation that functions only as an illustrative aid is governed by a different rule. State courts apply their own evidence rules. Whether and how any of this applies to your table is a question for the lawyer trying the case.

The other side can check your figures against the pages, and a table reconciled with notes survives that check. For assembling the whole set, see the guide to OCR for trial exhibits, and for the full list of export formats see the formats section of the homepage.