How to Create a Load File From Scanned PDFs

To create a load file from scanned PDFs, run OCR on each document, save its text as a file named after the first Bates number, and list every document in a delimited table with its Bates range, page count and text path. Add an image cross-reference file when you deliver single-page images.

That is a manageable job for a few dozen or a few hundred paper documents, which is the size of production a small firm usually has to make by hand. It stops being manageable in the thousands, and the last section covers what to do then. Before any of it, get the recipient's written specification. Every step below bends to what that document says.

What does a review platform need besides the PDFs?

A review platform needs four things besides the page images: the searchable text of each document, a data file listing every document and its fields, a cross-reference tying page images to documents, and a folder layout that matches the paths written in those files. A folder of PDFs alone carries none of that.

The Sedona Conference Glossary defines a load file as one that "indicates where individual pages or files belong together as documents" and "where each document begins and ends", and that may also hold metadata, coded data and extracted text. The definition is in the Sedona Conference Glossary, Fifth Edition. In short, the load file is the packing list, and the platform will not unpack the box without it.

A typical delivery folder looks like this:

VOL001/
  DATA/VOL001.dat
  DATA/VOL001.opt
  IMAGES/001/ABC0000001.TIF
  IMAGES/001/ABC0000002.TIF
  TEXT/001/ABC0000001.txt

Whether you may send PDFs at all is the first question to settle. The SEC's Data Delivery Standards show how strict a recipient can be. The agency prefers scanned paper "in a structured format including industry standard load files, Bates numbered image files, native files and searchable document-level text files", with black and white pages as 300 DPI single-page TIFF files. It takes PDF productions only "with prior approval", and then each PDF "must represent a discrete document" and must contain embedded text. Opposing counsel or your own platform may be far more relaxed. Ask, and keep the answer in writing.

The DAT and OPT files themselves, line by line, are explained in the post on the Concordance load file format. This post is about filling them from scans.

Which fields belong in a load file for scanned documents?

For scanned paper, five fields carry almost all the weight: first Bates number, last Bates number, page count, custodian, and the path to the document's text file. Paper has no file system behind it, so the author, sent date and file name columns that an email production fills stay empty unless someone reads the page and types them in.

The names in that table come from the SEC's field list, which also says the last Bates field should be filled in even for a single-page document. Your recipient may use other names. Match them exactly, because an importer maps columns by header.

Type the Bates numbers you see printed on the image. This load file is how a discovery production of electronically stored information enters the other side's review tool. A wrong begin number or end number opens the wrong document. Check one single-page document and one multi-page document, then fill the remaining rows the same way.

Two kinds of field need a decision.

Attachment ranges. If a cover letter was stapled to an enclosure, some specifications want that relationship recorded. The SEC asks that "children files should follow parent files sequentially in the Bates numbering" and uses BEGATTACH and ENDATTACH columns for the range. For loose paper with no attachments, leave them out or leave them blank, as the specification directs.

Coded fields. A document date or title typed in by a person is what the Sedona glossary calls coding. It takes real time per document. Supply coded fields only if the parties agreed to them, and never fill a metadata column with a guess. An empty field is honest. A wrong date in a date column will sort the document into the wrong month for every reviewer who uses it.

Building the load file from OCR exports

The build has two halves: produce one text file per document with OCR, then describe the set in a spreadsheet and convert that sheet to the delimited format. Do the work in this order so that each later step can be checked against an earlier one.

1. Split the scans into documents. One PDF per document, named by its first Bates number: ABC0000001.pdf. Cut at the boundaries of the original paper documents.
2. Run OCR and export plain text. In Schema Law, upload each PDF and choose the plain text export. You get one text file per uploaded document, with a page break character between pages. That is the document-level text file a load file points to.
3. Name each text file after the first Bates number. ABC0000001.txt sits beside ABC0000001.pdf in name, in a separate TEXT folder. The SEC requires this naming and requires the text as "separate document-level text files, not as fields within the .DAT file".
4. Record the page count. Read it from the PDF. If you also export JSON, the file carries a page_count value you can copy instead of counting.
5. Fill the spreadsheet. One row per document, one column per field, header row first. Format every column as text before typing so the program does not strip leading zeros from Bates numbers.
6. Convert the sheet to a DAT. Save it as tab-delimited text. In a text editor that supports search and replace with special characters, replace each tab with þ, then character 20 (hexadecimal 14), then þ. Add a þ at the start and end of every line. Save as UTF-8.
7. Write the OPT if you deliver single-page images. One line per page image, with a Y on each document's first page.
8. Assemble the folders and write paths relative to the top of the volume.

A worked example for three documents:

Step 6 is where hand-built files usually break, and you may be able to skip it. Relativity's load file specifications say it accepts .dat, .csv and .txt load files. If your recipient takes an ordinary CSV with a header row, save the sheet as CSV and stop.

Be clear about what the OCR tool contributes. Schema Law's CSV export is not a load file. For a document without tables it lists the recognized text line by line with a page number beside each line, which is handy for confirming which Bates stamp was read on which page. For a document with tables it holds the tables. The post on converting a scanned PDF to CSV describes that file. The table of documents above is something you build yourself. The CSV, JSON and plain text exports are included on every plan. The free plan stops at three pages a document, enough to test one short document.

Document boundaries and page counts

Every row in a load file is one document, so you must decide where each document starts and ends before you number anything. A 300-page PDF that came off the scanner as one file is a box of paper, and loading it as a single record makes it one enormous document that no reviewer can code sensibly.

The Sedona glossary calls this unitization and describes two methods. Physical unitization "uses actual objects such as staples, paper clips, and folders". Logical unitization has a person look at each page for cues such as "consecutive page numbering, report titles, similar headers and footers". Physical is quicker and easier to defend. If the paper is still in the office, note the staples and clips before anyone feeds the scanner.

The arithmetic is simple and unforgiving:

• Last Bates number = first Bates number + page count - 1.
• Each document's first number = the previous document's last number + 1.
• The page counts of all documents add up to the number of page images.

Several cases break those sums.

• Blank backs. A duplex scan of single-sided paper doubles the page count. Decide whether blank pages are dropped before stamping. Once a blank page carries a Bates number, it stays in the count.
• Separator sheets. Slip sheets and folder tabs scanned with the paper either get a number and belong to a document, or are removed first. Do not leave it to chance.
• Attachments. A letter and its enclosure are two rows whose numbers run consecutively, parent first.
• Oversized documents. An OCR service may cap pages per document. Schema Law's Solo plan reads up to 250 pages in one document and the Team plan up to 1,000, with a 20 MB limit per upload. A ledger longer than that has to be split for OCR and its text rejoined afterwards, which the post on batch OCR for discovery documents covers.

Getting stamps and text to agree page by page is its own subject, set out in the guide to OCR on Bates numbered documents.

Testing a load file before sending it

Test a load file by counting, then by loading. The counts in the data file, the cross-reference file and the folders must agree with each other, every path must lead to a real file, and a sample of documents must show the right text beside the right pages.

Run this checklist on the finished volume, after copying it to a different drive so that paths tied to your own machine fail now and not at the recipient's office.

1. Rows in the DAT below the header = number of documents = number of text files.
2. Lines in the OPT = number of image files = sum of the PGCOUNT column.
3. Y markers in the OPT = rows in the DAT.
4. Sort by FIRSTBATES and confirm that each row starts one number after the previous row ended.
5. Open the DAT in a plain text editor. The header row is there, and every value sits between þ characters.
6. Open five text files chosen at random. Each begins with the words on the page whose Bates number is in the file name.
7. No text file is empty unless its document is a photograph or a blank form with nothing to read.
8. File and folder names follow the specification. The SEC permits only letters, digits and underscores, and no more than 2,000 files in a folder.
9. The encoding is what was asked for. The SEC requires a byte order mark on a Unicode DAT.

If you have access to any review platform, load the volume into an empty workspace and open the first, last and longest documents. If you do not, send the recipient the first ten documents as a sample and ask them to confirm it loads before you deliver the rest. A failed import the day before a deadline costs more than the sample would have. The list of common mistakes that make a load file fail to import explains the errors these checks catch.

Having the load file produced for you

You can have the load file produced by someone else, and past a certain size you should. A hand-built table works for a banker's box. For thousands of documents, attachment families or redacted pages, one slipped row misnumbers everything after it, and a vendor or a platform service that generates the files from the documents is safer.

Three things shape that choice.

• A litigation support vendor. They scan, unitize, stamp, OCR and deliver a volume to specification. You pay per page or per hour and you still own the quality check.
• The review platform's own services. If the destination is a workspace you control, the platform may accept loose PDFs and run its own OCR, which removes the need for a load file. Ask its support team what it takes, and whether it will do the loading for you, before you build anything.
• The receiving party's specification. Whoever builds the file builds it to this document, so get it in writing first. While you are asking, ask whether the recipient would take searchable PDFs, one per document, or an ordinary CSV index in place of DAT and OPT files. A simpler form agreed in writing can remove the load file from the job.

Schema Law is not one of these routes. It does not produce load files on any plan, and it has no custom export layout. Every plan runs the same software and exports the same eight formats: Word, Excel, CSV, JSON, searchable PDF, plain text, Markdown and HTML. None of those is a DAT or an OPT. Three of them give you material to build one from:

• Plain text. One text file per uploaded document, with a page break character between pages. Renamed to the first Bates number, it is the document-level text file that the text path column points to.
• CSV. For a document without tables, each line of recognized text with its page number. Use it to confirm which Bates stamp was read on which page before you type the ranges.
• Searchable PDF. Your uploaded PDF with a hidden text layer added. Where the recipient accepts one PDF per document, it is the deliverable itself.

The table of documents, the Bates ranges and the paths remain yours, or your vendor's, to build. The pricing page lists the page allowance on each plan.

Whoever does the work, hand over the same four things: the written specification, the Bates prefix and starting number, the documents already split or clear instructions on where to split them, and the deadline. When the volume comes back, run the counting checklist yourself. Responsibility for what your firm produces does not transfer with the invoice.

The other formats a scan can become are set out under export formats on the home page.