Legal Document to Structured Fields: Write the Field List
By Schema LawPublished Updated
To turn scanned legal documents into structured fields, write the field list before you convert anything: each field's name, what counts as its value, its format, and where on the page it is found. Then convert the scans to text, fill one row per document, and record why any field is empty.
Going from a legal document to structured fields is two jobs. Reading the page into text is the first, and software does it. Deciding which words on the page are the lender, the principal sum or the maturity date is the second, and it is done by a person, or by a system that a person has to instruct and check. Both the person and the system need the same thing before they start, which is a written definition of every field.
The example in this guide is fictional. A firm acting for the buyer of a loan portfolio has 140 scanned promissory notes from Larkfield Credit Union and needs a table of them.
What are structured fields in a legal document?
Structured fields are named values taken from a document and stored one per slot: lender, principal_amount, maturity_date. Running text says the same things in sentences. The difference is that a field can be sorted, filtered, counted and compared across a thousand documents, and a sentence has to be read.
Here is a paragraph of the kind a scanned note contains:
FOR VALUE RECEIVED, the undersigned, Tobias Renner ("Borrower"), promises to pay to the order of Larkfield Credit Union ("Lender") the principal sum of Forty-Two Thousand Five Hundred Dollars ($42,500.00), with interest at 6.25% per annum, on or before June 30, 2029.
And the same content as fields:
| Field | Value |
|---|---|
borrower_name | Tobias Renner |
lender_name | Larkfield Credit Union |
principal_amount | 42500.00 |
interest_rate_percent | 6.25 |
maturity_date | 2029-06-30 |
Nothing in the table is new. Everything in it has been chosen, named and put in a fixed form.
It helps to keep three layers apart:
- The page image. What the scanner made.
- The recognized text. What OCR read from the image, page by page, with any tables as rows and cells.
- The fields. What someone decided the text says about the questions you care about.
A conversion tool gives you layer two. Schema Law, for example, reads pages into text and tables, and its JSON export holds each page's text and tables. It has no maturity_date in it. The post on scanned PDF to JSON sets out what JSON from a scanned document contains. Layer three is built on top of that text, and the field list is its design.
Writing the field list before any document is converted
Write the field list first because it decides everything after it: which documents you need, which pages matter, how long the job is and whether two people filling the table will agree. A field list written after the work has begun gets rewritten with every surprise, and the rows already filled no longer match the ones that follow.
Start from the questions
List what the table has to answer. For the Larkfield notes: which notes mature before the closing date, what is the total principal, which carry a guaranty, and which are missing a signature page. Every field should serve one of those questions. A field that serves none is cost without use.
Define each field in one row
A field list is itself a table. For each field write:
| Part of the definition | Example for one field |
|---|---|
| Name | maturity_date |
| Meaning | The date by which the principal is payable in full, as stated on the face of the note |
| Kind | Stated (read from the page) |
| Type and format | Date, written YYYY-MM-DD |
| Where to look | First paragraph; if absent, a section headed "Maturity" or "Payment" |
| If there are two candidates | Take the one in the note, not one in a cover letter; record the other in notes |
| Example | 2029-06-30 |
| Page reference | Required, in maturity_date_page |
Do not skip the "two candidates" line. A note with a later amendment attached may show two maturity dates, and two careful people can each pick a different one. The definition has to say which is recorded.
The fields every list needs
Whatever the documents are, include these:
- A document identifier. A Bates number or a control number of your own, as text.
- The source file name.
- The document type, from a short fixed list.
- A page reference for every value that someone may later have to prove.
- Who filled the row and when.
- A free-text notes field for whatever does not fit.
Keep the list short
Twelve defined fields that are filled the same way on every document are worth more than forty filled unevenly. You can add a field later and run a second pass for it. You cannot easily repair forty inconsistent ones.
For contracts, the question of which terms earn a field is covered in the post on contract data extraction, under which contract terms are worth extracting. For bills, there is a worked field list in the guide to billing records, under what columns a scanned invoice or time entry needs in Excel. For the objective fields used to index a production, such as author, recipient and document date, see the guide on how to index discovery documents.
Which fields can be read off the page and which need a lawyer's judgment?
A field can be read off the page when the document states the value in words or figures that you can point to. It needs judgment when the value is a conclusion drawn from the words. Mark every field in the list as stated, derived or judgment, because the three need different people and different checks.
| Kind | Test | Examples from the notes | Who fills it | How it is checked |
|---|---|---|---|---|
| Stated | You can put a finger on the value | Borrower name, principal sum, interest rate, date of the note | Anyone trained on the field list | Compare with the page |
| Derived | Computed from stated values by a fixed rule | Months from note date to maturity; total of scheduled payments | A formula, or a person following the rule | Recompute |
| Judgment | Requires reading and deciding | Whether the note is assignable without consent; whether a default clause is unusual | A lawyer | Review by a second lawyer, as the matter requires |
Borderline fields
Some fields look stated and are not.
- "Governing law." Stated, when a clause names a state. Judgment, when no clause does and someone infers it from the parties' addresses. Define the field as "the jurisdiction named in a governing law clause" and the inference never enters the table.
- "Signed." Whether ink appears on the signature line is something a person can see on the page image. Whether the note was validly executed is a legal conclusion. Name the field
signature_presentand it stays on the stated side. - "Maturity date." Stated when printed as a date. Derived when the note says "sixty months from the date hereof". Store the words in one field and the computed date in another, and say which rule was used. Date arithmetic in contracts has traps of its own, set out in the post on extracting contract terms from a scan.
Keep judgment fields apart
Put judgment fields in their own group of columns, with the reviewer's initials beside each. Two things follow. A paralegal can finish every stated field without waiting for a lawyer. And a table handed to a client or an expert can be cut down to the stated fields alone, so that no legal assessment travels with it by accident.
This guide is about handling documents. Which conclusions to draw from a clause is legal work, and it belongs to the lawyer on the matter.
One format for dates, names and amounts across every document
Choose one written form for each type of value and put it in the field list: one date format, one way of writing a name, one way of writing an amount. Documents state the same value in many forms, and a table that copies them all cannot be sorted or added up.
Dates
The notes print dates as "June 30, 2029", "6/30/29" and "the 30th day of June, 2029". Store one form. A year-month-day form with hyphens is a sound choice: the internet date standard RFC 3339 defines a full date as full-date = date-fullyear "-" date-month "-" date-mday, which gives 2029-06-30. Text written that way sorts in date order, and it cannot be mistaken for a day-first date.
Rules to write down:
- Four-digit years, always. Decide how a two-digit year on the page is expanded, and flag every one.
- A date that is only a month and year is stored in a text field as it stands, not padded with an invented day.
- The printed form goes in a companion field when the wording matters.
Names
A name has to match itself across documents. Store it twice:
| Field | Rule | Example |
|---|---|---|
borrower_name_as_written | Exactly as on the page, including errors | TOBIAS J. RENNER |
borrower_name | Your standard form | Renner, Tobias J. |
For organisations, fix how suffixes are written (LLC or L.L.C., Inc. with or without a comma) and whether "The" is kept. Do not correct a name that the document misspells. Record it as written and note the variance; a misspelling may matter.
On these promissory notes, store consideration as the principal figure the note states, and keep any "value received" wording in the notes field. Record an assignment only when the page names an assignee. When the note describes a material breach, or mentions the statute of frauds, copy that sentence into its own field and leave what it means to the lawyer.
Amounts
- Store the number alone: 42500.00, with no dollar sign and no thousands separator.
- Put the currency in its own field if more than one could occur.
- Where a document gives the sum in words and in figures, store the figures, and add a yes-or-no field for whether the two agree. Which one prevails when they differ is a legal question, so the table records the conflict and does not resolve it.
- Store a percentage as the number printed, 6.25, in a field whose name says it is a percentage.
Identifiers
Loan numbers, account numbers and Bates numbers are text. Stored as numbers, they lose leading zeros, and spreadsheet programs may round long ones. The CSV guide explains how to manage opening a CSV without Excel changing the data.
Fixed lists
Any field with a small set of answers gets a list: document_type is one of note, allonge, guaranty, security agreement, other. Free typing produces "Guaranty", "guarantee" and "Gty" for one thing.
Blank, illegible and not stated: three different empty fields
An empty field can mean three things. Blank: the document has a place for the value and nothing was filled in. Illegible: something is there and cannot be read. Not stated: the document does not address the point. Record which, because each one sends the reader somewhere different.
| Status | What the page shows | Example | What the reader does next |
|---|---|---|---|
| Blank | A line, box or space left empty | "Maturity date: ________" | Treats the gap as a fact about the document |
| Illegible | Marks that cannot be read with confidence | A rate obscured by a fax smear | Gets a better copy or the paper original |
| Not stated | No provision on the subject | A note with no guaranty reference | Looks to other documents or to the general law |
A fourth status, not applicable, is worth having when a field only makes sense for some document types.
How to record the status
Do not leave the cell empty, and do not type the reason into the value field of a date or number column, where it will break sorting and sums. Add a status field beside each field that can be empty:
interest_rate_percent | interest_rate_status |
|---|---|
| 6.25 | stated |
| blank | |
| illegible | |
| not_stated |
In JSON the same pattern applies. RFC 8259 gives JSON one value for nothing, null: "JSON can represent four primitive types (strings, numbers, booleans, and null)". One null cannot carry three meanings, so the reason goes in a second field.
Illegible is a human call
Whether a figure can be read is decided by a person looking at the page image. Do not rely on the converted text to tell you. Schema Law's homepage says recognition is strongest on clean print and weaker on faint faxes, stamps over text and handwriting, and that nothing marks a passage as uncertain. A smeared 6.25 may come back as a confident-looking 8.25. For any value that matters, the person filling the field looks at the scan.
When a value is partly legible, record what can be read with the doubt marked, in the notes field: "rate reads 6.?5, second digit obscured, p. 1". Do not put a guess in the value field.
Why the distinction is worth the effort
Count the statuses when the table is done. Thirty illegible maturity dates is a request for better copies. Thirty blank ones is a finding about how the notes were completed. Thirty not stated may mean the field was defined for the wrong document type. An undifferentiated column of empty cells tells you none of that.
How do you test a field list on ten documents before the full set?
Test the field list by having two people fill it independently for the same ten documents, then compare their answers field by field. Every disagreement is a fault in a definition, not in a person. Fix the definitions, record the changes, and only then start the full set.
Choosing the ten
Do not take the first ten in the box. Pick for variety:
- The oldest and the newest.
- The shortest and the longest.
- At least one of each document type on your list.
- One poor scan.
- One with handwriting or a filled-in form.
- One with an amendment or attachment.
Running it
- Convert the ten scans to text. A small test like this is also a fair trial of the conversion tool on your own documents; the three steps on the homepage show what that involves in Schema Law.
- Give two people the field list and the ten documents. They do not confer.
- Each fills every field, with page references and statuses.
- Put the two tables side by side and mark each cell as agree or differ.
- Count the differences by field.
Reading the result
| What the comparison shows | What it means | The fix |
|---|---|---|
| One field differs on most documents | The definition is ambiguous | Rewrite it; add the "two candidates" rule |
| Same value, different form | The format rule is missing or unclear | Tighten the format; add an example |
| One person filled it, one marked not stated | "Where to look" is too narrow | List more locations |
| Both left it empty on every document | The documents do not contain it | Drop the field, or accept a column of not stated |
| Both agree and both are wrong against the page | The converted text misled them | Require a look at the image for that field |
| Notes field is full of the same remark | A field is missing | Add it |
The fifth row needs a third check to find: someone compares a sample of agreed values with the page images.
Freezing the list
Number the revised field list as version 1, date it, and file it with the matter. If a definition has to change after the full run has started, change the version, note what changed, and decide whether rows already filled need revisiting for that field. A table whose rows were filled under different rules looks uniform and is not.
Then scale up. The ten test documents stay in the set as filled, and their rows are your reference examples for whoever joins the job later. For the plan limits that apply to a larger run, see the pricing page.