For extraction, the best model is the one with the lowest error rate on your own documents, measured field by field. A general ranking cannot tell you that. Extraction looks easy, because the answer is printed on the page. That is exactly why errors go unnoticed: a wrong invoice number or a swapped debit and credit looks plausible.
Last reviewed: 17 October 2026. Facts checked. This post makes no claim about any model's accuracy or ranking, and none was looked up, because rankings change and public benchmarks rarely use Indian invoices or bank statements. The test method below is general practice, not a quoted standard.
What extraction means in an audit
In audit work, extraction usually means turning a page into rows:
- Purchase and sales invoices into vendor, GSTIN, invoice number, date, taxable value, tax and total.
- Bank statements into date, narration, debit, credit and balance.
- Confirmation replies into balance, date and any stated differences.
- Agreements into dates, amounts and parties (see the model choice for contract review).
The output then feeds a sample, a reconciliation or a working paper. A mistake here carries forward into every test built on it.
The three kinds of tool
| Approach | Strength | Weakness |
|---|---|---|
| Classic OCR plus rules | Predictable, cheap, same result every run | Breaks when the layout changes |
| General chat model reading a page image | Handles unfamiliar layouts | Can fill a gap with a plausible guess |
| Purpose-built extraction service | Often returns a confidence score per field | Cost, and where the file goes needs checking |
Many practical setups combine them: OCR or a document service reads the text, and a language model organises it. Compare the whole chain, not just the model name.
What to test before you rely on one
1. Build a set from your own work. Take about 20 real documents and include the awkward ones: a skewed phone photo, a stamped and handwritten invoice, a multi-page bank statement with a continuation header, a scanned confirmation with a signature over the figure. Remove or mask client identifiers before sending anything to a tool you have not cleared.
2. Make the answer key by hand. Two people key the fields for each document. Where they differ, a third resolves it. This is slow, and it is the part that makes the result meaningful.
3. Score each field, not each document. A document that is "mostly right" can still have the wrong total. Count, per field type, how many were exact, how many were wrong and how many were left blank. Amounts and dates matter most; give them their own tally.
4. Look at the failure type. A blank is a safe failure because someone will look. A confident wrong value is the dangerous one. A model that leaves a field empty when unsure beats one that always answers.
5. Test the arithmetic. Ask whether line items add to the subtotal, whether tax plus value equals the total, and whether running balances on a statement tie from row to row. Checking these by formula, outside the model, catches many errors for free.
6. Repeat the run. Run the same ten documents three times. If a figure changes between runs with no change in input, the process is not stable enough for evidence-bearing work.
Indian-document traps to include
- GSTIN with look-alike characters (0 and O, 1 and I). A format and checksum check in a spreadsheet catches these.
- Dates written as 03/04/26. Decide the format rule first and test it.
- Amounts in the lakh and crore comma style, and amounts in words that disagree with the figures.
- Bank narrations that wrap across lines, and statements where a debit and credit are shown in one column with a "Dr" or "Cr" suffix.
- Multi-language invoices; see the model choice for multilingual documents.
Controls that matter more than the model
Whichever model wins, three controls do most of the work. First, the extracted total is tied back to a control total taken from the source, such as the statement's closing balance. Second, a person reviews a sample of the extracted rows against the page, and the review is recorded. Third, the model name, version and date are written in the working paper, because a later update can change behaviour; see model versioning and change control.
If extraction feeds a sample or reconciliation, remember that the auditor still owns the conclusion. The tool prepares data; it does not provide the evidence by itself. For the wider review step, see human-in-the-loop review.
Where the file goes
Extraction means uploading client documents, often full of PAN, GSTIN and bank details. Before any test with real files, check the tool's retention and training terms and whether the use fits your client engagement terms. Open-source, self-hosted or API key: where the data goes sets out the questions to ask.
A simple decision rule
Pick the lowest-cost option that clears your own bar. A reasonable bar is zero unexplained amount errors on the test set, a blank rather than a guess when unsure, and stable results across repeat runs. Re-run the 20-document set whenever the tool or model changes, and keep the score sheet in the file.
Frequently Asked Questions
Is OCR enough, or do I need a language model?
For fixed layouts, OCR with rules is cheaper and more repeatable. A language model helps when layouts vary. Test both on your documents.
How many documents make a fair test?
Twenty is enough to expose obvious problems, not to prove a precise error rate. Add more of the document types that matter most to your clients.
Can extracted data be used as audit evidence?
It can support a procedure, provided the source is retained and the extraction is checked against it. The auditor remains responsible for the evidence.
Related Articles
Statutory facts on this page are checked against their sources, and the page says where it relied on secondary reporting. How we verify · Report an error