A model that accepts a very long file has not necessarily read it all with equal care. For audit work the question is not how many pages fit. It is whether the one paragraph that changes your conclusion comes back, with its page number, when you ask for it. Every major model family now advertises a long context window, and the numbers keep changing. Treat them as a ceiling, then test your own files.
Why audit files stress long context
An audit file is unusual input. A single engagement can hold board minutes, loan agreements, a year of bank statements, lease deeds, confirmations, a trial balance and a pile of scanned invoices. The facts that matter are rarely repeated. A covenant breach may sit in one paragraph of a sanction letter. A related party may appear once, in a footnote to a minute book.
Models are generally better at the start and the end of a long prompt than at the middle. This is a widely reported behaviour in research on long-context models, and it varies by model and version. It is the reason the contract review post in this series asks you to place clauses in the middle of your longest documents when testing.
Three ways to give a model a long file
Put everything in one prompt. Simple and often fine for a single agreement or a set of minutes. The risk is quiet omission: the answer reads well and misses something.
Chunk and search. The file is cut into sections, an index is built, and only the relevant sections are sent with each question. This is what many document tools do behind the scenes. It scales to whole engagements but depends on the search step finding the right sections. If the search misses a section, the model never sees it, and it will not tell you.
Chunk and sweep. Every section is read in turn against the same question, and the results are combined. Slower and costlier, but it suits the question "find every instance of X", which is most of what an auditor asks.
None is best in general. Pick by question type. A question that has one answer in one place suits search. A question that asks for all exceptions suits a sweep.
Make the model show where it found things
The single most useful control is a citation requirement. Ask for the document name, page or section number and the exact sentence. Then open the page.
A workable instruction: "For every point, give the file name, page number and the exact sentence quoted. If you cannot find a quote, say not found. Do not paraphrase without a quote."
This does two things. It makes checking quick, because you jump straight to the page. It also gives you an early signal of trouble: if the quotes are paraphrases that do not appear on the page, the model is guessing. The wider pattern is covered in AI hallucinations in audit.
Page numbers need care. A scanned PDF may have printed page numbers that differ from the file's page count. Agree on one convention, such as file page number, and state it in the instruction.
A recall test you can run in an hour
Before relying on any model or tool for long files, build a test from a closed engagement, or from a sample file with client details removed.
- Take a long document set, say a loan agreement with schedules, plus board minutes.
- Choose five facts that appear once each. Place them at different positions: early, middle and late.
- Ask the model five direct questions, one per fact, and one open question such as "list every restriction on dividend payment".
- Score each answer as found with correct quote, found with wrong quote, or missed.
- Repeat with the same file cut into chunks and sent through your tool's search.
Compare the two runs. If the whole-file run finds all five and the search run misses two, your tool's retrieval is the weak point, not the model. Keep the results with the date, the model name and the version, because the answer can change after an update. The model versioning post explains how to log this.
Things that break long-file work in practice
- Scans without text. Poor scans produce garbled text, and the model reads the garble. Check text extraction on a few pages before blaming the model. See document extraction.
- Tables across pages. A bank statement table split over two pages can lose its header. Ask for totals and tie them to the statement's closing balance.
- Duplicates. The same agreement saved in three versions leads to three answers. Name versions clearly or remove the superseded ones.
- Mixed languages. Regional-language documents change quality sharply between models. See multilingual and handwritten documents.
- Hidden instructions. Text inside a client file can try to steer the model. Treat uploaded content as untrusted. See prompt injection and data leakage.
Numbers belong to the ledger, not the model
Long context is good for reading text. It is a poor way to add up a ledger. If a question needs totals, ageing or matching across thousands of rows, have the model write a query or a formula, run it in Excel or a database, and keep the output. Reading ten thousand rows pasted into a prompt invites arithmetic slips, and nothing in the answer shows whether it slipped.
Confidentiality still applies
A longer window means more client material sent in one go. Check where the tool processes and stores it before the file leaves your machine. Can you upload client ledgers to AI tools sets out the questions, and the DPDP-safe prompt library shows how to strip identifiers first.
A short routine
- State the question type: one answer, or every instance.
- Require quotes with file and page.
- Spot-check at least three citations per run against the source.
- Run the one-hour recall test whenever the model or tool changes.
- Keep the prompt, the model name and the date with the working paper.
Frequently Asked Questions
Does a bigger context window mean fewer missed items?
Not by itself. Recall inside a long input varies by model and version, so test it on your own documents.
Should I chunk the file or send it whole?
Send whole for one document with a single answer to find. Sweep in chunks when you need every instance, such as all restrictive covenants.
How many citations should I check?
Enough to trust the pattern. Three per run is a floor. Check more where the point affects a material balance or the audit opinion.
Last reviewed: 17 October 2026. Facts checked. No vendor page was opened in this session, so no context-window size, price, release date or benchmark is quoted. The point about weaker recall in the middle of long inputs is described from published research in general terms and was not re-opened here. Test on your own files.
Related Articles
Statutory facts on this page are checked against their sources, and the page says where it relied on secondary reporting. How we verify · Report an error