An anomaly test is only as useful as the number of flags a team can actually examine. A method that flags two thousand entries in a ledger of twenty thousand has not found risk. It has moved the work. Judge any approach, AI or otherwise, by how many flags it raises, how many are worth a look and how you will deal with the rest.
For journal entry testing itself, see journal entry testing procedures and where AI helps. For the digit-pattern test, see Benford's law in audit.
Rules and models answer different questions
Rules encode something you already know is risky: entries posted on a holiday, round amounts just under an approval limit, manual entries to revenue near year end, entries by users who rarely post. Rules are transparent. You can state why an item was flagged, and a reviewer can rerun the test.
Statistical and machine learning methods look for what is unusual compared with the rest of the data: an amount far from the usual range for that account, a vendor paid at an odd frequency, a pair of accounts rarely used together. They can find patterns you did not think to write as a rule. They are harder to explain, and the set of flags can change when the data or the model changes.
Language models sit in a third place. They read text well. They can scan narrations and descriptions for odd wording, vague explanations or mismatches between narration and account. They are not designed to compute outliers across thousands of numeric rows, and they should not be the engine for that part.
A sound design uses rules for known risks, statistics for the unknown, and a language model for narration review and for explaining flags in plain words.
Start with the false positive budget
Before running anything, decide how many flags a reviewer can examine properly. For a small engagement that may be thirty to fifty items. Then tune to that budget:
- Raise thresholds until the flag count fits.
- Rank by risk score and review from the top.
- Combine signals. An entry that is round, manual, near period end and posted by an unusual user is more interesting than one with a single trait.
A short, ranked list that gets fully examined beats a long one that is skimmed.
Know your data first
Anomalies depend on normal. A trading company with lumpy bulk sales will show different patterns from a service firm with monthly retainers. Before testing:
- Reconcile the ledger extract to the trial balance, so you know the population is complete.
- Check dates, user fields and entry types. Missing user data weakens many tests.
- Separate system-generated from manual entries.
- Look at the data by month and account to understand what is usual.
Poor data produces noise, as explained in garbage in, audit out.
Keep scoring outside the model chat
Compute scores with code or a tool you can rerun, and save the logic and the output. If a model proposes the logic, a person confirms it, the code runs, and the flagged list is saved with the date and data version. The model's role in this pattern is to write and explain, not to be the calculator. See deterministic versus probabilistic AI.
Testing the test
Plant known items in a copy of the data: a duplicate payment, a round-sum entry, an out-of-hours posting. Run the method. Did it flag them? If not, it will not find real ones either. Record the result, and rerun when the method or the model changes.
Also try running the method on a prior year, where you know the outcome. Items already identified by earlier work should surface.
What to do with a flag
A flag is a question, not a finding. For each flagged item:
- Look at the entry and its support: the invoice, the approval, the bank movement.
- Ask the person who posted it, where appropriate.
- Record the explanation and the evidence that backs it.
- Classify: error, explained and acceptable, or suspicious and escalated.
- Consider whether a pattern across flags changes the risk assessment.
Keep the record of cleared flags too. It shows what was looked at, and it saves time when the same pattern appears next year.
Where it connects to sampling
Flagged items are a targeted selection, not a random one. They tell you where risk is, but they do not support a conclusion about the unflagged remainder unless the test covers the full population and its design is sound. Think carefully about the wording in the file. See SA 530, AI and sampling and duplicate payments tests for concrete tests.
Fraud and the limits of the method
Anomaly methods find statistical oddities. Fraud is often designed to look ordinary. A well-hidden entry, backed by genuine-looking documents and posted within normal ranges, may never be flagged. Do not let a clean anomaly run reduce professional scepticism or the response to assessed fraud risks. Use it as one procedure among several.
Confidentiality and tools
Ledger extracts hold names, bank details and commercial terms. Process them inside tools whose data handling you have reviewed, or locally. See can you upload client ledgers to AI tools.
Frequently Asked Questions
Is machine learning better than rules for anomalies?
Neither is better in general. Rules cover known risks and are easy to explain. Statistical methods find unknown patterns and are harder to defend. Many teams use both.
How many flags are too many?
More than the team can properly examine. Set the number from reviewer capacity and tune to it.
Can a chat model find outliers in a pasted ledger?
It may produce a list, but it is not a reliable calculator. Run the test in code or a spreadsheet and keep the output.
Last reviewed: 17 October 2026. Facts checked. No vendor, standard or research page was opened in this session. The methods are described in general terms from common audit analytics practice. No detection rate, benchmark or model comparison is claimed. Test any method on planted items in your own data.
Related Articles
Statutory facts on this page are checked against their sources, and the page says where it relied on secondary reporting. How we verify · Report an error