Testing change management: program changes, configuration and emergency changes

• Obtain a complete population of changes from the system rather than the ticket tool
• Test authorization, testing and migration approval for sampled changes
• Evaluate segregation between development and production
• Handle emergency changes and configuration changes in the test

The risk

AS 2110.B4 lists unauthorized changes to systems or programs and failure to make necessary changes as risks from IT. If a program that calculates depreciation, revenue allocation or commissions changes without control, the automated control the auditor relied on may no longer work.

Complete the population first

The most common weakness in change testing is an incomplete population. The ticketing system lists changes that went through the process. The question is whether any change reached production without a ticket.

Ways to get a complete population:

• a system-generated list of objects changed in production (programs, tables, configuration) with dates;
• deployment tool logs showing every migration to production;
• version history in the code repository compared with what is running in production.

Reconcile the system list to the ticket list. A change in the system list with no ticket is an exception. Document how you obtained the list and why you believe it is complete.

Attributes to test for each sampled change

1. Request and authorization. A business owner requested and approved the change before development started.
2. Testing. The change was tested, and the business user accepted the result (user acceptance testing) before migration.
3. Migration approval. Someone authorized moving the change into production.
4. Segregation. The person who developed the change did not migrate it to production. If developers have production access, that is a design issue to evaluate even if every sampled change looks fine.

Configuration changes

Many packaged systems have no code changes but have configuration that drives accounting: posting rules, tax tables, approval limits, three-way match tolerances. A change in configuration can affect financial reporting as much as a code change. Ask how configuration changes are logged and approved, and include them in the population.

Emergency changes

Emergency changes skip some steps to fix production fast. The control is usually after-the-fact review and approval within a set time. Test a sample of emergency changes for documentation of the reason, the post-implementation approval, and timeliness. A high volume of emergency changes is itself a signal that the normal process is being bypassed.

Segregation of environments

Ask who has access to move code into production and who has developer rights. Obtain the list of users with production migration rights and compare it to the development team. Where a small team cannot separate roles, look for a detective control: for example, an independent monthly review of a system-generated list of all production changes, matched to approved tickets.

Sample sizes and period

Sample sizes follow firm methodology and depend on the population size and risk. Cover the full period of reliance. A small population (for example, five changes in a year) can be tested in full.

Packaged software with no client code

If the client cannot change source code (a packaged cloud general ledger), document that and focus on configuration and vendor release management. For vendor-hosted software, evidence about the vendor's own change management comes from its SOC 1 report, when one covers the system (lessons 10 to 13).

What to do with exceptions

For a change that skipped testing or approval, identify what changed and whether it touched a key automated control or report. If it did, test the control or report after the change to see that it still works, and evaluate the deficiency. If it did not, the deficiency may be limited, but note it.

Use the change management test worksheet to record the population reconciliation and each attribute.

Change population at Ashgrove Logistics