Practical guide
What Evidence Shows Change Control Is Operating?
Change control is operating when a defined population of changes can be traced from request through impact review, authorization, implementation, verification, release, and follow-up, including exceptions and failed changes. A policy, ticket template, or approval count shows design or activity; it does not by itself show that the control operated for the changes that mattered.
Change control is operating when a defined population of changes can be traced from request through impact review, authorization, implementation, verification, release, and follow-up, including exceptions and failed changes. A policy, ticket template, or approval count shows design or activity; it does not by itself show that the control operated for the changes that mattered.
Define the control claim and change population
Write the control objective first. For example: “Material changes to the AI-enabled customer-guidance workflow are identified, assessed, authorized, tested, released, and reviewed by accountable owners before they affect users, unless an approved emergency route applies.”
Define what counts as a change. Include relevant model, prompt, retrieval source, policy, data transformation, tool permission, integration, threshold, user experience, monitoring, and fallback changes. Record exclusions and who can approve them. Set the review period and identify the complete population source, such as a deployment register reconciled with code, configuration, model, content, and access-system records.
NIST SP 800-53 CM-3 calls for organizations to define configuration-controlled changes, review and approve or disapprove proposed changes with security and privacy impact analysis, document decisions, implement approved changes, retain records, monitor activity, and coordinate affected components. Tailor the control to the system and risk; do not present it as a universal legal requirement. NIST SP 800-53 Revision 5.1
Sources: [1]
Build an evidence chain for each sampled change
For every selected change, link the following records with stable identifiers:
The evidence should demonstrate sequence and substance. A timestamped approval that does not identify the reviewed version is weak evidence. A successful automated test is not evidence that a required domain review occurred. A ticket closed after deployment may show record completion but not prior authorization.
NIST's AI RMF Core treats governance as continuous across the AI lifecycle and includes post-deployment monitoring, incident response, recovery, and change management. That supports testing the whole operating chain rather than stopping at pre-release approval. NIST AI RMF Core
- Request and reason, including the initiating problem or requirement
- Affected components, users, data, policies, controls, and dependencies
- Risk, security, privacy, domain, and operational impact reviews as applicable
- Test plan, expected result, actual result, reviewer, and retained artifact
- Authorization with approver, scope, conditions, version, and time
- Release record that identifies what actually changed
- Verification after release, including monitoring and rollback readiness
- Incident, exception, correction, or retrospective record where relevant
Sources: [2]
Test design before operating effectiveness
First test whether the control is designed to address the stated risk. Confirm the change taxonomy, materiality rules, decision rights, segregation or challenge requirements, evidence fields, emergency route, rollback criteria, retention, and review cadence. A perfectly followed process can still be poorly designed.
Then test operation. Reconcile the change population to independent sources and sample by risk, type, owner, system, and outcome. Include ordinary, emergency, rejected, rolled-back, and failed changes rather than sampling only completed successes. For each sample, verify the required steps, their order, the people involved, the artifact versions, and whether approval conditions were satisfied.
Record each test as pass, exception, unavailable evidence, or not applicable with a reason. Missing evidence is not automatically proof that the action did not occur, but it prevents a supported operating claim. Investigate population completeness and instrumentation failures separately from process failures.
Examine exceptions and emergency changes
An exception record should name the requirement not met, reason, scope, risk owner, compensating action, expiry, and closure evidence. Test whether expired exceptions remain active and whether repeated exceptions reveal a broken standard process.
Emergency changes need a genuinely restricted trigger, an authorized initiator, minimum pre-release checks, logging, immediate monitoring, rollback authority, and a time-bounded retrospective review. Measure emergency use against the complete change population. A high or rising rate is a diagnostic that requires context, not automatic proof of failure.
The NIST AI RMF Playbook recommends connecting AI governance to existing organizational governance and risk controls, documenting roles, and maintaining review practices. Use those suggestions to integrate change evidence with existing systems rather than creating an isolated AI-only approval layer without a clear need. NIST AI RMF Govern Playbook
Sources: [3]
Worked test: policy-assistant release
Consider 18 changes to an internal policy assistant during one quarter. The population reconciliation finds 14 ordinary releases, two rejected requests, one rollback, and one emergency retrieval-source correction. The control owner samples six records, including the rollback and emergency change.
Four ordinary changes have linked requests, impact reviews, version-bound approvals, test artifacts, and post-release checks. One prompt update has an approval tied only to the ticket, while the prompt was edited again before release. The reviewer cannot show that the approver saw the deployed hash, so the sample is an exception even though the ticket says approved.
The emergency source correction has an authorized trigger and monitoring evidence but no retrospective domain review within the required two business days. The late review later finds no harmful output, but that does not erase the process exception. The control owner adds a release gate that invalidates approval when the prompt or source bundle hash changes and an alert for overdue emergency reviews.
The disposition is “operating with two exceptions requiring repair,” not simply effective or ineffective. The evidence supports the six tested records for the stated quarter. It does not establish legal compliance, the factual accuracy of every assistant response, or operation outside the defined population and period.
Record the review and next action
Close with the control objective, period, population source and reconciliation, sample method, evidence inspected, exceptions, unavailable evidence, compensating actions, responsible owners, due dates, residual risk, reviewer, and next review trigger. Preserve enough detail for another reviewer to reproduce the conclusion without exposing unnecessary sensitive content.
AI can help reconcile identifiers, compare timestamps, detect missing fields, and assemble a candidate sample. Human reviewers must judge materiality, source authority, required approvals, exception significance, and the supported conclusion.
Two Prune can place a participant in a bounded assessment where they examine change records, find evidence gaps, and produce a review artifact. It does not operate or certify an organization's control, make a legal-compliance determination, establish safe AI use, or make employment decisions.
Sources
- 1.NIST SP 800-53 Revision 5.1 · National Institute of Standards and Technology
- 2.NIST AI RMF Core · National Institute of Standards and Technology
- 3.NIST AI RMF Govern Playbook · National Institute of Standards and Technology