Practical guide
Quality assurance policy for AI-enabled work
A quality assurance policy for AI-enabled work should define which work needs assurance, who owns the result, what evidence must be retained, how material defects are handled, and when the workflow must be restricted or stopped. It should not declare a system “safe” or “effective” merely because a test was run or an average score was high. This educational policy model needs adaptation to the.
A quality assurance policy for AI-enabled work should define which work needs assurance, who owns the result, what evidence must be retained, how material defects are handled, and when the workflow must be restricted or stopped. It should not declare a system “safe” or “effective” merely because a test was run or an average score was high. This educational policy model needs adaptation to the organization’s sector, jurisdiction, risk tolerance, contracts, and operating environment. A policy document alone is not evidence that the control operates.
Policy objective and scope
Objective: provide repeatable, proportionate assurance that AI-enabled work meets its stated requirements before consequential use and remains within those requirements while operating.
Scope: list the covered teams, workflows, systems, vendors, data classes, geographies, and decisions. Include AI-generated or AI-transformed work even when a human performs the final action. Exclude only named uses with a recorded rationale and review date.
Principles: the accountable human or team retains ownership; assurance is proportional to consequence and uncertainty; material defects are visible rather than averaged away; test evidence is reproducible; exceptions expire; and changed context triggers review.
The NIST AI RMF Core calls for policies, procedures, documented roles, testing before deployment and during operation, measurement of uncertainty, independent review where useful, monitoring, and regular reassessment. NIST describes the framework as voluntary and adaptable rather than a certification. NIST AI RMF Core
Sources: [1]
Use the CLEAR control cycle
CLEAR converts the policy into five operating requirements.
### C: Classify the work and consequence
The workflow owner must document the intended purpose, affected people, final action, material requirements, plausible failure modes, and maximum acceptable consequence. Classification should account for data sensitivity, reversibility, scale, time pressure, human review, and whether an error can reach an external party.
At minimum, separate low-consequence drafting support from work that affects rights, safety, finances, access, legal obligations, public claims, or critical operations. A higher-risk classification requires stronger independence, broader cases, tighter defect rules, and more frequent review.
### L: Link requirements to tests
Each material requirement must link to at least one assurance method and a named evidence artifact. Methods may include source trace, deterministic checks, representative case review, adversarial testing, accessibility evaluation, calculation replay, human-factors review, red-team exercise, or monitored pilot.
The test plan must record the frozen system version, configuration, source set, case-selection rationale, expected result, defect taxonomy, evaluator, uncertainty, and handling of missing evidence. A passing aggregate cannot override a failed critical requirement.
### E: Execute independently and preserve evidence
The assurance owner must be sufficiently independent from the author or delivery team for the consequence involved. Test execution should preserve inputs, outputs, reviewer decisions, defects, retests, environment details, and deviations from the plan. Personal and sensitive data should be minimized and protected under applicable policy.
The U.S. Government Accountability Office’s AI Accountability Framework organizes review around governance, data, performance, and monitoring. It provides questions and audit procedures for entities, auditors, and third-party assessors. Use it as a primary-source accountability model, not as proof of compliance or as a substitute for jurisdiction-specific requirements. GAO AI Accountability Framework
### A: Authorize, restrict, or reject
The decision authority must review the evidence packet and select a recorded disposition:
Every exception needs an owner, rationale, compensating control, affected scope, expiry date, and approval. Teams may not relabel a failed material requirement as an accepted limitation without the designated authority.
### R: Review operation and change
The workflow owner must monitor signals appropriate to the use case, such as material defects, overrides, complaints, escalations, source failures, drift, response time, unauthorized use, and exception age. Review after a material model, prompt, source, data, integration, user-population, or policy change. Also review on a fixed schedule even when no incident is reported.
Monitoring must lead to an action rule. A dashboard without a disposition owner is observation, not control.
- authorize within the tested boundary
- authorize with restrictions and an expiry
- repair and retest before use
- pause an operating workflow
- reject or retire the use case
Sources: [2]
Required roles
| Role | Minimum accountability | | --- | --- | | Policy owner | maintains scope, definitions, review cadence, and approved exception process | | Workflow owner | defines intended use, requirements, operating boundary, and remediation | | Assurance owner | approves the test design, preserves evidence, and reports defects | | Reviewer or evaluator | performs assigned checks and records uncertainty and missingness | | Decision authority | authorizes, restricts, pauses, or retires the workflow | | Incident owner | coordinates containment, evidence preservation, communication, and recovery |
One person may hold several roles for a low-consequence use case, but the combination and resulting independence limitation must be explicit.
Minimum assurance packet
Retain a versioned requirement register, workflow and data-flow map, risk-control-test matrix, approved test plan, frozen test cases, execution results, defect and exception registers, authorization record, change log, monitoring reviews, and next review date. The packet should allow another qualified reviewer to reproduce the decision without relying on the original author’s memory.
Worked example: sourced client briefing
A consulting team uses an AI assistant to draft a client briefing from approved research. The work can contain public claims and recommendations, so the workflow is classified above ordinary internal brainstorming.
Classify: the intended use is first-draft assembly. The assistant cannot publish, contact a client, or add unapproved sources. A consultant owns the final briefing.
Link: the requirements include source completeness, citation accuracy, distinction between source fact and inference, date relevance, calculation correctness, confidentiality, and required caveats. The assurance plan uses a frozen set of representative briefs, source-trace review, calculation replay, and cases with conflicting or stale sources.
Execute: an independent reviewer records results by defect class. The overall acceptance rate is 93 percent, but two cases contain unsupported material claims and one omits a conflicting source. Those defects remain visible and prevent unrestricted authorization.
Authorize: the decision authority permits a restricted pilot in which every claim is source-checked and every briefing receives senior review. The team must repair retrieval filters and retest the three failed cases plus a broader challenge set. The exception expires in 30 days.
Review: monitoring records unsupported-claim defects, source-link failures, reviewer overrides, review time, and incidents. A change to the approved source collection triggers targeted retesting. The policy supports a restricted pilot with repair, not a declaration that quality assurance is effective in every context.
Exceptions, failures, and enforcement
Anyone performing covered work must report a material defect or boundary breach through the defined route. The workflow owner must contain the issue, preserve evidence, assess affected outputs, and notify the decision authority. Retaliation for good-faith reporting should be prohibited under the organization’s applicable policies.
The policy owner should review expired exceptions, repeated defects, missing evidence, overdue remediation, and unauthorized expansion. Hold or pause the workflow when a critical test fails, required evidence is unavailable, a material change is untested, an exception expires, or the accountable owner cannot be identified.
Two Prune can use quality-assurance work as a realistic assessment scenario and describe task-bounded evidence across framing, evidence navigation, AI use, verification, judgment, and communication. It does not certify control effectiveness, provide legal compliance, promise safe operation, or make employment decisions.
Sources
NIST AI Risk Management Framework Core
- U.S. Government Accountability Office: Artificial Intelligence Accountability Framework
Sources: [1] [2]
Sources
- 1.NIST AI RMF Core · National Institute of Standards and Technology
- 2.GAO AI Accountability Framework · U.S. Government Accountability Office