Practical guide
AI fluency for quality managers: review a recurring defect
Use a fictional document defect to separate correction, suspected cause and evidence that a preventive change actually worked.
Quality managers can practice AI fluency by reviewing a recurring defect without confusing a repaired output with a repaired process. This fictional document-control case follows an obsolete instruction through correction and follow-up. It offers a review worksheet, not a certification claim or a statistically established quality improvement.
State the defect precisely
Describe the mismatch between an expected output and the output observed.
The fictional team sends a welcome pack that still tells readers to use an old submission address. The approved source document contains the new address. Three recently prepared packs contain the obsolete instruction. Record the pack identifiers and affected sentence, rather than describing the entire onboarding process as broken.
Ask AI to compare the versions and identify the mismatched instruction. Verify that other differences are relevant before adding them to the defect record. A change in punctuation is not the same issue as an incorrect action route. The record should let another reviewer find the defect without guessing.
Separate correction from prevention
Fixing the three packs does not explain why the old instruction appeared.
A correction replaces the obsolete address in those packs and determines whether recipients need an updated version. A preventive change addresses how future packs are created. Keep those actions separate so closing the first does not automatically close the second. Do not actually contact anyone during this synthetic exercise.
The packet does not yet establish the cause. An old template, a copied prior pack or a transformation step are possible explanations. Ask AI to list the evidence needed to distinguish them, not to select the most familiar root-cause story. A confident label is not an investigation.
Test the suspected source of the defect
Reproducing the mismatch can narrow the explanation without proving every past instance.
Introduce two fictional templates: version 2 contains the old address and version 3 contains the new one. The shared folder still opens version 2 by default. A paper walkthrough using that default reproduces the obsolete sentence. This supports investigating the default template route; it does not prove all three packs used it.
Check the creation history where available and mark missing history as unknown. Propose changing the default route and clearly retiring the old template. Do not delete historical records that are needed to understand the incident. Keep any real retention or access decision with the responsible owner.
Define a follow-up check before declaring success
Evidence of implementation and evidence of effectiveness are different.
A screenshot of the new default establishes that a configuration changed, not that future packs are correct. For this exercise, inspect the next five synthetic packs produced through the normal route and the alternative copy-from-prior-pack route. Five is an illustrative walkthrough size, not a statistically justified sample.
If the alternative route still copies the obsolete sentence, the change has not addressed that path. Record the failure and revise the action rather than averaging it away. A small successful check cannot establish that the defect will never recur, especially through untested creation paths.
Close with evidence and a reopening condition
A useful quality note makes both the completed actions and their limits visible.
Deliver the defect description, affected versions, corrections, competing explanations, tested route and follow-up results. Include an owner and a reopening trigger, such as another pack with the obsolete instruction. Distinguish a proposed check from an executed one and keep the original faulty example for comparison.
Review whether the worker avoided jumping from three defects to an unsupported universal cause. The internal audit exercise asks whether a control was performed; this task asks whether a specific recurring problem was corrected and whether the chosen preventive action addressed the observed mechanism.
Sources and scope
NIST: AI risk management. Skills England: workplace AI foundations.
These references provide background, not validation or endorsement of this exercise. The case details, calculations and suggested review questions are original instructional material. Use them to discuss observable work, not to infer customer outcomes, professional credentials or performance in every setting. Before adapting the exercise, confirm the relevant facts, approved tools, data permissions and decision owners. If you change the case, revisit the expected answers and checks as well. These examples describe practice tasks, not a promise that a particular product includes the fictional features.
Sources: [1] [2]
Sources
- 1.AI RMF Core · NIST
- 2.AI foundation skills for work benchmark · Skills England