Reviewing AI-assisted engineering work without losing traceability

Make the source package visible before drafting

Imagine a synthetic facilities team comparing two replacement pumps. The source package contains an equipment schedule, a dated operating profile, two vendor data sheets, and an internal note about a possible future load increase. An AI assistant produces a recommendation that Pump B is “the best fit” because its rated flow exceeds the current schedule. The memo omits that the vendor sheet uses a different head condition and that the future load note is not an approved requirement. The problem is not grammar. It is an unexamined jump between unlike evidence.

A reviewer first creates a source register: document name, revision, owner, effective date, relevant unit, and status. The operating profile and equipment schedule may describe the current need; the vendor sheets provide conditional performance data; the future load note is a planning assumption until its owner confirms it. The register keeps the assistant from presenting every input as equally authoritative. Only an engineer with the appropriate responsibility can decide which design basis controls.

The NASA Systems Engineering Handbook discusses documenting requirement rationale and assumptions, checking traceability to stakeholder expectations, and asking whether requirements are verifiable. NASA's handbook is a primary engineering source, but its project rules do not automatically govern this fictional facilities case. The NIST traceability glossary defines traceability as a discernible association among entities such as requirements, system elements, and verifications. Those sources support the principle behind the register; they do not validate the pump choice.

Sources: [1] [2]

Turn each important sentence into a trace row

In the synthetic comparison, the row for “Pump B meets the duty point” should identify the duty-point requirement, the exact vendor curve revision, the unit conversion if one was made, and the reviewer who checked the curve at the required head. If the draft only points to a nameplate maximum flow, the row is incomplete. A separate row for “capacity for the future load” should be marked conditional because the future load is unapproved. The final memo can say that further confirmation is needed without pretending the uncertainty has disappeared.

The trace is not limited to numbers. A sentence claiming an option is easier to maintain needs an actual maintenance criterion and evidence. A sentence claiming it is permitted needs the applicable authority and an authorized interpretation. The AI assistant may help locate candidate passages or format the table, but it should not be asked to sign off its own assumptions. Version, unit, and scope checks are concrete ways to challenge a convincing but unsupported synthesis.

For a useful review meeting, the team can bring a one-page claim trace with columns for draft statement, governing input, revision, calculation or test, discrepancy, disposition, and owner. The reviewer can mark a statement supported, conditional, or removed. This artifact is deliberately different from a general “AI for engineers” checklist: it follows a specific decision memo down to the evidence needed for each claim.

Verify independently before the memo becomes a decision

An engineer should reproduce any consequential calculation from the original inputs, inspect the units and operating conditions, and compare the result with the AI-generated wording. A second reviewer can check that the source register includes the latest approved versions and that unresolved assumptions are conspicuous in the recommendation. If a requirement changes, the affected trace rows and conclusions must be reopened rather than silently carried forward.

NIST's AI Risk Management Framework Core calls for contextual evaluation, documented limitations, and human oversight of AI-system outputs. This is a general risk-management framework; the actual technical standard and approval path for any engineering work must come from its project and jurisdiction. The pump example is synthetic, with no tested outcome. Two Prune's assessment product may observe how a participant handles a bounded simulation and provide evidence for human interpretation; it does not approve engineering designs or predict workplace performance.

Sources: [3]