Security controls for AI-enabled knowledge work

Start with a falsifiable control statement

Write what the control is expected to prevent, detect, contain, or recover from. Name the protected asset, threat or failure, actor, system boundary, enforcement point, and conditions. “Access is secure” cannot be tested. “The retrieval service filters every document result against the requesting user's current repository permissions before model context is assembled” can be challenged.

Distinguish preventive, detective, corrective, and recovery controls. A prompt warning may guide behavior but does not enforce access. Output scanning may detect a pattern but cannot prove prohibited data never entered the model context. Human approval may stop publication while leaving unauthorized retrieval unresolved.

NIST's Cybersecurity, Privacy, and AI programme connects AI-specific work with broadly applicable resources such as the Cybersecurity Framework, Privacy Framework, Secure Software Development Framework, and security-control overlays. It supports integrating AI controls into existing risk programmes rather than treating them as a separate checklist.

Sources: [1]

Assemble the control evidence chain

For each control, record the risk statement, intended outcome, system and version, configuration source, identity or data boundary, owner, implementation evidence, test population, expected and observed result, failure severity, monitoring event, exception, compensating measure, retest, and review date.

Separate design, implementation, and operation. A configuration screenshot may show intended setup. A technical test may show that setup blocked one scenario. Production logs and sampled events may show whether it continued to operate. None alone establishes universal effectiveness.

Define the evidence cadence as part of the control. Configuration evidence may be collected after every material change, technical tests before release and on a scheduled basis, and operating samples over a stated period. Record gaps when telemetry is missing or a control cannot be exercised safely. A stale passing test should not silently substitute for evidence from the current system.

The preliminary NIST Cyber AI Profile uses Cybersecurity Framework outcomes to organize priorities for securing AI components, AI-enabled defense, and AI-enabled attacks. Because it is an initial preliminary draft, it should be treated as evolving guidance and not as a certification baseline.

Sources: [2]

Worked artifact: a synthetic source-isolation control

A synthetic knowledge assistant has a stated control: users may retrieve only documents they can open in the source repository. The evidence chain identifies the repository permission service as the authority, the retrieval filter as the enforcement point, and a cached semantic index as a dependency.

Tests use unique synthetic markers in general, finance, and executive folders. Direct questions respect permissions, but after an executive user's access is removed, the marker remains retrievable from a stale cache. The control fails its revocation condition. Engineers add event-driven invalidation and a short fallback expiry, purge derived entries, and rerun access, revocation, cross-document, export, and shared-link tests.

The review retains the failed evidence, correction, retest, remaining cache interval, owner, and narrow approval scope. This is a synthetic example; it does not establish effectiveness in any customer environment.

Verify operation and handle exceptions

Sample tests across identities, data classes, connectors, model paths, and change events. Reperform high-consequence controls independently. Check negative and positive behavior: protected content must remain inaccessible, while authorized work must still function. Inspect logs for disabled controls, bypass routes, processing failures, and permissions evaluated under a service identity.

Every exception needs scope, reason, approver, compensating measure, evidence, start and expiry dates, and a closure test. Reopen the chain after model, provider, orchestration, identity, repository, connector, retention, or monitoring changes.

Report test coverage, severe failures, unresolved exceptions, overdue retests, and monitoring gaps rather than a single “control score.” Two Prune can observe how a participant reasons about control evidence in a bounded assessment. It does not certify security, provide legal conclusions, operate customer controls, or make employment decisions.