Practical guide
What evidence shows a security control is operating
A policy, configuration screenshot, passing test, or clean dashboard can each support a security-control review, but none alone shows that the control operated throughout a stated period. A defensible conclusion connects the control requirement to current design evidence, observed operation, challenge tests, exceptions, and retests. Use the PROVE packet: Purpose, Record, Observe, Verify, and.
A policy, configuration screenshot, passing test, or clean dashboard can each support a security-control review, but none alone shows that the control operated throughout a stated period. A defensible conclusion connects the control requirement to current design evidence, observed operation, challenge tests, exceptions, and retests. Use the PROVE packet: Purpose, Record, Observe, Verify, and Exceptions.
P: State the control purpose as a testable claim
Name the protected asset, threat or failure, actor, system boundary, enforcement point, expected behavior, failure consequence, and accountable owner. Separate the control objective from the mechanism. “Prevent unauthorized retrieval” is an objective. Permission filtering at query time may be one mechanism; cache invalidation, export restrictions, and monitoring may be separate controls or dependencies.
Write a determination statement that can fail:
> For every retrieval request during the review period, the service evaluates the requesting identity's current source permissions before content enters model context, and denied content does not appear in responses or exports.
Define the evidence period and material change rule. A model, connector, identity provider, index, permission scheme, orchestration path, export route, or logging change may require a new assessment. Do not reuse a passing result from a materially different version.
NIST describes the Risk Management Framework Assess step as determining whether controls are implemented correctly, operating as intended, and producing the desired outcome relative to security and privacy requirements. That federal framework is not a certification for a private workflow, but its separation of implementation, operation, and outcome is useful. NIST RMF: Assess step
Sources: [1]
R: Record design and implementation evidence
Create a control record:
| Field | Required evidence | | --- | --- | | requirement | source, version, scope, and interpretation owner | | control objective | prevented, detected, corrected, or recovered condition | | mechanism | architecture, configuration source, code or service version | | enforcement point | exact place where the decision occurs | | dependencies | identity, data, connector, cache, monitor, and human action | | owner | operator, reviewer, risk acceptor, and escalation route | | change history | material change, date, approver, and retest status |
Inspect the actual mechanism rather than only the policy. Confirm that the documented configuration matches the running environment and that a bypass route is not outside the diagram. Preserve hashes or immutable references where practical, while avoiding secrets and unnecessary sensitive data.
NIST SP 800-53A Rev. 5 provides customizable procedures for assessing security and privacy controls and was updated with Release 5.2.0 in August 2025. It treats specifications, mechanisms, activities, and individuals as assessment objects. Tailor that structure to the local risk; do not present use of the publication as proof of compliance. NIST SP 800-53A Rev. 5
Sources: [2]
O: Observe operation over the stated period
Choose evidence that could reveal failure, not only evidence that the mechanism exists. Depending on the control, this may include signed configuration history, deployment records, access decisions, alerts, blocked events, sampled transactions, reviewer actions, exception tickets, recovery records, and monitoring gaps.
Define the population, sample, period, and selection method. Include high-consequence cases, ordinary cases, change events, and cases likely to exercise bypasses. Reconcile monitoring totals to a source population where possible. Missing telemetry is an evidence limitation, not evidence that no failure occurred.
Use an operating-evidence table:
| Evidence ID | Population and period | Selection | Expected signal | Observed result | Gap or confound | Reviewer | | --- | --- | --- | --- | --- | --- | --- | | | | | | | | |
Interview operators to understand practice and exceptions, but corroborate material statements. A person saying the control runs daily is not the same as a dated operating record. Conversely, a log entry may not show whether a human reviewed or resolved the event as required.
V: Verify with examine, interview, and test
Plan challenge procedures before seeing the result. NIST SP 800-53A organizes assessment methods around examine, interview, and test. Examination reviews specifications, mechanisms, and activities; interviews clarify understanding and responsibility; tests compare actual with expected behavior under stated conditions. Use all relevant methods rather than treating a document review as an operating test.
For each procedure, record the object, environment, version, precondition, input, expected result, observed result, evidence reference, severity, and disposition. Test positive and negative paths. A permission control should block unauthorized content without preventing authorized work. Test direct prompts, indirect retrieval, exports, shared links, cached results, permission revocation, service identities, and failure modes appropriate to the system.
Independence should match consequence. The operator may run frequent checks, while a separate reviewer reperforms a sample or high-risk scenario. Record conflicts and adjudication. Do not average a severe failure into a reassuring percentage.
E: Track exceptions, remediation, and retest
Every exception record needs scope, detected date, evidence, consequence, owner, compensating measure, approval authority, expiry, remediation, retest procedure, and closure decision. Preserve the failed result. A later pass does not erase the period in which the control failed.
Use four dispositions:
Report the conclusion with limitations: population, sample, period, versions, untested paths, monitoring gaps, open exceptions, and next review trigger. Avoid “effective” without the exact scope. A useful conclusion is: “The control passed the listed challenge tests on version 4.2 and no failures appeared in the sampled 14-day operating record; export paths and emergency access were not tested.”
- Pass: the bounded evidence supports the determination for the tested scope and period.
- Repair: a remediable gap requires correction and retest.
- Hold: evidence is incomplete, stale, or not interpretable.
- Stop: a severe failure or unauthorized condition makes continued operation unacceptable to the accountable owner.
Worked artifact: a synthetic source-permission control
A synthetic knowledge assistant claims to filter every retrieved document against current repository permissions. The PROVE packet identifies the repository as authority, the retrieval filter as enforcement point, a semantic index as dependency, and revocation within ten minutes as the expected behavior.
Design inspection shows the permission check in the query path. Operating evidence covers 1,840 synthetic retrieval events over 14 days, including 22 denied requests and two connector outages. The monitor records no filter bypass, but logging was unavailable for 47 minutes during one outage; that period is marked unobserved rather than clean.
Challenge tests place unique markers in general, finance, and executive folders. Direct access and cross-document questions behave as expected. A revocation test fails: an executive marker remains retrievable for 31 minutes through a stale index. The disposition is repair, not pass. The owner purges derived entries on permission-change events, adds a ten-minute fallback expiry, and creates an alert for invalidation failures.
Retesting passes direct, revocation, export, shared-link, and service-identity cases on the corrected version. The final conclusion remains bounded: the tested paths passed after remediation, the logging gap remains disclosed, and emergency access is scheduled for separate review. This synthetic example does not establish effectiveness in any customer environment.
Claims and product boundary
This packet supports a scoped operating conclusion; it does not establish legal compliance, security certification, absence of compromise, or universal control effectiveness. Current jurisdiction-specific requirements and authorized security judgment remain necessary. Two Prune can assess how a participant reasons about control evidence in a bounded work simulation. It does not operate customer controls, certify security, or make employment decisions.
Sources
NIST RMF: Assess step
- NIST SP 800-53A Rev. 5
- NIST AI RMF Core
Sources: [1] [2] [3]
Sources
- 1.NIST RMF: Assess step · National Institute of Standards and Technology
- 2.NIST SP 800-53A Rev. 5 · National Institute of Standards and Technology
- 3.NIST AI RMF Core · National Institute of Standards and Technology