Practical guide

Prompt-injection defense policy for AI-enabled work

This policy sets minimum rules for systems that combine model instructions with untrusted content or can call tools, retrieve data, or cause external effects. It assigns accountable owners, limits privileges, requires evidence-backed testing, and defines exception and incident routes. A policy is a governance baseline. It does not show that controls operate, replace system-specific threat.

By Two Prune

This policy sets minimum rules for systems that combine model instructions with untrusted content or can call tools, retrieve data, or cause external effects. It assigns accountable owners, limits privileges, requires evidence-backed testing, and defines exception and incident routes. A policy is a governance baseline. It does not show that controls operate, replace system-specific threat modelling, or establish that an AI system is secure.

Authority and scope boundary

NIST defines prompt injection as an attack that exploits combining untrusted input with a prompt constructed by a higher-trust party. Its Generative AI Profile describes direct and indirect prompt injection and frames risk management across design, development, deployment, and use. The UK National Cyber Security Centre's secure-AI guidance calls for lifecycle security, supply-chain controls, protected assets and logs, documented data, models and prompts, and managed access to sensitive data. These sources support the policy requirements below. They do not prescribe a complete defense for every system. NIST prompt-injection glossary NIST AI 600-1 Generative AI Profile NCSC secure AI system development

This policy applies to AI-enabled systems that accept user or third-party content, retrieve documents or webpages, use connectors or memory, invoke tools, access non-public data, or generate content for an external destination. It covers employees, contractors, service owners, developers, operators, reviewers, and suppliers within the approved system boundary.

Sources: [1] [2] [3]

Policy rules

### 1. Register the system and trust boundaries

Before use, the service owner must register the system's purpose, owner, model and version, system instructions, retrieval sources, data classes, tools, identities, permissions, output destinations, human approval points, suppliers, and supported users. The owner must map every route through which untrusted content can enter, including user text, files, webpages, email, connector results, memory, and model-generated content.

Unregistered high-impact tools or data paths must not be enabled.

### 2. Treat external content as data, not authority

Instructions found in retrieved or uploaded content must not override higher-trust instructions or authorization policy. The system must preserve provenance where practical and distinguish user intent, system policy, retrieved evidence, and tool results. Content filtering or instruction classifiers may support the control but must not be the only boundary for consequential effects.

### 3. Minimize privileges and separate authorization

Models and orchestration components must receive only the data and tool permissions needed for the approved task. High-impact actions require an authorization check outside the model. The check must validate the user, action, target, data scope, and current approval. Examples include sending messages, changing records, moving money, publishing content, granting access, or exporting restricted data.

Human approval must show the proposed action and material inputs. A generic approval at session start is insufficient for a later consequential action.

### 4. Validate tool inputs and observe effects

Tool schemas must constrain accepted operations and reject unexpected fields, destinations, and data classes. The service must log input provenance, retrieval events, policy decisions, tool requests, authorization results, approvals, effects, blocked attempts, and incident references. Logs must be protected and minimized according to their sensitivity.

A blocked final answer is not evidence of containment if a tool already acted or restricted data entered an unauthorized trace.

### 5. Test adversarial and benign behavior

Before release and after material change, the owner must run a versioned test set covering direct injection, indirect instructions in every supported content type, obfuscation, multi-turn attacks, data-exfiltration attempts, tool misuse, cross-context contamination, and representative benign content. Results must reconcile attempted, executed, passed, failed, inconclusive, and excluded cases.

The test record must state expected effects, coverage gaps, false positives, independent cases, prior regressions, and residual risk. Production monitoring and incident records must be reviewed with test results.

### 6. Handle exceptions, incidents, and change

An exception needs a named owner, business rationale, affected assets and users, compensating controls, residual risk, approving authority, expiry date, and retest plan. Exceptions must not silently become permanent design.

Suspected prompt injection must follow the security incident process. Preserve relevant evidence, contain affected capabilities, identify exposed versions and data, notify accountable owners, and retest repaired and representative unaffected paths before restoring scope.

Material changes to models, prompts, retrieval, tools, permissions, data, monitoring, suppliers, or threat intelligence trigger review and proportionate retesting.

Accountability matrix

| Role | Accountable for | | --- | --- | | executive or risk owner | risk appetite, high-impact exceptions, unresolved residual risk | | service owner | registration, approved purpose, operating evidence, renewal or withdrawal | | security owner | threat model, test adequacy, monitoring, incidents, retest disposition | | data owner | data classification, access, retention, disclosure conditions | | engineering owner | system boundaries, privilege controls, tool validation, versioning | | user or operator | approved use, review of proposed actions, reporting suspicious behavior | | independent reviewer | challenge of evidence, failures, exclusions, and limitations |

Required policy evidence

Maintain the current system register, architecture and trust-boundary map, data and tool inventory, control map, threat-set manifest, benign test set, execution logs, production monitoring sample, exception register, incident record, approval history, change log, and current operating disposition. Evidence must name its version, period, population, limitations, reviewer, and expiry.

Allowed dispositions are operating as tested, operating with limitations, not operating as intended, or insufficient evidence. Avoid a general “secure” label.

Worked policy decision: synthetic document assistant

A fictional document assistant summarizes approved internal files and can save notes to a project folder. A team asks to add webpage retrieval and email sending. The existing system has a direct-prompt filter but no map of indirect inputs and no authorization layer outside the model.

Under this policy, webpage retrieval can enter a restricted test environment because the data source, provenance, permissions, and threat set can be registered. Email sending cannot enter the same trial. It is a consequential external action, and the proposed generic session approval does not validate each recipient, message, attachment, or data class.

Testing of webpage retrieval includes instructions embedded in HTML, PDFs, metadata, and search snippets, plus benign security articles. Two cases place malicious instructions into draft notes, while the project-folder allowlist prevents an external write. The service owner records not operating as intended, disables the affected PDF path, repairs provenance handling, and retains the allowlist as an independent effect control. The exception request for email is denied until per-action authorization, input validation, logging, and adversarial tests exist. The policy decision does not state that the repaired assistant is secure.

Review and enforcement

The service owner must review this policy and each registered system at least annually and after material change or incident. Breaches may lead to capability restriction, access removal, containment, investigation, or system withdrawal under the organization's existing governance. Legal, regulatory, contractual, accessibility, employment, and sector-specific requirements remain separately applicable and need qualified review.

Two Prune boundary

Two Prune can assess whether a participant identifies prompt-injection risks, evaluates supplied policy and evidence, and writes a bounded control disposition in a simulation. It does not write or approve an organization's security policy, penetration-test or certify systems, provide legal compliance, authorize deployment, predict workplace performance, or make employment decisions.

Sources

NIST: Prompt injection glossary

  • NIST: Artificial Intelligence Risk Management Framework, Generative AI Profile
  • UK NCSC: Guidelines for secure AI system development

Sources: [1] [2] [3]

Sources

  1. 1.NIST prompt-injection glossary · National Institute of Standards and Technology
  2. 2.NIST AI 600-1 Generative AI Profile · National Institute of Standards and Technology
  3. 3.NCSC secure AI system development · www.ncsc.gov.uk