Practical guide
How to create an AI launch plan that survives contact with reality
A useful launch plan is a decision system, not a calendar decorated with milestones. It tells the team what is changing, for whom, what evidence permits each step, who can pause the release, and how the service will be supported after the first users arrive. AI can help assemble the plan, interrogate dependencies, draft test cases, and surface gaps. It should not silently decide whether a.
A useful launch plan is a decision system, not a calendar decorated with milestones. It tells the team what is changing, for whom, what evidence permits each step, who can pause the release, and how the service will be supported after the first users arrive. AI can help assemble the plan, interrogate dependencies, draft test cases, and surface gaps. It should not silently decide whether a product is ready. Readiness depends on observed evidence from the actual deployment context and an accountable owner. This guide uses the FLIGHT method: Frame, Limits, Instrumentation, Gates, Handoff, and Tracking.
1. Frame the launch decision
Write a one-sentence decision before building a schedule:
> Decide whether to release capability X to audience Y, through channel Z, on or after date D, subject to the listed readiness gates.
Then record:
The GOV.UK Service Manual treats movement into live service as a phase transition supported by evidence from discovery, alpha, and beta. It also expects teams to keep improving the service after launch, maintain security and availability, measure success, and continue quality and performance testing. That is a useful operating model even outside government: launch is a controlled change in exposure, not the end of delivery. GOV.UK: How the live phase works
- the user problem and target audience;
- the smallest release unit that creates a complete user journey;
- what will remain unavailable;
- the decision owner and launch commander;
- the people who can block or pause the release;
- the planned exposure sequence, such as staff, invited users, limited availability, then wider availability;
- the source version, model, prompt, retrieval index, policy, or other AI component that could change behavior.
Sources: [1]
2. Declare limits before benefits
Create a boundary card that sits at the top of the plan.
| Boundary | Launch statement | Evidence needed | | --- | --- | --- | | Intended use | What the capability assists a person to do | Tested task set and user journey | | Excluded use | Decisions or data it must not handle | Policy and interface controls | | Audience | Who may access the release | Identity and entitlement test | | Data | Permitted input classes and storage path | Data-flow review and deletion test | | Human authority | Who reviews, overrides, and accepts residual risk | Named owner and rehearsal record | | Known limits | Failure modes users must understand | Test results and user-facing wording |
For an AI-enabled capability, include the model or service dependency, fallback behavior, escalation route, and the conditions under which AI output must be independently checked. NIST's AI Risk Management Framework calls for context-specific testing before deployment, documented test methods and results, and post-deployment monitoring that includes feedback, override, incident response, recovery, and change management. NIST AI RMF Core
Sources: [2]
3. Design instrumentation around decisions
Do not begin with a dashboard wish list. Begin with the questions that determine whether to continue, repair, pause, or roll back.
Use four measurement layers:
For each metric, specify an owner, event definition, denominator, segment, collection window, acceptable missingness, and decision it informs. A metric without a linked decision is observation, not a launch control.
- Journey health: completion, abandonment, latency, error rate, and support contact at each critical step.
- AI behavior: task success on the release test set, unsupported-output rate, override frequency, source retrieval failures, and known-limit triggers.
- Operational burden: review time, escalation volume, incident handling time, and manual fallback capacity.
- User consequence: whether people understand the output, can challenge it, and can complete the intended task without hidden work.
4. Convert readiness into evidence gates
Build gates around claims the team must support. Avoid a single percentage score that lets a strong area conceal a critical failure.
### Gate A: complete journey
### Gate B: bounded AI behavior
### Gate C: controlled operations
### Gate D: accountable release
NIST's Generative AI Profile describes actions organizations can select based on their goals, risk tolerance, resources, and role in the AI lifecycle. Treat it as a source of risk-management prompts, not a universal checklist or proof that a release is acceptable. NIST AI 600-1: Generative AI Profile
- The intended user can complete the end-to-end path in the production-like environment.
- Accessibility, authentication, permissions, notifications, export, and support routes have been exercised.
- The fallback path works when the AI dependency is unavailable or produces an unusable response.
- Release-specific tasks cover ordinary, edge, adversarial, and out-of-scope inputs.
- Expected answers or review criteria are frozen before the final run.
- Results include uncertainty, error categories, and examples, not just an average.
- A person can identify AI involvement, review supporting material, and override where the workflow requires it.
- Monitoring alerts reach a named responder.
- The team has rehearsed pause, rollback, model or prompt reversion, and degraded-mode procedures.
- Support staff know the boundary card and escalation route.
- Dependencies, contact routes, access rights, and recovery steps have been checked.
- Security, privacy, legal, records, accessibility, product, and operational reviews are complete where applicable.
- Open exceptions state the owner, rationale, expiry date, exposure, and compensating action.
- The launch decision and dissenting views are recorded.
Sources: [3]
5. Plan the exposure sequence and handoff
For every stage, complete this card:
| Field | Entry | | --- | --- | | Audience and maximum exposure | | | Entry and exit time | | | Features and data allowed | | | Evidence required to enter | | | Metrics and review frequency | | | Pause or rollback trigger | | | Decision owner | | | Support and incident owner | |
Sequence by risk and learning value, not only by audience size. A small group that cannot reveal the hardest failure is not a meaningful test. Include the users, tasks, integrations, and data conditions that matter to the next decision.
The handoff packet should contain the boundary card, architecture and data-flow references, test report, open exceptions, runbook, support scripts, incident classification, rollback steps, change log, and the next review date. The GOV.UK service-assessment guidance asks teams to provide a service brief, recent prototype, high-level architecture, and an explanation of how AI is used. It also recommends bringing the people who can answer questions across ownership, technology, research, design, and performance. GOV.UK: Book a service assessment
Sources: [4]
6. Track decisions after launch
Create a launch decision log rather than a collection of chat messages.
| Time | Signal | Evidence link | Decision | Owner | Follow-up | Review date | | --- | --- | --- | --- | --- | --- | --- | | | | | continue / repair / pause / roll back | | | |
Review the first hours and days at a cadence matched to the possible consequence. Compare real behavior with the release test set. Record changes to models, prompts, retrieval sources, policies, interfaces, and thresholds. A launch remains controlled only while the evidence still describes the running system.
Worked example: launching an AI-assisted policy search
A 400-person organization plans to release an internal assistant that retrieves approved policy passages and drafts a short answer. It may not make access, disciplinary, compensation, or employment decisions. Users must see the cited passage and contact the policy owner when the source is missing or ambiguous.
The team first proposes a company-wide Monday launch because 93% of 120 test questions received a plausible answer. FLIGHT rejects that evidence as incomplete. The aggregate hides three failures: outdated passages remained in the index, denial-of-access questions produced confident summaries without the controlling exception, and the monitoring dashboard counted responses rather than completed policy tasks.
The revised plan has three stages:
The decision log records a repair during stage one when two policies have duplicate effective dates. The team pauses expansion, fixes the source manifest, reruns the affected tests, and records the new index hash. The launch date moves by two days. That is not a scheduling failure; it is the plan working as intended.
- Ten policy and support staff use a frozen index for five working days. Entry requires versioned sources, entitlement tests, and a working no-answer path.
- Fifty volunteers from three functions use the service for ordinary requests and four designed edge cases. Entry requires zero unresolved critical source-version errors and a successful rollback rehearsal.
- Wider internal access begins only after the owner reviews source-grounding errors, user challenges, support workload, and missing-source behavior.
How AI can help without owning the decision
AI can draft dependency questions, transform the boundary card into test cases, compare a runbook with incident scenarios, summarize evidence by gate, and identify unresolved owners or dates. Require every generated claim to link to a source or observed result. Keep acceptance criteria frozen, record material edits, and make the release owner sign the final gate record.
Two Prune can place a participant in a realistic launch-planning assessment with conflicting readiness evidence, operational constraints, and a required launch recommendation. It evaluates how the participant frames the decision, navigates evidence, uses AI, verifies claims, exercises judgment, and communicates the plan. It does not deploy products, approve releases, or make employment decisions.
Sources
GOV.UK Service Manual: How the live phase works
- GOV.UK Service Manual: Book a service assessment
- NIST AI Risk Management Framework Core
- NIST AI 600-1: Generative Artificial Intelligence Profile
Sources: [1] [4] [2] [3]
Sources
- 1.GOV.UK: How the live phase works · UK Government
- 2.NIST AI RMF Core · National Institute of Standards and Technology
- 3.NIST AI 600-1: Generative AI Profile · National Institute of Standards and Technology
- 4.GOV.UK: Book a service assessment · UK Government