AI-enabled pharmaceutical work: build a context-of-use evidence record

Define one context of use

Describe the task as an information transformation that supports a named decision. “Summarize study data” is incomplete. “Prepare a first-pass table of protocol deviations from an approved dataset for review by the study team” identifies the input, output, owner, and downstream use. Record the product or programme, lifecycle stage, population, jurisdiction, effective date, and decision authority.

Do not carry an approval from one context into another. A workflow used to retrieve public literature does not automatically qualify for clinical-data analysis, safety case processing, manufacturing investigations, promotional review, or regulatory evidence generation. Similar interfaces can hide different risks.

The FDA and EMA's current Guiding Principles of Good AI Practice in Drug Development emphasize a clear context of use, a risk-based approach, data governance and documentation, performance assessment, lifecycle management, and essential information. They are guiding principles for drug and biological product development, not a finding that a particular workflow is acceptable.

Sources: [1]

Build the evidence record

Create one record for each workflow and version. Include:

The record should make assumptions visible. If a source lacks a version, if a term is ambiguous, or if the AI cannot distinguish a draft from an approved document, the output should not silently fill the gap.

• Decision supported: the exact judgment, action, or document the output informs.
• Authorized users: the roles that may run, review, approve, and release the work.
• Approved inputs: source systems, document states, data classes, dates, and exclusions.
• Transformation: retrieval, extraction, classification, calculation, drafting, or prediction.
• Expected output: required fields, uncertainty labels, citations, and formatting.
• Principal failures: omissions, source confusion, leakage, unsupported inference, or stale information.
• Verification: test cases, reconciliations, calculations, specialist review, and acceptance rules.
• Traceability: source-to-output links, model and prompt version, corrections, and disposition.
• Change trigger: data, model, process, regulation, population, or purpose changes that require review.

Worked artifact: a synthetic protocol-deviation table

A fictional study operations team wants an AI assistant to create a first-pass protocol-deviation table. The input folder contains the current protocol, a superseded protocol, site notes, an export of visit dates, and an unsigned clarification memo.

The first output classifies three visits using the superseded visit window and treats the unsigned memo as an approved exception. The team records the workflow this way:

| Record field | Pilot decision | | --- | --- | | Decision supported | Reviewer prioritization of records for manual assessment | | Approved inputs | Current approved protocol, controlled visit export, dated site records | | Exclusions | Superseded protocol and unsigned clarification memo | | Transformation | Extract dates, calculate windows, identify records needing review | | Output boundary | First-pass table only; no final classification or regulatory conclusion | | Verification | Recalculate every date rule, sample source links, compare against labeled cases | | Traceability | Store protocol version, row-level source, calculation rule, reviewer correction | | Stop trigger | Wrong protocol version, missing source, changed rule, or unexplained classification |

The repaired workflow uses a controlled manifest that marks document status and effective date. It calculates visit windows from an independently checked rule table and routes ambiguous records to a study professional. The team retains the failed version because it shows what the controls must catch. This example is synthetic and is not clinical, regulatory, medical, or quality advice.

Test the evidence path, not only the prose

Use representative safe data and deliberate traps. Include a superseded document, a conflicting source, a missing value, a unit mismatch, a duplicate record, and an out-of-scope request. Check whether the workflow identifies the issue, preserves the original source, labels uncertainty, and routes the exception to the right owner.

The EMA's adopted reflection paper on AI in the medicinal product lifecycle discusses AI and machine learning across medicine development and post-authorization settings. It distinguishes contexts and discusses traceability, data, performance, and lifecycle considerations. Teams should consult the full paper, current applicable requirements, and relevant specialists rather than treating this resource as an implementation guide.

Measure the failure modes tied to the task. Useful measures may include source-link accuracy, extraction error by field, calculation reconciliation, unresolved-conflict rate, reviewer corrections, and reasons for escalation. A high average score can conceal a critical failure, so record errors by consequence and category.

Sources: [2]

Decide, monitor, and reopen the record

Assign the workflow a disposition of approve for the defined pilot, revise and retest, hold for missing evidence, or exclude. Approval should name the exact workflow version, data boundary, reviewer, acceptance rule, monitoring owner, and change triggers. It should not authorize adjacent use cases.

After deployment, sample outputs and corrections at a frequency proportionate to the task. Investigate changes in source quality, user behavior, error distribution, or tool configuration. Reopen the record before expanding to another lifecycle stage, dataset, population, team, or decision.

Assess pharmaceutical evidence judgment in a bounded simulation

A controlled assessment can provide a mixed-version protocol pack, tabular records, an AI-generated deviation table, and a deadline. Observable evidence may include whether the participant defines the context of use, excludes superseded sources, checks calculations, preserves traceability, escalates ambiguity, and communicates the limits of the output.

Report only what the participant demonstrated in the scenario and which evidence supports the assessment. Do not infer pharmaceutical expertise, professional authorization, regulatory acceptability, patient safety, product quality, or future work outcomes from one exercise. Two Prune can organize evidence from a synthetic or explicitly approved non-sensitive assessment. It does not conduct regulated pharmaceutical work, certify systems, or make employment decisions.