Learning experience platforms: separate learning activity from demonstrated AI capability

Define the learning decision first

Name the decision the programme owner needs to make. “Measure AI learning” is too broad. “Determine whether a cohort can apply source verification after a research module, using a common simulated task and human-reviewed evidence” identifies a construct, population, and evidence route.

Document the platform, edition, enabled integrations, identity model, content catalogue, recommendation logic, event standard, reporting fields, and data-retention rules. Confirm these in the live environment. Do not claim that a learning experience platform can or cannot record a behavior without checking its current product, plan, configuration, permissions, and connected tools.

The 1EdTech standards catalogue describes standards such as Learning Tools Interoperability, Caliper Analytics, and Question and Test Interoperability. These standards can support exchange of tools, activity data, assessments, and results. Standards describe structures and interfaces; they do not establish that a particular event measures AI fluency or proves learning transfer.

Sources: [1]

Build the activity-to-evidence map

For every programme claim, record:

The map prevents a familiar substitution: treating attendance, clicks, or completion as evidence of applied capability. Activity data may show exposure and participation. A work artifact may show what was submitted. A controlled task can create a comparable opportunity to observe how someone framed, checked, and delivered the work.

• Learning objective: the specific behavior the programme intends to develop.
• Learning opportunity: the lesson, practice, discussion, or coached exercise offered.
• Platform event: the activity record available, such as launch, view, response, submission, or completion.
• Work artifact: the actual output, revision, source check, calculation, or decision produced.
• Interpretation limit: what the event or artifact cannot establish on its own.
• Performance task: the bounded situation in which the behavior should be observable.
• Review rule: the rubric, evidence requirements, missingness treatment, and reviewer.
• Decision use: development conversation, curriculum revision, or another defined purpose.
• Recheck trigger: changed content, integration, event schema, rubric, population, or platform configuration.

Worked artifact: a synthetic source-verification module

A fictional AI academy offers a module on verifying claims. The platform records video completion, a knowledge-check score, discussion participation, and a downloaded checklist. Leaders initially report that everyone who completed the module can verify AI-generated research.

The team maps each signal:

| Map field | Recorded interpretation | | --- | --- | | Learning objective | Identify unsupported claims and verify them against authoritative sources | | Learning opportunity | Video, guided example, checklist, and practice discussion | | Platform events | Completion, quiz response, post, and checklist download | | Interpretation limit | Events show participation, not independent performance on a new task | | Performance task | Review a synthetic briefing with mixed-quality citations and a deadline | | Work artifact | Corrected briefing, claim-source table, unresolved issues, and rationale | | Review rule | Credit only checks visible in the submitted work and available process evidence | | Decision use | Target follow-up practice and revise the module, not rank participants |

The programme keeps its completion dashboard but stops calling it capability evidence. It adds a small post-learning simulation and reports observed verification behavior separately from platform activity. The example is synthetic and does not represent measured customer outcomes.

Check the event and identity chain

Test the actual data path from the learning tool to the platform and analytics store. Include duplicate launches, offline activity, a shared device, a late event, a retried quiz, an external tool, and a participant whose identity mapping changes. Confirm what each event means, which system emitted it, and whether ordering or duplication changes the report.

The 1EdTech Caliper Analytics overview explains that Caliper provides common vocabulary and profiles for learning activities such as assessments, media, reading, sessions, and tool use. It supports consistent activity records across environments. A Caliper event remains an activity description. Its educational meaning depends on the learning design, context, event implementation, and interpretation.

Where an integration claims standards support, verify the relevant version and certification rather than assuming every field or workflow is implemented. Preserve source system, event time, actor, activity, object, and transformation history where available.

Sources: [2]

Validate the inference, not only the dashboard

Write each programme claim beside the evidence that could support it. “Opened three resources” can be supported by valid activity events. “Selected the most authoritative evidence” requires a task where source choice is visible and evaluated. “Transfers the skill to work” requires broader evidence than one learning-platform report or one simulation.

Use dispositions such as activity supported, artifact supported, task-bounded performance observed, insufficient evidence, or remove. Track missing opportunities separately from poor performance. If the task never required a source check, absence of that check is not evidence of inability.

Set a narrow operating boundary

Approve the measurement design only for the named programme, objective, event path, task, and decision use. Record who can access learner data, who reviews work, how missing records are handled, and when the design must be recalibrated. Do not reuse a convenient activity metric as a proxy for unrelated capabilities.

Reassess after platform, content, integration, identity, event, rubric, population, or data-policy changes. A standards-conformant event stream can improve interoperability without validating the learning claim attached to it.

Assess learning transfer with a bounded simulation

A controlled assessment can give every participant the same synthetic research pack, AI-generated draft, verification requirement, and time limit. Observable evidence may include source selection, contradiction detection, claim correction, uncertainty management, and final communication.

Two Prune can report task-bounded evidence and confidence limitations from that simulation. It does not certify the learning platform, prove programme impact, establish long-term transfer, predict future work performance, or make employment decisions.