Performance Diagnosis and Experience Hypothesis · Lesson 3

Build a Falsifiable Experience Hypothesis

9 min read

Lesson objectives

  • Specify a mechanism connecting practice and support to an observable workplace decision, with success evidence and a possible disconfirming observation.
Reading tools

After diagnosis, ‘We will build an interactive experience’ is insufficient. It names a format without explaining how performance might change. A useful learning hypothesis connects an action in the experience to a decision at work through a specific mechanism. For example, if handoff staff practice distinguishing a cue that requires a stop from one that permits continuation, and receive feedback about why the choice matters, they may pass fewer incomplete cases without a check. Why might this work? It asks the learner to retrieve a criterion under conditions resembling the work moment, see the consequence of missing the cue, and try a revised decision. That chain is a prediction, not an established result. Specify who participates, which decision matters, where the cue appears and what could block transfer into work.

At a fictional logistics center, the morning team hands 120 sensitive shipments a week to the evening team. Each record should include temperature, measurement time and receiver signature. Reviewers examine 60 records from two weeks: 12 lack a field, and seven of those pass to the next stage without a completion request. The learning manager proposes a lesson on ‘the importance of safety’. The operations supervisor notes that the handoff screen allows a blank field to be submitted. Three workers say they do not know whether missing measurement time alone requires a stop. The evidence suggests two competing mechanisms: an unclear decision rule and a technical path that lets incomplete records through. Learning may address the first but cannot by itself fix the second. Before selecting content, ask whether an approved operating rule specifies when to stop and who may make that decision.

Write the hypothesis precisely: ‘For staff performing handoffs, practice on three graduated cases with a missing field and an ambiguous case, plus feedback explaining the stop-and-review rule, will improve correct stop decisions in fresh simulations and may reduce incomplete records passed at work, provided the interface displays the fields before submission and permits a completion request.’ The first result can be checked promptly in cases unseen during practice; the second needs workplace observation. The condition matters: staff cannot apply a criterion that the screen conceals. Do not promise a 30% safety gain without a basis for that estimate. Learner satisfaction with the activity is not a substitute for a correct handoff decision.

Now separate inputs, mechanism and outputs. Inputs are three short cases: a complete record, one missing measurement time and one with an illegible signature despite an otherwise acceptable reading. The acceptable range itself belongs to the fictional organization's approved procedure; do not invent a number and present it as a universal standard. The mechanism is repeated choice with immediate explanation. Staff select ‘continue’ or ‘stop and request completion’, then see which item of information controlled the decision. The proximal result is correct decisions on fresh cases with different field order and wording, so the check does not reward memorizing a screen. The distal result is the proportion of incomplete workplace records that pass, balanced against delay to valid shipments. If staff stop everything, missing-record errors may fall while operations suffer unnecessarily.

Disconfirmation is useful design information, not an embarrassing failure. Suppose practice improves simulated decisions but incomplete workplace cases still pass when shifts are busy. Investigate whether time, authority or incentives prevent application. If the simulation does not improve, inspect rule clarity, feedback and example variety before making a longer lecture. If passed incomplete records fall before any practice after the screen repair, do not attribute the result to an activity that has not happened. State these possibilities before seeing the results; otherwise almost any change can be recast as success afterward. Keep dates of screen changes, policy changes and practice delivery so the sequence remains interpretable.

Put a small evidence plan beside the hypothesis. The first row asks whether a worker can recognize a case that requires a stop. Its evidence is a fresh response scored against a stable criterion, not a count of attempts. The second asks whether the interface reveals the fields and allows a completion request. Its evidence is a task test recording what the worker actually sees and when the cue appears. The third asks whether the decision transfers to work. Its evidence is a review of real records with change dates and shift context. Do not compress these into one number. Learning can improve without transfer, and records can improve because of a system constraint without a skill change. When results diverge, the team gains a precise repair question instead of a vague demand for more training.

Protect the simulation against an easy pattern guess. If every case with a blank field means ‘stop’, a participant can learn the appearance of a question rather than the work rule. Include a case with a recorded measurement and legible signature that permits continuation, one requiring clarification because a record is unreadable, and one in which time pressure conflicts with the stop condition. Do not make an operational or ethical dispute mysteriously ambiguous; provide the relevant fictional policy in the case. Feedback identifies which condition was met and which was not, explaining why an acceptable temperature alone does not resolve an unknown measurement time. Then vary the surface details in a fresh case while retaining the same principle, so the check tests decision transfer rather than option-order memory.

For a before-and-after comparison, hold denominators as consistent as possible. Seven records passed from 60 reviewed in the first period, and one from 60 in the second. The descriptive difference is six cases or ten percentage points, but the mix of sensitive shipments or sampling method may have changed. Also record how many records were incomplete to begin with in each period. If the screen makes measurement time mandatory, fewer incomplete records may even be available for review, so the denominator ‘incomplete records’ alone cannot sustain a comparison. Explain the process change. A useful safety measure may be incomplete cases released without review divided by all eligible handoffs, accompanied by the counts so a small percentage is not read without scale.

End the hypothesis with a decision if a precondition fails. If operations has not approved the rule for an illegible signature, stop authoring a one-answer item and settle that policy first. If the application offers no completion-request action, do not punish workers in a scenario for failing to select a nonexistent button; redesign the task or specify the required product change. If managers will not allow time for review, record how that limits transfer and test a shorter work-embedded path. These are not excuses to abandon learning. They are accountability boundaries: what the activity can plausibly improve, what the organization must provide, and which evidence will separate those responsibilities during review.

Lesson summary

Specify a mechanism connecting practice and support to an observable workplace decision, with success evidence and a possible disconfirming observation.