Competing causal hypotheses and discriminating tests

If 40% of one segment's orders arrive late, that rate describes the outcome but does not explain it. Possible mechanisms include inaccurate available-stock records, an optimistic promise rule, insufficient packing capacity on peak days, or late event entry despite actual on-time receipt. Each mechanism predicts a pattern in the evidence; that is what makes it a useful hypothesis. “The warehouse is weak” is an accusation without a test. “Late orders with reserved stock cluster just after the pick-wave cutoff while active picking time remains stable” predicts something that can be confirmed or contradicted.

Build a small tree of possible causes, rather than an unlimited list: unrealistic promise, unavailable stock, processing capacity, transport and measurement quality. Specify what each branch would look like. A promise-rule problem may differ by displayed lead time despite similar execution. An inventory-record problem should show successful reservation followed by failed picking due to count or location. A capacity problem may show a queue rising with volume and falling when a shift is added. A carrier problem should appear after its receipt scan while preparation time stays steady. These signatures need not be independent; an inventory error may trigger rescheduling and then overload capacity.

Rank tests by discriminating power, cost and operational safety. Average transport-time reporting is cheap but may not separate an unrealistic promise from a slow carrier when delivery distance changes. Joining original promise dates to reservation, preparation and receipt events is more informative but requires data cleaning. A week of manual tracking may reveal a logging gap, yet workers may behave differently when watched. Select a test that could genuinely strengthen one explanation and weaken another. Before seeing results, write how each of two plausible outcomes would change the decision; this reduces the temptation to reinterpret every observation as support for the sponsor's favorite remedy.

A fair comparison controls important differences. When delays rise during a promotion, higher order volume alone does not prove inadequate capacity: the product mix may require more packing work, or carrier regions may differ. Compare orders from the same product, channel and region across days with different loads, checking that the promise rule stayed stable. If a supplier and software system both changed on the same day, a simple before-and-after result cannot credit one. A limited test may use comparable allocation groups if operations and fairness permit, or a staged rollout with a similar comparison segment; record remaining bias rather than claiming randomized proof.

Close the diagnosis with confidence tied to observable evidence rather than a feeling. For example: “Complete timestamps for 78 of 90 late orders suggest pick-release waiting is the largest path in this segment, but the 12 missing release events could change its size; an added permanent shift is not yet justified.” A sensible interim recommendation can be a measurement repair or a low-risk promise-rule adjustment instead of capital expenditure. When evidence is insufficient, postponing an expensive decision and specifying a discriminating test is a real consulting outcome, provided the test has an owner and decision date.

Set a stopping rule before the trial. If you change a promise rule for a limited segment, monitor cancellations and conversion as well as the reported late rate: a longer promise can improve the metric without speeding service. Run the trial long enough for the normal order cycle, fix the success definition and state when the decision returns to the sponsor. Look for contrary evidence. If lateness falls just as much in a segment whose promise was unchanged, seasonality or the carrier may explain the result. A test must be capable of disappointing your hypothesis; that is useful information before a larger investment.