Weigh Conflicting Evidence and Record a Bounded Decision
6 min read
Lesson objectives
- The learner builds an evidence-confidence ledger separating observed facts, interpretations and unrepresentative samples, then states an interim decision and evidence that could reverse it.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
A service problem rarely comes with one consistent source. A system may record 'document missing', applicants may report confusing wording, and workers may believe the review queue is the real cause. Do not resolve the conflict by choosing whichever source supports the preferred project. Use an evidence ledger with five columns: precise claim, source and period, what the source observed directly, its bias or gap, and what would change the decision. A return code is a fact about a worker's classification, not necessarily proof that a required document was absent. An applicant interview can explain an important experience but cannot alone estimate its prevalence among all applications. A worker can expose a process failure absent from the system but may know only a particular shift. Select a sample that follows an application from start to end, checking submitted evidence, the rule version and the time it passed between teams. If reason codes are wrong, improve data quality before making a wide policy change. Keep 'what we know', 'what we infer' and 'what remains unknown' in distinct parts of the brief.
The apparent conflict may lie in units rather than facts. A service log says 120 of 200 returned applications were coded as evidence problems. Twelve of 15 interviewees say they did not understand the request, while a supervisor says some cases waited days for review. The first figure is 60% of returned applications with that code, not 60% of all applicants. The second is 80% of fifteen people recruited for interviews, not a population rate. The third needs timestamps and case distribution. Put a denominator and selection method beside every finding. Choose tests that separate explanations: audit a distributed sample of coded cases, test understanding of evidence wording among varied users and compare receipt and review times. Where a case says 'evidence missing' even though a document arrived, reason coding or review has failed. Where users understand the words but cannot obtain the requested proof, the evidence rule itself may be burdensome. Give each interpretation provisional weight according to whether the source can answer that particular question; do not pretend that all sources are interchangeable votes.
State a decision rule before gathering more evidence so the team does not tailor it to a favored result. For example, do not change an eligibility condition until its policy owner confirms an acceptable alternative; try clearer wording without changing the rule; do not withdraw assistance while its users' abandonment remains unexplained. Define a small reversible action and review date: improve instructions, train staff on reason coding and inspect cases from every channel over a set period. Check whether the audit sample is random or selected from complaints; do not multiply its error rate into the entire population without justification. When evidence conflicts, 'choose a test that distinguishes causes' may be a stronger brief than 'choose a final solution'. State the cost of delay too: an unnecessary burden may continue, so identify what can safely improve now and what needs authority or study. Name an owner and stop condition for each step. After the trial, return to the ledger and record why the recommendation changed if it did.
An evidence ledger is not a numerical vote among sources. Ask which source fits the question. A complete transaction log is well suited to counting codes in a period, but weak on cause if coding is poor. A detailed interview can find where wording confuses people, but not estimate how frequently that happens. An independent file audit is closer to classification accuracy, but needs a defensible sampling frame and a checked rule. Qualitative and quantitative evidence together can guide a stronger test without making a causal claim by themselves. Put a 'claim not yet supportable' entry in the brief, such as '60% of applicants fail to provide the document'; that protects public language from a denominator error. Also state 'enough evidence for a limited action': finding documents in files coded missing justifies correction of those cases and prompt investigation of the process. Do not wait for a perfect study to correct a visible mistake, and do not broaden that correction into a rule change for everyone without authority and evidence.
Lesson summary
Compare what each source directly observes and cannot establish, correct confirmed cases promptly, and test wording against review delay before changing the policy rule.