Appraise evidence quality and conflicts

Service evidence appears contradictory when sources measure different things. Eight interview participants may call submission “easy” while records show many contacts after submission. Do not favor the larger number or louder stakeholder first; define the unit of each source. An interview captures a participant’s perception at a moment, analytics record a technically defined event, and support contacts depend on who knows that channel. Timing also matters: submission may have improved while later verification remains confusing. In a professional evidence ledger, give each row a precise claim, source, collection method, period, included and missing groups, measurement reliability and the decision it could affect. This turns disagreement into a research task instead of permission to select the sponsor’s preferred account.

Keep three layers apart. Recorded observation: “Of 50 classified contacts, 18 concern missing confirmation.” Interpretation: “Perhaps the message was not delivered, or visitors do not understand what confirmation means.” Decision: “Test delivery and comprehension before adding another channel.” The ledger cannot jump from observation to decision without a rival explanation. Contacts could mean no message was sent, it reached an unexpected folder, or its text omits the next step. Use a discriminating check for each: authorized send logs, consented observation of message receipt and a comprehension test. If management cites high average satisfaction, ask when it was measured and who could respond; someone unable to reach the survey is absent from its result. Different sampling frames should be made explicit, not treated as a personal failure of the researcher.

Evidence weight is not a fixed decorative confidence score. Declare criteria before ranking: proximity to the behavior at issue, clarity of definition, coverage of affected groups, recency, independent explanation and reproducibility. A small reversible decision may justify a bounded trial on a strong signal with guardrails. An expensive change or one that excludes a group calls for stronger checking. Many rows do not imply independent sources: three dashboards derived from the same event are one lineage. Record likely biases as well. Staff may see escalations only, an interviewee may remember a striking incident, and a system may lose events during network outages. These limitations do not automatically invalidate evidence; they delimit what it can support.

When a real conflict appears, write: “What observation would change my recommendation this week?” This makes the recommendation corrigible. Choose a test that distinguishes causes, not a large collection of additional data without a decision rule. If a team calls repeat applications “user error” but repeats rise after a step requiring internal interpretation, inspect the wording and data handoff before launching an education campaign. State what remains unknown and what a wrong decision could cost: a bounded test, extra work for staff or exclusion of an assisted channel. Name the owner of the conflict test, its review date and a threshold for reversing the choice. Evidence then becomes accountable rather than a set of persuasive slides.