Define categories before counting them
5 min read
Lesson objectives
- Apply define categories before counting them to a case with defined evidence.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
A manager asks for the number of “late cases.” Before extracting it, define a case and its clock. Intake staff may call a submitted web form a case; an operations team may start a case only when documents are complete. If one clock begins on submission and the other on acceptance, both groups can produce correct numbers for different definitions. Critical analysis begins by exposing that mismatch, not accusing either group of bad faith. An operational definition specifies how records enter the numerator and denominator.
Write an inclusion and exclusion rule another reviewer can apply to raw records. For example: “An eligible case is a service request submitted this quarter with complete documents, not withdrawn; time starts when it becomes complete and ends when the decision is communicated.” That may suit evaluation of the operations team, but it does not measure the customer's whole journey. To assess customer experience, begin at first submission and include time waiting for documentation, with withdrawn requests reported separately. No definition is best for every decision. It must fit the claim and remain stable across the comparison or have its change clearly explained.
Test boundary cases before using the definition widely. A request submitted on the last day of one quarter and completed in the next: which period includes it? A request returned twice for missing information: one case with three stages or three cases? Two requests from one person for different services: separate or combined? These are not cosmetic wording choices; they can materially change rates. Construct included, excluded, and uncertain examples, then ask a second reader to classify them from the rule alone. If readers disagree, repair the definition or create a review-needed category instead of concealing the ambiguity in the final count.
Watch for a changing meaning during discussion. “Late” may initially mean beyond a promise to the customer and later mean beyond a shorter internal target. Both sides may then argue correctly about different propositions. Write the two metrics explicitly, each with its own threshold and denominator. Likewise “quality” may mean decision correctness, completion speed, accessibility, or satisfaction. A claim about overall quality needs a stated relation among its dimensions, not an unsupported jump from one metric.
A definition can carry a value choice, not merely a neutral classification. A team might automatically close a request as “complete” when the customer has not replied for a day. Closure rate rises, but the underlying need may remain unresolved. The closure rule can still be legitimate, yet “administratively closed” is not equivalent to “customer need solved.” Separating the administrative event from the intended outcome avoids circular praise of performance when the performance label itself is set for easy attainment. Report closure and verified resolution separately where the decision requires both.
If two reviewers disagree, a single overall agreement rate can hide the problem. Suppose both classify most of one hundred cases as “not late” and agree on eighty-five, yet disagree on many of the ten cases near the time limit. A high overall rate may be driven by the common easy category, while the cases that trigger escalation remain contested. Inspect a cross-table of classifications and the disputed records. Is the timestamp missing, is the five-day limit ambiguous, or did one reviewer start at submission while another started at document completion? Repair the repeated source of disagreement and reclassify affected cases. Do not rewrite every old record for a problem confined to a narrow class, and do not treat renaming a category as a fix for a missing timestamp. This checks consistency of application; it still does not prove that the measure answers the customer's question.
Lesson summary
Define categories before counting them: the lesson links the check to a worked case and practice.