Involving users without claiming universal representation
17 min read
Lesson objectives
Involving users without claiming universal representation
Reading tools
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
You read meeting minutes describing an appointment interface as easy for everyone. Five users attended, selected by the product team from active account holders. A support employee asks who represented people who tried to register and withdrew. Nobody answers. Participants did not necessarily give wrong accounts; the decision claimed more from their accounts than they could support. Useful participation starts by identifying whose experience a decision needs and why. The OECD health data governance recommendation emphasizes stakeholder engagement and expectations. The CDC evaluation framework treats collaboration and equity as ongoing actions. We apply those principles to a service committee without treating a small session as a representative population survey or a source of legal authorization.
First establish what participants can influence. The organization may already have purchased the system and only be testing its instructions. State that boundary so users understand that this meeting will not reopen procurement. If the service scope remains open, allow discussion of alternatives, burden, and assistance. Prepare three actionable questions rather than asking for general satisfaction. Where do you stop while changing an appointment? What information would let you continue? Who do you expect to receive your request? These differ from asking whether someone likes the application. Answers can change a particular journey and reveal expectations exceeding actual capability. Preserve accounts contradicting the team's assumptions even when only a minority in the room raises them.
Purposeful participation and statistical sampling serve different questions. You may deliberately recruit six people experiencing a particular failure to understand how it happens. That is useful for investigating mechanisms, but it does not estimate prevalence. Estimating a rate needs a sampling design, denominator, and approach to missing responses. The two methods can work together: interviews explain a failure mechanism and separate measurement tests its extent. State which question each method answers. “Everyone who attended agreed” describes attendees and should include their number and recruitment method. “Users agreed” expands the claim to people never heard. Preserve the distinction in both languages so that the scope of a decision remains proportionate to its evidence.
Structure the session to observe actions as well as impressions. Give a participant a simple synthetic-record task, such as moving an appointment and finding its new status. Ask what they expect before selecting a control, then observe where they stop. Explaining the step during the test and later recording success as evidence of clarity would misrepresent the task. When help is needed, record its type, timing, and point of request. Ask afterward why the participant chose that action; screen movements alone cannot establish the reason. A pause might reflect unclear wording, fear of losing the appointment, or an unseen completion control. Different explanations lead to different repairs, making direct inquiry more useful than interpretation at a distance.
Time compensation and practical access need explicit arrangements. Session timing or travel cost may exclude people, creating apparent agreement because only those able to attend were heard. Choose arrangements consistent with institutional rules and explain them equally to invitees. Offer an alternative for people unable to attend, such as a short interview or assisted telephone task. Some participants need more time, readable material, or a trusted companion. Ask about participation needs without collecting unnecessary health details. Where appropriate, separate logistical records from participants' views. This protects privacy while preserving an explanation of who could attend, who was absent, and what the absence means for interpreting the evidence.
Analyze session material in traceable units. Record the observed action, the participant's explanation, the team's interpretation, and additional evidence needed. For example, a person presses Back twice, says they are looking for the original appointment, and the team hypothesizes that the page title caused confusion. A revised title or comprehension check is needed before that hypothesis becomes a finding. Group similar accounts into a clear theme while retaining contradictory cases alongside it. Most attendees may request a prominent confirmation button, while one reveals that a screen reader does not announce the changed status. Preserve important observations even when infrequent. Their relevance comes from potential consequence and connection to the decision, rather than vote count alone.
Link each theme to a proposed action, owner, and review point. If users describe a message as frightening, identify the wording or information order involved and test an alternative. If they request a feature outside scope, explain the decision and available alternative. Participation does not promise implementation of every suggestion, but it requires an understandable response describing what happened to the input. Use a “heard, decided, will test” record, with evidence for each statement. Avoid naming participants in a broadly shared presentation without an authorized purpose. A thematic summary can preserve disagreement without identifying detail. The data owner reviews disclosure, while the service owner confirms that promised actions are within actual operating capability.
Consider differences between sites. Users at a small location may know staff personally, while another location depends on a shared telephone number with changing receivers. Success at the first site might rely on human relationships rather than the screen. Ask what happened outside the system before task completion. Did an employee call the participant? Did the participant already know whom to ask for help? Record that context to identify what can transfer during expansion. Hiding assistance makes the interface appear independent of work that enabled success. When recommending broader use, specify the support accompanying the change. The same wording can be tested at another site with different support and its comprehension compared without claiming that this alone forms a complete causal experiment.
Close the participation loop by returning to the people heard. Explain what changed, what remains under test, and why some suggestions were deferred. Choose a format they can easily read or hear. If later review is planned, explain how to submit new observations and who receives them. Feedback on the summary may show that the team misunderstood an important point. Record that correction with the prior version's date. Returning to participants does not certify every conclusion; it creates an opportunity to detect a specific interpretive error. A practical sign of success is that participants can describe how their input affected the decision and its limits. If someone believes they gave blanket approval for the service, clarify the distinction and correct the wording creating that impression.
The committee can hear conflicting groups without putting other people's needs to a vote. Some users want more detail in reminders; others fear exposing the visit reason on a shared phone. Identify the underlying needs: remembering the task and protecting privacy. Design an alternative addressing both, such as a neutral reminder leading to protected detail. Qualified roles assess suitability for context. Test comprehension with each group to see whether the change solves the issue or shifts it elsewhere. Legitimate disagreement may remain and require a reasoned decision. Explain the tradeoff, the authority settling it, and evidence capable of changing it, rather than presenting artificial agreement that disappears when people use the service differently.
Lesson summary
Involving users without claiming universal representation.