Screening and exclusion trail
8 min read
Lesson objectives
- Screening and exclusion trail
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
After searching for the effect of remote education support, you have 26 full texts. Screening can look like quick deletion, but it decides which evidence enters the synthesis and which findings disappear. Eligibility must follow the decision question rather than the direction of a result. Here the population is adult learners in vocational programs; the intervention is remote educational support compared with usual support or a specified alternative; the outcome is completion of a task within one term. A report only on teachers' satisfaction may inform implementation but does not answer the completion-effect question. A study of schoolchildren does not automatically transfer to the intended population. Write inclusion and exclusion rules that another reviewer can apply before knowing a paper's result.
Distinguish record, report, and study to avoid artificial weight. A record is a database or register entry; several records can point to one paper. A report is a document describing a study. One study may have an initial paper and a follow-up report. The study is the underlying participant group and design. Counting R1 and R2 as two studies inflates the evidence base and risks counting the same people twice. Yet R2 should not be thrown away as merely duplicated; it may contain accessibility data absent from R1. Build a linkage table with study identifier, report identifiers, population and intervention, outcomes and times, and access status. R1 and R2 belong to study S1; R5 belongs to S2. Three documents may be useful for one or more questions, but only two independent studies inform the cross-study comparison.
At title and abstract screening, retain uncertain records for full-text inspection. At full text, give each excluded report a specific primary reason, keeping secondary reasons if they matter to a different question. “Irrelevant” is not enough. “Schoolchildren rather than adult learners” or “no comparison and no completion outcome” can be checked. Use an ordered reason hierarchy so two screeners do not encode the same decision under incompatible labels. An unavailable text is an access status, not a scientific exclusion. If a corrected article or supplement becomes available later, update the record with a date and version instead of erasing the earlier decision. The audit trail should show why the count changed.
The six-record solution also tests the precision of the rule. Suppose R5 measures completion after five weeks rather than at the end of a term. Should it be excluded? That depends on the recorded wording. If “within one term” is a maximum follow-up window, it may enter with its shorter observation time clearly shown. If the protocol requires a measure specifically at term end, its five-week result cannot enter the main comparison, though it may inform short-term context. Do not reinterpret the rule after seeing that R5 is negative. Document the decision and apply it to all similar papers. A vague phrase in eligibility criteria can create bias even when each individual screening decision sounds plausible.
If two reviewers disagree on R6, do not let the louder one decide. Identify the missing facts: is there a comparator, is the sample independent of R1, and does the outcome measure task completion rather than logging into the platform? Request a supplement or ask the author if the deadline allows, recording the request and reply date. If no answer arrives, keep R6 unresolved and show two scenarios: how might the conclusion change if it is eligible, and what remains if it is not? Do not place an estimate with an unknown denominator into a numerical average. Do not erase the record from the selection trail merely to make the flow look tidy. Disclosing unresolved eligibility is part of honest screening.
Give the decision table explicit columns: record identifier, source, report identifier, possible study identifier, population, intervention and comparator, outcome and time, eligibility judgment, reason, and reviewer. Leave the study identifier blank for R6 if identity is unknown. Add the question each document informs: completion effect, accessibility, or implementation cost. A report can enter one question and be excluded from another. The bare word “excluded” might falsely suggest that it has no use at all. For R2, a precise entry is “additional report of study S1, informs accessibility; not an independent effect study.” That is better than simply “duplicate,” which risks deleting unique information. Later, the linkage lets the synthesis identify which outcomes belong to which participant group.
Recheck a sample of screening decisions without showing the second reviewer whether each finding is favorable, if that can be done. Suppose negative studies were often excluded as “different setting” while positive studies from the same type of setting remained. Reapply the rule consistently. This check does not make the entire review independent; it tests a particular route by which preference could affect selection. Record how many rows were checked and what reviewers disputed. With six records, a disagreement on R4 and R5 is more informative than a bare “two of six disagree.” It may reveal ambiguity in population or follow-up wording. Repairing the rule can prevent similar future disagreements rather than merely settling two rows by authority.
A reason hierarchy helps when a report fails more than one condition. Agree in advance to check population, then intervention and comparator, then outcome and timing. R3 may lack both a comparator and a completion outcome. Record “no comparator” as primary and “no completion outcome” as secondary. If a later supplement supplies a comparator, R3 does not enter automatically; the missing outcome still requires checking. If reason order changes halfway through the dossier, category totals may change while the studies do not. Preserve the classification rule and revision date. The sequence does not hide secondary reasons; it lets the selection summary reconcile with row-level records.
After screening, only two eligible studies may remain, one favorable and one unfavorable. A small evidence base is not permission to select the director's preferred result. Ask whether the two measure the outcome with the same definition and whether setting or study quality might explain disagreement. Those are questions for later extraction and appraisal. The honest output of this lesson is a clear eligibility table retaining both findings. If R6 later proves to be an independent negative study, its recorded pending status allows the team to update the synthesis. A transparent exclusion trail is therefore useful for correction and updating, not just for producing a polished diagram.
Reconcile the six full-text records before drawing a flow diagram. R1 and R5 enter the completion-effect question. R2 is an additional report of R1's study and informs accessibility. R3 and R4 are excluded from completion for recorded reasons. R6 remains unresolved. Do not summarize this as “three studies included and three excluded”: report count and study count differ, and unresolved is not excluded. Label the counting unit in each diagram box, then ask a colleague to reproduce the totals from the rows. A mismatch signals a linkage or status error, not a cosmetic diagram problem. If a later accessibility question arises, R2 can be reused, but its eligibility for that new question must be checked explicitly.
Lesson summary
Screening and exclusion trail