Reproducible evidence search
7 min read
Lesson objectives
- Turn a practical problem into a researchable question and documented evidence-search plan
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
Once a question is defined, do not search with a few improvised words and then keep only papers that confirm an expectation. Translate question components into searchable concepts. For each concept, list synonyms and terms used in neighbouring fields. In the shift and departure question, an exposure may be called flexible scheduling or flexible working hours; the outcome may be employee turnover or retention. Do not connect every term with AND: that requires all terms to appear and can hide relevant studies. Group synonyms with OR within a concept, then combine essential concepts with AND, using parentheses appropriate to the database. Test the draft expression against a known relevant record. If it fails, log the change and its reason.
Choose sources before searching and explain their role. A disciplinary database may cover peer-reviewed studies in one field, whereas a broader index may reveal reports or work from adjacent fields. Coverage varies; no single selected source is “all the literature.” Record source names, search dates, language and year limits, and document types. Limits can save effort while excluding relevant evidence, so justify them against the question. A title-only search is not an abstract or full-text search. If the expression changes after results appear, retain the earlier expression and the count it returned rather than erasing the trial.
Keep retrieval separate from selection. Retrieval supplies candidate records, including duplicates and irrelevant items. Selection applies predeclared eligibility rules for population, study type, outcome, period and publication type. Write exclusions another reviewer can apply. “Weak study” is a premature and vague eligibility rule; “does not measure employee departure” is checkable. Title-and-abstract screening may precede full-text assessment. Record a reason when excluding a full text so another reader can reconstruct the decision. PRISMA 2020 describes transparent search and flow reporting for systematic reviews. Borrowing that transparency for a bounded evidence scan does not turn the scan into a systematic review.
Plan duplicate handling and updates. Keep a record identifier or URL, title, year and discovery source; flag duplicates before counting studies. One report may appear in two indexes. A conference abstract and a full article may describe the same study; similarity alone is insufficient to delete either without checking. Add a full-text status and reviewer field. If the decision will be made months later, schedule an update or state plainly that the evidence search covers only its original date. A report written today does not make an older search current.
Test the plan with three questions. Could a colleague reproduce the search from the log? Would the eligibility rules explain why one study remains and another goes? Does the report state what the selected sources did not cover? If not, repair the log or limit the conclusion. A large result count does not prove there is no evidence gap: it may reflect broad words or duplicate records. A research gap need not be a total absence of publications. Differences in population, measurement or design may leave existing studies unable to answer the present decision.
When search terminology differs across Arabic and English, translating one word is not enough. Keep field-specific variants and spellings for each language, and note which database supports which language or translated fields. An Arabic report may use “دوران العاملين” where an English paper says turnover. An administrative “departure” may include internal transfer. Bilingual retrieval widens the search; it does not automatically make records conceptually comparable. Define the construct during screening, not only at final counting.
Backward or forward citation searching can help: inspect references of a relevant paper or later work citing it where tools permit. Log that as a separate discovery route instead of pretending those records came from the original database query. Check duplicates and later versions too. A conference abstract may become a full article with changed outcome details. The record count is not the study count until records representing the same investigation are reconciled.
Declare boundaries before findings: unavailable databases, excluded languages, inaccessible full texts and cutoff date. If the commissioned task is a time-limited initial scan, name it that. Transparency does not require imitating every step of a large systematic review; it prevents readers from mistaking a bounded scan for a comprehensive one. If a decision needs more complete evidence, propose a systematic review with a protocol, team and adequate time. Escalation is a professional response to scope, not a failure of the initial scan.
Lesson summary
Reproducible evidence search: document concepts, synonyms, Boolean combinations, sources, dates and selection.