Skip to Main Content

Systematic and systematic-like review toolkit

Toolkit of resources to support researchers in the development of systematic and systematic-like reviews

Step 1: Formulating the research question

The first stage in a review is formulating the research question. The research question accurately and succinctly sums up the review's line of inquiry. This page outlines approaches to developing a research question that can be used as the basis for a review.


Research question frameworks

Systematic ReviewIntegrative ReviewRapid Review

It can be useful to use a framework to aid in the development of a research question. Frameworks can help you identify searchable parts of a question and focus your search on relevant results

A technique often used in research for formulating a clinical research question is the PICO model. Slightly different versions of this concept are used to search for quantitative and qualitative reviews.

The PICO/ PECO  framework is an adaptable approach to help you focus your research question and guide you in developing search terms. The framework prompts you to consider your question in terms of these four elements:

P: Patient/ Population/ Problem

I/E: Intervention/ Indicator/ Exposure/ Event

C: Comparison/ Control

O: Outcome

 

For more detail, there are also the PICOT and PICOS additions:

PICOT - adds Time  

PICOS - adds Study design

 

PICO example

Consider this scenario:

Current guidelines indicate that nicotine replacement therapies (NRTs) should not be used as an intervention in young smokers.  Counselling is generally the recommended best practice for young smokers, however youth who are at high risk for smoking often live in regional or remote communities with limited access to counselling services.  You have been funded to review the evidence for the effectiveness of NRTs for smoking cessation in Australian youths to update the guidelines.

 

The research question stemming from this scenario could be phrased in this way:

In (P) adolescent smokers, how does (I) nicotine replacement therapy compared with (C) counselling affect (O) smoking cessation rates?

PICO element Definition Scenario
P (patient/population/problem) Describe your patient, population, or problem adolescent smokers
I (intervention/indicator Describe your intervention or indicator Nicotine Replacement Therapy (NRT)
C (comparison/control) What is your comparison or control? counselling
O (outcome) What outcome are you looking for? smoking cessation / risk of continued nicotine dependency

Alternative frameworks

PICO is one of the most frequently used frameworks, but there are several other frameworks available to use, depending on your question.

Question type

Structuring qualitative questions?

Try PIC or SPIDER:

  • Population, Phenomena of Interest, Context
  • Sample, Phenomenon of Interest, Design, Evaluation, Research type   

Cooke, A., Smith, D., & Booth, A. (2012). Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qualitative health research, 22(10), 1435-1443.

Question about aetiology or risk? 

Try PEO:

  • Population, Exposure, Outcomes

Moola, Sandeep; Munn, Zachary; Sears, Kim; Sfetcu, Ralucac; Currie, Marian; Lisy, Karolina; Tufanaru, Catalin; Qureshi, Rubab; Mattis, Patrick; Mu, Peifanf. Conducting systematic reviews of association (etiology), International Journal of Evidence-Based Healthcare: September 2015 - Volume 13 - Issue 3 - p 163-169.

Evaluating an intervention, policy or service? 

Try SPICE:

  • Setting, Population or Perspective, Intervention, Comparison, Evaluation

Booth, A. (2006), "Clear and present questions: formulating questions for evidence based practice", Library Hi Tech, Vol. 24 No. 3, pp. 355-368. https://doi-org.ezproxy-b.deakin.edu.au/10.1108/07378830610692127

Investigating the outcome of a service or policy? 

Try ECLIPSE:

  • Expectation, Client group, Location, Impact, Professionals, SErvice  

Wildridge, V., & Bell, L. (2002). How CLIP became ECLIPSE: a mnemonic to assist in searching for health policy/management information. Health Information & Libraries Journal, 19(2), 113-115.


Working out prevalence or incidence? 

Try CoCoPop:

  • Condition, Context, Population

Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015). Methodological guidance for systematic reviews of observational epidemiological studies reporting prevalence and cumulative incidence data. International journal of evidence-based healthcare, 13(3), 147-153.

Determining prognosis?

Try PFO:

  • Population, Prognostic Factors, Outcome

Conducting an economic evaluation? 

Try PICOC:

  • Population, Intervention, Comparator/s, Outomes, Context

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: a practical guide. Blackwell Pub.


 

Tip

JBI recommends the PCC (Population (or Participants), Concept, and Context) search framework to develop the research question of a scoping review. In some instances, just the concept and context are used in the search.

The University of Notre Dame Australia provides information on some different frameworks available to help structure the research question.


Further Readings


Search for existing reviews

Systematic ReviewIntegrative ReviewRapid Review

Before you start searching, find out whether any systematic reviews have been conducted recently on your topic. This is because similar systematic reviews could help with identifying your search terms, and information on your topic. It is also helpful to know if there is already a systematic review on your topic as it may mean you need to change your question.  

Cochrane Library and Joanna Briggs Institute publish systematic reviews. You can also search for the term "systematic review" in any of the subject databases. You can also search PROSPERO, an international register of systematic reviews, to see if there are any related reviews underway but not yet published; there are additional review registers detailed below.
 

Watch this video to find out how to search for published systematic reviews


Protocols and Guidelines for reviews

It is recommended that authors consult relevant guidelines and create a protocol for their review.  

Protocols provide a clear plan for how the review will be conducted, including what will and will not be included in the final review. Protocols are widely recommended for any systematic review and are increasingly a requirement for publication of a completed systematic review.

Guidelines provide specific information on how to perform a review in your field of study. A completed review may be evaluated against the relevant guidelines by peer reviewers or readers, so it makes sense to follow the guidelines as best you can.


Click the headings below to learn more about the importance of protocols and guidelines.

Protocols

Your protocol (or plan for conducting your review) should include the rationale, objectives, hypothesis, and planned methods used in searching, screening and analysing identified studies used in the review. The rationale should clearly state what will be included and excluded from the review. The aim is to minimise any bias by having pre-defined eligibility criteria.

Base the protocol on the relevant guidelines for the review that you are conducting. PRISMA-P was developed for reporting and development of protocols for systematic reviews. Their Explanation and Elaboration paper includes examples of what to write in your protocol. York's CRD has also created a document on how to submit a protocol to PROSPERO.

There are several registers of protocols, often associated with the organisation publishing the review. Cochrane and Joanna Briggs Institute both have their own protocol registries, and PROSPERO is a wide-reaching registry covering protocols for Cochrane, non-Cochrane and non-JBI reviews on a range of health, social care, education, justice, and international development topics.

Before beginning your protocol, search within protocol registries such as those listed above, or Open Science Framework or Research Registry, or journals such as Systematic Reviews and BMJ Open. This is a useful step to see if a protocol has already been submitted on your review topic and to find examples of protocols in similar areas of research.    

While a protocol will contain details of the intended search strategy, a protocol should be registered before the search strategy is finalised and run, so that you can show that your intention for the review has remained true and to limit duplication of in progress reviews.  

A protocol should typically address points that define the kind of studies to be included and the kind of data required to ensure the systematic review is focused on the appropriate studies for the topic. Some points to think about are:

  • What study types are you looking for? For example, randomised controlled trials, cohort studies, qualitative studies
  • What sample size is acceptable in each study (power of the study)? 
  • What population are you focusing on? Consider age ranges, gender, disease severity, geography of patients.
  • What type of intervention are you focusing on?
  • What outcomes are of importance to the review, including how those outcomes are measured?
  • What context should you be looking for in a study? A lab, acute care, school, community...
  • How will you appraise the studies? What methodology will you use?
  • Does the study differentiate between the target population and other groups in the data? How will you handle it if it does not?
  • Is the data available to access if the article does not specify the details you need? If not, what will you do?
  • What languages are you able to review? Do you have funding to translate articles from languages other than English?
     

Further reading

PLoS Medicine Editors. (2011). Best practice in systematic reviews: the importance of protocols and registration. PLoS medicine, 8(2), e1001009.

Systematic Review guidelines

The Cochrane handbook of systematic reviews of interventions is a world-renowned resource for information on designing systematic reviews of intervention.  

Many other guidelines have been developed from these extensive guidelines.

General systematic reviews

Meta-analyses

  • An alternative to PRISMA is the Meta‐analysis Of Observational Studies in Epidemiology (MOOSE) for observational studies. It is a 35‐item checklist. It pays more attention to certain aspects of the search strategy, in particular the inclusion of unpublished and non‐English‐language studies.

Surgical systematic reviews

Nursing/Allied Health systematic reviews

Nutrition systematic reviews

Occupational therapy

Education/Law/ Sociology systematic reviews

Other

Integrative Review guidelines

Integrative reviews may incorporate experimental and non-experimental data, as well as theoretical information.  They differ from systematic reviews in the diversity of the study methodologies included.

Guidelines:

Rapid Review guidelines

Rapid reviews differ from systematic reviews in the shorter timeframe taken and reduced comprehensiveness of the search.

Cochrane has a methods group to inform the conduct of rapid reviews with a bibliography of relevant publications.

A modified approach to systematic review guidelines can be used for rapid reviews, but guidelines are beginning to appear:

Scoping Review guidelines

Scoping reviews can be used to map an area, or to determine the need for a subsequent systematic review. Scoping reviews tend to have a broader focus than many other types of reviews, however, still require a focused question.

Guidelines:

Umbrella reviews

Meta-syntheses

Living reviews

Qualitative systematic reviews

Mixed methods systematic review

Realist reviews

Other