Have you ever seen a news piece about a scientific breakthrough and wondered how accurate the reporting is? Or wondered about the research behind the headlines? This is the beginning of critical appraisal: thinking critically about what you see and hear, and asking questions to determine how much of a 'breakthrough' something really is.
The article "Is this study legit? 5 questions to ask when reading news stories of medical research" is a succinct introduction to the sorts of questions you should ask in these situations, but there's more than that when it comes to critical appraisal. Read on to learn more about this practical and crucial aspect of evidence-based practice.
Critical appraisal forms part of the process of evidence-based practice.
“Evidence-based practice across the health professions” outlines the fives steps of this process. Critical appraisal is step three:
Critical appraisal is the examination of evidence to determine applicability to clinical practice. It considers (1):
If practitioners hope to ‘stand on the shoulders of giants’, practicing in a manner that is responsive to the discoveries of the research community, then it makes sense for the responsible, critically thinking practitioner to consider the reliability, influence, and relevance of the evidence presented to them.
While critical thinking is valuable, it is also important to avoid treading too much into cynicism; in the words of Hoffman et al. (1):
… keep in mind that no research is perfect and that it is important not to be overly critical of research articles. An article just needs to be good enough to assist you to make a clinical decision.
Evidence-based practice is intended to be practical. To enable this, critical appraisal checklists have been developed to guide practitioners through the process in an efficient yet comprehensive manner.
Critical appraisal checklists guide the reader through the appraisal process by prompting the reader to ask certain questions of the paper they are appraising. There are many different critical appraisal checklists but the best apply certain questions based on what type of study the paper is describing. This allows for a more nuanced and appropriate appraisal. Wherever possible, choose the appraisal tool that best fits the study you are appraising.
Like many things in life, repetition builds confidence and the more you apply critical appraisal tools (like checklists) to the literature the more the process will become second nature for you and the more effective you will be.
Identifying the study type described in the paper is sometimes a harder job than it should be. Helpful papers spell out the study type in the title or abstract, but not all papers are helpful in this way. As such, the critical appraiser may need to do a little work to identify what type of study they are about to critique. Again, experience builds confidence but having an understanding of the typical features of common study types certainly helps.
To assist with this, the Library has produced a guide to study designs in health research.
The following selected references will help also with understanding study types but there are also other resources in the Library’s collection and freely available online:
In order to encourage consistency and quality, authors of reports on research should follow reporting guidelines when writing their papers. The EQUATOR Network is a good source of reporting guidelines for the main study types.
While these guidelines aren't critical appraisal tools as such, they can assist by prompting you to consider whether the reporting of the research is missing important elements.
Once you've identified the study type at hand, visit EQUATOR to find the associated reporting guidelines and ask yourself: does this paper meet the guideline for its study type?
Determining which checklist to use ultimately comes down to finding an appraisal tool that:
Below are some sources of critical appraisal tools. These have been selected as they are known to be widely accepted, easily applicable, and relevant to appraisal of a typical journal article. You may find another tool that you prefer, which is acceptable as long as it is defensible:
The information on this page has been compiled by the Medical Librarian. Please contact the Library's Health Team (libhealth@deakin.edu.au) for further assistance.
Reference list
1. Hoffmann T, Bennett S, Del Mar C. Evidence-based practice across the health professions. 2nd ed. Chatswood, N.S.W., Australia: Elsevier Churchill Livingston; 2013.
2. Greenhalgh T. How to read a paper : the basics of evidence-based medicine. 5th ed. Chichester, West Sussex: Wiley; 2014.
3. Harris M, Jackson D, Taylor G. Clinical evidence made easy. Oxfordshire, England: Scion Publishing; 2014.
4. Aronoff SC. Translational research and clinical practice: basic tools for medical decision making and self-learning. New York: Oxford University Press; 2011.