critical appraisal literature review example

Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

Critical Appraisal tools

Critical appraisal worksheets to help you appraise the reliability, importance and applicability of clinical evidence.

Critical appraisal is the systematic evaluation of clinical research papers in order to establish:

• Does this study address a  clearly focused question ?
• Did the study use valid methods to address this question?
• Are the valid results of this study important?
• Are these valid, important results applicable to my patient or population?

If the answer to any of these questions is “no”, you can save yourself the trouble of reading the rest of it.

This section contains useful tools and downloads for the critical appraisal of different types of medical evidence. Example appraisal sheets are provided together with several helpful examples.

Critical Appraisal Worksheets

• Systematic Reviews  Critical Appraisal Sheet
• Diagnostics  Critical Appraisal Sheet
• Prognosis  Critical Appraisal Sheet
• Randomised Controlled Trials  (RCT) Critical Appraisal Sheet
• Critical Appraisal of Qualitative Studies  Sheet
• IPD Review  Sheet

Chinese - translated by Chung-Han Yang and Shih-Chieh Shao

• Systematic Reviews  Critical Appraisal Sheet
• Diagnostic Study  Critical Appraisal Sheet
• Prognostic Critical Appraisal Sheet
• RCT  Critical Appraisal Sheet
• IPD reviews Critical Appraisal Sheet
• Qualitative Studies Critical Appraisal Sheet 

German - translated by Johannes Pohl and Martin Sadilek

• Systematic Review  Critical Appraisal Sheet
• Diagnosis Critical Appraisal Sheet
• Prognosis Critical Appraisal Sheet
• Therapy / RCT Critical Appraisal Sheet

Lithuanian - translated by Tumas Beinortas

• Systematic review appraisal Lithuanian (PDF)
• Diagnostic accuracy appraisal Lithuanian  (PDF)
• Prognostic study appraisal Lithuanian  (PDF)
• RCT appraisal sheets Lithuanian  (PDF)

Portugese - translated by Enderson Miranda, Rachel Riera and Luis Eduardo Fontes

• Portuguese – Systematic Review Study Appraisal Worksheet
• Portuguese – Diagnostic Study Appraisal Worksheet
• Portuguese – Prognostic Study Appraisal Worksheet
• Portuguese – RCT Study Appraisal Worksheet
• Portuguese – Systematic Review Evaluation of Individual Participant Data Worksheet
• Portuguese – Qualitative Studies Evaluation Worksheet

Spanish - translated by Ana Cristina Castro

• Systematic Review  (PDF)
• Diagnosis  (PDF)
• Prognosis  Spanish Translation (PDF)
• Therapy / RCT  Spanish Translation (PDF)

Persian - translated by Ahmad Sofi Mahmudi

• Prognosis  (PDF)
• PICO  Critical Appraisal Sheet (PDF)
• PICO Critical Appraisal Sheet (MS-Word)
• Educational Prescription  Critical Appraisal Sheet (PDF)

Explanations & Examples

• Pre-test probability
• SpPin and SnNout
• Likelihood Ratios

Best Practice for Literature Searching

• Literature Search Best Practice
• What is literature searching?
• What are literature reviews?
• Hierarchies of evidence
• 1. Managing references
• 2. Defining your research question
• 3. Where to search
• 4. Search strategy
• 5. Screening results
• 6. Paper acquisition
• 7. Critical appraisal
• Further resources
• Training opportunities and videos
• Join FSTA student advisory board This link opens in a new window
• Chinese This link opens in a new window
• Italian This link opens in a new window
• Persian This link opens in a new window
• Portuguese This link opens in a new window
• Spanish This link opens in a new window

Deciding what to include in your review through critical appraisal

Once you have narrowed down your pool of results, it's time to begin critically appraising your articles.  Using a checklist helps you scrutinise articles in a consistent, structured way.  

Questions to consider include: 

• Are the aims of the study clearly stated?
• Is the study design suitable for the aims?
• Are the measurements and methods used clearly described?
• Are the correct measurement tools used?
• Are the statistical methods described?
• Was the sample size adequate? 
• Are the methods overall described in enough detail that you could replicate the study?
• Does the discussion overall reflect the results?
• Who funded this study?
• What are the specific limitations of what can be concluded from the study?

Working through the questions will help you identify the strengths and weakness of each article, and also identify points to draw on when you write about the literature. 

• DOWNLOAD THE CRITICAL APPRAISAL CHECKLIST

Additional critical appraisal checklists

REFLECT provides a  checklist for evaluating randomized control trials in livestock and food safety. 

CASP provides  checklists  for critical appraisal of studies related to health.

JBI provides checklists for critical appraisal of studies related to health.

Documenting critical appraisal decisions

As you closely examine full articles, you will be making judgements about why to include  or exclude  each study from your review.  Documenting your reasoning will help you reassure yourself and demonstrate to others that you have been systematic and unbiased in your appr aisal decisions.

Keeping track of what you have excluded, and why, will be very helpful if you must defend your work—for instance, if your literature review is part of a dissertation or thesis. 

Pulling all the literature you will include in your review into a single chart is a good way to begin to synthesise the literature. 

• DOWNLOAD THE FULL TEXT SCREENING CHART

Best practice!

BEST PRACTICE RECOMMENDATION :  If you include any direct quotes in your chart (or in any notes) be sure to use quotation marks so that you don’t later mistake the words for your own.

BEST PRACTICE RECOMMENDATION: The more carefully you record each of the steps of your process, the more easily reproducible it will be. This is especially important for research abstracts and articles found in conference proceedings.

• << Previous: 6. Paper acquisition
• Next: Further resources >>
• Last Updated: May 17, 2024 5:48 PM
• URL: https://ifis.libguides.com/literature_search_best_practice

• University of Texas Libraries
• UT Libraries

Systematic Reviews & Evidence Synthesis Methods

Critical appraisal.

• Types of Reviews
• Formulate Question
• Find Existing Reviews & Protocols
• Register a Protocol
• Searching Systematically
• Search Hedges and Filters
• Supplementary Searching
• Managing Results
• Deduplication
• Coding and Data Extraction
• Glossary of Terms
• Video tutorials This link opens in a new window
• Librarian Support
• Systematic Review & Evidence Synthesis Boot Camp

Some reviews require a critical appraisal for each study that makes it through the screening process. This involves a risk of bias assessment and/or a quality assessment. The goal of these reviews is not just to find all of the studies, but to determine their methodological rigor, and therefore, their credibility.

"Critical appraisal is the balanced assessment of a piece of research, looking for its strengths and weaknesses and them coming to a balanced judgement about its trustworthiness and its suitability for use in a particular context." 1

It's important to consider the impact that poorly designed studies could have on your findings and to rule out inaccurate or biased work.

Selection of a valid critical appraisal tool, testing the tool with several of the selected studies, and involving two or more reviewers in the appraisal are good practices to follow.

1. Purssell E, McCrae N. How to Perform a Systematic Literature Review: A Guide for Healthcare Researchers, Practitioners and Students. 1st ed. Springer ;  2020.

Evaluation Tools

• The Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II) The Appraisal of Guidelines for Research & Evaluation Instrument (AGREE II) was developed to address the issue of variability in the quality of practice guidelines.
• Centre for Evidence-Based Medicine (CEBM). Critical Appraisal Tools "contains useful tools and downloads for the critical appraisal of different types of medical evidence. Example appraisal sheets are provided together with several helpful examples."
• Critical Appraisal Skills Programme (CASP) Checklists Critical Appraisal checklists for many different study types
• Critical Review Form for Qualitative Studies Version 2, developed out of McMaster University
• Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS) Downes MJ, Brennan ML, Williams HC, et al. Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS). BMJ Open 2016;6:e011458. doi:10.1136/bmjopen-2016-011458
• Downs & Black Checklist for Assessing Studies Downs, S. H., & Black, N. (1998). The Feasibility of Creating a Checklist for the Assessment of the Methodological Quality Both of Randomised and Non-Randomised Studies of Health Care Interventions. Journal of Epidemiology and Community Health (1979-), 52(6), 377–384.
• GRADE The Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group "has developed a common, sensible and transparent approach to grading quality (or certainty) of evidence and strength of recommendations."
• Grade Handbook Full handbook on the GRADE method for grading quality of evidence.
• MAGIC (Making GRADE the Irresistible choice) Clear succinct guidance in how to use GRADE
• Joanna Briggs Institute. Critical Appraisal Tools "JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers." Includes checklists for 13 types of articles.
• Latitudes Network This is a searchable library of validity assessment tools for use in evidence syntheses. This website also provides access to training on the process of validity assessment.
• Mixed Methods Appraisal Tool A tool that can be used to appraise a mix of studies that are included in a systematic review - qualitative research, RCTs, non-randomized studies, quantitative studies, mixed methods studies.
• RoB 2 Tool Higgins JPT, Sterne JAC, Savović J, Page MJ, Hróbjartsson A, Boutron I, Reeves B, Eldridge S. A revised tool for assessing risk of bias in randomized trials In: Chandler J, McKenzie J, Boutron I, Welch V (editors). Cochrane Methods. Cochrane Database of Systematic Reviews 2016, Issue 10 (Suppl 1). dx.doi.org/10.1002/14651858.CD201601.
• ROBINS-I Risk of Bias for non-randomized (observational) studies or cohorts of interventions Sterne J A, Hernán M A, Reeves B C, Savović J, Berkman N D, Viswanathan M et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions BMJ 2016; 355 :i4919 doi:10.1136/bmj.i4919
• Scottish Intercollegiate Guidelines Network. Critical Appraisal Notes and Checklists "Methodological assessment of studies selected as potential sources of evidence is based on a number of criteria that focus on those aspects of the study design that research has shown to have a significant effect on the risk of bias in the results reported and conclusions drawn. These criteria differ between study types, and a range of checklists is used to bring a degree of consistency to the assessment process."
• The TREND Statement (CDC) Des Jarlais DC, Lyles C, Crepaz N, and the TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. Am J Public Health. 2004;94:361-366.
• Assembling the Pieces of a Systematic Reviews, Chapter 8: Evaluating: Study Selection and Critical Appraisal.
• How to Perform a Systematic Literature Review, Chapter: Critical Appraisal: Assessing the Quality of Studies.

Other library guides

• Duke University Medical Center Library. Systematic Reviews: Assess for Quality and Bias
• UNC Health Sciences Library. Systematic Reviews: Assess Quality of Included Studies
• Last Updated: Nov 22, 2024 1:46 PM
• URL: https://guides.lib.utexas.edu/systematicreviews

• UWF Libraries

Literature Review: Conducting & Writing

• Sample Literature Reviews
• Steps for Conducting a Lit Review
• Finding "The Literature"
• Organizing/Writing
• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window

Sample Lit Reviews from Communication Arts

Have an exemplary literature review.

Note: These are sample literature reviews from a class that were given to us by an instructor when APA 6th edition was still in effect. These were excellent papers from her class, but it does not mean they are perfect or contain no errors. Thanks to the students who let us post!

• Literature Review Sample 1
• Literature Review Sample 2
• Literature Review Sample 3

Have you written a stellar literature review you care to share for teaching purposes?

Are you an instructor who has received an exemplary literature review and have permission from the student to post?

Please contact Britt McGowan at [email protected] for inclusion in this guide. All disciplines welcome and encouraged.

• << Previous: MLA Style
• Next: Get Help! >>
• Last Updated: Oct 18, 2024 9:43 AM
• URL: https://libguides.uwf.edu/litreview

Critical appraisal

What is critical appraisal.

Critical appraisal involves critically examining a study to determine its validity, reliability and relevance. It is an important and essential part of the EBP 5-step process . For a more comprehensive overview refer to the critical appraisal for health guide .

It is useful to ask the following three questions when conducting an appraisal:

• Are the results of the study valid?
• Are the results of the study reliable?
• Are the results applicable to my situation? (Saewert & Hagler, 2015 1 )

Critical appraisal tools & guides

There are many different appraisal tools that can be used depending on the type of study. See below for some examples.

• Critical Appraisal Skills Programme (CASP) checklists This set of critical appraisal tools are designed to be used when reviewing health research. They include checklists for systematic reviews, randomised controlled trials, cohort studies etc.
• Centre for Evidence Based Medicine - Critical appraisal tools Contains useful tools and worksheets for the critical appraisal of different types of medical evidence.
• Joanna Briggs Institute - critical appraisal tools JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers.
• PRISMA checklist A 27-item checklist addressing the different sections of systematic reviews regardless of the design of included studies. Primarily designed for systematic reviews that evaluate the effects of health interventions, PRISMA can also be used for systematic reviews evaluating societal or educational interventions as well as those evaluating aetiology, prevalence, or prognosis.
• Additional critical appraisal tools and guides

 1.  Saewert, K. J., & Hagler, D. (2015). Generating the evidence for health professional education: The five As of the scholarship of learning and teaching - ask, answer, access, appraise and apply. In T. Brown & B. Williams (Eds.), Evidence-based education in the health professions: Promoting best practice in the learning and teaching of students (pp. 67-80). London: Radcliffe.

• << Previous: Covidence