The next step in the systematic review process is to undertake a quality assessment of the results retrieved. Critically appraising a piece of research combines analysis of the design of the study, the validity of the findings in relation to the design of the study, the likelihood of bias, and the relevance of the overall results to other current research. These are the key steps as outlined in Greenhalgh, J. and Brown, T. Quality assessment: where do I begin? In: A. Boland, M.G. Cherry and R. Dickson (eds.) Doing a systematic review: a student's guide, 2nd edn. London: SAGE, 107-130.
Step 1: Note the design(s) of the studies to be included in your review
Step 2: Identify the type(s) of quality assessment tool(s) to suit your review
Step 3: Source appropriate quality assessment tool(s)
Step 4: Carry out quality assessment using the appropriate tool(s)
Step 5: Tabulate and summarize the results of your quality assessment
Step 6: Think about how the quality assessment results might impact on the recommendations and conclusions of your systematic review
The Critical Appraisal Skills Programme has produced a set of eight checklists which can be used when appraising studies for inclusion/exclusion in the systematic review. The checklists provide a series of study-specific questions to use when reading research to enable the critical appraisal of each item.
Available from https://casp-uk.net/
BestBETs provide a range of Critial Appraisal (CA) Worksheets for a range of study types, including:
Available from https://bestbets.org/
SIGN (Scottish Intercollegiate Guidelines Network)
SIGN, part of the Evidence Directorate of Healthcare Improvement Scotland, has produced a number of checklists to help critically evaluate systematic reviews, randomised controlled trials, etc.
These are available at: https://www.sign.ac.uk/what-we-do/methodology/checklists/
Centre for Evidence-Based Medicine (CEBM)
The Centre for Evidence-Based Medicine (CEBM), based at the University of Oxford, has developed tools and worksheets to help with the critical appraisal of different types of medical evidence.
These are available at: https://www.cebm.ox.ac.uk/resources/ebm-tools/critical-appraisal-tools
JBI: Joanna Briggs Institute Critical Appraisal Tools
JBI's Critical Appraisal Tools can assist with assessing the trustworthiness, relevance and validity of research papers. Includes:
Available from https://jbi.global/critical-appraisal-tools
Mixed Methods Appraisal Tool (MMAT)
The Mixed Methods Appraisal Tool (MMAT) is intended to be used as a checklist for appraising and/or describing studies included in systematic mixed studies reviews (reviews including original qualitative, quantitative and mixed methods studies).
Available from http://mixedmethodsappraisaltoolpublic.pbworks.com/
A PDF user guide for MMAT is available for download
The Centre for Evidence-Based Medicine (CEBM)'s brief guide on Study designs can help you spot the difference and to identify the advantages/disadvantages of different study types. The guide is available from https://www.cebm.ox.ac.uk/resources/ebm-tools/study-designs
Appraising for Anti-Racism
Created by Ramona Naicker from Northern Care Alliance NHS Group
Barts Health NHS Trust's Knowledge and Library Services have produced a series of short videos taking you through how to critically appraise a Randomised Controlled Trial (RCT) using the CASP checklist. The video below is exploring how to answer question 1: Did the study address a clear focused question?
This video is a hands on introduction to using the CASP checklist to appraise systematic reviews facilitated by Dr Becky Gould, hosted by Cochrane UK in September 2021.