Critical Appraisal in Nursing Research | CASP, JBI & PRISMA Guide

How to write A systematic Review and Meta-Analysis

Complete academic guide aligned with NMC standards and PRISMA 2020 systematic review methodology.

Key Sources

The Cochrane Handbook for Systematic Reviews of Interventions (v6.4, 2024), the JBI Manual for Evidence Synthesis (Aromataris et al., 2024), PRISMA 2020 (Page et al., BMJ), GRADE guidelines (Guyatt et al., 2011 series), and CASP (2024). Key corrections: (1) I² thresholds now reflect the Cochrane Handbook's four overlapping bands.

Key Takeaways

1. Critical analysis is not summary. it evaluates a study's validity, reliability, and clinical transferability.
2. Use CASP for structured critical appraisal of individual papers- In assignments, literature reviews, and within systematic reviews. Use the full JBI methodology when conducting a formal systematic review at master's or PhD level that requires a complete JBI framework.
3. The PICO/PICo framework is always the first step — it defines your question before any appraisal begins.
4. A systematic review follows the same sequence as a published research paper: Background → Introduction (PICO) → Methods (protocol, search, screening, appraisal) → Results (PRISMA + synthesis) → Discussion (critique, flaws, ethics) → Conclusion.

Background

The NMC Code requires all registrants to base practice on the best available evidence and understanding critical appraisal is how you demonstrate this professionally.

Article Structure:  Background → Introduction → Methods → Results → Discussion → Conclusion, aligned with PRISMA 2020 reporting standards.

Why Critical Analysis Matters, and What the NMC Demands

The NMC Code (2018) is unambiguous: nurses must "always practise in line with the best available evidence." This is not aspirational language, it is a professional and legal obligation. Reading a paper is straightforward. Critically analysing it, which involve interrogating the methodology, identifying bias, evaluating evidence quality, and determining whether findings transfer to your patient population is the sophisticated skill the NMC expects.

Professional Obligation

The NMC Code (clause 6) requires you to "take account of current evidence and knowledge when assessing, planning, delivering and evaluating care." Critical appraisal is how you operationalise this. See also NICE Safe Staffing Guideline (SG1) and RCN library and evidence-based practice support.

Critical Analysis vs. Descriptive Summary

Markers identify the failure to move beyond description as the most common weakness in research assignments at every level. The pattern of critical analysis is always: Observation → Methodological Significance → Evaluative Consequence.

Introduction

Forming Your Research Question (PICO/PICo)

A dissertation or thesis opens by defining the scope of the inquiry. In nursing research, this means constructing a formal research question using PICO (quantitative questions) or PICo (qualitative questions). These frameworks are embedded in both the JBI methodology and the CASP framework, everything that follows derives from them.

PICO (Quantitative Research)

 PICo (Qualitative Research)

 ü  EXPERT TIP

Every keyword in your database search strings must map back to a PICO/PICo element. If a search term cannot be linked to P, I, C, or O, it does not belong in your search strategy.

Methods

Choosing Your Appraisal Framework (CASP vs JBI)

This is the section students get wrong. CASP and JBI are not interchangeable; they serve different purposes at different levels of study.

 Practical guide: CASP or JBI?

CASP checklists are versatile tools used for appraising individual papers and for quality appraisal within systematic reviews. JBI offers a complete methodological ecosystem: when conducting a formal systematic review following JBI methodology, JBI provides the full framework: study-specific checklists, evidence hierarchy, FAME concepts, and synthesis guidance. When in doubt, check what your institution or supervisor requires — both are evidence-based and widely cited.

The CASP Three-Pillar Framework (for Individual Paper Appraisal)

Pillar 1. Are the results valid? This concerns internal validity. For quantitative studies: was randomisation properly performed? Were participants blinded? Was a power calculation conducted? For qualitative studies: was the methodology congruent with the epistemological stance? Were Lincoln and Guba's (1985) trustworthiness criteria (credibility, transferability, dependability, confirmability) addressed?

Pillar 2. What are the results? Interrogate beyond statistics. For quantitative work, examine clinical significance (effect size, NNT, ARR) alongside statistical significance (p-value). A wide 95% confidence interval signals imprecision. For qualitative work, check whether findings emerge organically from the data and whether negative cases are acknowledged.

Pillar 3.  Will the results help locally? Students mostly undervalue this external validity question. Were participants similar to your patient population? Is the intervention feasible within NHS resource constraints? Are findings recent enough to align with current NICE or NHS England guidance?

The JBI Model: Four Core FAME Concepts (for Systematic Reviews)

 JBI also provides study design-specific evidence hierarchies. For effectiveness/intervention evidence, the hierarchy runs: Level 1 (systematic review of RCTs) → Level 2 (single well-designed RCT) → Level 3 (quasi-experimental) → Level 4 (observational/case series) → Level 5 (expert opinion and bench research). Importantly, JBI has separate hierarchies for different evidence types — diagnosis, prognosis, costs, and meaningfulness each have their own level frameworks.

 💡 EXPERT TIP

Each JBI checklist item is answered Yes / No / Unclear / Not Applicable — confirmed in the JBI Critical Appraisal Checklist documentation. You must report JBI appraisal scores for every included study in a dedicated table in your Methods appendix. CASP checklists use a different response format — Yes / No / Can't Tell — without a formal aggregate scoring protocol. JBI tools can also be used beyond formal systematic reviews: including Critically Appraised Topics (CATs), journal clubs, and educational settings.

 Conducting the Systematic Review

The methodology chapter of your dissertation or thesis must demonstrate that your review is transparent, reproducible, and free from selection bias. A systematic review follows a fixed sequence and each phase must be completed before the next begins.

STEP 1

Develop and Register a Protocol

Specify your review question, eligibility criteria, search strategy, databases, data extraction plan, appraisal approach, and synthesis method before searching. Register your protocol with PROSPERO — free, time-stamped, and required by most peer-reviewed journals. Registration prevents post-hoc manipulation of eligibility criteria or outcome definitions.

STEP 2

Define Eligibility Criteria

Derived directly from your PICO/PICo question. Cover: study design, population, intervention/phenomenon, comparator, outcome measures, language, publication date, and publication type. Present these in a structured table in your Methods section. Transparent criteria are what distinguish a systematic review from a narrative literature review.

STEP 3

Comprehensive Database Search

Search at least four to six databases. Standard databases for UK nursing research: CINAHL (nursing-specific, most important), MEDLINE/PubMed (biomedical), PsycINFO (mental health), Cochrane Library (existing reviews and RCTs), EMBASE (European/pharmacological focus), and NICE Evidence Search (UK guidelines). Also search grey literature via NHS reports and PROSPERO for existing reviews. Construct Boolean search strings using AND, OR, NOT with MeSH/CINAHL controlled vocabulary. Document every search string, date searched, and number of results — this goes in your Methods appendix.

STEP 4

Screening (Two Phases)

Phase 1: Two independent reviewers screen titles and abstracts against eligibility criteria using Covidence or Rayyan. Disagreements are resolved by consensus or a third reviewer. Phase 2: Studies passing Phase 1 are retrieved in full text and re-evaluated. All excluded full-text studies must be listed with their reason for exclusion — a mandatory PRISMA 2020 requirement.

STEP 5

Quality Appraisal

Apply the appropriate JBI or CASP checklist to every included study (see section above). Conduct appraisal in duplicate. Do not exclude studies solely on quality unless pre-specified in your protocol — instead, present quality scores in a table and account for them in your synthesis and conclusions.

Results

 PRISMA 2020 Reporting Standard

PRISMA 2020 is the internationally mandated reporting standard for systematic reviews. Adherence is required by every peer-reviewed journal and expected systematic reviews. The centrepiece is the PRISMA flow diagram, which is a three-phase visual record of how studies moved through your review. The official PRISMA 2020 diagram has three labelled phases (Identification, Screening, Included) — the eligibility/full-text assessment step sits within the Screening phase, not as a separate fourth phase as in the older 2009 version:

 PRISMA 2020 Update: KEY Changes

PRISMA 2020 added requirements absent from the 2009 version: you must now report searches of trial registries, updated guidance on citation searching, and new items on risk of bias tools and synthesis methods. Use the official PRISMA 2020 checklist (27 items) alongside the PRISMA 2020 flow diagram. Items most commonly omitted by nursing reviewers: Item 7 (full search strategy for all databases), Item 8 (selection process), Item 11 (risk of bias assessment tool and process), and Item 22 (certainty of evidence — e.g. GRADE/ConQual rating for each outcome).

 Data Extraction and Evidence Synthesis

Design your data extraction form before you begin and pilot it on two or three studies. Extract in duplicate. Capture: study identification (author, year, country, funding, COI), study design and JBI evidence level, participant characteristics, intervention/phenomenon description, outcomes and measurement tools, all quantitative results or qualitative themes, and quality appraisal scores.

 Choosing a Synthesis Method

Narrative synthesis is used when studies are too heterogeneous to pool statistically. Organise findings thematically, describe the direction and consistency of results, and discuss sources of variability. This must be structured and not a sequential summary of papers.

 Meta-analysis is used when quantitative studies are sufficiently similar in population, intervention, comparator and outcome. It produces a pooled effect size with confidence interval, displayed in a forest plot. Assess heterogeneity using the I² statistic. The Cochrane Handbook (Higgins et al., 2024) provides four overlapping rough guide bands: 0–40% might not be important; 30–60% may represent moderate heterogeneity; 50–90% may represent substantial heterogeneity; 75–100% indicates considerable heterogeneity. Note: these are not rigid cutoffs, the importance of I² depends on the magnitude and direction of effects and the strength of evidence from the chi-squared test. High I² does not automatically preclude pooling, but requires careful methodological explanation. In  addition, assess publication bias with funnel plots and Egger's test when ten or more studies are included.

Meta-aggregation is the JBI-preferred approach for qualitative evidence. It groups findings (statements from individual studies) into categories (similar meanings), then develops synthesised findings (credible summary statements) or Themes. Use the ConQual tool or the JBI's equivalent of GRADE for qualitative evidence to rate confidence in each synthesised finding across dependability and credibility.

 💡 EXPERT TIP

Use the GRADE approach for quantitative evidence certainty: High, Moderate, Low, or Very Low, based on five domains: risk of bias, inconsistency, indirectness, imprecision, and publication bias. GRADE is required by NICE and expected in doctoral systematic reviews. Use GRADEpro GDT to create your Summary of Findings tables — it integrates directly with RevMan Web.

 Discussion

Writing the Critique and Interpreting Your Findings

The Discussion is where your scholarly contribution is most visible. It does not re-describe findings, it interprets them in relation to theory, practice, and policy. A doctoral-level Discussion moves through six stages:

a.     Summary of principal findings — direction of evidence, overall GRADE/ConQual quality, consistency across studies

b.     Contextualising within existing literature — relation to Cochrane Reviews, NICE guidelines, JBI Evidence Summaries; explain divergence if it exists

c.     Implications for nursing practice — name the specific intervention, care pathway, or protocol that would need to change; at what scale; what resource implications

d.     Implications for nursing education — link to NMC Standards of Proficiency (2018) and NHS England workforce frameworks

e.     mplications for future research — which populations are under-researched? which comparators untested? recommend specific, feasible trial designs

f.      Limitations of the review itself — language restrictions, date restrictions, grey literature completeness, residual reviewer bias

 Academic Tone: The Core Rules

The 7 Most Common Research Flaws to Identify

FLAW 1

Absence of a Power Calculation

"The absence of a reported a priori power calculation renders it impossible to determine whether the study was adequately powered to detect the primary outcome, thus limiting confidence in the null result reported."

 FLAW 2

Selection Bias and Convenience Sampling

"The recruitment of participants exclusively from a single NHS Trust, without multi-site sampling, introduces selection bias and substantially restricts the external validity of these findings."

 FLAW 3

Lack of Blinding

"The open-label design, in which both participants and outcome assessors were aware of group allocation, introduces a material risk of performance and detection bias as defined by the Cochrane Risk of Bias 2 (RoB 2) tool."

FLAW 4

Correlation Reported as Causation

"The cross-sectional design precludes causal inference, and the contribution of unmeasured confounders — including patient acuity and shift patterns — remains unaddressed."

 FLAW 5

Funding Bias and Conflict of Interest

"The study's sponsorship by the manufacturer of the intervention device, combined with the absence of a pre-registered analysis plan, raises legitimate concerns about outcome reporting bias."

 FLAW 6

Attrition and Loss to Follow-Up

"The 28% dropout rate, without intention-to-treat analysis or imputation of missing data, introduces a substantive risk of attrition bias and may have biased the reported outcomes in the direction of benefit."

 FLAW 7

Statistical vs. Clinical Significance

"While the difference reached statistical significance (p=0.04), the mean difference of 0.3 points on a 10-point scale falls substantially below the minimum clinically important difference (MCID) of 1.5–2 points, raising questions about the clinical meaningfulness of this outcome."

 Ethical Considerations in Nursing Research

Doctoral-level appraisal interrogates ethical quality — not merely noting "ethical approval was obtained." All research involving NHS patients, staff, or data must receive approval from an NHS Health Research Authority Research Ethics Committee. When appraising, examine six domains:

 Conclusion

The conclusion is the tightest section of the entire document. It must directly answer the original PICO/PICo research question, state the overall certainty of evidence (GRADE/ConQual), avoid overstating findings ("this review provides preliminary evidence that..." not "this review proves..."), make a qualified statement about practice implications, and specify the type and rationale for future research needed.

Example of Conclusion

"This systematic review of seven studies (n=1,243) provides low-to-moderate certainty evidence that structured repositioning protocols at two-hourly intervals are associated with a statistically significant reduction in Stage II+ pressure injury incidence in adult NHS inpatients (RR 0.61, 95% CI 0.45–0.82). Given the high risk of performance bias and limited follow-up in the included studies, these findings should be interpreted with caution. Further multi-site, adequately powered RCTs with pre-registered protocols are required before these findings can justify revision of current NICE pressure ulcer prevention guidelines (CG179)."

 Supplementary

Academic Phrase Bank

Acknowledging Strengths

"A notable methodological strength of this study is the use of a randomised, double-blind design, which substantially reduces the risk of performance and detection bias."

"The large, multi-site sample (n=842 across six NHS Trusts) enhances the external validity of these findings and their applicability to UK clinical practice."

Identifying Limitations

·       "The short follow-up period of 4 weeks is insufficient to evaluate the sustainability of the observed effect beyond the immediate post-intervention period."

·       "The authors' reliance on self-reported outcome data, in the absence of triangulation with objective clinical measures, introduces the potential for recall bias and social desirability bias."

Evaluating Conclusions

·       "The authors' assertion that the intervention 'definitively reduces' the primary outcome is not warranted by the data presented, which demonstrates a statistically significant but clinically modest effect."

·       "These preliminary findings should be interpreted as hypothesis-generating rather than conclusive, pending replication in larger, adequately powered trials."

Transferability to UK / NHS Practice

·       "Given the significant structural differences between the North American healthcare system and the NHS, these findings cannot be uncritically applied to UK nursing practice."

·       "In the context of the NMC Code's requirement to practise in accordance with the best available evidence, these findings, if replicated, would have substantive implications for nursing practice in the United Kingdom."

Essential Tools, Databases & Software

💡 EXPERT TIP

NHS OpenAthens gives every NHS employee free access to CINAHL, PsycINFO, EMBASE, the BMJ, the Lancet, and thousands of other journals. Register at openathens.nice.org.uk using your NHS email — one of the most underused resources in UK nursing.

Frequently Asked Questions

Q: When should I use CASP and when should I use JBI?

Use CASP when you need a structured, accessible checklist to critically appraise a study — whether for a taught assignment, an essay, or as part of the appraisal stage of a systematic review. CASP checklists cover RCTs, qualitative studies, cohort studies, case-control studies, economic evaluations, and more. Use JBI methodology when you are conducting a formal systematic review that follows the JBI framework end-to-end — this means using the JBI protocol, JBI study-specific appraisal checklists, the FAME framework, JBI's evidence hierarchy, and JBI's synthesis approach. The key distinction is not simply 'one paper vs. many papers' — it is whether you are following the full JBI methodological framework or using checklists more flexibly within another approach.

Q: What is the difference between a systematic review and a literature review?

A literature review is a broad, often narrative synthesis of existing knowledge — it may be selective and does not require a pre-registered protocol. A systematic review follows a rigorous, pre-specified, reproducible methodology: registered protocol (typically on PROSPERO), comprehensive multi-database searching, dual-reviewer screening, structured quality appraisal (using JBI or CASP checklists), and transparent synthesis. A narrative literature review may be acceptable for a taught master's module; a systematic review is expected at doctoral level and for publication in peer-reviewed journals.

Q: Do I need to register my protocol on PROSPERO?

Registration on PROSPERO is strongly recommended and required by many peer-reviewed journals as a condition of publication. For a PhD thesis, your research governance committee will typically require evidence of registration. It is free, time-stamped, and protects against post-hoc changes to eligibility criteria or outcome definitions. Note that PROSPERO does not accept protocols for reviews that have already completed data extraction.

Q: How many studies should a systematic review include?

There is no minimum or maximum. A well-conducted review may include as few as three or four studies if the evidence base is genuinely sparse, or hundreds if not. What matters is that your search strategy was comprehensive, your eligibility criteria were applied consistently, and your PRISMA flow diagram documents everything transparently. A five-study review with a documented six-database search and clear exclusion reasons is methodologically stronger than a 40-study review with no documented search strategy.

Q: How do I handle conflicting findings across studies in my synthesis?

Conflicting findings are a feature, not a flaw, of systematic reviews — addressing them rigorously demonstrates doctoral-level skill. First, investigate whether the conflict is explained by methodological differences (sample characteristics, intervention dose, follow-up duration, outcome instruments). Then assess whether heterogeneity is statistical (I² in meta-analysis) or clinical/conceptual. If conflicts cannot be resolved through subgroup or sensitivity analysis, report them transparently in your Discussion and acknowledge this as a limitation. Never selectively report only the studies that support your hypothesis.

Common Questions Answered in this Guide 

Section 1

Why do nurses need to critically analyse research papers?

What does the NMC Code say about evidence-based practice?

What is the difference between critical analysis and summary?

Section 2

What is PICO in nursing research?

What is the difference between PICO and PICo?

How do I form a research question for a nursing dissertation?

Section 3

What is CASP vs JBI in nursing?

When should I use CASP and when should I use JBI?

Which critical appraisal tool is best for a nursing assignment?

Section 4

How do you structure a systematic review in nursing?

What databases should I search for a nursing systematic review?

What is PROSPERO and do I need to register my review?

Section 5

What is PRISMA 2020 and how do I use it?

How do I create a PRISMA flow diagram for my systematic review?

What are the phases of the PRISMA 2020 flow diagram?

Section 6

What is meta-analysis vs narrative synthesis in nursing?

What does I² mean in a systematic review?

How do I use GRADE in a nursing systematic review?

Section 6

How do I write a discussion section for a nursing systematic review?

What should a doctoral-level discussion include?

How do I link findings to NMC standards and NICE guidelines?

Section 7

What are the most common flaws in nursing research papers?

What is selection bias in nursing research?

How do I identify performance bias and detection bias?

Section 8

What ethical issues should I discuss in a nursing research critique?

What is NHS Health Research Authority approval?

Section 9

How do I write a conclusion for a nursing systematic review?

What should a doctoral-standard conclusion include?

How do I state GRADE certainty of evidence in a conclusion?

 References and Further Reading

Aromataris, E., Lockwood, C., Porritt, K., Pilla, B., & Jordan, Z. (Eds.). (2024). JBI Manual for Evidence Synthesis. JBI. synthesismanual.jbi.global

Cochrane Collaboration. (2024). Cochrane Handbook for Systematic Reviews of Interventions (Version 6.4). training.cochrane.org/handbook/current

Cochrane Collaboration. (2024). Review Manager (RevMan Web) [Computer program]. The Cochrane Collaboration. Note: RevMan 5 (desktop) is no longer supported by Cochrane. revman.cochrane.org

Critical Appraisal Skills Programme. (2024). CASP Checklists. CASP UK. casp-uk.net/casp-tools-checklists/

Evidence Prime. (2024). GRADEpro GDT [Computer program]. McMaster University and Evidence Prime. gradepro.org

Guyatt, G., Oxman, A. D., Akl, E. A., et al. (2011). GRADE guidelines: 1. Introduction. Journal of Clinical Epidemiology, 64(4), 383–394.

Health Research Authority. (2023). UK Policy Framework for Health and Social Care Research. hra.nhs.uk

Higgins, J. P. T., et al. (2023). RoB 2: A revised tool for assessing risk of bias in randomised trials. Cochrane. methods.cochrane.org

Joanna Briggs Institute. (2023). Critical Appraisal Tools. jbi.global/critical-appraisal-tools

Kolaski, K., Logan, L. R., & Ioannidis, J. P. A. (2023). Guidance to best tools and practices for systematic reviews. Systematic Reviews, 12, 96. doi.org/10.1186/s13643-023-02255-9

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage Publications.

National Institute for Health and Care Excellence. (2022). Developing NICE guidelines: the manual [PMG20]. NICE. nice.org.uk/process/pmg20

Nursing and Midwifery Council. (2018). Future Nurse: Standards of Proficiency for Registered Nurses. NMC. nmc.org.uk/standards/standards-for-nurses/

Nursing and Midwifery Council. (2018). The Code: Professional Standards of Practice and Behaviour for Nurses, Midwives and Nursing Associates. NMC. nmc.org.uk/standards/code/

Page, M. J., McKenzie, J. E., Bossuyt, P. M., et al. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. doi.org/10.1136/bmj.n71

PROSPERO. (2023). International Prospective Register of Systematic Reviews. University of York. crd.york.ac.uk/prospero/

Ready to Put This Into Practice?

UKNurses.net provides resources, CPD modules, and community support for nurses at every stage of their research journey — from understanding a single paper to completing a doctoral thesis.

Explore Research Resources → uknurses.net/resources/research