level evidence literature review

• Library databases
• Library website

Evidence-Based Research: Levels of Evidence Pyramid

Introduction.

One way to organize the different types of evidence involved in evidence-based practice research is the levels of evidence pyramid. The pyramid includes a variety of evidence types and levels.

• systematic reviews
• critically-appraised topics
• critically-appraised individual articles
• randomized controlled trials
• cohort studies
• case-controlled studies, case series, and case reports
• Background information, expert opinion

Levels of evidence pyramid

The levels of evidence pyramid provides a way to visualize both the quality of evidence and the amount of evidence available. For example, systematic reviews are at the top of the pyramid, meaning they are both the highest level of evidence and the least common. As you go down the pyramid, the amount of evidence will increase as the quality of the evidence decreases.

Text alternative for Levels of Evidence Pyramid diagram

EBM Pyramid and EBM Page Generator, copyright 2006 Trustees of Dartmouth College and Yale University. All Rights Reserved. Produced by Jan Glover, David Izzo, Karen Odato and Lei Wang.

Filtered Resources

Filtered resources appraise the quality of studies and often make recommendations for practice. The main types of filtered resources in evidence-based practice are:

Scroll down the page to the Systematic reviews , Critically-appraised topics , and Critically-appraised individual articles sections for links to resources where you can find each of these types of filtered information.

Systematic reviews

Authors of a systematic review ask a specific clinical question, perform a comprehensive literature review, eliminate the poorly done studies, and attempt to make practice recommendations based on the well-done studies. Systematic reviews include only experimental, or quantitative, studies, and often include only randomized controlled trials.

You can find systematic reviews in these filtered databases :

• Cochrane Database of Systematic Reviews Cochrane systematic reviews are considered the gold standard for systematic reviews. This database contains both systematic reviews and review protocols. To find only systematic reviews, select Cochrane Reviews in the Document Type box.
• JBI EBP Database (formerly Joanna Briggs Institute EBP Database) This database includes systematic reviews, evidence summaries, and best practice information sheets. To find only systematic reviews, click on Limits and then select Systematic Reviews in the Publication Types box. To see how to use the limit and find full text, please see our Joanna Briggs Institute Search Help page .

You can also find systematic reviews in this unfiltered database :

To learn more about finding systematic reviews, please see our guide:

• Filtered Resources: Systematic Reviews

Critically-appraised topics

Authors of critically-appraised topics evaluate and synthesize multiple research studies. Critically-appraised topics are like short systematic reviews focused on a particular topic.

You can find critically-appraised topics in these resources:

• Annual Reviews This collection offers comprehensive, timely collections of critical reviews written by leading scientists. To find reviews on your topic, use the search box in the upper-right corner.
• Guideline Central This free database offers quick-reference guideline summaries organized by a new non-profit initiative which will aim to fill the gap left by the sudden closure of AHRQ’s National Guideline Clearinghouse (NGC).
• JBI EBP Database (formerly Joanna Briggs Institute EBP Database) To find critically-appraised topics in JBI, click on Limits and then select Evidence Summaries from the Publication Types box. To see how to use the limit and find full text, please see our Joanna Briggs Institute Search Help page .
• National Institute for Health and Care Excellence (NICE) Evidence-based recommendations for health and care in England.
• Filtered Resources: Critically-Appraised Topics

Critically-appraised individual articles

Authors of critically-appraised individual articles evaluate and synopsize individual research studies.

You can find critically-appraised individual articles in these resources:

• EvidenceAlerts Quality articles from over 120 clinical journals are selected by research staff and then rated for clinical relevance and interest by an international group of physicians. Note: You must create a free account to search EvidenceAlerts.
• ACP Journal Club This journal publishes reviews of research on the care of adults and adolescents. You can either browse this journal or use the Search within this publication feature.
• Evidence-Based Nursing This journal reviews research studies that are relevant to best nursing practice. You can either browse individual issues or use the search box in the upper-right corner.

To learn more about finding critically-appraised individual articles, please see our guide:

• Filtered Resources: Critically-Appraised Individual Articles

Unfiltered resources

You may not always be able to find information on your topic in the filtered literature. When this happens, you'll need to search the primary or unfiltered literature. Keep in mind that with unfiltered resources, you take on the role of reviewing what you find to make sure it is valid and reliable.

Note: You can also find systematic reviews and other filtered resources in these unfiltered databases.

The Levels of Evidence Pyramid includes unfiltered study types in this order of evidence from higher to lower:

You can search for each of these types of evidence in the following databases:

TRIP database

Background information & expert opinion.

Background information and expert opinions are not necessarily backed by research studies. They include point-of-care resources, textbooks, conference proceedings, etc.

• Family Physicians Inquiries Network: Clinical Inquiries Provide the ideal answers to clinical questions using a structured search, critical appraisal, authoritative recommendations, clinical perspective, and rigorous peer review. Clinical Inquiries deliver best evidence for point-of-care use.
• Harrison, T. R., & Fauci, A. S. (2009). Harrison's Manual of Medicine . New York: McGraw-Hill Professional. Contains the clinical portions of Harrison's Principles of Internal Medicine .
• Lippincott manual of nursing practice (8th ed.). (2006). Philadelphia, PA: Lippincott Williams & Wilkins. Provides background information on clinical nursing practice.
• Medscape: Drugs & Diseases An open-access, point-of-care medical reference that includes clinical information from top physicians and pharmacists in the United States and worldwide.
• Virginia Henderson Global Nursing e-Repository An open-access repository that contains works by nurses and is sponsored by Sigma Theta Tau International, the Honor Society of Nursing. Note: This resource contains both expert opinion and evidence-based practice articles.
• Previous Page: Phrasing Research Questions
• Next Page: Evidence Types
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Systematic Reviews

• Levels of Evidence
• Evidence Pyramid
• Joanna Briggs Institute

The evidence pyramid is often used to illustrate the development of evidence. At the base of the pyramid is animal research and laboratory studies – this is where ideas are first developed. As you progress up the pyramid the amount of information available decreases in volume, but increases in relevance to the clinical setting.

Meta Analysis  – systematic review that uses quantitative methods to synthesize and summarize the results.

Systematic Review  – summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate st atistical techniques to combine these valid studies.

Randomized Controlled Trial – Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Cohort Study – Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Case Control Study – study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Case Series   – report on a series of patients with an outcome of interest. No control group is involved.

• Levels of Evidence from The Centre for Evidence-Based Medicine
• The JBI Model of Evidence Based Healthcare
• How to Use the Evidence: Assessment and Application of Scientific Evidence From the National Health and Medical Research Council (NHMRC) of Australia. Book must be downloaded; not available to read online.

When searching for evidence to answer clinical questions, aim to identify the highest level of available evidence. Evidence hierarchies can help you strategically identify which resources to use for finding evidence, as well as which search results are most likely to be "best".                                             

Image source: Evidence-Based Practice: Study Design from Duke University Medical Center Library & Archives. This work is licensed under a Creativ e Commons Attribution-ShareAlike 4.0 International License .

The hierarchy of evidence (also known as the evidence-based pyramid) is depicted as a triangular representation of the levels of evidence with the strongest evidence at the top which progresses down through evidence with decreasing strength. At the top of the pyramid are research syntheses, such as Meta-Analyses and Systematic Reviews, the strongest forms of evidence. Below research syntheses are primary research studies progressing from experimental studies, such as Randomized Controlled Trials, to observational studies, such as Cohort Studies, Case-Control Studies, Cross-Sectional Studies, Case Series, and Case Reports. Non-Human Animal Studies and Laboratory Studies occupy the lowest level of evidence at the base of the pyramid.

• << Previous: What is a Systematic Review?
• Next: Locating Systematic Reviews >>
• Getting Started
• What is a Systematic Review?
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: Aug 6, 2024 10:17 AM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

Evidence Based Practice Toolkit

• What is EBP?
• Asking Your Question

Levels of Evidence / Evidence Hierarchy

Evidence pyramid (levels of evidence), definitions, research designs in the hierarchy, clinical questions --- research designs.

• Evidence Appraisal
• Find Research
• Standards of Practice

Levels of evidence (sometimes called hierarchy of evidence) are assigned to studies based on the research design, quality of the study, and applicability to patient care. Higher levels of evidence have less risk of bias . 

Levels of Evidence (Melnyk & Fineout-Overholt 2023)

*Adapted from: Melnyk, & Fineout-Overholt, E. (2023).  Evidence-based practice in nursing & healthcare: A guide to best practice   (Fifth edition.). Wolters Kluwer.

Levels of Evidence (LoBiondo-Wood & Haber 2022)

Adapted from LoBiondo-Wood, G. & Haber, J. (2022). Nursing research: Methods and critical appraisal for evidence-based practice (10th ed.). Elsevier.

" Evidence Pyramid " is a product of Tufts University and is licensed under BY-NC-SA license 4.0

Tufts' "Evidence Pyramid" is based in part on the  Oxford Centre for Evidence-Based Medicine: Levels of Evidence (2009)

• Oxford Centre for Evidence Based Medicine Glossary

Different types of clinical questions are best answered by different types of research studies.  You might not always find the highest level of evidence (i.e., systematic review or meta-analysis) to answer your question. When this happens, work your way down to the next highest level of evidence.

This table suggests study designs best suited to answer each type of clinical question.

• << Previous: Asking Your Question
• Next: Evidence Appraisal >>
• Updated: Jul 4, 2024 12:54 PM
• URL: https://libguides.winona.edu/ebptoolkit

• Evidence-Based Medicine
• Finding the Evidence
• eJournals for EBM

Levels of Evidence

• JAMA Users' Guides
• Tutorials (Learning EBM)
• Web Resources

Resources That Rate The Evidence

• ACP Smart Medicine
• Agency for Healthcare Research and Quality
• Clinical Evidence
• Cochrane Library
• Health Services/Technology Assessment Texts (HSTAT)
• PDQ® Cancer Information Summaries from NCI
• Trip Database

Critically Appraised Individual Articles

• Evidence-Based Complementary and Alternative Medicine
• Evidence-Based Dentistry
• Evidence-Based Nursing
• Journal of Evidence-Based Dental Practice

Grades of Recommendation

Critically-appraised individual articles and synopses include:

Filtered evidence:

• Level I: Evidence from a systematic review of all relevant randomized controlled trials.
• Level II: Evidence from a meta-analysis of all relevant randomized controlled trials.
• Level III: Evidence from evidence summaries developed from systematic reviews
• Level IV: Evidence from guidelines developed from systematic reviews
• Level V: Evidence from meta-syntheses of a group of descriptive or qualitative studies
• Level VI: Evidence from evidence summaries of individual studies
• Level VII: Evidence from one properly designed randomized controlled trial

Unfiltered evidence:

• Level VIII: Evidence from nonrandomized controlled clinical trials, nonrandomized clinical trials, cohort studies, case series, case reports, and individual qualitative studies.
• Level IX: Evidence from opinion of authorities and/or reports of expert committee

Two things to remember:

1. Studies in which randomization occurs represent a higher level of evidence than those in which subject selection is not random.

2. Controlled studies carry a higher level of evidence than those in which control groups are not used.

Strength of Recommendation Taxonomy (SORT)

• SORT The American Academy of Family Physicians uses the Strength of Recommendation Taxonomy (SORT) to label key recommendations in clinical review articles. In general, only key recommendations are given a Strength-of-Recommendation grade. Grades are assigned on the basis of the quality and consistency of available evidence.
• << Previous: eJournals for EBM
• Next: JAMA Users' Guides >>
• Last Updated: Jan 25, 2024 4:15 PM
• URL: https://guides.library.stonybrook.edu/evidence-based-medicine
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Systematic Reviews: Levels of evidence and study design

Levels of evidence.

"Levels of Evidence" tables have been developed which outline and grade the best evidence. However, the review question will determine the choice of study design.

Secondary sources provide analysis, synthesis, interpretation and evaluation of primary works. Secondary sources are not evidence, but rather provide a commentary on and discussion of evidence. e.g. systematic review

Primary sources contain the original data and analysis from research studies. No outside evaluation or interpretation is provided. An example of a primary literature source is a peer-reviewed research article. Other primary sources include preprints, theses, reports and conference proceedings.

Levels of evidence for primary sources fall into the following broad categories of study designs   (listed from highest to lowest):

• Experimental : RTC's (Randomised Control Trials)
• Quasi-experimental studies (Non-randomised control studies, Before-and-after study, Interrupted time series)
• Observational studies (Cohort study, Case-control study, Case series) 

Based on information from Centre for Reviews and Dissemination. (2009). Systematic reviews: CRD's guidance for undertaking reviews in health care. Retrieved from http://www.york.ac.uk/inst/crd/index_guidance.htm

Hierarchy of Evidence Pyramid

"Levels of Evidence" are often represented in as a pyramid, with the highest level of evidence at the top:

Types of Study Design

The following definitions are adapted from the Glossary in " Systematic reviews: CRD's Guidance for Undertaking Reviews in Health Care " , Centre for Reviews and Dissemination, University of York :

• Systematic Review The application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic and research question. 
• Meta-analysis A systematic review which uses quantitative methods to summarise the results
• Randomized control clinical trial (RCT) A group of patients is randomised into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.
• Cohort study Involves the identification of two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.
• Case-control study Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking to see if they had the exposure of interest.
• Critically appraised topic A short summary of an article from the literature, created to answer a specific clinical question.

EBM and Study Design

• << Previous: SR protocol
• Next: Searching for systematic reviews >>
• Getting started
• Types of reviews
• Formulate the question
• SR protocol
• Levels of evidence and study design
• Searching for systematic reviews
• Search strategies
• Subject databases
• Keeping up to date/Alerts
• Trial registers
• Conference proceedings
• Critical appraisal
• Documenting and reporting
• Managing search results
• Statistical methods
• Journal information/publishing
• Contact a librarian
• Last Updated: May 15, 2024 11:15 AM
• URL: https://ecu.au.libguides.com/systematic-reviews

Edith Cowan University acknowledges and respects the Noongar people, who are the traditional custodians of the land upon which its campuses stand and its programs operate. In particular ECU pays its respects to the Elders, past and present, of the Noongar people, and embrace their culture, wisdom and knowledge.

Understanding Evidence Levels in Evidence-Based Medicine: A Guide for Healthcare Professionals

• Teaching Hospital Badulla University of Colombo Sri Lanka

• Teaching Hospital, Badulla, SriLanka

• Ministry of Health, Sri Lanka
• This person is not on ResearchGate, or hasn't claimed this research yet.

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• PLAST RECONSTR SURG

• Sandra V. Spilson
• James E McCarthy

• Patricia B Burns
• Kevin C Chung

• DeLaine Schmitz
• Loren Lipworth
• Robert E. Tarone
• Joseph K. McLaughlin
• Kevin C. Chung
• Ashwin N Ram
• INT J CANCER

• D L Sackett
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Nursing-Johns Hopkins Evidence-Based Practice Model

Jhebp model for levels of evidence, jhebp levels of evidence overview.

• Levels I, II and III

Evidence-Based Practice (EBP) uses a rating system to appraise evidence (usually a research study published as a journal article). The level of evidence corresponds to the research study design. Scientific research is considered to be the strongest form of evidence and recommendations from the strongest form of evidence will most likely lead to the best practices. The strength of evidence can vary from study to study based on the methods used and the quality of reporting by the researchers. You will want to seek the highest level of evidence available on your topic (Dang et al., 2022, p. 130).

The Johns Hopkins EBP model uses 3 ratings for the level of scientific research evidence 

• true experimental (level I)
• quasi-experimental (level II)
• nonexperimental (level III) 

The level determination is based on the research meeting the study design requirements  (Dang et al., 2022, p. 146-7).

You will use the Research Appraisal Tool (Appendix E) along with the Evidence Level and Quality Guide (Appendix D) to analyze and  appraise research studies . (Tools linked below.)

N onresearch evidence is covered in Levels IV and V.

• Evidence Level and Quality Guide (Appendix D)
• Research Evidence Appraisal Tool (Appendix E)

Level I Experimental study

randomized controlled trial (RCT)

Systematic review of RCTs, with or without meta-analysis

Level II Quasi-experimental Study

Systematic review of a combination of RCTs and quasi-experimental, or quasi-experimental studies only, with or without meta-analysis.

Level III Non-experimental study

Systematic review of a combination of RCTs, quasi-experimental and non-experimental, or non-experimental studies only, with or without meta-analysis.

Qualitative study or systematic review, with or without meta-analysis

Level IV Opinion of respected authorities and/or nationally recognized expert committees/consensus panels based on scientific evidence.

Clinical practice guidelines

Consensus panels

Level V Based on experiential and non-research evidence.

Literature reviews

Quality improvement, program, or financial evaluation

Case reports

Opinion of nationally recognized expert(s) based on experiential evidence

These flow charts can also help you detemine the level of evidence throigh a series of questions.

Single Quantitative Research Study

Summary/Reviews 

These charts are a part of the Research Evidence Appraisal Tool (Appendix E) document.

Dang, D., Dearholt, S., Bissett, K., Ascenzi, J., & Whalen, M. (2022). Johns Hopkins evidence-based practice for nurses and healthcare professionals: Model and guidelines. 4th ed. Sigma Theta Tau International

• << Previous: Start Here
• Next: Levels I, II and III >>
• Last Updated: Feb 8, 2024 1:24 PM
• URL: https://bradley.libguides.com/jhebp

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 21, Issue 4
• New evidence pyramid
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• M Hassan Murad ,
• Mouaz Alsawas ,
• http://orcid.org/0000-0001-5481-696X Fares Alahdab
• Rochester, Minnesota , USA
• Correspondence to : Dr M Hassan Murad, Evidence-based Practice Center, Mayo Clinic, Rochester, MN 55905, USA; murad.mohammad{at}mayo.edu

https://doi.org/10.1136/ebmed-2016-110401

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• EDUCATION & TRAINING (see Medical Education & Training)
• EPIDEMIOLOGY
• GENERAL MEDICINE (see Internal Medicine)

The first and earliest principle of evidence-based medicine indicated that a hierarchy of evidence exists. Not all evidence is the same. This principle became well known in the early 1990s as practising physicians learnt basic clinical epidemiology skills and started to appraise and apply evidence to their practice. Since evidence was described as a hierarchy, a compelling rationale for a pyramid was made. Evidence-based healthcare practitioners became familiar with this pyramid when reading the literature, applying evidence or teaching students.

Various versions of the evidence pyramid have been described, but all of them focused on showing weaker study designs in the bottom (basic science and case series), followed by case–control and cohort studies in the middle, then randomised controlled trials (RCTs), and at the very top, systematic reviews and meta-analysis. This description is intuitive and likely correct in many instances. The placement of systematic reviews at the top had undergone several alterations in interpretations, but was still thought of as an item in a hierarchy. 1 Most versions of the pyramid clearly represented a hierarchy of internal validity (risk of bias). Some versions incorporated external validity (applicability) in the pyramid by either placing N-1 trials above RCTs (because their results are most applicable to individual patients 2 ) or by separating internal and external validity. 3

Another version (the 6S pyramid) was also developed to describe the sources of evidence that can be used by evidence-based medicine (EBM) practitioners for answering foreground questions, showing a hierarchy ranging from studies, synopses, synthesis, synopses of synthesis, summaries and systems. 4 This hierarchy may imply some sort of increasing validity and applicability although its main purpose is to emphasise that the lower sources of evidence in the hierarchy are least preferred in practice because they require more expertise and time to identify, appraise and apply.

The traditional pyramid was deemed too simplistic at times, thus the importance of leaving room for argument and counterargument for the methodological merit of different designs has been emphasised. 5 Other barriers challenged the placement of systematic reviews and meta-analyses at the top of the pyramid. For instance, heterogeneity (clinical, methodological or statistical) is an inherent limitation of meta-analyses that can be minimised or explained but never eliminated. 6 The methodological intricacies and dilemmas of systematic reviews could potentially result in uncertainty and error. 7 One evaluation of 163 meta-analyses demonstrated that the estimation of treatment outcomes differed substantially depending on the analytical strategy being used. 7 Therefore, we suggest, in this perspective, two visual modifications to the pyramid to illustrate two contemporary methodological principles ( figure 1 ). We provide the rationale and an example for each modification.

• Download figure
• Open in new tab
• Download powerpoint

The proposed new evidence-based medicine pyramid. (A) The traditional pyramid. (B) Revising the pyramid: (1) lines separating the study designs become wavy (Grading of Recommendations Assessment, Development and Evaluation), (2) systematic reviews are ‘chopped off’ the pyramid. (C) The revised pyramid: systematic reviews are a lens through which evidence is viewed (applied).

Rationale for modification 1

In the early 2000s, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group developed a framework in which the certainty in evidence was based on numerous factors and not solely on study design which challenges the pyramid concept. 8 Study design alone appears to be insufficient on its own as a surrogate for risk of bias. Certain methodological limitations of a study, imprecision, inconsistency and indirectness, were factors independent from study design and can affect the quality of evidence derived from any study design. For example, a meta-analysis of RCTs evaluating intensive glycaemic control in non-critically ill hospitalised patients showed a non-significant reduction in mortality (relative risk of 0.95 (95% CI 0.72 to 1.25) 9 ). Allocation concealment and blinding were not adequate in most trials. The quality of this evidence is rated down due to the methodological imitations of the trials and imprecision (wide CI that includes substantial benefit and harm). Hence, despite the fact of having five RCTs, such evidence should not be rated high in any pyramid. The quality of evidence can also be rated up. For example, we are quite certain about the benefits of hip replacement in a patient with disabling hip osteoarthritis. Although not tested in RCTs, the quality of this evidence is rated up despite the study design (non-randomised observational studies). 10

Rationale for modification 2

Another challenge to the notion of having systematic reviews on the top of the evidence pyramid relates to the framework presented in the Journal of the American Medical Association User's Guide on systematic reviews and meta-analysis. The Guide presented a two-step approach in which the credibility of the process of a systematic review is evaluated first (comprehensive literature search, rigorous study selection process, etc). If the systematic review was deemed sufficiently credible, then a second step takes place in which we evaluate the certainty in evidence based on the GRADE approach. 11 In other words, a meta-analysis of well-conducted RCTs at low risk of bias cannot be equated with a meta-analysis of observational studies at higher risk of bias. For example, a meta-analysis of 112 surgical case series showed that in patients with thoracic aortic transection, the mortality rate was significantly lower in patients who underwent endovascular repair, followed by open repair and non-operative management (9%, 19% and 46%, respectively, p<0.01). Clearly, this meta-analysis should not be on top of the pyramid similar to a meta-analysis of RCTs. After all, the evidence remains consistent of non-randomised studies and likely subject to numerous confounders.

Therefore, the second modification to the pyramid is to remove systematic reviews from the top of the pyramid and use them as a lens through which other types of studies should be seen (ie, appraised and applied). The systematic review (the process of selecting the studies) and meta-analysis (the statistical aggregation that produces a single effect size) are tools to consume and apply the evidence by stakeholders.

Implications and limitations

Changing how systematic reviews and meta-analyses are perceived by stakeholders (patients, clinicians and stakeholders) has important implications. For example, the American Heart Association considers evidence derived from meta-analyses to have a level ‘A’ (ie, warrants the most confidence). Re-evaluation of evidence using GRADE shows that level ‘A’ evidence could have been high, moderate, low or of very low quality. 12 The quality of evidence drives the strength of recommendation, which is one of the last translational steps of research, most proximal to patient care.

One of the limitations of all ‘pyramids’ and depictions of evidence hierarchy relates to the underpinning of such schemas. The construct of internal validity may have varying definitions, or be understood differently among evidence consumers. A limitation of considering systematic review and meta-analyses as tools to consume evidence may undermine their role in new discovery (eg, identifying a new side effect that was not demonstrated in individual studies 13 ).

This pyramid can be also used as a teaching tool. EBM teachers can compare it to the existing pyramids to explain how certainty in the evidence (also called quality of evidence) is evaluated. It can be used to teach how evidence-based practitioners can appraise and apply systematic reviews in practice, and to demonstrate the evolution in EBM thinking and the modern understanding of certainty in evidence.

• Leibovici L
• Agoritsas T ,
• Vandvik P ,
• Neumann I , et al
• ↵ Resources for Evidence-Based Practice: The 6S Pyramid. Secondary Resources for Evidence-Based Practice: The 6S Pyramid Feb 18, 2016 4:58 PM. http://hsl.mcmaster.libguides.com/ebm
• Vandenbroucke JP
• Berlin JA ,
• Dechartres A ,
• Altman DG ,
• Trinquart L , et al
• Guyatt GH ,
• Vist GE , et al
• Coburn JA ,
• Coto-Yglesias F , et al
• Sultan S , et al
• Montori VM ,
• Ioannidis JP , et al
• Altayar O ,
• Bennett M , et al
• Nissen SE ,

Contributors MHM conceived the idea and drafted the manuscript. FA helped draft the manuscript and designed the new pyramid. MA and NA helped draft the manuscript.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Linked Articles

• Editorial Pyramids are guides not rules: the evolution of the evidence pyramid Terrence Shaneyfelt BMJ Evidence-Based Medicine 2016; 21 121-122 Published Online First: 12 Jul 2016. doi: 10.1136/ebmed-2016-110498
• Perspective EBHC pyramid 5.0 for accessing preappraised evidence and guidance Brian S Alper R Brian Haynes BMJ Evidence-Based Medicine 2016; 21 123-125 Published Online First: 20 Jun 2016. doi: 10.1136/ebmed-2016-110447

Read the full text or download the PDF:

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

Levels of evidence in research

• 5 minute read
• 114.8K views

Table of Contents

Level of evidence hierarchy

When carrying out a project you might have noticed that while searching for information, there seems to be different levels of credibility given to different types of scientific results. For example, it is not the same to use a systematic review or an expert opinion as a basis for an argument. It’s almost common sense that the first will demonstrate more accurate results than the latter, which ultimately derives from a personal opinion.

In the medical and health care area, for example, it is very important that professionals not only have access to information but also have instruments to determine which evidence is stronger and more trustworthy, building up the confidence to diagnose and treat their patients.

5 levels of evidence

With the increasing need from physicians – as well as scientists of different fields of study-, to know from which kind of research they can expect the best clinical evidence, experts decided to rank this evidence to help them identify the best sources of information to answer their questions. The criteria for ranking evidence is based on the design, methodology, validity and applicability of the different types of studies. The outcome is called “levels of evidence” or “levels of evidence hierarchy”. By organizing a well-defined hierarchy of evidence, academia experts were aiming to help scientists feel confident in using findings from high-ranked evidence in their own work or practice. For Physicians, whose daily activity depends on available clinical evidence to support decision-making, this really helps them to know which evidence to trust the most.

So, by now you know that research can be graded according to the evidential strength determined by different study designs. But how many grades are there? Which evidence should be high-ranked and low-ranked?

There are five levels of evidence in the hierarchy of evidence – being 1 (or in some cases A) for strong and high-quality evidence and 5 (or E) for evidence with effectiveness not established, as you can see in the pyramidal scheme below:

Level 1: (higher quality of evidence) – High-quality randomized trial or prospective study; testing of previously developed diagnostic criteria on consecutive patients; sensible costs and alternatives; values obtained from many studies with multiway sensitivity analyses; systematic review of Level I RCTs and Level I studies.

Level 2: Lesser quality RCT; prospective comparative study; retrospective study; untreated controls from an RCT; lesser quality prospective study; development of diagnostic criteria on consecutive patients; sensible costs and alternatives; values obtained from limited stud- ies; with multiway sensitivity analyses; systematic review of Level II studies or Level I studies with inconsistent results.

Level 3: Case-control study (therapeutic and prognostic studies); retrospective comparative study; study of nonconsecutive patients without consistently applied reference “gold” standard; analyses based on limited alternatives and costs and poor estimates; systematic review of Level III studies.

Level 4: Case series; case-control study (diagnostic studies); poor reference standard; analyses with no sensitivity analyses.

Level 5: (lower quality of evidence) – Expert opinion.

By looking at the pyramid, you can roughly distinguish what type of research gives you the highest quality of evidence and which gives you the lowest. Basically, level 1 and level 2 are filtered information – that means an author has gathered evidence from well-designed studies, with credible results, and has produced findings and conclusions appraised by renowned experts, who consider them valid and strong enough to serve researchers and scientists. Levels 3, 4 and 5 include evidence coming from unfiltered information. Because this evidence hasn’t been appraised by experts, it might be questionable, but not necessarily false or wrong.

Examples of levels of evidence

As you move up the pyramid, you will surely find higher-quality evidence. However, you will notice there is also less research available. So, if there are no resources for you available at the top, you may have to start moving down in order to find the answers you are looking for.

• Systematic Reviews: -Exhaustive summaries of all the existent literature about a certain topic. When drafting a systematic review, authors are expected to deliver a critical assessment and evaluation of all this literature rather than a simple list. Researchers that produce systematic reviews have their own criteria to locate, assemble and evaluate a body of literature.
• Meta-Analysis: Uses quantitative methods to synthesize a combination of results from independent studies. Normally, they function as an overview of clinical trials. Read more: Systematic review vs meta-analysis .
• Critically Appraised Topic: Evaluation of several research studies.
• Critically Appraised Article: Evaluation of individual research studies.
• Randomized Controlled Trial: a clinical trial in which participants or subjects (people that agree to participate in the trial) are randomly divided into groups. Placebo (control) is given to one of the groups whereas the other is treated with medication. This kind of research is key to learning about a treatment’s effectiveness.
• Cohort studies: A longitudinal study design, in which one or more samples called cohorts (individuals sharing a defining characteristic, like a disease) are exposed to an event and monitored prospectively and evaluated in predefined time intervals. They are commonly used to correlate diseases with risk factors and health outcomes.
• Case-Control Study: Selects patients with an outcome of interest (cases) and looks for an exposure factor of interest.
• Background Information/Expert Opinion: Information you can find in encyclopedias, textbooks and handbooks. This kind of evidence just serves as a good foundation for further research – or clinical practice – for it is usually too generalized.

Of course, it is recommended to use level A and/or 1 evidence for more accurate results but that doesn’t mean that all other study designs are unhelpful or useless. It all depends on your research question. Focusing once more on the healthcare and medical field, see how different study designs fit into particular questions, that are not necessarily located at the tip of the pyramid:

• Questions concerning therapy: “Which is the most efficient treatment for my patient?” >> RCT | Cohort studies | Case-Control | Case Studies
• Questions concerning diagnosis: “Which diagnose method should I use?” >> Prospective blind comparison
• Questions concerning prognosis: “How will the patient’s disease will develop over time?” >> Cohort Studies | Case Studies
• Questions concerning etiology: “What are the causes for this disease?” >> RCT | Cohort Studies | Case Studies
• Questions concerning costs: “What is the most cost-effective but safe option for my patient?” >> Economic evaluation
• Questions concerning meaning/quality of life: “What’s the quality of life of my patient going to be like?” >> Qualitative study

Find more about Levels of evidence in research on Pinterest:

17 March 2021 – Elsevier’s Mini Program Launched on WeChat Brings Quality Editing Straight to your Smartphone

Professor anselmo paiva: using computer vision to tackle medical issues with a little help from elsevier author services, you may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Research and Evidence-based Practice: Levels of Evidence and Study Designs

• MCHS Published Research
• Levels of Evidence and Study Designs
• Searching for the Evidence & Critical Appraisal
• Reference and Citation Management
• Writing and Publication
• Searching Grey Literature

Evidence Pyramid

An  evidence pyramid   visually depicts the evidential strength of different research designs. The image below is one of several available renderings of an evidence pyramid. Studies with the highest internal validity, characterized by a high degree of quantitative analysis, review, analysis, and stringent scientific methodoloy, are at the top of the pyramid. Observational research and expert opinion reside at the bottom of the pyramid. 

Which Research Designs for Which Questions?

Different types of research studies are better suited to answer different categories of clinical questions. You might not always find the highest level of evidence (i.e., systematic review or meta-analysis) to answer your question.  When this happens, work your way down the Evidence Pyramid to the next highest level of evidence.

Therapy : Which treatment does more harm than good?

RCT > Cohort Study  >  Case Control > Case Series

Diagnosis : Which diagnostic test should I use?

Prospective, blind comparison to a gold standard, ie. A controlled trial that looks at patients with varying degrees of an illness and administers both diagnostic tests -- the test under investigation and the "gold standard" test -- to all of the patients in the study group.

Prognosis : What is the patient's likely clinical course over time?

Cohort Study > Case Control > Case Series

Etiology / Harm : What are the causes of this disease or condition?

RCT > Cohort Study > Case Control > Case Series

Prevention : How do we reduce the chance of disease by identifying and modifying risk factors?

RCT > Cohort Study > Case Control > Case Series

Cost : Is one intervention more cost-effective than another?

Economic Analysis

Quality of Life : What will be the patient's quality of life following an intervention?

Qualitative Study

Levels of Evidence

Types of Study Designs

Systematic Review:    A summary of the clinical literature. A systematic review is a critical assessment and evaluation of all research studies that address a particular clinical issue. The researchers use an organized method of locating, assembling, and evaluating a body of literature on a particular topic using a set of specific criteria. A systematic review typically includes a description of the findings of the collection of research studies. Cochrane Reviews are the gold standard!  (AHRQ Glossary of Terms)

Meta-Analysis :   A work consisting of studies using a quantitative method of combining the results of independent studies (usually drawn from the published literature) and synthesizing summaries and conclusions which may be used to evaluate therapeutic effectiveness, plan new studies, etc. It is often an overview of clinical trials. It is usually called a meta-analysis by the author or sponsoring body and should be differentiated from reviews of literature. (PubMed)

Evidence Guideline:   Systematically developed statement to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances (Institute of Medicine).  These have a rigorous development process.  An example is AHRQ Guidelines at guidelines.gov or Lippincott  Procedures .

Evidence Summary:   A summary of the evidence.

Randomized Controlled Trial:    A controlled clinical trial that randomly (by chance) assigns participants to two or more groups. There are various methods to randomize study participants to their groups. (AHRQ Glossary of Terms)

Controlled Clinical Trial:   A type of clinical trial comparing the effectiveness of one medication or treatment with the effectiveness of another medication or treatment. In many controlled trials, the other treatment is a placebo (inactive substance) and is considered the "control." (AHRQ Glossary of Terms)

Cohort Study:   A clinical research study in which people who presently have a certain condition or receive a particular treatment are followed over time and compared with another group of people who are not affected by the condition. (AHRQ Glossary of Terms)

Case Control Study :    The observational epidemiologic study of persons with the disease (or other outcome variable) of interest and a suitable control (comparison, reference) group of persons without the disease. The relationship of an attribute to the disease is examined by comparing the diseased and nondiseased with regard to how frequently the attribute is present or, if quantitative, the levels of the attribute, in each of the groups. (OCEBM Table of Evidence Glossary)

Case Series:   A group or series of case reports involving patients who were given similar treatment. Reports of case series usually contain detailed information about the individual patients. This includes demographic information (for example, age, gender, ethnic origin) and information on diagnosis, treatment, response to treatment, and follow-up after treatment. (OCEBM Table of Evidence Glossary)

Case Study :    An investigation of a single subject or a single unit, which could be a small number of individuals who seem to be representative of a larger group or very different from it. (Dictionary of Nursing Theory and Research, Fourth Edition)

Editorial:    Work consisting of a statement of the opinions, beliefs, and policy of the editor or publisher of a journal, usually on current matters of medical or scientific significance to the medical community or society at large. The editorials published by editors of journals representing the official organ of a society or organization are generally substantive. (PubMed)

Opinion:   A belief or conclusion held with confidence but not substantiated by positive knowledge or proof. (The Free Dictionary)

Animal Research:   A laboratory experiment using animals to study the development and progression of diseases. Animal studies also test how safe and effective new treatments are before they are tested in people.(NCI Dictionary of Cancer Terms)

In Vitro Research:   In the laboratory (outside the body). The opposite of in vivo (in the body). (NCI Dictionary of Cancer Terms)

• << Previous: PICO
• Next: Searching for the Evidence & Critical Appraisal >>
• Last Updated: May 30, 2023 4:20 PM
• URL: https://marshfieldclinic.libguides.com/Research_and_EBP

MSU Libraries

• Need help? Ask Us

Nursing Literature Reviews

• Literature and Other Types of Reviews
• Starting Your Search
• Developing a Research Question and the Literature Search Process
• Conducting a Literature Search

Levels of Evidence

• Creating a PRISMA Table
• Literature Table and Synthesis
• Other Resources

Levels of evidence (sometimes called hierarchy of evidence) are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care. These decisions gives the grade (or strength) of recommendation. Just because something is lower on the pyramid doesn't mean that the study itself is lower-quality, it just means that the methods used may not be as clinically rigorous as higher levels of the pyramid. In nursing, the system for assigning levels of evidence is often from Melnyk & Fineout-Overholt's 2011 book,  Evidence-based Practice in Nursing and Healthcare: A Guide to Best Practice .  The Levels of Evidence below are adapted from Melnyk & Fineout-Overholt's (2011) model.  

Melnyk & Fineout-Overholt (2011)

• Meta-Analysis:  A systematic review that uses quantitative methods to summarize the results. (Level 1)
• Systematic Review:  A comprehensive review that authors have systematically searched for, appraised, and summarized all of the medical literature for a specific topic (Level 1)
• Randomized Controlled Trials:  RCT's include a randomized group of patients in an experimental group and a control group. These groups are followed up for the variables/outcomes of interest. Examples of RCTs are clinical trials that compare the effects of drugs, surgical techniques, medical devices, diagnostic procedures, diets or other medical treatments. (can be Level 2 or Level 4, depending on how expansive the study)
• Non-Randomized Controlled Trials:  A clinical trial in which the participants are not assigned by chance to different treatment groups. Participants may choose which group they want to be in, or they may be assigned to the groups by the researchers.
• Cohort Study:  Identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest. ( Level 5)
• Case-Control Study:  Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking to see if they had the exposure of interest.
• Background Information/Expert Opinion:  Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition.  While background information presents a convenient summary, often it takes about three years for this type of literature to be published. (Level 7)
• << Previous: Conducting a Literature Search
• Next: Creating a PRISMA Table >>
• Last Updated: May 10, 2024 9:36 AM
• URL: https://libguides.lib.msu.edu/nursinglitreview

Evidence Based Practice: Study Designs & Evidence Levels

• Databases to Search
• EBP Resources
• Study Designs & Evidence Levels
• How Do I...

Introduction

This section reviews some research definitions and provides commonly used evidence tables.

Levels of Evidence Johns Hopkins Nursing Evidence Based Practice

Dang, D., & Dearholt, S. (2017). Johns Hopkins nursing evidence-based practice: model and guidelines. 3rd ed. Indianapolis, IN: Sigma Theta Tau International. www.hopkinsmedicine.org/evidence-based-practice/ijhn_2017_ebp.html

Identifying the Study Design

The type of study can generally be figured out by looking at three issues:

Q1. What was the aim of the study?

• To simply describe a population (PO questions)  = descriptive
• To quantify the relationship between factors (PICO questions)  =  analytic.

Q2. If analytic, was the intervention randomly allocated?

• Yes?  =  RCT 
• No? = Observational study  

For an observational study, the main type will then depend on the timing of the measurement of outcome, so our third question is:

Q3. When were the outcomes determined?

• Some time after the exposure or intervention? = Cohort study ('prospective study')
• At the same time as the exposure or intervention? = Cross sectional study or survey
• Before the exposure was determined? = Case-control study ('retrospective study' based on recall of the exposure)

Centre for Evidence-Based Medicine (CEBM)

Definitions of Study Types

Case report / Case series:  A report on a series of patients with an outcome of interest. No control group is involved.

Case control study:  A study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Cohort study:  Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Randomized controlled clinical trial:  Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Systematic review:  A summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Meta-analysis:  A systematic review that uses quantitative methods to synthesize and summarize the results.

Meta-synthesis: A systematic approach to the analysis of data across qualitative studies. -- EJ Erwin, MJ Brotherson, JA Summers. Understanding Qualitative Meta-synthesis. Issues and Opportunities in Early Childhood Intervention Research, 33(3) 186-200 .

Cross sectional study:  The observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously.

Prospective, blind comparison to a gold standard:  Studies that show the efficacy of a diagnostic test are also called prospective, blind comparison to a gold standard study. This is a controlled trial that looks at patients with varying degrees of an illness and administers both diagnostic tests — the test under investigation and the “gold standard” test — to all of the patients in the study group. The sensitivity and specificity of the new test are compared to that of the gold standard to determine potential usefulness.

Qualitative research:  answers a wide variety of questions related to human responses to actual or potential health problems.The purpose of qualitative research is to describe, explore and explain the health-related phenomena being studied.

Retrospective cohort:  follows the same direction of inquiry as a cohort study.  Subjects begin with the presence or absence of an exposure or risk factor and are followed until the outcome of interest is observed.  However, this study design uses information that has been collected in the past and kept in files or databases.  Patients are identified for exposure or non-exposures and the data is followed forward to an effect or outcome of interest.

(Adapted from CEBM's Glossary and Duke Libraries' Intro to Evidence-Based Practice )

American Association of Critical Care Nursing-- Levels of Evidence

Level A   Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention or treatment

Level B  Well designed controlled studies, both randomized and nonrandomized, with results that consistently support a specific action, intervention, or treatment

Level C   Qualitative studies, descriptive or correlational studies, integrative reviews, systematic reviews, or randomized controlled trials with inconsistent results

Level D Peer-reviewed professional organizational standards, with clinical studies to support recommendations

Level E Theory-based evidence from expert opinion or multiple case reports

Level M  Manufacturers’ recommendations only  

Armola RR, Bourgault AM, Halm MA, Board RM, Bucher L, Harrington L, Heafey CA, Lee R, Shellner PK, Medina J. (2009) AACN levels of evidence: what's new ?  J.Crit Care Nurse. Aug;29(4):70-3.

Flow Chart of Study Designs

Figure: Flow chart of different types of studies (Q1, 2, and 3 refer to the three questions below  in "Identifying the Study Design" box.) Centre for Evidence-Based Medicine (CEBM)

What is a "Confidence Interval (CI)"?

A confidence interval (CI) can be used to show within which interval the population's mean score will probably fall. Most researchers use a CI of 95%. By using a CI of 95%, researchers accept there is a 5% chance they have made the wrong decision in treatment. Therefore, if 0 falls within the agreed CI, it can be concluded that there is no significant difference between the two treatments. When 0 lies outside the CI, researchers will conclude that there is a statistically significant difference.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics.  Journal Of Wound Care ,  22 (5), 248-251.

What is a "p-value?"

Categorical (nominal) tests This category of tests can be used when the dependent, or outcome, variable is categorical (nominal), such as the dif­ference between two wound treatments and the healing of the wound (healed versus non­healed). One of the most used tests in this category is the chi­squared test (χ2). The chi­squared statistic is calculated by comparing the differences between the observed and the expected frequencies. The expected frequencies are the frequencies that would be found if there was no relationship between the two variables. 

Based on the calculated χ2 statistic, a probability (p ­value) is given, which indicates the probability that the two means are not different from each other. Researchers are often satisfied if the probability is 5% or less, which means that the researchers would conclude that for p < 0.05, there is a significant difference. A p ­value ≥ 0.05 suggests that there is no significant difference between the means.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics. Journal Of Wound Care, 22(5), 248-251.

• << Previous: EBP Resources
• Next: How Do I... >>
• Last Updated: Jul 31, 2024 2:57 PM
• URL: https://mcw.libguides.com/evidencebasedpractice

MCW Libraries 8701 Watertown Plank Road Milwaukee, WI 53226 (414) 955-8300

Contact Us Locations & Hours Send Us Your Comments

Levels of Evidence and Study Design: Levels of Evidence

Levels of evidence.

• Study Design
• Study Design by Question Type
• Rating Systems

This is a general set of levels to aid in critically evaluating evidence. It was adapted from the model presented in the book, Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice  (Melnyk & Fineout-Overholt, 2019). Some specialties may have adopted a slightly different and/or smaller set of levels.

Evidence from a clinical practice guideline based on systematic reviews or meta-analyses of randomized controlled trials. Is this is not available, then evidence from a systematic review or meta-analysis of random controlled trials.

Evidence from randomized controlled studies with good design.

Evidence from controlled trials that have good design but are not randomized.

Evidence from case-control and cohort studies with good design.

Evidence from systematic reviews of qualitative and descriptive studies.

Evidence from qualitative and descriptive studies.

Evidence from the opinion of authorities and/or the reports of expert committees. 

Evidence Pyramid

The pyramid below is a hierarchy of evidence for quantitative studies. It shows the hierarchy of studies by study design; starting with secondary and reappraised studies, then primary studies, and finally reports and opinions, which have no study design. This pyramid is a simplified, amalgamation of information presented in the book chapter “Evidence-based decision making” (Forest et al., 2019) and book Evidence-Based Practice in Nursing and Healthcare: A Guide to Best Practice  (Melnyk & Fineout-Overholt, 2019).

Evidence Table for Nursing

Advocate Health - Midwest provides system-wide evidence based practice resources. The Nursing Hub* has an Evidence-Based Quality Improvement (EBQI) Evidence Table , within the Evidence-Based Practice (EBP) Resource. It also includes information on evidence type, and a literature synthesis table.

*The Nursing Hub requires access to the Advocate Health - Midwest SharePoint platform.

Forrest, J. L., Miller, S. A., Miller, G. W., Elangovan, S., & Newman, M. G. (2019). Evidence-based decision making. In M. G. Newman, H. H. Takei, P. R. Klokkevold, & F. A. Carranza (Eds.),  Newman and Carranza's clinical periodontology  (13th ed., pp. 1-9.e1). Elsevier.

• Melnyk, B. M., & Fineout-Overholt, E. (2019).  Evidence-based practice in nursing and healthcare: A guide to best practice  (4th ed.). Wolters Kluwer. 
• << Previous: Overview
• Next: Study Design >>
• Last Updated: Dec 29, 2023 2:03 PM
• URL: https://library.aah.org/guides/levelsofevidence

University Libraries

• Ohio University Libraries
• Library Guides

Evidence-based Practice in Healthcare

• Levels of Evidence
• EBP Tutorials
• Question- PICO
• Definitions
• Systematic Reviews
• Finding Evidence
• Filter by Study Type
• Too Much or Too Little?
• Critical Appraisal
• Quality Improvement (QI)
• Performing a Literature Review
• Contact - Need Help?

Places to Search for Evidence

• TRIP Find several levels of evidence, including guidelines
• PubMed Can limit search by type of study
• Cochrane Library Find lots of systematic reviews here

Library Session Activities

Is all evidence created equal.

Is All Evidence C reated Equal? No.

The medical literature is immense, but only a small portion of it is immediately useful in answering clinical questions. The literature reports the whole spectrum of the scientific research process -- the long journey from in-vitro studies to double-blind randomized controlled trials. This has been called the "wedge of evidence" or the " pyramid of evidence." (See below) Also see the Oxford Centre for Evidence-based Medicine for another chart .

An understanding of how various levels of evidence are reported and how this literature is organized will help the searcher retrieve the highest levels of evidence for a particular clinical question. High levels of evidence may not exist for all clinical questions because of the nature of medical problems and research and ethical limitations.

Pyramid of Evidence

• PDF of Pyramid of Evidence
• << Previous: Systematic Reviews
• Next: Finding Evidence >>

Nursing - Systematic Reviews: Levels of Evidence

• Levels of Evidence
• Meta-Analyses
• Definitions
• Citation Search
• Write & Cite
• Give Feedback

"How would I use the 6S Model while taking care of a patient?" .cls-1{fill:#fff;stroke:#79a13f;stroke-miterlimit:10;stroke-width:5px;}.cls-2{fill:#79a13f;} The 6S Model is designed to work from the top down, starting with Systems - also referred to as computerized decision support systems (CDSSs). DiCenso et al. describes that, “an evidence-based clinical information system integrates and concisely summarizes all relevant and important research evidence about a clinical problem, is updated as new research evidence becomes available, and automatically links (through an electronic medical record) a specific patient’s circumstances to the relevant information” (2009). Systematic reviews lead up to this type of bio-available level of evidence.

What are systematic reviews, polit–beck evidence hierarchy/levels of evidence scale for therapy questions.

"Figure 2.2 [in context of book] shows our eight-level evidence hierarchy for Therapy/intervention questions. This hierarchy ranks sources of evidence with respect the readiness of an intervention to be put to use in practice" (Polit & Beck, 2021, p. 28). Levels are ranked on risk of bias - level one being the least bias, level eight being the most biased. There are several types of levels of evidence scales designed for answering different questions. "An evidence hierarchy for Prognosis questions, for example, is different from the hierarchy for Therapy questions" (p. 29).

Advantages of Levels of Evidence Scales

"Through controls imposed by manipulation, comparison, and randomization, alternative explanations can be discredited. It is because of this strength that meta-analyses of RCTs, which integrate evidence from multiple experiments, are at the pinnacle of the evidence hierarchies for Therapy questions" (p. 188).

"Tip: Traditional evidence hierarchies or level of evidence scales (e.g., Figure 2.2), rank evidence sources almost exclusively based on the risk of internal validity threats" (p. 217).

Systematic reviews can provide researchers with knowledge that prior evidence shows. This can help clarify established efficacy of a treatment without unnecessary and thus unethical research. Greenhalgh (2019) illustrates this citing Dean Fergusson and colleagues (2005) systematic review on a clinical surgical topic (p. 128).

Limits of Levels of Evidence Scales

Regarding the importance of real-world clinical practice settings, and the conflicting tradeoffs between internal and external validity, Polit and Beck (2021) write, "the first (and most prevalent) approach is to emphasize one and sacrifice another. Most often, it is external validity that is sacrificed. For example, external validity is not even considered in ranking evidence in level of evidence scales" (p. 221). ... From an EBP perspective, it is important to remember that drawing inferences about causal relationships relies not only on how high up on the evidence hierarchy a study is (Figure 2.2), but also, for any given level of the hierarchy, how successful the researcher was in managing study validity and balancing competing validity demands" (p. 222).

Polit and Beck note Levin (2014) that an evidence hierarchy "is not meant to provide a quality rating for evidence retrieved in the search for an answer" (p. 6), and as the Oxford Center for Evidence-Based Medicine concurs that evidence scales are, 'NOT intended to provide you with a definitive judgment about the quality of the evidence. There will inevitably be cases where "lower-level" evidence...will provide stronger than a "higher level" study (Howick et al., 2011, p.2)'" (p. 30).

Level of evidence (e.g., Figure 2.2) + Quality of evidence = Strength of evidence .

The 6S Model of Levels of Evidence

"The 6S hierarchy does not imply a gradient of evidence in terms of quality , but rather in terms of ease in retrieving relevant evidence to address a clinical question. At all levels, the evidence should be assessed for quality and relevance" (Polit & Beck, 2021, p. 24, Tip box).

The 6S Pyramid proposes a structure of quantitative evidence where articles that include pre-appraised and pre-synthesized studies are located at the top of the hierarchy (McMaster U., n.d.).

It can help to consider the level of evidence that a document represents, for example, a scientific article that summarizes and analyses many similar articles may provide more insight than the conclusion of a single research article. This is not to say that summaries can not be flawed, nor does it suggest that rare case studies should be ignored. The aim of health research is the well-being of all people, therefore it is important to use current evidence in light of patient preferences negotiated with clinical expertise.

Other Gradings in Levels of Evidence

While it is accepted that the strongest evidence is derived from meta-analyses, various evidence grading systems exist. for example: The Johns Hopkins Nursing Evidence-Based Practice model ranks evidence from level I to level V, as follows (Seben et al., 2010): Level I: Meta-analysis of randomized clinical trials (RCTs); experimental studies; RCTs Level II: Quasi-experimental studies Level III: Non-experimental or qualitative studies Level IV: Opinions of nationally recognized experts based on research evidence or an expert consensus panel Level V: Opinions of individual experts based on non-research evidence (e.g., case studies, literature reviews, organizational experience, and personal experience) The American Association of Critical-Care Nurses (AACN) evidence level system , updated in 2009, ranks evidence as follows (Armola et al., 2009): Level A: Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention, or treatment Level B: Well-designed, controlled randomized or non-randomized studies with results that consistently support a specific action, intervention, or treatment Level C: Qualitative, descriptive, or correlational studies, integrative or systematic reviews, or RCTs with inconsistent results Level D: Peer-reviewed professional organizational standards, with clinical studies to support recommendations Level E: Theory-based evidence from expert opinion or multiple case reports Level M: Manufacturers’ recommendations (2017)

EBM Pyramid and EBM Page Generator

Unfiltered are resources that are primary sources describing original research. Randomized controlled trials, cohort studies, case-controlled studies, and case series/reports are considered unfiltered information.

Filtered are resources that are secondary sources which summarize and analyze the available evidence. They evaluate the quality of individual studies and often provide recommendations for practice. Systematic reviews, critically-appraised topics, and critically-appraised individual articles are considered filtered information.

Armola, R. R., Bourgault, A. M., Halm, M. A., Board, R. M., Bucher, L., Harrington, L., ... Medina, J. (2009). AACN levels of evidence. What's new? Critical Care Nurse , 29 (4), 70-73. doi:10.4037/ccn2009969

DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. BMJ Evidence-Based Nursing , 12 (4) https://ebn.bmj.com/content/12/4/99.2.short

Fergusson, D., Glass, K. C., Hutton, B., & Shapiro, S. (2005). Randomized controlled trials of Aprotinin in cardiac surgery: Could clinical equipoise have stopped the bleeding?. Clinical Trials , 2 (3), 218-232.

Glover, J., Izzo, D., Odato, K. & Wang, L. (2008). Evidence-based mental health resources . EBM Pyramid and EBM Page Generator. Copyright 2008. All Rights Reserved. Retrieved April 28, 2020 from https://web.archive.org/web/20200219181415/http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html Note. Document removed from host. Old link used with the WayBack Machine of the Internet Archive to retrieve the original webpage on 2/10/21 http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html

Greenhalgh, T. (2019). How to read a paper: The basics of evidence-based medicine and healthcare . (Sixth ed.). Wiley Blackwell.

Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The “4S” evolution of services for finding current best evidence. BMJ Evidence-Based Medicine , 6 (2), 36-38.

Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. BMJ Evidence-Based Medicine , 11 (6), 162-164.

McMaster University (n.d.). 6S Search Pyramid Tool https://www.nccmt.ca/capacity-development/6s-search-pyramid

Polit, D., & Beck, C. (2019). Nursing research: Generating and assessing evidence for nursing practice . Wolters Kluwer Health.

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus

Seben, S., March, K. S., & Pugh, L. C. (2010). Evidence-based practice: The forum approach. American Nurse Today , 5 (11), 32-34.

• Systematic Review from the Encyclopedia of Nursing Research by Cheryl Holly Systematic reviews provide reliable evidential summaries of past research for the busy practitioner. By pooling results from multiple studies, findings are based on multiple populations, conditions, and circumstances. The pooled results of many small and large studies have more precise, powerful, and convincing conclusions (Holly, Salmond, & Saimbert, 2016) [ references in article ]. This scholarly synthesis of research findings and other evidence forms the foundation for evidence-based practice allowing the practitioner to make up-to-date decisions.

Standards & Guides

• Cochrane Handbook for Systematic Reviews of Interventions The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.
• Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.
• Systematic Reviews by The Centre for Reviews and Dissemination "The guidance has been written for those with an understanding of health research but who are new to systematic reviews; those with some experience but who want to learn more; and for commissioners. We hope that experienced systematic reviewers will also find this guidance of value; for example when planning a review in an area that is unfamiliar or with an expanded scope. This guidance might also be useful to those who need to evaluate the quality of systematic reviews, including, for example, anyone with responsibility for implementing systematic review findings" (CRD, 2009, p. vi, "Who should use this guide")

• Carrying out systematic literature reviews: An introduction by Alan Davies Systematic reviews provide a synthesis of evidence for a specific topic of interest, summarising the results of multiple studies to aid in clinical decisions and resource allocation. They remain among the best forms of evidence, and reduce the bias inherent in other methods. A solid understanding of the systematic review process can be of benefit to nurses that carry out such reviews, and for those who make decisions based on them. An overview of the main steps involved in carrying out a systematic review is presented, including some of the common tools and frameworks utilised in this area. This should provide a good starting point for those that are considering embarking on such work, and to aid readers of such reviews in their understanding of the main review components, in order to appraise the quality of a review that may be used to inform subsequent clinical decision making (Davies, 2019, Abstract)
• Papers that summarize other papers (systematic reviews and meta-analyses) by Trisha Greenhalgh ... a systematic review is an overview of primary studies that: contains a statement of objectives, sources and methods; has been conducted in a way that is explicit, transparent and reproducible (Figure 9.1) [ Table found in book chapter ]. The most enduring and reliable systematic reviews, notably those undertaken by the Cochrane Collaboration (discussed later in this chapter), are regularly updated to incorporate new evidence (Greenhalgh, 2020, p. 117, Chapter 9).
• A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals by Juxia Zhang et al. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was released as a standard of reporting systematic reviewers (SRs). However, not all SRs adhere completely to this standard. This study aimed to evaluate the reporting quality of SRs published in the Cochrane Library and paper-based journals (Zhang et al., 2019, Abstract).

Cochrane [Username]. (2016, Jan 27). What are systematic reviews? YouTube. https://www.youtube.com/watch?v=egJlW4vkb1Y

Davies, A. (2019). Carrying out systematic literature reviews: An introduction. British Journal of Nursing , 28 (15), 1008–1014. https://doi-org.ezproxy.simmons.edu/10.12968/bjon.2019.28.15.1008

Greenhalgh, T. (2019). Papers that summarize other papers (systematic reviews and meta-analyses). In How to read a Paper : The basics of evidence-based medicine and healthcare . (Sixth ed., pp. 117-136). Wiley Blackwell.

Holly, C. (2017). Systematic review. In J. Fitzpatrick (Ed.), Encyclopedia of nursing research (4th ed.). Springer Publishing Company. Credo Reference.

Zhang, J., Han, L., Shields, L., Tian, J., & Wang, J. (2019). A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals. Medicine , 98 (49), e18099. https://doi.org/10.1097/MD.0000000000018099

• << Previous: Start
• Next: Meta-Analyses >>
• Last Updated: Nov 3, 2023 1:19 PM
• URL: https://simmons.libguides.com/systematic-reviews

Evidence Based Practice (EBP) Research

• 5-Minute Librarian Consult Registration [synchronous]
• Articles & Search Tools
• Books & Guidelines
• Clinical & Research Sources

Levels of Evidence Based Resources

• Literature Review Types
• Understanding Research & Citing Sources
• Videos & Tutorials

A Quick Way to Find...

Primary research, key primary research, guidelines, clinical trials, systematic reviews, and more!

Trip Database Pro

Critical Appraisal Articles

• A Guide to Critical Appraisal Resources  
• Determining the Level of Evidence: Nonexperimental Research Designs
• Implementation of an enhanced recovery after surgery program (ERAS)

Evidence Hierarchies

Finding and evaluating evidence is the second phase in the Johns Hopkins Evidence-Based Practice Model (JHEBP). Evidence hierarchies guide identifying the best evidence for decision-making based on the rigor of the methods used (level) and the execution of the study or reporting (quality). Appraisal begins with identifying the level of evidence and then the quality.  The combination of level and quality determines the overall determination of the strength of the evidence. 

Finding and Using Health Statistics

This course for librarians and students in health sciences describes different types of health statistics, how they are collected, and where they can be found.

The U.S. Preventive Services Task Force is an independent, volunteer panel of national experts in disease prevention and evidence-based medicine. The Task Force works to improve the health of people nationwide by making evidence-based recommendations about clinical preventive services.

Trip Pro (Robot Analyzer Tool)  

Trip Scores:  Measures the  risk of bias  in randomized controlled and clinical trial studies, and expert opinion or reviews (narrative) versus evidence based literature.

Curtis, Alexa Colgrove PhD, MPH, FNP, PMHNP; Keeler, Courtney PhD. Sampling Design in Nursing Research . AJN, American Journal of Nursing: March 2021 - Volume 121 - Issue 3 - p 53-57.  

The most and least effective studies.

An overview of how research works.

• Johns Hopkins Hierarchy of Evidence Guide 2022 Evidence Levels 1-5.
• Critique Appraisal Example
• Risk of Bias Tools [Cochrane]

Spence J. D. (2020). The need for clinical judgement in the application of evidence-based medicine.  BMJ evidence-based medicine ,  25 (5), 172–177.  https://doi.org/10.1136/bmjebm-2019-111300  

• << Previous: Clinical & Research Sources
• Next: Literature Review Types >>
• Last Updated: Jul 30, 2024 4:55 PM
• URL: https://samuelmerritt.libguides.com/find-ebp

Evidence Synthesis: Systematic Reviews, Scoping Reviews, Etc.

Systematic review stages - an overview, definition and background, courses and tutorials, tips for getting started, attribution and thanks.

• How Can the Libraries Help?
• What's the Difference between Systematic and Scoping Reviews?
• Rapid Review
• Formulate Your Research Question
• Create and Register Your Protocol
• Inclusion and Exclusion Criteria
• Choose Your Databases
• Devise and Translate Your Search Strategy
• Other Evidence Sources
• Screen Your Articles
• Chart/Extract Data
• Present and Summarize Your Results
• Develop Your Research Question
• Critical Appraisal
• Extract Data
• Report Your Results
• Grey Literature
• Software Tools

Social Sciences Librarian

Image Source: What authors DO. Designed by Jessica Kaufman, Cochrane Consumers & Communication Review Group, Centre for Health Communication & Participation, La Trobe University, 2011.

Evidence synthesis is ". ..a type of research method that allows researchers to bring together all relevant information on a research question. This can be useful to identify gaps in knowledge, establish an evidence base for best-practice guidance, or help inform policymakers and practitioners" ( London School of Hygiene and Tropical Medicine ).

There are many different types of evidence synthesis.

Evidence synthesis methods like systematic reviews and scoping reviews have traditionally been used in the health sciences, but have become more prevalent in the social sciences in recent years. Scholars across the social sciences have continued to develop, test, and standardize best practices to use for these emerging methods.

• Evidence Synthesis Institute Videos From University of Minnesota.
• Systematic Reviews and Meta-Analysis A free Open Learning Initiative course from Carnegie Mellon University.

Some things to consider as you embark on a writing an evidence synthesis review.

What type of review will you do?

See the page of this guide " Types of Evidence Synthesis ".

What databases will you search?

How will you track the searching you have already done?  **Link to literature review spreadsheet.**

Use library guides and talk to librarians.

Has a recent review already been done about the topic of your research?.

Search to see if the review you want to write has already been written, or to determine if your review can extend the literature in some way. 

Don't forget to check if relevant review articles you find have been cited. This is one way (among others) to discover new literature, including other reviews. 

Consider a protocol and pre-registration

Look at sample reviews in your subject area and see what protocols they use.

Additional ideas/tips:

• Find a small set of "seed articles" that are directly related to your topic and use them to test your search strategy.
• Consider setting up an email alert to monitor new literature about your subject and new items that cite literature of interest.

Special thanks go to Julia Maxwell at Rutgers University, who has graciously allowed her guide to serve as a template.

This guide would not have been possible without the guidance, support, and feedback of RBHS librarians Yingting Zhang and Matthew Bridgeman .

This guide has benefitted from the excellent work of many other university libraries, including Monash University , University of South Australia, University of North Carolina , Cornell University , University of British Columbia , University of Melbourne , CQ University , University of Maryland , and Duquesne University  Libraries.

• Next: How Can the Libraries Help? >>
• Last Updated: Aug 6, 2024 10:53 AM
• URL: https://libraryguides.lehigh.edu/systematicreviews

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Bone Radiation-Induced Sarcomas: Outcomes Based on Histology and Surgical Treatment: A Systematic Review of the Literature

Affiliations.

• 1 Facultad de Medicina, Universidad Peruana Cayetano Heredia, Lima, Peru.
• 2 Division of Orthopaedic Oncology, Department of Orthopaedic Surgery, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts.
• 3 Division of Orthopedic Oncology, Miami Cancer Institute, Baptist Health System South Florida, Plantation, Florida.
• PMID: 39102470
• DOI: 10.2106/JBJS.RVW.24.00066

Background: Bone radiation-induced sarcomas (B-RIS) are secondary neoplasms with reportedly worse overall survival than de novo bone sarcoma. Treatment strategy for these neoplasms remains uncertain. Our systematic review sought to assess overall survival based on histology and surgical intervention.

Methods: A systemic review was conducted following Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines and registered in PROSPERO (438415). Studies describing oncologic outcomes of patients with B-RIS in the appendicular and axial skeleton were included. The Strengthening the Reporting of Observational Studies in Epidemiology checklist was used for quality assessment. Survival analysis by histologic subtype and surgery type was performed in a subset of 234 patients from 11 articles with individualized data. A total of 20 articles with a total of 566 patients were included. The most frequent location was the pelvis (27.7%), and the main histological types were osteosarcoma (69.4%), undifferentiated pleomorphic sarcoma (14.1%), and fibrosarcoma (9.2%). Limb-salvage and amputation were performed in 68.5% and 31.5% of cases, respectively.

Results: Local recurrence was 13%, without difference between limb-salvage surgery and amputation (p = 0.51). The metastasis rate was 42.3%. Five-year OS was 43.7% (95% confidence interval [CI], 33.3%-53.5%) for osteosarcoma, 31.5% (95% CI, 11.3%-54.2%) for UPS, and 28.1% (95% CI, 10.6%-48.8%) for fibrosarcoma. Five-year OS was 49.2% (95% CI, 35.3%-61.6%) for limb-salvage and 46.9% (95% CI, 29.1%-62.9%) for amputation. There was no difference in 5-year OS between histologic subtypes (p = 0.18) or treatment type (p = 0.86).

Conclusion: B-RIS demonstrated poor OS at 5 years after initial management regardless of histology. Limb-salvage surgery was not associated with lower 5-year OS compared with amputation. Future studies should compare both groups while controlling for confounders.

Level of evidence: Level III. See Instructions for Authors for a complete description of levels of evidence.

Copyright © 2024 by The Journal of Bone and Joint Surgery, Incorporated.

PubMed Disclaimer

Conflict of interest statement

Disclosure: The Disclosure of Potential Conflicts of Interest forms are provided with the online version of the article (http://links.lww.com/JBJSREV/B124).

• Beck A. Zur Frage des Rontgensarkoms, zugleich ein Beitrag zur Pathogenese des Sarkoms. Munchener Med Wochenschr. 1922;69:623-5.
• Cahan WG, Woodard HQ, Higinbotham NL, Stewart FW, Coley BL. Sarcoma arising in irradiated bone; report of 11 cases. Cancer. 1948;1(1):3-29.
• Yap J, Chuba PJ, Thomas R, Aref A, Lucas D, Severson RK, Hamre M. Sarcoma as a second malignancy after treatment for breast cancer. Int J Radiat Oncol Biol Phys. 2002;52(5):1231-7.
• Spałek MJ, Czarnecka AM, Rutkowski P. The management of radiation-induced sarcomas: a cohort analysis from a sarcoma tertiary center. J Clin Med. 2021;10(4):694.
• Lewis VO, Raymond K, Mirza AN, Lin P, Yasko AW. Outcome of postradiation osteosarcoma does not correlate with chemotherapy response. Clin Orthop Relat Res. 2006;450:60-6.

Publication types

• Search in MeSH

Related information

Linkout - more resources, full text sources.

• Wolters Kluwer
• MedlinePlus Health Information

Research Materials

• NCI CPTC Antibody Characterization Program

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Sports Phys Ther
• v.7(5); 2012 Oct

LEVELS OF EVIDENCE IN MEDICINE

Peter mcnair.

1 Health and Rehabilitation Research Institute, Auckland University of Technology, Auckland, New Zealand

Levels of evidence allow clinicians to appreciate the quality of a particular research paper quickly. The levels are generally set out in a hierarchical order, which is based largely upon the experimental design. While there are ideal designs for studies examining the effects of interventions, risk factors for a clinical condition or diagnostic testing, in most instances researchers have had to make compromises and these subsequently decrease the quality of their work. This paper provides information concerning how those compromises relate to subsequent levels that are given to a piece of research. It also provides an understanding of issues related to evaluating papers, and suggest ways in which the reader might discern how relevant a paper might be to one's clinical practice.

INTRODUCTION

During the 1990s, the term evidence based medicine (EBM) became notably more apparent in research and clinical literature. As the name suggests, it referred to examining the research evidence for making clinical decisions, and as such it was more firmly grounded in the assessment of the science supporting clinical decision‐making, rather than a reliance on the experiences and subjective perceptions of so called authorities or experts. 1 For EBM to have credibility, there needed to be a systematic manner in which clinical research was assessed, and this demanded the development of levels of evidence to ultimately appreciate and assess the quality of research available in answering a particular clinical question. Initially, efforts on assessment of quality were focused upon intervention studies, examining the degree of effectiveness of treatments for clinical disorders, however, in recent years such efforts have expanded to include other key clinical research areas such as diagnosis and risk factors. The purpose of this paper is to describe the key elements that determine the levels of evidence that subsequently allow the most appropriate or efficacious clinical decision to be made for the patient.

STUDY DESIGN HIERARCHIES PROVIDE AN INITIAL STARTING POINT

Physical Therapists are often interested in studies that involve treatment interventions, identifying risk factors for succumbing to an injury or disease, and diagnosis of clinical conditions. In each of these areas, there are a number of different study designs that can be implemented. These designs may dictate the potential importance of the studies findings in its field. The design that a researcher chooses should be that which most appropriately answers the question being posed. 2 However in many cases, it reflects the resources that researchers have at their disposal and the practicalities of undertaking the research. Resources required for studies may involve physical space and equipment, expertise in data collection, administrative processing of data, statisticians for analyzing data, and patient availability. In most cases, a researcher does not have the opportunity to cover all of these resources to the maximum level possible. Because of this, compromises are made and these often affect the choice of design to be utilised during the research process.

In studies concerning interventions, risk factors and diagnosis, the strength of an experimental paper's design is rated upon a scale that has 4‐5 levels and may be regarded as a hierarchy with level 1 being the highest. In the current paper, the hierarchies presented are based on those recommended by the National Health and Medical Research Council of Australia. 3 However, there are others 4 and they generally follow the same pattern, being different only in the alphanumeric nomenclature given to the levels of the hierarchy (eg: 1a or IIa etc). While one design may be high in the hierarchy for a particular question to be answered, it may not fare so well for a different question. For instance, while a prospective cohort study may be very effective at identifying risk factors, such a design does not provide professionals with the best evidence of a treatment's effect on a particular clinical condition. For the latter, a randomised controlled trial (RCT) would be more appropriate. Thus, it is important to recognise that different study designs have particular features that may make them advantageous for answering a certain type of research question.

If possible, always look for systematic reviews when searching the literature. A Level 1 rating is reserved for a systematic review of the experimental papers. In such a paper, the quality of the designs and the findings of all the individual experimental papers are assessed in a systematic manner to provide an overall assessment or answer for a particular study question. However, it should be noted that not all systematic reviews automatically reach Level 1. If the papers that were reviewed were primarily of studies with poor designs, then the strength of evidence for the providing the answer to the question posed is lower, and the systematic review no matter how well it was conducted will not receive Level 1 status. 5 Thus, the experimental papers upon which the review is based should determine the validity and strength of the review's findings.

Even when a systematic review has utilised papers with the strongest possible designs, the professional needs to appreciate a number of other factors that will influence its importance. These include the number of papers that have been reported upon, and the consistency of the results across papers. One should also appreciate the degree to which the findings apply to the clinical population of interest and what the implications are in respect to applying them in clinical practice, that is, could they be reasonably implemented. On the above‐mentioned scale, the highest quality experimental designs are rated with a Level 2 and lesser‐rated designs receive Levels that decline to 4‐5.

Interventions

For studies examining treatment interventions, randomised controlled trials (RCTs) provide Level II evidence, the strongest level of evidence below a systematic review. Not surprisingly, the two key criteria for these study designs are the incorporation of at least one control group and the randomisation of participants. 6 Without a control group, it is impossible to determine how participants would have changed over time without the experimental intervention. For instance, changes may have occurred due to disease progression or spontaneous recovery. The specific conclusions that can be drawn regarding the experimental intervention are critically dependent on what the control group receives. For example, researchers could compare the effects of icing on acute knee pain to a control group who received no specific intervention, or they could give the control group a bag of peas that are at room temperature to place over their knee for the same period of time. In the first example, the only conclusion that could be drawn is that icing is more effective at reducing pain than no treatment, whereas in the latter example, by controlling for effects associated with receiving a physical intervention to the knee and for the time of application, a researcher could therefore make more specific conclusions regarding the effects of ice itself. In terms of randomisation, the crucial criterion for a RCT is that neither the participant nor the experimenter should be able to predict which group the participant will be allocated to. Commonly accepted randomisation procedures include a coin toss, random number generator, drawing group allocation from an envelope. While researchers may design more complex procedures to ensure that group characteristics are matched on important factors and that participant numbers are balanced between groups, the final determination of group allocation for each participant should be due to chance alone.

One step down from an RCT is a pseudo‐RCT, which provides Level III‐1 evidence. In these study designs, there is still an appropriate control group but group allocation is not strictly randomised. Group allocation in pseudo‐RCTs is dictated by a set rule such as date of birth or participant number. These are weaker randomisation procedures as the experimenter can have knowledge of the group to which a participant will be assigned. The ability to predict group allocation introduces bias into the study as this knowledge can affect the decision about whether to enter the participant into the trial, which may bias the results of the trial overall.

The next level of evidence, Level III‐2, incorporates non‐randomised controlled trials and two types of observational studies. Non‐randomised controlled trials have marked group selection bias. For example, participants may allocate themselves into groups by choosing to receive a treatment, or participants presenting to a particular treatment provider might be always allocated to the experimental intervention and those that present to another treatment provider might receive a control intervention only. Observational designs include cohorts in which a group of people who are exposed to a particular intervention are followed over time and their health outcomes compared to a similar group of people who were not exposed to the intervention. Another example of an observational study is the case‐control design, in which people with a selected condition are identified and their history of exposure to an intervention is compared to a similar group of people who do not have the condition. In all of these study designs, the researchers are not in control of group randomisation and thus the potential for selection bias is substantially higher than in RCTs. This selection bias means that there will be an inherent risk that confounding factors, or factors other than the intervention of interest, are influencing the results of the study. However, it is important to recognise that there are some research questions and interventions to which researchers cannot apply the principles of randomisation and have subjects assigned to different groups., e.g. abortion or obesity, or whether parachutes are an effective life saver. In such situations, the observational designs are the best or only alternative, and hence they can be extremely valuable. 7

The final group of studies providing Level III evidence (Level III‐3) are comparative studies with non‐controlled designs. These are non‐randomised studies where a group of people receiving the intervention of interest are compared with previous or historical information, or to another group receiving another intervention in another study. The key limitation of these studies is the lack of a concurrent control group, and thus it is not possible to determine the specific effects of the intervention in the population as there is not a suitable comparative group. The attempt to make up for the lack of a control group by comparing to historical data or other studies provides an improvement over non‐comparative studies (see case series below), but is still limited. For example, comparison to historical data on disease progression may be confounded by changes in disease management, specific characteristics of the participants tested, or variations in the assessment of outcome measures.

The lowest level of evidence (Level IV) is provided by case series that have no comparison group. These are usually pre‐test ‐ post‐test comparisons of outcomes following an intervention in a single group. Obviously, the lack of a control comparison severely limits the strength of the findings and the conclusions that could be drawn. These study designs will often incorporate the addition of a second pre‐test measure following a baseline, control period. This control period and additional baseline measure marginally strengthen the design of the study by enabling participants to serve “as their own control”. Case series study designs are commonly used for feasibility studies to demonstrate the potential efficacy, safety, or practicality of an intervention before implementation in a larger, more robust study. 8

Risk factors

In the intervention section above, we described observational study designs such as the prospective cohort and the case control. While not the best choice of design for examining interventions where subjects can be randomised into groups, they can be very powerful in the study of risk factors associated with the development of clinical conditions. 9 In the aetiology hierarchy, the strongest of the observational studies is the prospective cohort receiving level II. As the name suggests, it follows a group of similar individuals (eg: forestry workers) over time to examine whether a particular factor (eg: vibration from chain saw use) influences the occurrence of an outcome (osteoarthritis in the hand). A key point is that the occurrence of the outcome has not occurred at the commencement of the study. Such a design allows a consistent measurement of exposure across all the study participants and consistent measurement of the criteria that determines the outcome (eg: the presence of osteoarthritis in the hand). Cohort designs can be prospective or retrospective with the latter being at a lower hierarchal level. The key difference is that the data related to the exposure and the outcome has already been collected in the retrospective design. In many instances, the risk factor and/or outcome of interest was not the reason for the original study. 10 For example, while a prospective study may have primarily been run to examine vibration levels as a risk factor for osteoarthritis of the hand in forestry workers, data might also have been collected on specific safety procedures and injuries that occurred in this cohort. Such data can be linked retrospectively and associations between variables can provide important findings. However, because the retrospective study was not the original intention, the same degree of standardisation of the data collection procedures and the precision in which they were collected is unlikely to have been undertaken and therefore the design is not as strong as a prospective study.

At the next level in the hierarchy of designs for examining risk factors is the case‐control study. In this design two groups are identified, one that has a clinical condition of interest, and another that does not. For instance, a group of forestry workers with osteoarthritis of the hand would be the case group and they would be compared to a group of forestry workers without osteoarthritis of the hand. That comparison might involve examining potential physical risk factors, (e.g. tools used, tasks performed, times and volume of work) that were undertaken by both groups over a specified time to highlight a risk factor or set of factors that are different across the groups. This design is weaker than the cohort design as only the outcome (osteoarthritis of the hand) has the potential to have been measured in a standardised and precise manner. 10 Even then, one of the most notable criticisms of this design is that the criteria for being included in either the control or case groups may be insufficient to accurately represent those of the wider population with and without the condition of interest. 9 This is particularly so, when the case‐control design is targeting risk factors for a rare condition. Characterising risk factors associated with rare conditions is a key strength of the case control. The alternative, if one were to use a prospective cohort, means waiting for sufficient cases to contract a disease so that its risk factors might be characterised well, and that may never eventuate.

Cross sectional study designs and case series form the lowest level of the aetiology hierarchy. In the cross sectional design, data concerning each subject is often recorded at one point in time. For instance, a questionnaire might be sent to a district where forestry is a predominant industry. It might ask about the presence of osteoarthritis in the hand. In doing so, the prevalence of the disorder can be established. Some information related to exposure might also be collected and associations might be observed, but it is difficult to be confident in the validity of these associations. Thus, information gained from the cross‐sectional study is often a starting point that provides the impetus to use a more powerful design to substantiate the initial findings.

For diagnostic studies, the basic design utilized is very similar across most studies, and the higher levels of the hierarchy are based on meeting specific methodological criteria within that design. To receive Level II strength, the design is usually a prospective cohort, and the comparison it makes between a diagnostic test and a reference standard requires the following criteria: 11 All subjects should receive the reference standard, and that standard should be the best evidence available for determining whether the condition of interest is present. For studies, involving primary care, this will often be a scanning or electrophysiological procedure and might also include an anaesthetic block, while in studies involving tertiary care patients, the reference standard is often what is observed at surgery. The diagnostic test and the reference standard should also be completely independent of one another. It is crucial that the reference standard and the diagnostic tests are clearly described so that others can replicate them. The persons performing the diagnostic tests on the patients should not have knowledge of the results of the reference standard and similarly those performing the reference standard should have no knowledge of the results of the diagnostic test. The patients participating in the study must be well described, and represent those with mild as well as severe levels of the condition of interest who are recruited in a consecutive manner, and at the end of the study they are all accounted for.

Studies where the subjects are not consecutively recruited are assigned level III‐1 strength. When the criteria relating to reference standards are partially compromised, a study is regarded as level III‐2. When a study uses a group of subjects that don't include a wide spectrum of those likely to have the condition, or don't identify specific potential sub‐groupings that might affect the results, it is assigned level III‐3. Such studies are often case‐control designs where there are narrow criteria for inclusion in either the case or control groups, which can ultimately affect the generalizability of the results. 12 The lowest level (IV) is reserved for those studies that lack a reference standard.

IRRESPECTIVE OF DESIGN, THE QUALITY OF STUDIES IS IMPORTANT

While hierarchies provide the professional with a guide to how well a study design might answer a question, one must also consider how well that design has been implemented. 5 Within each design, there is a set of criteria that should be subscribed to, to make the design as robust as possible. The RCT may be at level II on the design hierarchy, and hence a good choice of design for studies examining the effects of an intervention. However, if that RCT has insufficient subject numbers to detect a reasonable difference across groups or blinding of subjects was not undertaken, or there were notable dropouts, then one should question the value of the results from that study, despite the design being the most appropriate. A study with a design lower on the hierarchy that has been undertaken well may provide more valid information.

There are numerous scales or checklists to choose from within the literature to assess the quality of individual research studies across the domains of interventions, aetiology, and diagnosis. The key sources of bias that might threaten the validity of the results of studies generally relates to the selection of patients, randomization, therapeutic regime, withdrawals, blinding, and statistical analyses. 6 Be aware that some checklists are extremely extensive 13 and include questions on issues that may not actually have the potential to bias the results, which is the primary reason for your assessment of the methodological quality.

The answers to checklist questions concerning methodological issues may be categorical (eg: bias present or not) or may be graded (e.g. 1 to 4). In some instances, the answers are weighted according to how important the checklist developer thought the bias might affect the results. Generally, the weightings of checklist questions have been subjectively applied with little if any empirical support, and subsequently total scores across checklists can be quite different. 6 Where weighting has not been applied across questions, the assumption is that all issues are of the same value and that is arguably not so. In light of these potential issues, at the Cochrane Collaboration Higgins et al 14 have indicated that readers refrain from giving an overall score to a paper on its methodological quality, but rather to identify whether methodological quality criteria have been met or not met, and in the latter case, how relevant the issue might be to the size of the effects observed in the study. This strategy makes it much harder for an individual to discern whether a particular paper is one that should be given more or less consideration, in respect to clinical decisions to be made. If clinicians are expected to assess the merits of individual experimental papers, this is an area that must be addressed further for more types of studies. Key sources of questionnaires for assessing the quality of intervention, risk factor and diagnostic studies are provided by Higgins et al, 14 Hayden et al, 15 Bossuyt et al, 16 and Whiting et al, 17 respectively.

APPLYING WHAT IS FOUND IN THE LITERATURE TO CLINICAL SCENARIOS

Assuming that papers have been identified that perform well from a methodological perspective, and their designs are well placed on the hierarchy for answering a particular question, finding papers that include participants who are similar to the patient(s) of interest to the professional is important. Such consideration should include an assessment of the level of severity of the groups under study (eg: mildly, moderately or severely affected), together with the amount of treatment they were being given, and the timing of that treatment within their disease/injury healing process. Furthermore, check when the researchers made their assessments to determine change in the participant's status. Ask whether these are realistic time points to do an assessment, and if the follow up was appropriate to determine the longer‐term effects.

It is also important that clinicians look beyond the treatment effect of an intervention to get a balanced view of its merits. Consideration should be made not only of the benefits but also the potential harm associated with a particular treatment. For instance, a new regime for treating acute muscle tears might be developed and shown in a well‐conducted RCT to allow players to return to sports much earlier than anything currently available. However, that same regime may induce side effects, perhaps a greater likelihood of the injury recurring 6‐12 months later due to the laying down of excessive scar tissue in the early stages of the rehabilitation regime. Examination of such points will allow the professional to make a better judgement concerning the relevance of the papers to the clinical decision at hand.

GUIDELINES PROVIDE A SYSTEMATIC REVIEW AND A SET OF RECOMMENDATIONS

Based on the information presented above, it would seem a monumental task for therapists to assess a series of individual papers and thereafter make an informed decision concerning every clinical problem that they face, particularly those where the patient is atypical, and does not resemble the subjects presented in studies. To make the task easier, guidelines have been developed to answer specific clinical problems/questions and provide recommendations. Because of the resources required, guidelines are usually initiated by organisations such as specialist groups in a field of medicine/allied health or a national health agency. These organisations convene a guidelines panel that is usually composed of scientists, clinical specialists, statisticians, patients and lay people, and they are supported by data analysts and administrators. Their first step is to identify the question of interest and the key outcomes associated with that question. They then assess systematic reviews (Level 1 evidence in the hierarchy) that have been previously published or specifically undertake their own systematic review. In doing so, they provide a summary of the quality of the research undertaken, the consistency of the results across studies, the magnitudes of the intervention's effects observed in patient subgroups, the benefits versus the potential harm associated with a treatment, and whether the health benefits of a treatment are worth the costs of providing them. 18 Most importantly though, guidelines include recommendations and these are often quite definitive, being categorised as ‘strong’ or ‘weak’. Guyatt et al 19 describe these as reflecting a trade off between the benefits of treatment against the burdens of receiving it together with its risks; while taking into account the accuracy and strength of the data supporting the intervention. If the data analysed from experimental papers indicates that an intervention has a large effect and the risks and burdens associated with the treatment are low, then a strong recommendation can be made to implement it. Where there are inconsistencies in findings or small treatment effects or notable risks, the recommendation for the treatment/intervention might be regarded as ‘weak’, and the patient's particular circumstances may then play a greater role in whether a particular treatment is implemented.

Given the extent and thoroughness behind the construction of guidelines and the inclusion of recommendations, they are an important source for guiding clinical decision making and should be searched for early in your examination of the literature.

THINK BEYOND THE SCIENCE

While the current paper has focused upon the quantitative assessment of evidence, it cannot be regarded as the sole means by which professionals make clinical decisions. It is important that therapists continue to appreciate the individuality of each patient and the personal circumstances that they bring with their pathophysiological issues. While at present, qualitative research does not have a formal place in levels of evidence, there is without doubt evidence for its importance in providing insights into patients' viewpoints on how the clinical condition and its treatment has influenced the lives that they lead.

Therefore, professionals must continue to value highly how we interact and react to each patient's situation, continually striving to be effective listeners and communicators, as well as being advocates of the best research evidence to help all patients improve the quality of their lives.

Key Points summary.

• Open access
• Published: 01 August 2024

Literature review of complementary and alternative therapies: using text mining and analysis of trends in nursing research

• Jihye Nam   ORCID: orcid.org/0000-0002-5534-2660 1 ,
• Hyejin Lee   ORCID: orcid.org/0000-0002-8501-0560 1 ,
• Seunghyeon Lee   ORCID: orcid.org/0009-0005-6411-364X 1 &
• Hyojung Park   ORCID: orcid.org/0000-0002-7804-0593 1  

BMC Nursing volume  23 , Article number:  526 ( 2024 ) Cite this article

211 Accesses

Metrics details

This study aimed to review the literature on complementary and alternative therapies, utilizing text mining and trend analysis in nursing research. As CAM becomes increasingly prevalent in healthcare settings, a comprehensive understanding of the current research landscape is essential to guide evidence-based practice, inform clinical decision-making, and ultimately enhance patient outcomes.

This study aimed to identify CAM-related literature published from 2018 to 2023. Using the search terms 'complementary therap*', 'complementary medicine', 'alternative therap*', and 'alternative medicine', we performed a comprehensive search in eight databases, including EMBASE, Cochrane Central, PubMed Central, Korea Education and Research Information Service (RISS), Web of Science, KMbase, KISS, and CINAHL. From the text network and topic modeling analysis of 66,490 documents, 15 topics were identified. These topics were classified into two nursing-related topics through an academic classification process involving three doctors with doctoral degrees, three nurses, and three pharmacists. Based on the classified topics, research trends were comparatively analyzed by re-searching the database for 12 nursing and 22 non-nursing literature.

This study found that in nursing literature, yoga is used to improve mental symptoms such as stress and anxiety. In non-nursing literature, most of the experimental studies on complementary and alternative therapies were conducted in a randomized manner, confirming that a variety of physiological and objective indicators were used. Additionally, it was discovered that there were differences in the diversity of research subjects and research design methods for the same intervention method. Therefore, future research should focus on broadening the scope of subjects and measurement tools in nursing studies. Additionally, such studies should be conducted with randomization and generalizability in the experimental design in mind.

This study employed text network analysis and text mining to identify domestic and international CAM research trends. Our novel approach combined big data-derived keywords with a systematic classification method, proposing a new methodological strategy for trend analysis. Future nursing research should focus on broadening the scope of subjects, diversifying measurement tools, and emphasizing randomization and generalizability in experimental designs.

Peer Review reports

The World Health Organization (WHO) defines complementary and alternative medicine (CAM) as healthcare practices outside a country's traditional or conventional medicine [ 1 ]. According to the National Center for Complementary and Integrative Health (NCCIH), CAM encompasses nutritional approaches (e.g., herbs), psychological methods (e.g., mindfulness), physical therapies (e.g., massage), integrated mind–body practices (e.g., yoga or auricular acupressure), and modalities that combine psychology and nutrition [ 2 ]. This definition suggests CAM may facilitate holistic nursing by addressing both psychological and physical aspects [ 3 ]. Consequently, substantial CAM research is conducted in nursing internationally [ 4 , 5 ], spanning areas like pain, depression, anxiety, chronic disease symptoms, sleep disturbances, and vomiting [ 4 , 5 , 6 ]. Classification systems exist, with the Korean Nursing Association (2023) delineating 12 CAM subcategories [ 6 ] and NCCIH outlining 76 therapies across major categories like nutrition, body, and psychotherapies [ 4 , 5 , 6 , 7 ]. The multitude of CAM types has prompted trend identification research, including reviews on Chinese medicine for allergic rhinitis, aromatherapy, auricular acupressure, and CAM for COVID-19 [ 6 , 8 ]. However, many previous studies have significant limitations in comprehensively identifying overall research trends in CAM. First, they tend to focus narrowly on specific diseases or treatments, lacking a broader perspective on the field as a whole [ 6 , 8 ]. Second, the use of search queries containing keywords from a specific discipline or arbitrarily selected by researchers introduces bias and hinders the identification of overarching trends [ 9 , 10 ]. These limitations highlight the need for a more systematic and data-driven approach to analyzing CAM research trends [ 11 , 12 , 13 ]. A previous study [ 14 , 15 ] suggested the use of text mining technique as an approach for literature review [ 16 ]. To date, the analysis on research trend in nursing has been conducted more than five years after publication or has only been conducted with partial analyses through literature reviews and text mining [ 17 , 18 , 19 ].

The overarching goal was to extract keywords identifying domestic and international CAM research trends using text network analysis and analyze these trends within the nursing field. Specific objectives were: 1) Identify frequency, degree centrality, closeness centrality, and betweenness centrality for keywords appearing in domestic and international CAM studies; 2) Identify key themes within these studies; 3) Discern nursing keywords among sub-topic groups; 4) Analyze and compare nursing and other disciplinary literature based on findings; and 5) Analyze the trend of CAM in nursing based on extracted nursing keywords.

Study design and methodological framework

This study employs a novel methodological framework that combines text mining techniques with expert validation to identify and analyze CAM research trends in a comprehensive and data-driven manner. The framework consists of the following key steps.

Data collection: A comprehensive search of multiple databases is conducted to collect a broad range of CAM-related literature across various disciplines.

Text preprocessing involves several techniques to prepare the data for analysis. These include natural language processing, stopword removal, and synonym standardization.

Keyword extraction and network analysis: Text mining techniques, including term frequency-inverse document frequency (TF-IDF) and centrality analysis, are applied to extract key topics and analyze their relationships within the literature.

Topic modeling: Latent Dirichlet Allocation (LDA) is used to identify latent topics within the literature and visualize their proportions and relationships.

Expert validation: An expert panel of physicians, nurses, and pharmacists is consulted to validate the relevance and credibility of the identified topics and classify them into respective academic fields.

Focused literature analysis: Based on the expert-validated nursing-related topics, a focused re-search and analysis of the literature are conducted to identify trends specific to nursing research on CAM.

This multi-step framework allows for a more comprehensive and less biased exploration of CAM research trends by leveraging text mining techniques to process large volumes of literature, identify key topics, and uncover patterns that may not be apparent through traditional review methods [ 14 , 15 , 16 ]. The integration of expert validation ensures the relevance and credibility of the findings, while the focused analysis of nursing literature provides insights specific to the nursing discipline within the broader context of CAM research. The process of selecting studies for our analysis is illustrated in Fig.  1 , which provides a clear visual representation of the key steps involved, from the initial database search to the final classification of studies into nursing and other disciplines. This multi-step approach, combined with the visual aid, enhances the clarity and transparency of our methodology, allowing readers to better understand and contextualize the subsequent data analysis steps.

Flow diagram for literature selection process

Literature selection

This study focused on complementary and alternative medicine studies conducted in the fields of medicine, public health, and nursing in Korea and abroad. After specifying the research title and abstract as the search scope to extract the literature and build a database, the literature related to nursing was classified based on the topics derived through text network analysis and then, the literature that met the selection criteria was secondarily extracted and analyzed through the abstract screening. The three researchers checked the consistency of the study selection process and if there was any discrepancy, the final decision was made through consensus among the researchers.

The selection criteria for the literature were: (1) domestic and foreign studies published within the last five years (January 2018 to September 2023) that conducted studies on complementary and alternative medicine; and

The exclusion criteria for the literature were grey literatures, dissertations, and studies for which original texts are not available.

Data collection strategies

In this study, the database was selected by referring to the COSI (Core, Standard, Ideal) [ 20 ] model presented by the National Library of Medicine for literature search. EMBASE, Cochrane Central, and PubMed Central were selected as the core databases.

On the other hand, the standard databases selected were the Cumulative Index to Nursing and Allied Health Literature (CINAHL); and Korean database services such as the Research Information Sharing Service (RISS), KMbase, and Korean studies Information Service System (KISS). These Korean databases were included to ensure a comprehensive coverage of potentially relevant studies published in South Korea, as they index a wide range of domestic and international journals, conference proceedings, and dissertations across various disciplines, including those related to CAM. However, it is important to note that the inclusion of these Korean databases does not limit the scope of our study to Korean literature only, as the majority of our analysis focuses on studies published in English and indexed in the core and standard international databases.

In addition, the Web of Science was selected to include a wider range of literature for the ideal database, and the period of literature search focused on the last five years, from 1 January 2018 to 15 September 2023. to capture the most recent trends in CAM research following the last comprehensive analysis of CAM research trends conducted in 2018 by Sung et al. [ 19 ]. This time frame was chosen to provide an updated and comprehensive analysis of CAM research trends, building upon the findings of previous studies and identifying new patterns and areas of focus that have emerged in recent years, given the rapid evolution of CAM research and the increasing integration of CAM into mainstream healthcare.

The data collection procedure was limited for both domestic and foreign studies. In case of foreign studies, ‘English’ was limited as the search language, ‘abstract and title’ were identified as the field, ‘article’ was set as the document form, and the keywords were ‘complementary therap*,’ ‘complementary medicine,’ ‘alternative therap*,’ and ‘alternative medicine.’ For the Korean studies, ‘Korean’ was limited as the search language, ‘abstract and title’ were identified as the field, ‘article’ was set as the document form, and the search keywords used were ‘보완대체,’ ‘대체요법,’ and ‘대체의학.’ In searching for the secondary literature, studies in the field of nursing were presented to a group of nine experts including physicians, nurses, and pharmacists with a master's degree or higher, and then the relevant areas were classified to extract the keywords. These keywords were then used in the text mining search. Topic words, the majority of which were classified as nursing, were re-searched in the collected database. The literature selection and classification process were carried out independently by three researchers and promoted through discussions between the researchers.

Data analysis process

Data extraction.

A comprehensive literature search was conducted across eight databases: CINAHL, Cochrane, EMBASE, KISS, Kmbase, PubMed, RISS, and Web of Science. This extensive search yielded a total of 77,062 studies. To ensure the integrity and non-redundancy of our dataset, we employed a rigorous two-step deduplication process. First, we utilized the 'Find Duplicates' function in EndNote software for initial automatic deduplication. This function systematically identifies and groups potential duplicate records based on shared metadata such as title, authors, year, and DOI. Through this automated process, 12,107 duplicate records were identified and removed.

Following the automated process, we conducted a manual review to identify and remove any remaining duplicates that the software might have missed. This careful manual screening allowed us to catch subtle duplicates that automated systems might overlook, such as those with slight variations in titles or author names. Through this manual review, an additional 465 duplicate records were identified and removed. In total, our rigorous two-step deduplication process resulted in the removal of 12,572 duplicate records. Of these, 12,107 were removed through automated deduplication and 465 through manual review. After deduplication, 64,490 unique studies were retained for further analysis. These studies were systematically organized by title and subjected to a thorough text preprocessing phase. During this phase, unstructured words were sorted and cleaned using the social networking program Netminer 4.3.3 and text editor Notepad + + (version 8.5.8).

Also, stopwords such as pronouns, adverbs, and numbers were deleted through natural language processing, while exception list, defined words, and thesaurus were registered. The exception list and thesaurus were selected by the three researchers, and if they failed to reach a unanimous agreement, the keywords were refined through consultation and the abstracts and preambles were reviewed again to examine the context in which the words were used. In case of the exception list, literature search keywords or stopwords such as pronouns, adverbs, numbers, and special symbols were considered, while ‘complementary,’ ‘medicine,’ ‘alternative,’ ‘therapeutic,’ ‘therapy,’ ‘therap,’ ‘therapies,’ ‘the,’ ‘a,’ ‘and,’ ‘of,’ ‘for,’ ‘in,’ ‘to,’ and ‘among’ were excluded. Special symbols like ‘’,:'"()&-?# <  >  + "",‘ were excluded as well. As for defined words, ‘cells → cell,’ ‘effects → effect,’ ‘staphylococcus aureus → staphylococcus,’ ‘aureus → staphylococcus,’ ‘characteristics → characterization,’ ‘efficacy → effect,’ ‘rat → mice,’ ‘radio → radiation,’ ‘systems → system,’ ‘agents → agent,’ ‘activity → activation,’ ‘carcinoma → cancer,’ ‘cases → case,’ ‘mouse → mice,’ ‘practices → practice,’ ‘radio sensitization → radiation,’ ‘years → year,’ ‘α → alpha,’ and ‘β → beta’ were selected, and data sorting for synonyms was performed. As a result of the analysis, a database consisting of 464,625 words was constructed.

Data analysis

In this study, text mining and topic modeling analysis were employed using textom and RStudio (4.3) to identify keywords related to CAM. Word analysis, TF-IDF, and degree centrality analysis were performed through text mining, with results presented via visualization. TF-IDF determines if a keyword holds actual significance within a document, as words with high TF and TF-IDF values appear frequently and are more likely keywords or important terms [ 21 , 22 ]. Following previous studies [ 22 , 23 ], the minimum word length was set to two, with the top 20 words extracted per topic. Text network analysis created word networks expressing co-occurrence frequency as links [ 24 ]. To gauge word occurrence frequency, words were converted to word-word one-mode, and degree centrality analysis identified highly influential network words. The results of these analyses, including frequency, TF-IDF, degree centrality, closeness centrality, and betweenness centrality of core keywords, can be found in Table 1 .

This study utilized Latent Dirichlet Allocation (LDA) for topic modeling, a statistical method that estimates the probability distribution of topics within documents based on the Document Term Matrix (DTM). Following established practices in the literature, we set the Markov Chain Monte Carlo (MCMC) parameters to alpha = 1.44, beta = 0.001, and iterations = 1,000 [ 25 ]. To determine the optimal number of topics, we iteratively tested configurations ranging from 1 to 20 topics. Through a combination of silhouette clustering analysis and researcher consensus, we identified that a 15-topic model best represented the research trends in our corpus.

LDA visualization indicated that larger topic sizes represented greater proportions within the analyzed studies [ 25 ]. We confirmed that the ideal number of topics, where topics do not overlap and have distinct boundaries, is 15, as shown in Fig.  2 . To validate the relevance and credibility of the topic modeling results, we consulted an expert panel consisting of physicians, nurses, and pharmacists with master's or doctoral degrees. The panel members were asked to classify the 15 derived topics into their respective academic fields. Based on the survey results, two topics (Topics 4 and 7) were identified as nursing-related, with the majority of the expert panel categorizing them as such.

LDA topic modeling visualization

Using the words from these two nursing-related topics, a keyword search was conducted within the database to identify the final set of literature containing these terms. The selected literature was then classified as either nursing-related or non-nursing-related based on the following criteria: (1) the study was published by a nursing school or department, (2) the authors were nurses or nursing researchers, (3) the authors were hospital-affiliated nurses, or (4) the study was published in a nursing journal. The classification process was carried out independently by three authors, and the final categorization was determined through a verification process among them.

Literature review

After the three researchers re-searched the database built based on the sub-words of the extracted topics, a total of 35 articles were selected, including 13 nursing-related literatures and 22 other discipline-related literatures. The sub-words used for the re-search were derived from Topic 4 and Topic 7 in Table 2  and were classified using the PICO (Population, Intervention, Comparison, Outcome) framework. The population-related sub-words included 'patient,' 'students,' and 'nursing.' The intervention-related sub-words were 'yoga,' 'treatment,' 'radiation,' 'acupuncture,' 'education,' and 'cam.' The comparison-related sub-word was 'placebo,' and the outcome-related sub-words included 'anxiety,' 'depression,' 'symptoms,' 'knowledge,' 'attitudes,' and 'perceptions.' These PICO-classified sub-words were used to conduct the database re-search.

In order to examine the research trends in nursing and other related fields, general characteristics (author, country of publication, year of publication) and research characteristics (research design model, statistical method, research subject, intervention method, outcome variable, measurement instruments) were identified, presented, and compared. Meanwhile, the three researchers independently prepared a characteristic table to ensure the accuracy of the extracted contents and if there was any discrepancy, one data was selected through the discussion process until a consensus was reached and a characteristic table was constructed.

To assess the quality of the selected studies, we employed the Mixed Methods Appraisal Tool (MMAT), a concise tool designed to evaluate various study designs, including qualitative, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed methods studies [ 26 ]. This comprehensive tool allowed us to systematically evaluate the methodological rigor of our diverse selection of studies. Each study was evaluated against five MMAT criteria specific to its design, focusing on aspects such as research question appropriateness, data collection methods, and result interpretation. Our assessment revealed varying levels of methodological quality. Among nursing studies (A1-A12), 25% were high quality (5/5 criteria met), 58.3% moderate quality (4/5 criteria), and 16.7% low quality (3/5 criteria). Importantly, all included studies met at least 3 out of the 5 MMAT criteria, indicating an overall moderate to high quality across the selected literature. This suggests that the studies included in our analysis provide a reliable foundation for drawing conclusions. Studies that did not meet all criteria were carefully reviewed, and their potential limitations were considered when interpreting their findings. The MMAT provided a useful overview of study quality and was deemed suitable for assessing methodological rigor while maintaining the feasibility of our analysis. This approach ensured a balanced and nuanced interpretation of the evidence in the field of complementary and alternative medicine. The detailed results of this quality assessment can be found in Tables 3  and 4 .

Data collection and ethical considerations

Since the data used in this study did not contain information that can identify individuals, the study was conducted after obtaining an IRB approval (IRB No: ewha-202311–0008-01) from the Institutional Review Board of Ewha Womans University.

Analysis of word frequency and centrality

The frequency and percentage of the top 20 words related to complementary and alternative medicine are shown in Table 1 . The frequency and percentage of the top 20 words related to complementary and alternative medicine are shown in Table 1 . The table presents the top 20 keywords ranked by frequency, TF-IDF, degree centrality, closeness centrality, and betweenness centrality. The frequency column indicates the number of times each keyword appears in the analyzed documents, while the TF-IDF column represents the importance of each keyword within the entire document set. Degree centrality, closeness centrality, and betweenness centrality are network analysis measures that indicate the importance and influence of each keyword within the text network. The words with the highest frequency included ‘cell’ (7,653 times), ‘patient’ (6,910 times), ‘treatment’ (6,851 times), ‘cancer’ (6,722 times), ‘study’ (6,295 times), and ‘effect’ (6,203 times). The words with the highest values of TF-IDF, in order, were ‘cell,’ ‘effect,’ ‘cancer,’ ‘patient,’ ‘treatment,’ and ‘study.’ As a result of centrality analysis, the top six common words, in order, were ‘effect,’ ‘treatment,’ ‘study,’ ‘analysis,’ ‘disease,’ and ‘approach.’ Except for common words, the words with the highest values in the centrality analysis, in order, were ‘model,’ ‘patient,’ ‘activation,’ and ‘use.’ The words with the highest values for closeness centrality were ‘factor,’ ‘model,’ ‘patient,’ and ‘activation,’ while the words with the highest values for betweenness centrality were ‘factor,’ ‘model,’ ‘type,’ and ‘activation.’

Results of the topic modeling

The LDA visualization provides insights into the relative importance and distinctiveness of identified topics. In this visualization, the size of each topic circle is proportional to its prevalence within the analyzed corpus, with larger circles indicating topics that are more frequently discussed across the literature. Interestingly, we observed that some topics, despite being represented by smaller circles, were positioned at considerable distances from other topics. This spatial separation suggests that these topics, while perhaps less prevalent, possess high discriminant validity and represent distinct thematic areas within the field of complementary and alternative medicine research. This interpretation is consistent with established principles in topic modeling, where spatial relationships in visualizations can indicate semantic distinctiveness. An expert panel of 9 individuals (3 doctors, 3 nurses, and 3 pharmacists), each holding a master's or doctoral degree, conducted a survey to classify the topics based on the keywords. The topic that received the most votes from the panel was designated as the representative field for that topic. Based on the resulting values of the topic modeling, 20 sub-words for each topic were presented and provided in Table 2 , Topics 1–3, 5–6, and 9–15 were classified as Medicine, Topics 4 and 7 as Nursing, and Topics 8 and 10 as Pharmacology.

The process of selecting studies for our analysis is illustrated in Fig.  2 . To determine the optimal number of topics for our analysis, we conducted Latent Dirichlet Allocation (LDA) visualization. As Greene et al. [ 25 ] suggest, larger topic sizes in LDA visualization indicate a greater proportion of that topic within the analyzed studies. We tested topic numbers ranging from 1 to 20, seeking a configuration where topics were visually distinct and non-overlapping. This approach aligns with Liu et al. [ 24 ], who note that topics with high discriminant validity appear as small but clearly separated clusters. After careful visual analysis, we determined that 15 topics provided the most coherent and distinct groupings, as shown in Fig.  2 . This visualization demonstrates the independence and non-overlapping nature of our identified topics, supporting the robustness of our topic modeling approach. Based on the resulting values of the topic modeling, 20 sub-words for each topic were presented and provided in Table 2 . The expert panel's classification suggested that Topics 4 and 7 had relevance to nursing research. However, upon closer examination of the keywords included in these topics, it became apparent that they also encompassed literature from other medical disciplines. While the expert panel's classification indicated these topics were nursing-related, the presence of medical terminology suggested a broader interdisciplinary scope. This highlighted the limitations in identifying nursing-specific research using the current topic modeling approach. To address this issue and clarify the nursing-specific research within these topics, a further refinement of the literature search was conducted using the PICO framework. The keywords from Topics 4 and 7 were used to formulate a focused research question and search strategy. This targeted approach yielded a final selection of 12 nursing-specific articles and 22 articles from other disciplines. By employing the PICO framework and leveraging the keywords from the identified nursing-related topics, it was possible to isolate the nursing research within the broader interdisciplinary landscape.

The words included in topic 4 were the following: ‘trial,’ ‘effect,’ ‘yoga,’ ‘treatment,’ ‘radiation,’ ‘phage,’ ‘protocol,’ ‘anxiety,’ ‘dose,’ ‘zinc,’ ‘symptoms,’ ‘depression,’ ‘placebo,’ ‘acupuncture,’ ‘feasibility,’ ‘training,’ ‘insights,’ ‘toxicity,’ ‘mri,’ and ‘emergency.’ The words included in the topic 7 were: ‘role,’ ‘survey,’ ‘practice,’ ‘evidence,’ ‘failure,’ ‘utilization,’ ‘heart,’ ‘students,’ ‘cam,’ education,’ ‘healthcare,’ ‘valve,’ ‘knowledge,’ ‘communication,’ ‘narrative,’ ‘practitioners,’ ‘attitudes,’ ‘nursing,’ ‘perceptions,’ and ‘pseudomonas.’

The characteristics of the 12 studies included in the literature review analysis are shown in Table 3 .

Of the 12 final literature selections in nursing, there were four randomized controlled trials [A2] [A4] [A7] [A8], three non-randomized comparative trials [A3] [A5] [A6], four descriptive survey studies [A1] [A9] [A10] [A11], and one qualitative study [A12]. Regarding the country of the study’s publication, there were five studies from the United States, three from the United Kingdom, two from Germany and Turkey, and one from Australia. As for the statistical techniques that appeared with high frequency, 10 studies, which were [A1] [A2] [A3] [A4] [A5] [A7] [A8] [A9] [A10] [A11,] used independent t-test, and it was used in most studies. On the other hand, χ2 test was used in seven studies [A3] [A4] [A7] [A8] [A9] [A10] [A11] and one-way analysis of variance was used in four studies [A1] [A9] [A10] [A11]. Regarding the studies that were conducted targeting patients, there was one study conducted on cancer patients [A5], one study on women with post-traumatic stress disorder caused by a car accident [A8], one study on hypertension patients [A7], and one study on breast cancer patients undergoing chemoradiotherapy [A4]. There were seven studies conducted on medical staffs [A1] [A3] [A6] [A9] [A10] [A11] [A12] and one study conducted on nursing students [A2]. Among the interventional therapies used in clinical trials, the most common one was yoga, which was identified in three studies. Specifically, there was one study that used yoga therapy for chemotherapy patients [A5], laughter yoga for nursing students [A2], and yoga therapy for women with post-traumatic disorder [A8]. There were also studies conducted on virtual cancer education program [A6], education on complementary and alternative medicine [A3], auricular acupressure for hypertensive patients [A7], and music therapy for those with breast cancer [A4]. In the studies conducted among medical professionals and nursing students, knowledge [A1] [A3] [A6] [A9] [A10] [A11], attitudes [A1] [A3] [A10] [A11], and usage surveys [A1] [A11] were identified as measurement variables, whereas depression [A8], pain [A7], quality of life [A7], and anxiety [A8] [A4] were identified as the measurement variables in the studies conducted on patients.

Other disciplines

The detailed characteristics of these studies, including the study design, sample, intervention, statistical methods, and outcome measures, are presented in Table 4 .

Of the 22 final literature selections in other disciplines, there were 20 randomized controlled trials [B1] [B2] [B3] [B5] [B6] [B7] [B8] [B9] [B10] [B11] [B12] [B13] [B14] [B15] [B16] [B17] [B18] [B20] [B21] [B22], one pre- and post-hoc comparative study [B4], and one scoping review [B19]. The detailed characteristics of these studies, including the study design, sample, intervention, statistical methods, and outcome measures, are presented in Table 4 . Regarding the country of the study's publication, there were seven studies from the United States of America and the United Kingdom, three studies from China, two studies from the Netherlands, and one study each from Germany, India, and Hong Kong. As for the statistical techniques that appeared with high frequency, there were 10 studies that used independent t-test [B2] [B3] [B5] [B6] [B8] [B11] [B13] [B15] [B18] and one-way ANOVA [B3] [B6] [B7] [B9] [B11] [B14] [B18] [B21] [B20] [B22], while seven studies used repeated measures ANOVA [B2] [B4] [B10] [B11] [B15] [B20] [B22]. All studies for the literature review were conducted on patients. The most common intervention used was auricular acupressure, which was applied on patients with Parkinson’s disease [B11], poststroke depression [B6] [B14], insomnia and depression [B20] [B21], carpal tunnel syndrome [B7], soldiers with PTSD [B19], migraine [B15], pelvic organ prolapse [B8], and gallbladder stones [B22]. The second most common intervention used was yoga therapy, and the subjects were those with active arthritis [B18], generalized anxiety disorder [B17], hemodialysis [B4], and hypertension [B2]. Other subjects and interventions shown in the studies were the following: irritable bladder syndrome patients treated with cinnamon patch [B13]; depression patients treated with bouldering psychotherapy [B12]; dementia patients treated with aromatherapy [B10]; insomnia patients treated with Tai-chi and meridian pressure [B9]; Crohn’s disease patients treated with moxibustion [B3]; HIV patients treated with green tea [B5]; and peripheral arterial disease patients treated with laser acupuncture [B1]. On the other hand, the following were identified as the measurement variables for yoga intervention: level of depression, arthritis stage, anxiety level, quality of life, treatment response rate, sleep, and autonomic function [B2] [B4] [B16] [B17] [B18]. Measurement variables for auricular acupressure included level of depression, sleep quality, level of pain, physical and psychological symptoms, severity of depressive symptoms pelvic organ prolapse, and gastrointestinal symptoms [B3] [B6] [B7] [B8] [B11] [B14] [B15] [B19] [B20] [B21] [B22].

In the study conducted using cinnamon patches, the overactive bladder symptom scores and residual urine volume after urination were identified [B13]. In the study which used green tea, the level of depression was assessed while measuring the severity of depressive symptoms through bouldering [B12]. In the study that used aromatherapy, the behavior, psychology, daily living ability, and cognitive function of the patients with dementia were also assessed [B10].

The present study employed text mining techniques to analyze the literature on CAM published over the past five years and identify trends in nursing research. The text network analysis revealed keywords with high TF-IDF and degree centrality, such as 'cell', 'patient', 'treatment', 'cancer', 'study', and 'effect', suggesting a strong focus on cellular mechanisms, patient-centered approaches, and treatment effects, particularly in the context of cancer [ 22 , 23 ]. The high centrality of these keywords indicates their importance and influence within the broader network of CAM research [ 24 , 25 ]. The topic modeling approach identified 15 major topics, providing a comprehensive overview of the key areas of focus in recent CAM research. This data-driven method offers a more nuanced understanding of research trends compared to previous studies that relied on arbitrary searches or focused on narrow populations or interventions [ 27 , 28 , 29 , 30 , 31 ]. By employing this systematic approach, the present study captures the breadth and diversity of CAM research, overcoming the limitations of previous nursing studies.

An expert panel of 9 individuals (3 doctors, 3 nurses, and 3 pharmacists), each holding a master's or doctoral degree, conducted a survey to classify topics based on keywords. According to the expert classification results shown in Table 2 , Topics 1–3, 5–6, and 9–15 were classified as Medicine, Topics 4 and 7 as Nursing, and Topics 8 and 10 as Pharmacology. While Topics 4 and 7 were found to be nursing-related, closer examination revealed the presence of literature from other medical disciplines within these topics. To address this issue and clarify the nursing-specific research, a further refinement of the literature search was conducted using the PICO framework. The keywords from Topics 4 and 7 were used to formulate a focused research question and search strategy, yielding a final selection of 34 articles, with 12 nursing-specific articles and 22 articles from other disciplines. Analyzing trends in nursing and interdisciplinary studies within the context of the existing literature provides a more comprehensive understanding of CAM research trends. From a nursing perspective, the identification of topics related to patient care, such as symptom management, quality of life, and patient education, highlights the potential for CAM interventions to improve patient outcomes and experiences. The prominence of keywords such as 'patient', 'treatment', and 'effect' highlights the need for evidence-based practice and the need for rigorous studies to evaluate the efficacy and safety of CAM interventions in nursing care. Furthermore, the expert panel's validation of Topics 4 and 7 as relevant to nursing research emphasizes the relevance of these areas within the nursing discipline. Topic 4, which includes keywords such as 'trial', 'effect', 'yoga', 'anxiety', and 'depression', suggests a focus on the psychological benefits of CAM interventions, particularly in the context of clinical trials. This aligns with the growing recognition of the importance of holistic, patient-centered care in nursing practice [ 3 , 4 ]. Topic 7, which includes keywords such as 'practice', 'evidence', 'education', 'knowledge', and 'attitudes', highlights the importance of evidence-based practice and the need for nurse education and training in CAM. As CAM interventions become increasingly popular among patients, it is crucial for nurses to have the knowledge and skills needed to provide safe and effective care [ 5 , 6 ]. The insights gained from this study highlight the potential of text mining and topic modeling techniques for investigating research trends in various fields [ 11 , 12 , 13 ]. By leveraging these methods, researchers can systematically analyze large volumes of literature, identify key areas of focus, and uncover patterns and trends that may not be apparent through traditional review methods [ 14 , 15 ]. This approach can lead to a more comprehensive understanding of the current state of research and inform future directions for investigation.

In conclusion, the present study demonstrates the value of text mining and topic modeling techniques in analyzing research trends, particularly in the field of CAM [ 9 , 10 ]. The systematic approach employed in this study allowed for a more comprehensive and data-driven exploration of the research landscape, overcoming the limitations of previous studies and providing valuable insights into the trends in nursing research on CAM. The findings of this study have significant implications for nursing practice, highlighting the need for evidence-based approaches, patient-centered care, and the integration of CAM interventions into nursing education and training. Future studies should consider adopting similar methodological approaches to investigate research trends in other fields, as this can lead to a more complete understanding of the current state of research and inform future directions for investigation.

The trends analysis of nursing and interdisciplinary studies on CAM revealed notable differences in research design, subject characteristics, intervention types, and assessment methods. Nursing studies exhibited a more balanced distribution of research designs, including randomized controlled trials [A2, A4, A7, A8], non-randomized comparative trials [A3, A5, A6], descriptive survey studies [A1, A9-A11], and a qualitative study [A12]. In contrast, other disciplines predominantly utilized experimental designs, with 95.2% of the studies being randomized controlled trials [B1-B3, B5-B18, B20-B22]. This disparity suggests that nursing research on CAM should expand its focus on experimental studies to enhance the evidence base and align with the methodological approaches of other disciplines.

The subject characteristics of nursing studies differed significantly from those of other disciplines, with nursing research primarily focusing on healthcare professionals and students [A1, A3, A6, A9-A12], while other disciplines exclusively studied patient populations [B1-B22]. This highlights the need for nursing research to diversify its study subjects and investigate the effects of CAM interventions on patients and healthcare providers [ 28 , 29 ], as well as broader community and general health populations [ 3 , 6 ]. By expanding its scope, nursing research can provide valuable insights into the effectiveness and applicability of CAM interventions in promoting health and well-being across diverse settings and populations [ 4 , 5 , 7 , 8 ]. Nurses, as frontline healthcare providers, are uniquely positioned to bridge the gap between healthcare settings and the community, engaging with patients and community members to assess their health needs and provide evidence-based recommendations for CAM interventions [ 1 , 2 ]. This expanded focus, coupled with interdisciplinary collaboration and knowledge exchange [ 9 , 10 ], can lead to the development of innovative, culturally sensitive, and evidence-based CAM interventions that address the complex health needs of individuals and communities alike.

A closer examination of the intervention types in nursing studies reveals that although they focused on a relatively limited range of CAM modalities, such as yoga [A2, A5, A8] and auricular acupressure [A7], these interventions demonstrated promising potential for managing various symptoms and conditions. For instance, yoga was found to be effective in reducing psychological symptoms and cortisol levels in college students [A2], alleviating chemotherapy-related symptoms in cancer patients [A5], and improving post-traumatic stress disorder among traffic accident survivors [A8]. Similarly, auricular acupressure was shown to help decrease angina symptoms in hypertensive patients [A7]. These research findings suggest that even though the scope of CAM interventions in nursing research may be limited, they can provide significant benefits to diverse patient populations [ 2 , 4 , 22 ]. In contrast, the wide array of CAM interventions investigated in other disciplines, such as aromatherapy for dementia [B10], green tea for depression in HIV patients [B5], laser acupuncture for peripheral arterial disease [B1], cinnamon patch for irritable bladder syndrome [B13], bouldering psychotherapy for depression [B12], Tai-chi and meridian pressure for insomnia [B9], and moxibustion for Crohn's disease [B3], demonstrates the potential for nursing research to explore and apply new therapies. The safety, efficacy, and potential of these diverse CAM modalities, as evidenced in other disciplines [ 23 , 24 ], should encourage nursing researchers to investigate their applicability in patient care. By conducting rigorous studies on the safety and efficacy of various CAM interventions, nursing research can provide valuable evidence to support the integration of complementary therapies into nursing practice [ 2 , 4 , 22 ]. Moreover, this trends analysis emphasizes the importance of studying CAM interventions for chronic disease management. With the increasing prevalence of chronic conditions [ 1 , 9 , 10 ], nursing research can play a pivotal role in evaluating the effectiveness of CAM for managing these diseases. Studies on yoga for hypertension [B2] and arthritis [B18], auricular acupressure for insomnia and depression [B20, B21], and moxibustion for Crohn's disease [B3] demonstrate the potential of CAM in improving patient outcomes and quality of life. As nurses have more direct and prolonged contact with patients compared to other healthcare professionals, they are well-positioned to assess the effectiveness of CAM interventions in both clinical and community settings [ 3 , 5 ]. By conducting well-designed studies on the safety and efficacy of various CAM modalities, nursing research can provide the necessary evidence to support the integration of complementary therapies into chronic disease management plans, ultimately enhancing patient care and outcomes across diverse settings. Leveraging their unique role in patient care and conducting rigorous studies on the safety and efficacy of various CAM interventions, particularly for chronic disease management, can enable nursing research to make significant contributions to the integration of complementary therapies into nursing practice. This approach has the potential to not only improve patient outcomes and experiences but also strengthen the evidence base for CAM in healthcare, fostering interdisciplinary collaboration in CAM research and advancing the field of nursing.

The analysis of assessment methods revealed that nursing studies heavily relied on self-developed measurement instruments (58.3%) [A3, A5, A6, A9-A12], while other disciplines predominantly used previously validated tools [B1-B22]. Furthermore, nursing studies rarely incorporated physiological indicators (8.3%) [A2], in contrast to the more frequent use of such measures in other disciplines (36.3%) [B1-B22]. These findings underscore the importance of utilizing validated assessment tools and physiological indicators in nursing research to enhance the reliability and validity of study results [ 31 ]. By incorporating these objective measures, nursing research can more clearly identify significant factors and strengthen the level of evidence, ultimately improving the credibility and applicability of the results.

The trends analysis of statistical techniques revealed a higher prevalence of independent t-tests in nursing research (83.3%) [A1-A5, A7-A11], while other disciplines showed a more balanced use of various techniques, including one-way ANOVA (45.5%) [B3, B6, B7, B9, B11, B14, B18, B20-B22] and repeated measures ANOVA (31.8%) [B2, B4, B10, B11, B15, B20, B22]. This difference can be attributed to the nature of the dependent variables assessed in each field, with nursing studies primarily focusing on single assessments of knowledge, attitudes, education, beliefs, and symptoms [A1, A3-A11], whereas other disciplines frequently employed repeated measures of pain, depression, response rate, serum levels, and neurological outcomes [B2-B4, B6-B8, B10, B11, B14-B22]. These findings underscore the importance of aligning the choice of statistical techniques with the nature of the outcome measures to ensure the validity and reliability of the research findings.

In conclusion, the trends analysis of nursing and interdisciplinary studies on CAM highlights the need for nursing research to expand its focus on experimental designs, diversify study subjects, explore various CAM interventions, utilize validated assessment tools and physiological indicators, and employ robust statistical techniques. By addressing these methodological considerations, nursing research can strengthen the evidence base for CAM interventions, facilitate their integration into nursing practice, and contribute to interdisciplinary dialogue in the field of CAM research [ 11 , 12 , 13 ]. As CAM use becomes increasingly prevalent among patients, particularly those with chronic conditions [ 1 , 9 , 10 ], nursing research has a crucial role to play in investigating the safety and efficacy of various CAM modalities [ 2 , 4 , 22 ]. This approach not only has the potential to improve patient outcomes and experiences but also enables nursing research to make valuable contributions to interdisciplinary collaboration in the field of CAM [ 3 , 5 ]. By embracing the diversity of CAM interventions and fostering interdisciplinary interactions, nursing research can broaden its scope, enhance the efficiency of patient-focused care, and move closer to providing truly holistic care that addresses the multifaceted needs of patients. Also, the integration of CAM into nursing practice, supported by robust research evidence, has the power to transform healthcare delivery and improve the lives of patients, particularly those with chronic conditions who stand to benefit greatly from a more comprehensive and individualized approach to care.

The trends analysis of nursing and interdisciplinary studies on CAM highlights the potential for nursing research to draw inspiration from the diverse CAM interventions studied in other disciplines and adapt them for nursing practice. For example, the use of aromatherapy for dementia [B10], green tea for depression in HIV patients [B5], and cinnamon patch for irritable bladder syndrome [B13] could be explored in nursing research to assess their feasibility and effectiveness in nursing care settings. By learning from the experiences of other disciplines and adapting promising CAM interventions for nursing practice, researchers can expand the scope of nursing research on CAM and contribute to the development of innovative, evidence-based complementary therapies for various patient populations. Given the current trends in nursing research on CAM, it is essential for future studies to consider the research directions and methodologies employed in other disciplines to guide the advancement of nursing science in this field. In summary, this trends analysis emphasizes the need for nursing research to embrace a more diverse and rigorous approach to CAM research, drawing inspiration from the methodologies and interventions studied in other disciplines. By expanding the focus on experimental designs, diversifying study subjects, exploring novel CAM interventions, utilizing validated assessment tools and physiological indicators, nursing research can strengthen the evidence base for CAM interventions, facilitate their integration into nursing practice.

Limitations

This study aimed to identify research trends in CAM through text network analysis and to analyze nursing research trends based on the findings. The use of text mining and big data analysis allowed for a more comprehensive and less biased approach to data collection and processing compared to arbitrary search strategies. However, there were still limitations in defining each field intuitively due to the diverse and wide-ranging areas of CAM used in different disciplines. Future studies should focus on analyzing overall topics across various fields as well as keyword extraction through text mining to gain a more holistic understanding of CAM research trends. Another limitation of this study is that the search languages were restricted to Korean and English. This may have excluded relevant studies published in other languages and might limit the generalizability of the findings. As CAM is rooted in diverse cultures and traditions worldwide, it is important to include studies conducted in various languages for a comprehensive understanding. Future research should incorporate more languages to provide a global perspective on CAM research trends.

Despite these limitations, this study offers a novel methodological strategy for trend analysis by combining keywords extracted using big data rather than relying on researchers' arbitrary settings. The keyword-based classification and literature analysis provide a new approach to identifying research trends and directions. The trends analysis between nursing literature and other disciplines revealed differences in subject selection, study design, statistical techniques, and measurement of dependent variables, highlighting the need for nursing research to broaden the range of subjects and measurement tools while considering randomization and generalization in experimental designs. Furthermore, this study emphasizes the importance of using design techniques that facilitate the sharing of research results beyond the nursing community.

Conclusions

This study significantly advances CAM research in nursing by providing a comprehensive, data-driven overview of research trends. We have identified key areas for improvement, such as the need for more randomized controlled trials and broader subject diversity, and have proposed innovative methodological strategies. Our findings underscore the importance of interdisciplinary collaboration and the adoption of diverse, rigorous research approaches. By addressing these gaps, nursing research in CAM can be strengthened, ultimately enhancing the integration of evidence-based CAM practices in nursing care and improving patient outcomes.

Availability of data and materials

The data and materials of this study are available from the corresponding author upon reasonable request.

Abbreviations

Complementary and Alternative Medicine

World Health Organization

National Center for Complementary and Integrative Health

Research Information Sharing Service

Korean studies Information Service System

Cumulative Index to Nursing and Allied Health Literature

Term Frequency-Inverse Document Frequency

Latent Dirichlet Allocation

Markov Chain Monte Carlo

Document Term Matrix

Mixed Methods Appraisal Tool

Institutional Review Board

World Health Organization. Traditional complementary and integrative medicine. Geneva: WHO; 2023. Available from:  https://www.who.int/health-topics/traditional-complementary-and-integrative-medicine#tab=tab_1 . Cited 7 Oct 2023.

Google Scholar  

National Center for Complementary and Integrative Health. Complementary, alternative, or integrative health: what's in a name? Bethesda (MD): NCCIH; 2023. Available from: https://www.nccih.nih.gov/health/complementary-alternative-or-integrative-health-whats-in-a-name . Cited 7 Oct 2023.

Shin GR. Development of Korean nursing intervention: alternative (complementary) therapy. J Korean Acad Nurs. 1999;29(6):1403–18.

Article   Google Scholar  

Kang MA, Kim BH, Kim HJ. Complementary and alternative therapy use and health promotion activities among adult women in urban areas. J Korean Wellness Soc. 2017;12(1):677–87. https://doi.org/10.21097/ksw.2017.02.12.677 .

Balouchi A, Mahmoudirad G, Hastings-Tolsma M, Shorofi SA, Shahdadi H, Abdollahimohammad A. Knowledge, attitude and use of complementary and alternative medicine among nurses: A systematic review. Complement Ther Clin Pract. 2018;31:146–57. https://doi.org/10.1016/j.ctcp.2018.02.008 .

Article   PubMed   Google Scholar  

Korean Nurses Association of Complementary and Alternative Therapies. Introduction to the divisions of the Korean Nurses Association of complementary and alternative therapies. Seoul: KNACAT; 2023. Available from:  http://catnurse.co.kr/html/frontend/ .

Oh K, Kim KS, Kwon SH, Park JW. Research trend of complementary and alternative medicine. J Korean Acad Nurs. 2006;36(5):721–31. https://doi.org/10.4040/jkan.2006.36.5.721 .

Pratt M, Wieland S, Ahmadzai N, Butler C, Wolfe D, Pussagoda K, et al. A scoping review of network meta-analyses assessing the efficacy and safety of complementary and alternative medicine interventions. Syst Rev. 2020;9:97. https://doi.org/10.1186/s13643-020-01328-3 .

Article   PubMed   PubMed Central   Google Scholar  

Baek EM, Lee HJ. Analysis of research trends and articles related to work-related overwork using text mining. Occup Health Res. 2023;5(2):79–90. https://doi.org/10.35861/KJOH.2023.5.2.79 .

Kim EJ, Kim JY. Exploring the online news trends of the metaverse in South Korea: A data-mining-driven semantic network analysis. Sustainability. 2023;15(23): 16279. https://doi.org/10.3390/su152316279 .

Karami A, Spinel MY, White CN, Ford K, Swan S. A systematic literature review of sexual harassment studies with text mining. Sustainability. 2021;13(12): 6589. https://doi.org/10.3390/su13126589 .

Lee, HJ, Youn KH. The Analysis of Research Trends in Social Service Quality Using Text Mining and Topic Modeling. J Int Things Converg. 2022;8(3):29–40. https://doi.org/10.20465/KIOTS.2022.8.3.029 .

Kim HY, Seo DH. A Topic Modeling Approach to the Analysis of Domestic Performance. Official J Korean Soc Dance Sci. 2019;36(3):67–78. https://doi.org/10.21539/Ksds.2019.36.3.99 .

O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4(1):1–22.  http://www.systematicreviewsjournal.com/content/4/1/5 .

Kim HJ, Jang HJ, Shin MS. Analysis of research trends on stuttering in adolescence using text mining and literature review. Lang Ther Res. 2020;29(4):1–12. https://doi.org/10.15724/jslhd.2020.29.4.001 .

Gurcan F, Cagiltay NE. Research trends on distance learning: A text mining-based literature review from 2008 to 2018. Interact Learn Environ. 2023;31(2):1007–28. https://doi.org/10.1080/10494820.2020.1815795 .

Yang NY, Shin KL. Analysis of research trends in nursing papers on complementary and alternative therapies in Korea. J Adult Nurs. 2003;15(2):226–35. https://doi.org/10.7739/jkafn.2021.28.2.174 .

Han KS, Lee PS, Choi WH, Ahn KH. The analysis of trends in complementary and alternative therapy (CAT) in nursing research in Korea. J Korean Acad Fundam Nurs. 2003;10(3):392–8.

Sung HR, Kim YS, Kim HY, Ha NY, Chae SM. Research trend about complementary and alternative therapy in Korea using text network analysis. Korean J Rehabil Nurs. 2018;21(2):61–70. https://doi.org/10.7587/kjrehn.2018.61 .

Bidwell S, Jensen MF. Using a search protocol to identify sources of information: The COSI model. In: Topfer LA, Auston I, editors. Etext on Health Technology Assessment Information Resources [Internet]. Bethesda (MD): National Library of Medicine; 2006. Available from: https://www.nlm.nih.gov/archive/20060905/nichsr/ehta/chapter3.html#COSI . Cited 2023 Mar 20.

Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res. 2003;3:993–1022.

Lee SJ, Kim HJ. Keyword extraction from news corpus using modified TF-IDF. J Soc e-Bus Stud. 2009;14(4):59–73.

Lee JH. Semantic network analysis of 2019 Gangwon-do wild fire news reporting: focusing on media agenda analysis. J Korea Contents Assoc. 2019;19(11):153–67. https://doi.org/10.5392/JKCA.2019.19.11.153 .

Liu S, Zhou MX, Pan S, Song Y, Qian W, Cai W, et al. Tiara: Interactive, topic-based visual text summarization and analysis. ACM Trans Intell Syst Technol. 2012;3(2):1–28. https://doi.org/10.1145/2089094.2089101 .

Greene D, O'Callaghan D, Cunningham P. How many topics? stability analysis for topic models. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2014; 2014 Sep 15–19; Nancy, France: Springer Berlin Heidelberg; 2014. p. 498–513. https://doi.org/10.1007/978-3-662-44848-9_32

Leung L. Validity, reliability, and generalizability in qualitative research. J Family Med Prim Care. 2015;4(3):324–7. https://doi.org/10.4103/2249-4863.161306 .

Park S, Lee JL. Research trend analysis of Korean new graduate nurses using topic modeling. J Korean Acad Soc Nurs Educ. 2021;27(3):240–50. https://doi.org/10.5977/jkasne.2021.27.3.240 .

Kim HJ, Lee HN, Oh HS, Yang YJ, Shin SH. Trend Analysis of Articles Published in the Journal of East-West Nursing Research. J East-West Nurs Res. 2014;20(2):167–76. https://doi.org/10.14370/jewnr.2014.20.2.167 .

v SH. Analysis of research trends in complementary and alternative therapy studies published in the Journal of Adult Nursing (2009-2018). In: Adult Nursing Academic Conference. 2019. p. 179–80.

Shin KR, Ha JY. The analysis of research trends about reflexology in Korea. Korean J Adult Nurs. 2006;18(4):603–11.

Lee HY, Kim SY. The trends of nursing research on aromatherapy in Korea. J East-West Nurs Res. 2010;16(2):85–95. https://doi.org/10.14370/JEWNR.2010.16.2.085 .

Download references

Acknowledgements

The authors thank all participants who accepted to be part of this work.

This study was supported by the National Research Foundation of Korea (NRF-2022R1F1A1071533)

Author information

Authors and affiliations.

College of Nursing, Ewha Womans University, 52, Ewhayeodae-Gil, Seodaemun-Gu, Seoul, 03760, South Korea

Jihye Nam, Hyejin Lee, Seunghyeon Lee & Hyojung Park

You can also search for this author in PubMed   Google Scholar

Contributions

Study design: JN, HP, and HL; Data collection and Data analysis : JN and HL ;Study supervision: HP; Manuscript writing: JN, HL, and SL Critical revisions for important intellectual content: HP. All authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Hyojung Park .

Ethics declarations

Ethics approval and consent to participate.

This study was granted an exemption from requiring ethics approval, as it utilized secondary data. The exemption was approved by the Institutional Review Board of Ewha Womans University (IRB No: ewha-202304–0016-01). Informed consent was not required due to the nature of the study, which involved the analysis of existing data that did not include any personally identifiable information. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nam, J., Lee, H., Lee, S. et al. Literature review of complementary and alternative therapies: using text mining and analysis of trends in nursing research. BMC Nurs 23 , 526 (2024). https://doi.org/10.1186/s12912-024-02172-9

Download citation

Received : 01 April 2024

Accepted : 11 July 2024

Published : 01 August 2024

DOI : https://doi.org/10.1186/s12912-024-02172-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Alternative
• Complementary
• Text network

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Case report
• Open access
• Published: 09 August 2024

Primary aldosteronism with hypokalemic rhabdomyolysis: a case report and review of the literature

• Pingan Shi 1 , 3 ,
• Chao Wang 2 &
• Yuanjun Lyu 1 , 3  

Journal of Medical Case Reports volume  18 , Article number:  362 ( 2024 ) Cite this article

Metrics details

Hypokalemic rhabdomyolysis is a rare clinical manifestation of primary aldosteronism, making its diagnosis challenging, particularly when it becomes the primary presenting symptom. Herein, we present a case of primary aldosteronism with hypokalemic rhabdomyolysis and conduct a related literature review.

Case presentation

We report the case of a 54-year-old Chinese male patient who presented with intermittent weakness over the past year and was admitted with sudden limb paralysis for 2 days. The final diagnosis was primary aldosteronism accompanied by hypokalemic rhabdomyolysis syndrome. By reviewing the related Chinese and English literature, we noticed that only a few cases were published since 1978. After excluding irrelevant literatures, we summarized and analyzed 43 patients of with primary aldosteronism accompanied by hypokalemic rhabdomyolysis syndrome. All patients showed good recovery, with normalized blood potassium levels, and a majority achieved normalized blood pressure. Some patients still required medication for blood pressure control.

Conclusions

Primary aldosteronism rarely causes rhabdomyolysis; the occurrence of severe hypokalemia and rhabdomyolysis should prompt consideration of primary aldosteronism in the differential diagnosis. Early detection and treatment are crucial for determining patient prognosis.

Peer Review reports

Primary aldosteronism (PA), commonly known as Conn’s syndrome, is the most common cause of secondary hypertension. Approximately 5–10% of adults with hypertension are estimated to have primary aldosteronism [ 1 ]. Its main characteristic is excessive secretion of aldosterone, which leads to increased tubular reabsorption of sodium and enhanced excretion of potassium in the kidneys. Consequently, elevated blood pressure and hypokalemia ensue, elevating the risk of cardiovascular events and target organ damage. Although hypokalemia is a common manifestation of primary aldosteronism, hypokalemic rhabdomyolysis represents a rare presentation of this disease. Rhabdomyolysis (RM) is a syndrome characterized by the breakdown of skeletal muscle, leading to the release of electrolytes, myoglobin, creatine kinase, and other substances. This can result in electrolyte imbalances, metabolic acidosis, renal failure, and other complications [ 2 ]. Rhabdomyolysis can lead to hyperkalemia, but overall, potassium deficiency can occur. However, the mechanism of rhabdomyolysis induced by hypokalemia has not been determined.

We report a case of primary aldosteronism combined with hypokalemic rhabdomyolysis and summarize the relevant literature to enhance clinicians' understanding of this condition. Early diagnosis and treatment are crucial for improving patient outcomes.

A 54-year-old Chinese male who was engaged in physical labor and admitted to the hospital due to intermittent fatigue for 1 year and sudden onset of paraplegia for 2 days. The patient experienced intermittent fatigue without apparent triggers for the past year, with improvement after rest. A total of 3 days prior, after exertion, he once again experienced fatigue accompanied by pain in the limbs and lower back. Family members provided massage to the lumbar and leg areas, resulting in bruising on the left lower limb. Symptoms improved with rest, and the patient had not previously sought medical attention. A total of 2 days prior, while working, he experienced heightened fatigue and developed paraplegia. He presented to our emergency department, where a head magnetic resonance imaging (MRI) scan showed no significant abnormalities, while a lumbar spine MRI revealed disc protrusion at L3–L4.

The electrolyte results revealed a blood potassium level of 1.83 mmol/L and a blood sodium level of 151.4 mmol/L, and liver function indicators revealed a total bilirubin level of 29.2 g/L and an aspartate aminotransferase (AST) level of 49.6 U/L. Potassium supplements were administered, and subsequent potassium monitoring revealed levels maintained between 2.0 and 2.3 mmol/L. Owing to the necessity for further treatment of hypokalemia, the patient was admitted to our department. Since the onset of symptoms, the patient’s diet and sleep have been normal, with no significant changes in urine or stool. The patient had a history of hypertension for 3 years, with the highest recorded blood pressure being 160/100 mmHg. Currently, the patient is receiving a daily regimen of 5 mg amlodipine and 150 mg/12.5 mg valsartan/hydrochlorothiazide, maintaining blood pressure levels of approximately 120–130/80–85 mmHg. No family history of hypertension was reported, and the individual had no significant personal medical history. On admission, the patient’s examination revealed the following: a temperature of 36.5 °C, heart rate of 100 beats of minute, respiratory rate of 19 breaths per minute, and blood pressure of 150/90 mmHg. The patient was alert with clear speech. Tenderness was noted in the soft tissues on the inner side of the left thigh, with a visible contusion measuring 20 cm × 15 cm. The thyroid was not palpably enlarged and no abnormalities were detected during the physical examination of the heart, lungs, or abdomen. Muscle strength and tone did not significantly differ among the four limbs. Neurological examination revealed no obvious abnormalities, and the lower extremities exhibited no edema. After admission, comprehensive laboratory investigations were conducted, including tests for adrenocorticotropic hormone (ACTH), cortisol (COR), thyroid function, renal function (repeatedly checked), coagulation profile, complete blood count, urinalysis, stool analysis, lipid profile, and blood glucose; all of these tests showed no abnormalities. Table 1 presents electrolyte levels, liver function, myocardial enzymes, urinalysis, and 24-hour urine potassium levels. The changes in blood potassium and creatine kinase levels at different timepoints after patient admission were presented (Fig.  1 ). The renin, angiotensin II, and aldosterone levels are detailed in Table  2 . In this case, since the patient did not discontinue the valsartan/hydrochlorothiazide tablets for 2 weeks according to the standard posture test of the renin–angiotensin–aldosterone system, the positive results obtained supported the diagnosis. Blood gas analysis revealed a pH of 7.441, a HCO 3− concentration of 30.4 mmol/L, a standard base excess of 6.2 mmol/L, and a blood potassium concentration of 1.8 mmol/L. The electrocardiogram showed sinus rhythm with a prolonged QT interval.

Serum potassium and creatine kinase levels over time in patients after admission

An adrenal computed tomography (CT) revealed a low-density mass in the medial limb of the left adrenal gland, with a CT value of approximately 0 Hounsfield unit (HU) and dimensions of approximately 22 mm × 16 mm (Fig.  2 A). Enhanced CT demonstrated a circular lesion in the left adrenal gland, with a maximum cross-sectional area of approximately 22 mm × 16 mm, clear borders, and enhancement phases with CT values of approximately 51 HU, 64 HU, 45 HU, and 21 HU, respectively (Fig.  2 B). Bilateral adrenal vein sampling yielded no successful results. Upon admission, the patient received oral and intravenous potassium supplementation, intravenous fluids, and symptomatic treatment for hypertension. After confirming the diagnosis of primary aldosteronism, the patient was prescribed spironolactone 20 mg twice a day in combination with sustained-release potassium chloride 2 g three times a day orally. Following the complete resolution of rhabdomyolysis, the patient underwent urological surgery at an external hospital one month later. Pathology revealed nodular cortical hyperplasia in the left adrenal gland, with active local cell proliferation and focal medullary hyperplasia (Fig.  3 ). During follow-up at 1 month, 6 months, and 1 year, the patient reported relief of fatigue and muscle pain symptoms, with no recurrence of limb paralysis. Blood potassium levels returned to normal, and the patient required daily administration of 30 mg of nifedipine controlled-release tablets to maintain a blood pressure of approximately 120/80 mmHg.

A Plain renal CT image showing a low-density mass in the medial limb of the left adrenal gland, with a CT value of approximately 0 HU (yellow arrow). B Adrenal enhanced CT image demonstrating a round-like mass in the left adrenal gland, with a maximum cross-sectional area of approximately 22 mm × 16 mm, indicating mild enhancement during the enhanced scan (red arrow)

Pathological findings suggestive of nodular hyperplasia with features of adrenal cortical adenoma were characterized by locally active cellular growth and focal medullary hyperplasia

Literature search and screening

Using the keywords “primary aldosteronism and rhabdomyolysis” or “primary aldosteronism and myopathy,” a search was conducted on English databases (PubMed, Scopus, Web of science, and Embase) and Chinese databases, including China National Knowledge Infrastructure (CNKI), the Wanfang Medical Database, and the VIP Technology Journal Database. The search spanned from 1978 to 2023.

Two authors independently conducted searches in the Chinese and English databases using a predefined search strategy. After the literature was retrieved, duplicates were removed using EndNote X9 software. Subsequently, the two authors individually assessed the titles and abstracts of the remaining articles, excluding irrelevant articles. Relevant literature on primary aldosteronism combined with rhabdomyolysis was identified. In cases of disagreement between the two authors, decisions were made by the research team. Moreover, while reviewing the relevant literature, references were scrutinized, and citation tracking searches were conducted to address any omissions.

A total of 39 articles were retrieved, comprising 9 in Chinese and 30 in English (Fig.  4 ). All the included articles were case reports, resulting in a total of 43 patients (Table  3 ). Three articles reported two patients each. A total of 19 of these patients were male and 24 were female, with an age range of 14–73 years and an average age of 47.6 years. Muscle weakness was the common clinical symptom among all patients and was accompanied by additional manifestations such as muscle pain, muscle atrophy, and, in some cases, vomiting. All patients exhibited hypokalemia, with the lowest potassium level recorded being 1.04 mmol/L. Creatine kinase (CK) levels increased to varying degrees upon admission. Only one patient among all the patients did not have hypertension. Acute kidney injury occurred in eight patients, but they recovered to normal function after treatment. Nine patients, including the patient in the present case, were treated with diuretics before being diagnosed.

Flowchart and results of the literature screening process. A total of 39 articles were included, including 30 in English and 9 in Chinese. CNKI, China National Knowledge Infrastructure; VIP, China Science and Technology Journal Database; Wanfan Digital Database, China Wanfang Data Knowledge Service Platform

Among the 43 patients, 32 underwent surgical excision of the lesions, with pathology revealing adenomas in 29 patients and adrenal hyperplasia in 3 patients (Table  3 ). Additionally, 11 patients were treated with oral spironolactone (40–120 mg). All patients recovered well, with normal blood potassium levels observed in all patients and normalization of blood pressure in the majority of patients. A few patients still required medication for blood pressure control. A summary of clinical features, diagnosis data, and treatment of all cases with primary hyperaldosteronism combined with rhabdomyolysis in both the literature and the present case were summarized in the Supplementary Table.

Discussion and conclusions

The causes of primary aldosteronism (PA) include aldosterone-producing adenoma (APA), idiopathic hyperaldosteronism (also referred to as idiopathic aldosteronism), primary adrenal cortical hyperplasia (also termed unilateral adrenal hyperplasia), familial hyperaldosteronism, adrenal cortical carcinoma secreting aldosterone, and ectopic aldosterone-secreting tumors [ 3 ]. The various etiologies of primary aldosteronism necessitate distinct treatment approaches. For aldosterone-producing adenomas, unilateral adrenal hyperplasia, and adrenal cortical carcinoma secreting aldosterone, surgery is the preferred treatment. However, for idiopathic hyperaldosteronism and glucocorticoid-remediable aldosteronism, medication is the preferred treatment. Currently, adrenal vein sampling (AVS) is recognized as the gold standard for diagnosing and subtyping primary aldosteronism. According to literature review, a total of nine patients underwent successful adrenal vein sampling (AVS) [ 4 ]. Among them, one patient reported bilateral adrenal masses, and AVS suggested unilateral dominance, leading to surgical excision [ 5 ]. In a case reported by Zavatto, imaging indicated bilateral adrenal nodules, and comprehensive AVS confirmed the diagnosis of idiopathic hyperaldosteronism and was managed with oral medication [ 6 ]. Another patient showed no imaging abnormalities in the adrenal glands, but AVS detected increased aldosterone secretion from the right adrenal gland, prompting treatment with oral spironolactone [ 7 ]. Unfortunately, AVS was unsuccessful in the present patient. Pathology indicated nodular adrenal cortical hyperplasia, not suggesting aldosterone-producing adenoma. A patient reported by Kotsaftis also presented with a unilateral nodule, without AVS, and the pathology revealed adrenal hyperplasia [ 8 ]. These cases underscore the importance of AVS.

The International Consensus on the Pathological Diagnosis of Unilateral Primary Aldosteronism emphasizes the pivotal role of aldosterone synthase (CYP11B2) immunohistochemical staining in the pathological diagnosis and subtyping of unilateral primary aldosteronism [ 9 ]. Therefore, for patients with unilateral primary aldosteronism who are unable to undergo surgery or who have failed adrenal vein sampling (AVS), performing CYP11B2 immunohistochemical staining on pathological specimens is recommended. Among the patients described in this article, a total of 32 patients underwent surgical treatment [ 5 , 6 , 8 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ], with three of whom exhibited adrenal hyperplasia [ 8 , 33 ]. Additionally, 11 patients received oral spironolactone (40–120 mg) [ 4 , 7 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. None of the 32 surgically treated patients exhibited CYP11B2 immunohistochemical staining in the pathological reports.

Rhabdomyolysis (RM) can result from various underlying causes, such as trauma, ischemia, metabolic disorders, infection, drugs, and electrolyte imbalances [ 44 ]. In this particular case, low potassium levels constitute the primary reason for RM. Furthermore, the patient in this instance was taking diuretics, which potentially aggravated her hypokalemia. A literature review in this context included nine patients treated with diuretics [ 4 , 6 , 7 , 19 , 20 , 37 , 41 ]. Rhabdomyolysis-induced muscle breakdown releases a substantial amount of potassium into the circulation. However, when clinical syndrome manifests as both hypokalemia and rhabdomyolysis, the absolute concentration of potassium falls below detectable levels. Hence, substantial potassium supplementation and intravenous fluid administration are imperative. Reports in the literature highlight a markedly elevated risk of rhabdomyolysis when blood potassium levels fall below 2.0 mmol/L [ 21 ]. Additionally, reports indicate that severe hypokalemia, defined as a blood potassium concentration < 2.5 mmol/L, often leads to severe muscle weakness or RM [ 17 ]. According to literature review, 33 patients had initial blood potassium levels < 2.0 mmol/L, constituting approximately 76.7% of the total. There were 41 patients with blood potassium levels ≤ 2.5 mmol/L, representing 95.3% of the population, thereby affirming the previously drawn conclusion. However, the mechanisms by which hypokalemia leads to RM remain unclear. Severe hypokalemia (serum potassium < 2.5 mmol/L) may crucially contribute to muscle injury for three reasons [ 4 , 19 ]: (1) microvascular constriction leading to reduced muscle blood supply and subsequent muscle cell dissolution, (2) inhibition of glycogen synthesis and storage, and (3) disruption of cell membrane ion transport.

Additionally, it is crucial to consider the relationship between serum transaminases and RM. Several studies have shown a correlation between changes in serum AST and alanine transaminase (ALT) and serum creatine kinase (CK) levels, suggesting that the increase in transaminase levels in patients with rhabdomyolysis may be a marker of skeletal muscle damage rather than liver cell damage [ 45 , 46 ]. Of course, it is essential to inquire about the patient's history of liver disease, and if this information is missing, elevated transaminase levels may be indicative of skeletal muscle injury. In this literature review, 20 patients provided AST and ALT values, and these patients had no history of liver disease, yet transaminase values showed varying degrees of elevation.

In our case, the patient’s blood pressure did not return to normal after surgery. Considering the patient’s middle-aged onset and short history of hypertension without a family history of hypertension, three different types of antihypertensive drugs were required preoperatively to control blood pressure. Postoperatively, only one antihypertensive drug was needed to control blood pressure. Combined with pathological findings suggesting adenomatous nodular hyperplasia, the possibility of a subtle lesion on the contralateral adrenal gland cannot be ruled out. Hypertension is considered to be secondary to aldosterone excess. The diagnosis and treatment of this case align with the 2020 expert consensus on the diagnosis and treatment of primary aldosteronism in China [ 47 ]. Unfortunately, the patient experienced AVS failure, and CYP11B2 immunohistochemical staining was not performed on the pathological specimen. Postoperative pathology indicates the disease is likely due to idiopathic hyperaldosteronism. Considering the possibility of recurrence of the disease, the levels of blood potassium and renin–angiotensin–aldosterone should be closely monitored after surgery.

Strengths of this case include clear diagnosis and timely treatment based on the patient’s clinical presentation, laboratory results, and imaging examinations. Spironolactone was administered preoperatively to correct hypokalemia. Moreover, the diagnostic and treatment processes were standardized, aligning with the 2020 expert consensus on the diagnosis and treatment of primary aldosteronism in China [ 47 ]. However, some limitations of our presentation case merit consideration. Firstly, the patient's adrenal vein sampling (AVS) was unsuccessful, leading to insufficient grounds for surgical treatment. Secondly, postoperative pathological specimens did not undergo CYP11B2 immunohistochemical staining.

In this paper, 43 cases of primary aldosteronism with hypokalemic rhabdomyolysis were retrospectively analyzed to improve clinicians’ understanding of the disease. We found that severe hypokalemia (potassium ≤ 2.5 mmol/L) may be a significant predictor of rhabdomyolysis. However, in the literature, the diagnosis and treatment of patients with primary aldosteronism complicated by hypokalemic rhabdomyolysis was not completely standardized. In this literature review, only nine cases underwent adrenal vein sampling (AVS) according to guidelines for the management of primary aldosteronism to determine the cause [ 48 ]. Pathological testing, including CYP11B2 immunohistochemical staining, is recommended for patients who have not had AVS or who have failed AVS according to the International Consensus on Pathological Diagnosis of Unilateral Primary Aldosteronism [ 9 ]. The literature did not identify case–control studies regarding the incidence rate of rhabdomyolysis associated with primary aldosteronism, such as stratifying by serum potassium levels (> 2.5 mmol/L and ≤ 2.5 mmol/L). Therefore, multicenter case-series control study will be our next research direction in the future.

Based on published cases, it can be concluded that hypokalemic rhabdomyolysis is a rare clinical manifestation of primary aldosteronism and is more commonly observed in middle-aged females (< 60 years old). Patients with hypertension and severe hypokalemia should be highly suspected. Early diagnosis and treatment can improve patient prognosis, reducing the frequency of complications such as rhabdomyolysis and acute kidney injury. Furthermore, for patients with PA, AVS should be conducted. In instances of unilateral PA, testing for CYP11B2 is essential for enhancing the accuracy of subtype diagnosis. Moreover, emphasis should be placed on the correlation between serum transaminase levels and RM.

Availability of data and materials

The datasets collected and analyzed during the current study are available from the corresponding author upon reasonable request.

Abbreviations

• Primary aldosteronism
• Rhabdomyolysis

Acute kidney injury

Adrenal vein sampling

Aldosterone synthase

Computed tomography

Funder JW, Carey RM. Primary aldosteronism: where are we now? where to from here? Hypertension. 2022;79:726–35.

Article   CAS   PubMed   Google Scholar  

Torres PA, Helmstetter JA, Kaye AM, Kaye AD. Rhabdomyolysis: pathogenesis, diagnosis, and treatment. Ochsner J. 2015;15:58–69.

PubMed   PubMed Central   Google Scholar  

Young WF Jr. Diagnosis and treatment of primary aldosteronism: practical clinical perspectives. J Intern Med. 2019;285:126–48.

Article   PubMed   Google Scholar  

Zavatto A, Concistrè A, Marinelli C, Zingaretti V, Umbro I, Fiacco F, et al . Hypokalemic rhabdomyolysis: a rare manifestation of primary aldosteronism. Eur Rev Med Pharmacol Sci. 2015;19:3910–6.

CAS   PubMed   Google Scholar  

Eguchi T, Miyauchi S. A case of primary aldosteronism with secondary hyperparathyroidism and bilateral adrenal tumors. Endocrinol Diabetes Metab Case Rep. 2015. https://doi.org/10.1530/EDM-15-0029 .

Article   PubMed   PubMed Central   Google Scholar  

Pecnik P, Müller P, Vrabel S, Windpessl M. Two cases of hypokalaemic rhabdomyolysis: same but different. BMJ Case Rep. 2018. https://doi.org/10.1136/bcr-2017-223609 .

Dominic JA, Koch M, Guthrie GP Jr, Galla JH. Primary aldosteronism presenting as myoglobinuric acute renal failure. Arch Intern Med. 1978;138:1433–4.

Kotsaftis P, Savopoulos C, Agapakis D, Ntaios G, Tzioufa V, Papadopoulos V, et al . Hypokalemia induced myopathy as first manifestation of primary hyperaldosteronism—an elderly patient with unilateral adrenal hyperplasia: a case report. Cases J. 2009;2:6813.

Williams TA, Gomez-Sanchez CE, Rainey WE, Giordano TJ, Lam AK, Marker A, et al . International histopathology consensus for unilateral primary aldosteronism. J Clin Endocrinol Metab. 2021;106:42–54.

Finsterer J, Lässer S. Severe hypokalemic paralysis as a manifestation of a mitochondrial disorder. Tohoku J Exp Med. 2013;231:9–12.

Martínez JJ, Oliveira CL, Meneses AL, Rodríguez SA, Corrales PP, López AH, et al . Rhabdomyolysis due to primary hyperaldosteronism. Endocrinol Nutr. 2009;56:431–4.

Goto A, Takahashi Y, Kishimoto M, Minowada S, Aibe H, Hasuo K, et al . Primary aldosteronism associated with severe rhabdomyolysis due to profound hypokalemia. Intern Med. 2009;48:219–23.

Chow CP, Symonds CJ, Zochodne DW. Hyperglycemia, lumbar plexopathy and hypokalemic rhabdomyolysis complicating Conn’s syndrome. Can J Neurol Sci. 1997;24:67–9.

Díaz-López EJ, Villar-Taibo R, Rodriguez-Carnero G, Fernandez-Pombo A, Garcia-Peino R, Blanco-Freire MN, et al . Should we suspect primary aldosteronism in patients with hypokalaemic rhabdomyolysis? A systematic review. Front Endocrinol (Lausanne). 2023;14:1257078.

Han R, Jiang X. Hypokalemia-induced rhabdomyolysis as the first symptom of primary aldosteronism: a case report and literature review. Ann Palliat Med. 2022;11:2778–84.

Sirkeci O, Sirkeci EE, Kucukciloglu Y. Severe hypokalemia and rhabdomyolysis caused by Conn syndrome. Clin Ter. 2021;172:407–9.

PubMed   Google Scholar  

Maung AC, Kerwen AK, Ching LP. Hypokalaemic rhabdomyolysis as initial presentation of primary aldosteronism. J R Coll Physicians Edinb. 2021;51:149–52.

Kollipara S, Ravindra S, Pai K, Shetty S. Quadriplegia and rhabdomyolysis as a presenting feature of Conn’s syndrome. BMJ Case Rep. 2021. https://doi.org/10.1136/bcr-2020-234686 .

Chen CT, Wang YC, Lin CM. Hypokalemia-induced rhabdomyolysis caused by adrenal tumor-related primary aldosteronism: a report of 2 cases. Am J Case Rep. 2021;22:e929758.

Lee JH, Kim E, Chon S. Hypokalemia-induced rhabdomyolysis by primary aldosteronism coexistent with sporadic inclusion body myositis. Ann Rehabil Med. 2015;39:826–32.

Wen Z, Chuanwei L, Chunyu Z, Hui H, Weimin L. Rhabdomyolysis presenting with severe hypokalemia in hypertensive patients: a case series. BMC Res Notes. 2013;6:155.

Cooray MS, Bulugahapitiya US, Peiris DN. Rhabdomyolysis: a rare presentation of aldosterone-producing adenoma. Indian J Endocrinol Metab. 2013;17:S237–9.

Karagüzel G, Bahat E, Imamoğlu M, Ahmetoğlu A, Yildiz K, Okten A. An unusual case of an aldosterone-producing adrenocortical adenoma presenting with rhabdomyolysis. J Pediatr Endocrinol Metab. 2009;22:1087–90.

Mahdyoon H, Mermiges DN, Wisgerhof M. Conn’s syndrome with rhabdomyolysis mimicking deep vein thrombophlebitis. South Med J. 1990;83:346–7.

Tan HY, Gong LL, Wang Y, Bai J, Ren W. Primary aldosteronism associated with rhabdomyolysis:a report of 2 cases and review of the literature. J Chongq Med Univ. 2018;6:763–7.

Google Scholar  

Gao PL, Yang ZM. A case of primary aldosteronism complicated by rhabdomyolysis and literature review. J Shanxi Med Univ. 2018;4:439–41.

Zhang J, Li H, Zhang XB, Guo LX. The first clinical manifestation of Rhabdomyolysis due to severe hypokalemic of a patient with primary aldosteronism. Int J Endocrinol Metabol. 2017;5:358–60.

Chen L, Han XM, Liu GX, Zang XP, Chen M. A case report of primary aldosteronism complicated by hypokalemic rhabdomyolysis. Chin J Urol. 2013;9:719–819.

Wang Y, Chen J, Yang K, Ren Q, Chen MH. Primary aldosteronism-induced profound hypokalemia and rhabdomyolysis: a case report. J Internal Intens Med. 2012;6:383–4.

Yao B, Qin Z, Tan Y, et al . Rhabdomyolysis in primary aldosteronism: a case report and review of the literature. AACE Clin Case Rep. 2015;1:e21–7.

Article   Google Scholar  

Zhang L, Shi HJ. Primary aldosteronism with rhabdomyolysis: a case report. J Musculoskeletal Pain. 2014;22:102–5.

Demir K, Sonmez O, Kayrak M, et al . Severe hypokalemia-associated rhabdomyolise and unusual poliuria in patient with primary aldosteronism. Eur J Gen Med. 2012;9:211–3.

Gonerska-Szadkowska A, Wittych-Długosz J, Makarewicz J, et al . Rhabdomyolysis following severe hypokalemia caused by Conn’s syndrome. Arch Med Sci. 2007;3(3):274–7.

Li RH, Cui XF, Wang XY. Primary aldosteronism characterized by habdomyolysis: a case report. J China-Japan Friendship Hospital. 2023;37:116–8.

Dyrmishi B, Olldashi T, Rista E, Fureraj T, Ylli D, Ylli A. Severe hypokalemia induced rhabdomyolysis by primary hyperaldosteronism coexistent with recurrent bilateral renal calculi. Acta Endocrinol (Buchar). 2017;13:228–31.

Tsai WT, Chen YL, Yang WS, Lin HD, Chien CC, Lin CL. Primary aldosteronism associated with severe hypokalemic rhabdomyolysis. Hormones (Athens). 2012;11:505–6.

Ozgür B, Kürsat S. Hypokalemic rhabdomyolysis aggravated by diuretics complicating Conn’s syndrome without acute renal failure. Clin Nephrol. 2002;57:89–91.

Schady W, Yuill GM. Myopathy and primary hyperaldosteronism. Neurology. 1981;31:225–6.

Kang JJ, Su PX, Deng BM, Yang HJ, Wang ZC. A case of adenoma-associated primary aldosteronism complicated by rhabdomyolysis. J Diagnost Concepts Pract. 2019;18:583–4.

Zhang HM, Zou YB, Zhang YQ, Wu HY. A case of primary aldosteronism complicated by rhabdomyolysis. Chin Circulat J. 2006;6:477–577.

Petidis Anna K, Douma S, Aslanidis S, et al . Hypertension associated with rhabdomyolysis. J Clin Hypertens. 2007;9:60–2.

Cakir I, Senol S, Simsek Y, et al . Primary hyperaldosteronism presenting with rhabdomyolysis in emergency room—case report. J Acute Dis. 2016;5(3):264–6.

Ruan ZF, He WQ. Severe hypokalemia induced rhabdomyolysis: a case report. Chin J Difficult Complic Cases. 2012;6:474–574.

Stahl K, Rastelli E, Schoser B. A systematic review on the definition of rhabdomyolysis. J Neurol. 2020;267:877–82.

Jo KM, Heo NY, Park SH, Moon YS, Kim TO, Park J, et al . Serum aminotransferase level in rhabdomyolysis according to concurrent liver disease. Korean J Gastroenterol. 2019;74:205–11.

Weibrecht K, Dayno M, Darling C, Bird SB. Liver aminotransferases are elevated with rhabdomyolysis in the absence of significant liver injury. J Med Toxicol. 2010;6:294–300.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Chinese Society of Endocrinology. Expert consensus on the diagnosis and treatment of primary aldosteronism (2020). Chin J Endocrinol Metab. 2020;36:727–36.

Funder JW, Carey RM, Mantero F, et al . The management of primary aldosteronism: case detection, diagnosis, and treatment: an endocrine society clinical practice guideline. J Clin Endocrinol Metab. 2016;101:1889–916.

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Department of Endocrinology, Tianjin Hospital, 406 Jiefang South Road, Tianjin, 300210, China

Pingan Shi & Yuanjun Lyu

Department of Urology, Tianjin Hospital, Tianjin, China

Department of Endocrinology, Tianjin Hospital of Tianjin University, Tianjin, China

You can also search for this author in PubMed   Google Scholar

Contributions

P.S. conceived, designed, and wrote the review. P.S. and Y.L. treated the patient and contributed to the data collection. C.W. and Y.L. commented and revised the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to Yuanjun Lyu .

Ethics declarations

Ethics approval and consent to participate.

The reported investigations did not require ethics approval since they were conducted as part of routine clinical care with therapeutic intent. Prior to undergoing any clinical procedures, the patient provided written informed consent.

Consent for publication

Written informed consent was obtained from the patient for publication of this case report and any accompanying images. A copy of the written consent is available for review by the Editor-in-Chief of this journal.

Competing interests

The authors declare that the article content was composed in the absence of any commercial or financial relationships that could be construed as a potential competing interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Shi, P., Wang, C. & Lyu, Y. Primary aldosteronism with hypokalemic rhabdomyolysis: a case report and review of the literature. J Med Case Reports 18 , 362 (2024). https://doi.org/10.1186/s13256-024-04708-8

Download citation

Received : 10 April 2024

Accepted : 14 July 2024

Published : 09 August 2024

DOI : https://doi.org/10.1186/s13256-024-04708-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Hypokalemia
• Literature analysis

Journal of Medical Case Reports

ISSN: 1752-1947

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

SYSTEMATIC REVIEW article

Direct health implications of e-cigarette use: a systematic scoping review with evidence assessment.

• 1 One Health Research Group, Faculty of Medicine, Universidad de las Américas, Quito, Ecuador
• 2 Facultad de Medicina, Fundación Universitaria Autónoma de las Américas, Pereira, Colombia
• 3 Fundacion Universitaria San Martín, Bogota, Colombia
• 4 Facultad de Ciencias de la Salud, Universidad del Quindío, Armenia, Colombia
• 5 Cancer Research Group (CRG), Faculty of Medicine, Universidad de las Americas, Quito, Ecuador

Background: E-cigarettes are often marketed as a less harmful alternative to traditional tobacco cigarettes. Despite their popularity, the evidence regarding their effects on human health remains unclear and is filled with complexities.

Objectives: This systematic review aims to elucidate the direct effects of electronic cigarette use on human health, carefully distinguishing between the specific characteristics of the populations studied.

Methodology: Adhering to the PRISMA guidelines, we conducted a comprehensive search in PubMed/Medline, Web of Science, Scopus, and Google Scholar databases without date restrictions, including articles in both Spanish and English. This approach enabled the identification and analysis of primary studies to understand the direct effect of electronic cigarettes on human health.

Results: A total of 33 studies were included that evaluated cardiovascular, pulmonary, renal, weight and fertility effects. Only five studies analyzed e-cigarettes in healthy populations and seven studies compared healthy individuals against smokers. The effects evaluated on smokers or former tobacco smokers were apparently positive, however, among healthy individuals, increased heart rate, mean arterial pressure, oxidative stress, alteration of respiratory epithelial cells and increased airflow resistance were found.

Conclusion: Smokers or former smokers who switch to e-cigarettes may reduce their exposure to carcinogens and lower their risk of developing severe health issues associated with conventional smoking. However, in healthy individuals who have never smoked traditional cigarettes, the use of e-cigarettes introduces several cardiovascular and respiratory adverse effects. These findings suggest that while e-cigarettes can be a strategic harm reduction tool for smokers, they are not a safe option for non-smokers.

1 Introduction

Smoking has been associated with several negative health effects since the early 1930s ( 1 ). It was not until the 1950s that Dr. Richard Doll and Dr. A. Bradford Hill published a series of influential studies highlighting the negative health effects of tobacco, including the renown British Doctors Study ( 2 ). Since then, public health advocates have actively sought methods to reduce tobacco consumption and its associated risks. This effort has been supported by numerous initiatives from the anti-tobacco industry aimed at decreasing smoking rates. Strategies have included the implementation of taxation, prohibition of smoking in public areas, strict regulatory controls, and the promotion of pharmacological nicotine replacement therapies ( 3 , 4 ). In 2004, electronic cigarettes (EC) were introduced to the market as a healthier alternative for chronic smokers dependent on nicotine, allowing them to smoke without the risks of tar and other toxic tobacco compounds ( 5 ). Due to their ability to generate vapor instead of smoke, electronic cigarettes (ECs) have gained popularity among those seeking to quit traditional smoking ( 6 ). They also appeal to adolescents and young adults, attracted by marketing campaigns that tout ECs as a safer alternative to traditional cigarettes and an effective tool for cessation ( 7 – 10 ). This perceived safety stems from the fact that traditional cigarettes require the combustion of paper and tobacco to generate smoke, which carries tar, nicotine, carbon monoxide, and other harmful substances into the lungs. In contrast, ECs do not involve combustion ( 8 , 11 ).

The global e-cigarette market is projected to grow from $22.5 billion in 2022 to $47.5 billion in 2028, with a compound annual growth rate (CAGR) of 13.5% from 2023 to 2028 ( 12 ). Prevalence among adults is approximately 10%, while it reaches 11.8% among middle and high school students. Notably, usage rates in these student groups surged by 10.5 and 27.5%, respectively, in 2019 ( 13 , 14 ). In 2020, it was estimated that 68 million adults used electronic cigarettes, predominantly those aged 18–24 years ( 15 , 16 ).

Although vaping electronic cigarettes is generally considered safer than smoking traditional tobacco, there is significant concern regarding the variety of compounds that can be combined in these devices. Most liquids used in e-cigarettes contain propylene glycol and glycerol, which are irritants to the respiratory tract ( 17 ). Furthermore, the degradation of e-cigarette vapor can produce formaldehyde, a carcinogenic substance. Studies have also shown an increase in inflammatory markers in the respiratory tract up to 10 times greater than those found in traditional cigarette smokers ( 18 , 19 ). Some e-cigarette cartridges also contain flavor enhancers like diacetyl, which, when inhaled, not only increases the risk of addiction but can also cause tissue damage such as bronchiolitis obliterans, a serious respiratory condition often referred to as “popcorn lung” ( 7 ). Despite some ECs containing no nicotine, the most popular ones feature nicotine levels ranging from 6 to 24 mg/mL, and even up to 100 mg/mL, making them highly addictive and resulting in higher blood nicotine levels compared to traditional smoking ( 7 , 11 ). These high levels of nicotine not only raise concerns about addiction but may also contribute to the numerous health issues associated with EC use.

The safety of e-cigarette consumption is not well-established, and evidence shows that ECs can cause lung inflammation and damage, disrupt lung epithelial cell function, and irritate the eyes, nose, and throat. Additionally, aerosolized nicotine from vaping has been linked to increased thromboembolic activity and impaired dilation and relaxation of small blood vessels ( 7 , 11 ).

Given the ongoing debates about the health implications of electronic cigarette use, this systematic review aims to thoroughly investigate the direct effects of electronic cigarette use on human health, considering the characteristics of the populations studied and the duration of exposure.

2 Materials and methods

2.1 study design.

We conducted a systematic review comprising primary source studies, including clinical trials, cross-sectional studies, cohorts, case controls, case series, and clinical case reports. Secondary source studies such as systematic reviews, meta-analyses, literature reviews, and narrative reviews were excluded, as were letters to the editor, comments, special articles, and editorials. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, a recommended guide for conducting systematic reviews and meta-analyses. This review did not have a registered protocol in the International prospective register of systematic reviews (PROSPERO), which does not accept registrations for scoping reviews, literature reviews, or mapping reviews.

2.2 Search strategies

We conducted an in-depth bibliographic search in English to encompass the broadest scope of academic literature. We utilized several key databases and libraries, including PubMed/Medline, Web of Science, Scopus, and Google Scholar. Additionally, we employed a snowball strategy to review the reference lists of relevant articles for any overlooked studies. Our literature search targeted primary studies published before June 2023. To execute the search, we used the following index terms, keywords, and Boolean operators: (“Electronic Nicotine Delivery Systems” OR “E-Cigarettes” OR “Electronic Cigarettes” OR “Vaping” OR “Nicotine Vaping” OR “Vape”) AND (“Health effects” OR “Toxicity” OR “Health Risk” OR “Physiology”) AND (“Effects, Acute” OR “Effects, Long-Term”) in the title (TI) or abstract (AB).

2.3 Selection criteria

2.3.1 inclusion criteria.

Primary studies examining the direct short-term or long-term health effects of using or consuming e-cigarettes, including impacts on the cardiovascular, respiratory, and other systems.

Studies conducted on human subjects.

2.3.2 Exclusion criteria

Studies examining the health effects of e-cigarette use conducted on animals.

Research focusing on the indirect health effects of e-cigarette use, such as assistance in quitting traditional tobacco cigarettes.

Studies that investigate the effects of consuming substances other than e-cigarettes, such as drugs or tobacco.

Secondary research or reviews on the health impacts of e-cigarette use.

The bibliographic search initially yielded a total of 137 papers. In the first screening phase, 74 studies were eliminated, primarily due to the type of document ( n = 38), and 17 studies were eliminated due to duplicates. Of the 46 remaining papers, four studies were excluded due to limitations found in the title or abstract. Finally, the 42 eligible papers were reviewed in their entirety, and 33 studies were included in this investigation. Figure 1 shows the selection process based on the PRISMA flow chart of the studies analyzed in this manuscript.

Figure 1 . PRISMA flowchart illustrating the study selection process for this systematic review, detailing the number of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage.

2.4 Bias assessment

To minimize the risk of bias, four members of the research team (JSIC, MH, EML, and PNL) independently performed the data extraction process at different times. Any discrepancies encountered during the data collection phase were resolved through discussion until consensus was reached among all team members. This method was implemented to ensure the accuracy and reliability of our findings.

2.5 Data synthesis

A comprehensive review was conducted on all manuscripts that met the established selection criteria. For cohort studies, we utilized the Newcastle-Ottawa Quality Assessment Scale for quantitative analysis, the Newcastle-Ottawa Scale was developed to assess the quality of nonrandomized studies with its design, content and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results ( 20 ). For cross-sectional studies, the Joanna Briggs Institute (JBI) critical appraisal checklist for analytical cross-sectional studies was employed, the JBI’s critical appraisal tools assist in assessing the trustworthiness, relevance and results of published papers ( 21 ). Additionally, for Randomized Controlled Trials (RCTs), we applied the JBI critical appraisal checklist specific to RCTs ( 21 ) (see Supplementary material 1 ).

The information from these manuscripts was meticulously organized and synthesized into descriptive tables. This format was chosen to present our findings clearly and concisely, facilitating ease of understanding for the reader.

3.1 Literature review and quality assessment

A total of 33 studies were included in this systematic review, of which 15 were cohort studies. Among these, 11 were of good quality and four were of acceptable quality. In addition, 17 studies were randomized controlled trials, with 10 of high quality and seven of moderate quality. Finally, only one cross-sectional study was identified, and it was classified as high quality.

3.2 Cardiovascular effects

3.2.1 effects on hemodynamics.

E-cigarette use has been linked to increased heart rate and blood pressure. This was observed in a combined analysis of nine randomized studies ( 22 – 30 ) and one prospective study ( 31 ) which investigated the short-term effects of e-cigarettes on healthy subjects, both with and without a history of tobacco use. Significant increases in heart rate, systolic and diastolic blood pressure, and mean arterial pressure were noted following acute inhalation of e-cigarettes, regardless of nicotine content. Conversely, a randomized study by D’Ruiz et al. involving 105 subjects who switched either completely or partially from tobacco cigarettes to electronic cigarettes demonstrated that electronic cigarette use over 5 days led to reduced blood pressure and heart rate in most participants ( 32 ).

Conversely, among users who transitioned from tobacco cigarettes to e-cigarettes, evidence from three studies, including a prospective randomized controlled trial involving active tobacco smokers, indicates a significant decrease in systolic blood pressure and resting heart rate after 1 month of e-cigarette use, with or without nicotine. This effect was particularly noted in participants who had smoked more than 20 pack-year of tobacco ( 33 ). Similarly, a randomized controlled clinical trial involving 263 tobacco smokers who switched to e-cigarettes demonstrated a statistically significant decrease in blood pressure and heart rate at the 1-month follow-up. However, these results did not maintain statistical significance at the 3-month follow-up ( 34 ).

Regarding long-term exploration through post hoc analysis of the ECLAT study, a 12-month prospective, randomized, controlled, double-blind trial in smokers with no intention to quit tobacco smoking who switched to e-cigarettes with and without nicotine, it was found that those who switched to e-cigarettes experienced a statistically significant reduction in long-term systolic blood pressure, with this reduction being more pronounced in smokers with elevated baseline blood pressure ( 35 ). Regarding the long-term effects of e-cigarette smoking on cardiac autonomic tone, a cross-sectional study comparing cases (e-cigarette users) and controls (healthy individuals with no history of smoking) who had used nicotine-containing e-cigarettes on most days for at least 1 year, showed a significant increase in sympathetic activity compared to the control group ( 30 ). Additionally, an evaluation of effects in tobacco smokers found an improvement in baseline acetylcholine and mean arterial pressure 3 and 6 months after starting e-cigarette replacement ( p  < 0.05) ( 36 ) ( Table 1 ).

Table 1 . Systematic summary of evidence of studies assessing the cardiovascular effects of e-cigarette use.

3.2.2 Impact on arterial vasculature

Regarding the effects on vasculature, three studies were included. A randomized crossover study of occasional tobacco cigarette users demonstrated an acute increase in vascular stiffness after inhaling e-cigarettes with and without nicotine, as evidenced by increased AI75 (pulse wave augmentation index at 75%) and PWV (pulse wave velocity) ( 24 ). However, George et al. reported that in tobacco smokers, arterial stiffness (PWV-dependent) improved after 1 month of switching from tobacco cigarettes to e-cigarettes, with or without nicotine. They also noted a significant enhancement in endothelial function as measured by flow-mediated dilatation (FMD) ( 33 ). In contrast, Biondi et al. observed a significant deterioration in FMD immediately after using nicotine-containing e-cigarettes ( 23 ).

Furthermore, long-term e-cigarette use in smokers attempting to quit has been associated with improved flow-mediated dilation at 3 and 6 months ( p  < 0.001) ( 36 ) ( Table 1 ).

3.2.3 Influence on cardiovascular biomarkers

Acute effects on biomarkers across five studies involving healthy populations with a history of smoking, including a randomized crossover study between tobacco smokers and non-smokers, demonstrated that active e-cigarette use in smokers and passive use in healthy individuals did not affect blood count markers. However, both active and passive tobacco users exhibited an increase in white blood cell, lymphocyte, and granulocyte counts for at least 1 h ( 38 ). Additionally, Biondi et al. revealed that both tobacco and electronic cigarette inhalation significantly increased markers of oxidative stress (sNox2-dp, H2O2, and 8-iso-PGF2a levels) and platelet activity (sCD40L and soluble P-selectin), and decreased markers of NO bioavailability and antioxidants (Vitamin E) in tobacco smokers classified as healthy ( 23 ), Similarly, Mobarrez et al. observed an increase in endothelial and platelet-derived vesicles (expressing increased P-selectin) 4 h after the consumption of nicotine-containing electronic cigarettes, and noted an increase in CD40 ligand even in nicotine-free e-cigarettes ( 39 ). Furthermore, Nocella et al. comparing the acute impact of tobacco versus e-cigarettes with equivalent nicotine content, found that within 5 min of smoking or vaping, both tobacco smokers and healthy individuals exhibited a statistically significant increase in sCD40L, sP-selectin, and platelet aggregation levels in a single-blind crossover trial ( 40 ). Conversely, a randomized single-blind, three-period measurement in smokers showed that exposure to nicotine-free electronic cigarettes did not lead to acute changes in cardiovascular oxidative stress parameters; however, exposure with nicotine resulted in increased arterial stiffness and plasma myeloperoxidase ( p  < 0.05) ( 29 ). Additionally, a prospective cross-sectional study reported an increase in platelet microparticles (PMPs) ( p  < 0.001) following electronic cigarette consumption ( 31 ).

In a randomized crossover trial comparing healthy subjects who smoked tobacco (maximum of 10 cigarettes per month), the acute use of nicotine-containing e-cigarettes significantly increased levels of circulating endothelial progenitor cells (flow cytometry) to the same magnitude as after smoking a traditional cigarette ( 41 ). This group also showed an increase in endothelial vesicles of platelet and endothelial origin 4 h after exposure to nicotine-containing e-cigarettes ( 39 ).

Regarding long-term effects, Moheimani et al. found that low-density lipoprotein (LDL) oxidizability was higher in e-cigarette users compared to the control group (healthy) ( p  = 0.78), while paraoxonase-1 activity, which protects against oxidative stress, tended to be lower in e-cigarette users ( p  = 0.72). Furthermore, markers of inflammation such as high-density lipoprotein antioxidant index, fibrinogen, and C-reactive protein did not differ between groups ( 30 ) ( Table 1 ).

3.2.4 Effects on cardiac function

The immediate cardiac effects of e-cigarettes compared to traditional tobacco were assessed through echocardiography. The results indicated that the e-cigarette group exhibited a lower heart rate-corrected isovolumetric relaxation time (IVRTc) ( p  = 0.011) and tissue Doppler flow (MPIt) ( p  = 0.019). These findings suggest a lack of immediate relaxing effects on the left ventricular musculature typically observed in smokers ( 42 ). Furthermore, an analysis of cross-sectional data from the US National Health Interview Surveys (NHIS) explored the relationship between e-cigarette use and myocardial infarction. This study found that daily e-cigarette use was independently associated with increased odds of having suffered a myocardial infarction ( 43 ) ( Table 1 ).

3.3 Impact on respiratory health

3.3.1 changes in respiratory epithelial cells.

Two randomized trials have evaluated the effects of e-cigarettes on respiratory epithelial cells. In a randomized pilot trial, Song et al. studied healthy non-smoking individuals exposed to nicotine-free, unflavored e-cigarettes containing 50% propylene glycol and 50% vegetable glycerin. They found that propylene glycol caused low-grade lung inflammation after 1 month of exposure, but no changes in gene expression in lung cells were observed ( 45 ). Conversely, Staudt et al., in a trial with healthy non-smoking subjects exposed to both nicotine-containing and nicotine-free e-cigarettes, noted genetic changes in small airway epithelial cells, alveolar macrophages, and lower airway mononuclear phagocytes after short exposure to EC aerosols. This study also reported alterations in alveolar macrophages, including changes in the systemic inflammatory response, impaired phagocytic capacities, and increased susceptibility to bacteria, particularly S. pneumoniae ( 46 ) ( Table 2 ).

Table 2 . Systematic summary of evidence of studies assessing the respiratory effects of e-cigarette use.

3.3.2 Effects on lung function

Among the acute effects on lung function, five studies evaluated them from different perspectives. Ferrari et al. observed that non-smokers using nicotine-free electronic cigarettes (EC) had a lower exhaled fraction of carbon monoxide (FeCO) compared to smokers; however, the exhaled fraction of nitric oxide (FeNO) showed no significant difference. Significant increases in both FVC (0.5–3.1%) and FEV1 (1.5–6%) were observed for both groups (tobacco and electronic cigarette users) over 5 days under various conditions. Furthermore, among non-tobacco smokers, exposure to electronic cigarettes resulted in a decrease in forced expiratory flow at 75% (FEF75), indicating that nicotine-free ECs do not produce significant short-term changes in lung functions in non-smokers but have small effects on different lung functions in tobacco smokers ( 47 ). Two randomized trials examined individuals who switched from tobacco cigarettes to electronic cigarettes, revealing that using ECs for 5 days did not lead to negative respiratory health outcomes, with spirometry findings showing no significant effect on airflow obstruction or lung function after EC use ( 32 ). Chaumont et al. reported that cessation of e-cigarette use appeared to improve the lung inflammation profile, though it did not affect spirometry variables (FEV1), lung membrane diffusing capacity, or lung capillary volume; additionally, nicotine-containing e-cigarettes did not decrease forced expiratory flow (FEF) compared with cessation of e-cigarette use ( 48 ). Flouris et al., in a study on the short-term impact of active and passive use of nicotine-containing ECs versus conventional tobacco, found similar cotinine levels in users as in conventional tobacco smokers, with most changes occurring in smokers. They noted small changes in pulmonary function (increase in FEV 1 /FVC, FVC, PEF, FEF 25 - 75 ), an increase in exhaled carbon monoxide (CO), and serum nicotine, and a decrease in exhaled fraction of nitric oxide (FeNO), although these values were lower than those found in regular tobacco users ( 49 ).

Regarding long-term effects in the general population with a history of smoking, a controlled 1-year follow-up study found that changes in exhaled nitric oxide (FeNO) were significantly correlated with those of exhaled carbon monoxide (COe) throughout the follow-up, mainly a relative increase in the first 3 months followed by a significant increase at 6 and 12 months ( 53 ).

Regarding short-term respiratory symptoms, Dicpinigaitis et al. found a significant inhibition of cough reflex sensitivity as measured by the cough provocation test (capsaicin inhalation) in 30 individuals with no history of smoking after a single exposure to e-cigarette vapor; this effect was transient, as cough reflex sensitivity returned 24 h after EC use ( 50 ). They also examined 20 HIV-positive smokers who completely switched from tobacco cigarettes to EC during an 8-week follow-up, demonstrating clinical improvement in symptoms such as coughing, wheezing, and shortness of breath, a decrease in daily cigarette consumption, and complete cessation in seven participants ( 51 ). Campagna et al. noted that, in the long term (1 year), respiratory symptoms rapidly disappeared with the use of electronic cigarettes, both in individuals who quit smoking completely and those who reduced tobacco consumption ( 53 ) ( Table 2 ).

3.3.3 Effects on respiratory biomarkers

Regarding the effects on respiratory biomarkers, two studies explored this influence in the short term. A randomized trial of dual use of tobacco and e-cigarettes (with different flavors) showed a decrease in harmful or potentially harmful biomarkers for polycyclic aromatic hydrocarbons (PAHs such as pyrene) in users of electronic cigarettes, which were reduced by 62–69%. The levels of Tobacco Specific Nitrosamines (TSNAs), such as 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanol (NNAL) and N-nitrosinornicotine (NNN), were reduced by 62–64 and 87–93%, respectively, as well as a reduction in the levels of volatile organic compounds. As for CO and NO measurements, the decrease in the e-cigarette cessation group saw a reduction of 89%. In addition, exhaled nitric oxide measurements evidenced an increase from day 1 to 5 mainly in the EC use and tobacco cessation groups ( 52 ). In contrast, Veldheer et al. evaluated the influence of electronic vs. non-electronic cigarette use (non-aerosol producing so-called cigarette substitutes) in tobacco smoking subjects, finding no relevant changes in terms of lung function after 3 months of e-cigarette use. In addition, it was determined that EC consumption reduced the number of daily cigarettes consumed and cigarette dependence in smoking subjects ( 34 ) ( Table 2 ).

3.4 Effects on other tissues and organs

3.4.1 renal effects.

Two randomized trials evaluated the short-term effects of e-cigarette use at the urinary level in former tobacco smokers and current nicotine e-cigarette users (at least 1 year). The effects of e-cigarette cessation with and without nicotine at the urinary level showed a decrease in the excretion of propylene glycol, 3-hydroxyisovalerate, and pyruvate, for those who performed the complete cessation sessions of vaping. While within the nicotine vaping sessions, the urinary excretion of trimethylamine oxide and hippurate were lower. Finally, the excretion of N-phenylacetyl glycine was lower in nicotine sessions compared to complete cessation of vaping ( 48 ). Similarly, among current tobacco smokers, e-cigarette smokers, and controls, it was evidenced that urinary excretion of biomarkers of exposure in individuals who ceased regular and electronic cigarette smoking decreased by 66–98%, including NNAL (4-(methylnitrosamino)-1-(3-pyridyl)-1-butanol), carboxyhemoglobin, nicotine, and its metabolites ( 52 ) ( Table 3 ).

Table 3 . Systematic summary of evidence on the effects of e-cigarettes on various tissues and organs.

3.4.2 Effects on body mass

Regarding effects on body mass and body weight, e-cigarette use apparently has no effect, both in the short term as found by Veldheer et al. in their randomized controlled study in which there were no statistically significant variations in weight ( 34 ), and in the long term as demonstrated by Polosa et al. in their prospective study with a follow-up of 3.5 years in which no statistically significant changes in body weight and e-cigarette consumption were evident, even in participants who consumed e-cigarettes with nicotine ( 37 ).

3.4.3 Effects on fertility

Despite the vast panorama of studies in animal models, only one report was found on e-cigarette and fertility in humans. Harlow et al. ( 54 ) found a small effect from a cohort study in US women, determined by a fertility index = 0.84; 95% CI: 0.67, 1.06 in current versus never users of e-cigarettes.

4 Discussion

Since their emergence as an alternative to tobacco smoking cessation, electronic nicotine delivery systems (ENDS) have been gaining popularity globally, with increasing use among non-tobacco smoking groups such as youth and even children ( 7 , 16 , 55 ). In addition to their well-known yet controversial role in facilitating tobacco smoking cessation and the associated health benefits ( 8 , 56 – 58 ), their growing commonality may foster erroneous beliefs about the harmlessness of e-cigarette use.

The industry has evolved rapidly, marked by indiscriminate use, which in less than two decades has led to the development of devices with various characteristics, including disposable options ( 59 ). These devices commonly use e-liquid, which contains various substances that may be flavored and may contain nicotine. Undoubtedly, these features have made these devices appealing to a broad audience and have complicated their study ( 60 , 61 ).

Research on the effects of e-cigarette use faces challenges due to these variations. This review has attempted to systematize the findings from research that aims to objectively assess the effects of e-cigarette use on various organs and tissues, identifying 33 studies conducted in different settings and with various comparisons.

Although many experts support the use of electronic cigarettes, it is important to recognize that many positive findings on human health are based on comparisons with traditional cigarettes, as shown in this review. Of all the included studies, only seven evaluated the effects of electronic cigarettes in a healthy non-tobacco smoking population, these studies revealed significant effects, including increases in heart rate, mean arterial blood pressure, arterial stiffness, and oxidative stress, as well as respiratory system changes such as alterations in the transcriptomes of the respiratory epithelium and alveolar macrophages, and a sudden increase in airflow resistance suggesting airway obstruction. Many of these effects were attributed to the presence of nicotine compared to nicotine-free electronic cigarettes ( 22 , 24 , 27 , 30 , 45 , 46 , 48 ) ( Figure 2 ). Additionally, five studies compared tobacco smokers with non-tobacco smokers, finding increases in vascular molecules and markers, including Scd40L, sP-selectin, platelet aggregation, significant reductions in pulmonary function (FEF-25), and a reduction in fertility incidence in women ( 37 , 40 , 47 , 49 , 54 ) ( Figure 2 ).

Figure 2 . Synthesis of the investigated health effects of e-cigarettes in healthy individuals.

The current literature included in this review discusses the acute effects of electronic cigarettes compared to traditional tobacco cigarettes. It is clear that most cigarette smokers find a risk reduction when switching to e-cigarettes. Although vaping is not harmless, it improves the health of those with chronic and continuous smoking. For instance, several studies highlighted positive outcomes associated with e-cigarette use; for instance, five studies reported reductions in blood pressure and heart rate among individuals who switched from smoking to using e-cigarettes, without immediate adverse effects on ventricular musculature or risk of acute myocardial infarction ( 32 – 34 , 42 , 43 ). However, three studies presented conflicting results on arterial stiffness ( 23 , 33 , 41 ). Increases in oxidative stress markers and endothelial and platelet vesicles were observed in terms of cardiovascular cellularity ( 23 , 31 , 39 , 49 ). Respiratory health findings were predominantly positive, showing reductions in polycyclic aromatic hydrocarbons, specific tobacco nitrosamines, and improvements in symptoms such as cough, wheezing, and shortness of breath, with no significant changes in pulmonary function ( 32 , 47 , 51 ). Positive renal effects were noted in former smokers who switched to e-cigarettes, indicated by decreased urinary biomarkers of tobacco smoke exposure ( 52 ). Population differences, product use patterns, and study designs and methodologies contribute to the mixed results observed in research on the effects of electronic cigarettes (e-cigarettes) compared to traditional cigarettes ( 23 , 33 , 41 ). Some studies included healthy individuals, while others involved participants with pre-existing conditions. Additionally, the frequency and duration of e-cigarette use varied, impacting health outcomes. Variations in study designs, such as cross-over versus parallel designs, and differences in endpoints measured also influenced the results ( 23 , 33 , 41 ). Lastly, methodologies for measuring these endpoints, including techniques and timing of biomarker assessments, varied across studies. Despite these findings, it would be incorrect to universally deem the effects of e-cigarette use as positive. Future assessments should focus on healthy, non-smoking populations, as previous studies suggest that while e-cigarettes pose a lower cardiovascular risk than tobacco cigarettes, the risk remains significant ( 61 , 62 ).

After carefully review and analyzed the literature, we found that some experts suggest that many of the harmful effects of e-cigarettes may be attributed to nicotine due to its direct impact and high addictiveness ( 63 – 65 ). Studies distinguishing between e-cigarettes with and without nicotine found increases in endothelial progenitor cells, endothelial vesicles, plasma myeloperoxidase, arterial stiffness, changes in nicotine receptors, alveolar macrophages, and mononuclear phagocytes, leading to altered inflammatory responses and increased pulmonary susceptibility in users of nicotine-containing e-cigarettes ( 29 , 41 , 46 ). These serious effects of nicotine are supported by the findings of Flouris et al., who noted the same cotinine levels in healthy non-tobacco smoking patients using nicotine e-cigarettes as in tobacco smokers ( 49 ). As this is a relatively new area of knowledge, the information is still conflicting. While some reviews emphasize the role of nicotine, other systematic reviews have shown that it is complicated to assign blame solely to nicotine ( 64 , 66 , 67 ).

Only five studies have sought to evaluate the long-term effects of e-cigarettes over periods ranging from 1 month to 3.5 years. These studies identified significant impacts on healthy participants with no smoking history, such as increased oxidability of low-density lipoproteins, decreased paraoxonase-1 activity, and lung inflammation attributed to propylene glycol ( 30 , 45 , 53 ). In contrast, among patients with a history of smoking, there were no significant changes in lung function 3 months after switching to e-cigarettes, although there was an increase in exhaled FeNO COe and improvements in clinical respiratory symptoms, including cough, phlegm, shortness of breath, and wheezing after 1 year ( 53 ). Comparisons between tobacco smokers and e-cigarette users showed no significant changes in body weight after 3.5 years of follow-up ( 37 ). The chronic effects of e-cigarettes remain unclear due to variability in study populations and a lack of consensus on what constitutes long-term exposure—ranging from as soon as 1 month to several years. Some experts believe the real long-term effects of e-cigarettes may only become apparent after decades ( 68 ).

Previous reviews have attempted to clarify the health effects of e-cigarettes, but this research addresses several identified limitations, including deficiencies in systematic search techniques and evidence evaluation. Notably, many studies did not differentiate between long and short-term effects or distinguish between populations that, despite being healthy, use tobacco ( 61 , 68 ). The evaluation of evidence in this research exposed limitations, especially in the randomized controlled trials that assessed the effects of e-cigarettes. These trials showed deficiencies in treatment allocation (e-cigarette, tobacco, and placebo), comparative groups, and follow-up. Meanwhile, the cohort studies demonstrated relatively acceptable quality, with insufficient follow-up being their main limitation ( Supplementary Tables 1 – 3 ).

The outlook for e-cigarettes on public health is concerning. This research underscores the need for further investigations to clarify misconceptions about e-cigarettes, including their real effects on populations not exposed to tobacco smoke, the impacts of long-term use, the effects of components other than nicotine like flavorings or moisturizers, and their actual effectiveness in smoking cessation ( 8 ).

The future concerns about e-cigarettes should mirror those historically associated with tobacco cigarettes. While this review focuses on the direct health impacts of e-cigarettes, it is crucial not to overlook their potential to attract very young non-smokers to substance use, thereby affecting public health significantly. We now can mitigate these long-term effects on global health and particularly on younger populations.

5 Conclusion

While traditional cigarettes are harmful to health, vaping offers some risk reduction. However, notable adverse effects, especially from nicotine-containing e-cigarettes on cardiovascular and respiratory function, challenge the perception that e-cigarettes are harmless. Studies in healthy non-smokers show significant adverse outcomes, suggesting e-cigarettes are not safe for non-smokers and could be harmful long-term. Moreover, the long-term effects remain uncertain, with potential risks similar to traditional smoking.

This review highlights key findings: the impact of e-cigarettes on oxidative stress, endothelial function, and platelet activation varies with smoking history and health status. Synthesizing data across smokers, ex-smokers, and non-smokers provides a comprehensive overview not covered in previous reviews. By considering these differences, our review offers a nuanced understanding of e-cigarette impacts. The analysis of markers and outcomes provides new insights into potential health risks, emphasizing the need for targeted public health policies. This information is crucial, especially for non-smokers and youth misled by e-cigarette safety profiles and flavors. Our review underscores the need for more research to define e-cigarette health impacts, including long-term effects and the impact of various e-liquid components.

Data availability statement

The original contributions presented in the study are included in the article/ Supplementary material , further inquiries can be directed to the corresponding author.

Author contributions

JI-C: Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing, Conceptualization, Data curation, Formal analysis, Investigation. PN-L: Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft. EM-L: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Resources, Validation, Visualization, Writing – original draft. MH: Data curation, Formal analysis, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft. AT-D-l-T: Data curation, Formal analysis, Investigation, Methodology, Resources, Visualization, Writing – original draft. EV-G: Data curation, Investigation, Visualization, Writing – original draft. CS-S: Formal analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft. VL-G: Data curation, Investigation, Project administration, Validation, Writing – original draft. WR: Data curation, Formal analysis, Investigation, Resources, Writing – original draft. DB: Data curation, Formal analysis, Investigation, Resources, Validation, Writing – original draft. MG: Data curation, Formal analysis, Investigation, Resources, Validation, Writing – original draft. AL-C: Investigation, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. EO-P: Investigation, Methodology, Project administration, Resources, Validation, Visualization, Writing – review & editing.

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1427752/full#supplementary-material

1. Proctor, RN . The history of the discovery of the cigarette–lung cancer link: evidentiary traditions, corporate denial, global toll. Tob Control . (2012) 21:87–91. doi: 10.1136/tobaccocontrol-2011-050338

PubMed Abstract | Crossref Full Text | Google Scholar

2. Doll, R, and Hill, AB. The mortality of doctors in relation to their smoking habits. Br Med J . (1954) 1:1451–5. doi: 10.1136/bmj.1.4877.1451

Crossref Full Text | Google Scholar

3. Cummings, KM, and Hyland, A. Impact of nicotine replacement therapy on smoking behavior. Annu Rev Public Health . (2005) 26:583–99. doi: 10.1146/annurev.publhealth.26.021304.144501

4. Pierce, JP, White, VM, and Emery, SL. What public health strategies are needed to reduce smoking initiation? Tob Control . (2012) 21:258–64. doi: 10.1136/tobaccocontrol-2011-050359

5. Peralta, AR, and Guntur, VP. Safety and efficacy of electronic cigarettes: a review. Mo Med . (2014) 111:238.

Google Scholar

6. Padon, AA, Maloney, EK, and Cappella, JN. Youth-targeted e-cigarette marketing in the US. Tob Regul Sci . (2017) 3:95–101. doi: 10.18001/TRS.3.1.9

7. Feeney, S, Rossetti, V, and Terrien, J. E-cigarettes—a review of the evidence—harm versus harm reduction. Tob Use Insights . (2022) 15:1179173X221087524. doi: 10.1177/1179173X221087524

8. Marques, P, Piqueras, L, and Sanz, M-J. An updated overview of e-cigarette impact on human health. Respir Res . (2021) 22:151. doi: 10.1186/s12931-021-01737-5

9. National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Population Health and Public Health Practice, Committee on the Review of the Health Effects of Electronic Nicotine Delivery SystemsEaton, DL, Kwan, LY, and Stratton, K (2018). E-Cigarette Devices, Uses, and Exposures. Public Health Consequences of E-Cigarettes. National Academies Press (US). Available at: https://www.ncbi.nlm.nih.gov/books/NBK507187/ (Accessed April 28, 2023)

10. Izquierdo-Condoy, JS, Naranjo-Lara, P, Morales-Lapo, E, Puglla-Mendoza, A, Hidalgo, MR, Tello-De-la-Torre, A, et al. E-cigarette use among Ecuadorian adults: a national cross-sectional study on use rates, perceptions, and associated factors. Tob Induc Dis . (2024) 22:1–13. doi: 10.18332/tid/187878

11. Reasoner, JJ, Regier, BA, Beckendorf, R, and McAllister, RK. Update on the risks of electronic cigarettes—vaping. Ochsner J . (2020) 20:2–4. doi: 10.31486/toj.20.0012

12. BCC Research (2024). Global E-cigarette Market Expected to Reach 47.5 Billion by 2028. https://www.bccresearch.com/pressroom/fod/global-e-cigarette-market-expected-to-reach-475-billion-by-2028 (Accessed April 28, 2024)

13. Hammond, D, Reid, JL, Rynard, VL, Fong, GT, Cummings, KM, McNeill, A, et al. Prevalence of vaping and smoking among adolescents in Canada, England, and the United States: repeat national cross sectional surveys. BMJ . (2019) 365:l2219. doi: 10.1136/bmj.l2219

14. Jamal, A, Gentzke, A, Hu, SS, Cullen, KA, Apelberg, BJ, Homa, DM, et al. Tobacco use among middle and high school students—United States, 2011–2016. Morb Mortal Wkly Rep . (2017) 66:597–603. doi: 10.15585/mmwr.mm6623a1

15. Cornelius, ME, Wang, TW, Jamal, A, Loretan, CG, and Neff, LJ. Tobacco product use among adults-United States, 2019. MMWR Morb Mortal Wkly Rep . (2020) 69:1736–42. doi: 10.15585/mmwr.mm6946a4

16. Jerzyński, T, Stimson, GV, Shapiro, H, and Król, G. Estimation of the global number of e-cigarette users in 2020. Harm Reduct J . (2021) 18:109. doi: 10.1186/s12954-021-00556-7

17. Kienhuis, AS, Soeteman-Hernandez, LG, Bos, PM, Cremers, HW, Klerx, WN, and Talhout, R. Potential harmful health effects of inhaling nicotine-free shisha-pen vapor: a chemical risk assessment of the main components propylene glycol and glycerol. Tob Induc Dis . (2015) 13:15. doi: 10.1186/s12971-015-0038-7

18. Bonner, E, Chang, Y, Christie, E, Colvin, V, Cunningham, B, Elson, D, et al. The chemistry and toxicology of vaping. Pharmacol Ther . (2021) 225:107837. doi: 10.1016/j.pharmthera.2021.107837

19. Shields, PG, Berman, M, Brasky, TM, Freudenheim, JL, Mathe, E, McElroy, JP, et al. A review of pulmonary toxicity of electronic cigarettes in the context of smoking: a focus on inflammation. Cancer Epidemiol Biomark Prev Publ Am Assoc Cancer Res Cosponsored Am Soc Prev Oncol . (2017) 26:1175–91. doi: 10.1158/1055-9965.EPI-17-0358

20. Wells, GA, Shea, B, O’Connell, D, Peterson, J, Welch, V, Losos, M, et al. (2021). The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute. Available at: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp (Accessed June 19, 2024).

21. JBI (2024). JBI critical appraisal tools. JBI Global. Available at: https://jbi.global/critical-appraisal-tools (Accessed June 19, 2024).

22. Gonzalez, JE, and Cooke, WH. Acute effects of electronic cigarettes on arterial pressure and peripheral sympathetic activity in young nonsmokers. Am J Physiol Heart Circ Physiol . (2021) 320:H248–55. doi: 10.1152/ajpheart.00448.2020

23. Biondi-Zoccai, G, Sciarretta, S, Bullen, C, Nocella, C, Violi, F, Loffredo, L, et al. Acute effects of heat-not-burn, electronic vaping, and traditional tobacco combustion cigarettes: the Sapienza University of Rome-vascular assessment of Proatherosclerotic effects of smoking (SUR-VAPES) 2 randomized trial. J Am Heart Assoc . (2019) 8:e010455. doi: 10.1161/JAHA.118.010455

24. Antoniewicz, L, Brynedal, A, Hedman, L, Lundbäck, M, and Bosson, JA. Acute effects of electronic cigarette inhalation on the vasculature and the conducting airways. Cardiovasc Toxicol . (2019) 19:441–50. doi: 10.1007/s12012-019-09516-x

25. Yan, XS, and D’Ruiz, C. Effects of using electronic cigarettes on nicotine delivery and cardiovascular function in comparison with regular cigarettes. Regul Toxicol Pharmacol . (2015) 71:24–34. doi: 10.1016/j.yrtph.2014.11.004

26. Dimitriadis, K, Narkiewicz, K, Leontsinis, I, Konstantinidis, D, Mihas, C, Andrikou, I, et al. Acute effects of electronic and tobacco cigarette smoking on sympathetic nerve activity and blood pressure in humans. Int J Environ Res Public Health . (2022) 19:1–11. doi: 10.3390/ijerph19063237

27. Cooke, WH, Pokhrel, A, Dowling, C, Fogt, DL, and Rickards, CA. Acute inhalation of vaporized nicotine increases arterial pressure in young non-smokers: a pilot study. Clin Auton Res . (2015) 25:267–70. doi: 10.1007/s10286-015-0304-z

28. Spindle, TR, Talih, S, Hiler, MM, Karaoghlanian, N, Halquist, MS, Breland, AB, et al. Effects of electronic cigarette liquid solvents propylene glycol and vegetable glycerin on user nicotine delivery, heart rate, subjective effects, and puff topography. Drug Alcohol Depend . (2018) 188:193–9. doi: 10.1016/j.drugalcdep.2018.03.042

29. Chaumont, M, de Becker, B, Zaher, W, Culié, A, Deprez, G, Mélot, C, et al. Differential effects of E-cigarette on microvascular endothelial function, arterial stiffness and oxidative stress: a randomized crossover trial. Sci Rep . (2018) 8:10378. doi: 10.1038/s41598-018-28723-0

30. Moheimani, RS, Bhetraratana, M, Peters, KM, Yang, BK, Yin, F, Gornbein, J, et al. Sympathomimetic effects of acute E-cigarette use: role of nicotine and non-nicotine constituents. J Am Heart Assoc . (2017) 6:e006579. doi: 10.1161/JAHA.117.006579

31. Kerr, DMI, Brooksbank, KJM, Taylor, RG, Pinel, K, Rios, FJ, Touyz, RM, et al. Acute effects of electronic and tobacco cigarettes on vascular and respiratory function in healthy volunteers: a cross-over study. J Hypertens . (2019) 37:154–66. doi: 10.1097/HJH.0000000000001890

32. D’Ruiz, CD, O’Connell, G, Graff, DW, and Yan, XS. Measurement of cardiovascular and pulmonary function endpoints and other physiological effects following partial or complete substitution of cigarettes with electronic cigarettes in adult smokers. Regul Toxicol Pharmacol . (2017) 87:36–53. doi: 10.1016/j.yrtph.2017.05.002

33. George, J, Hussain, M, Vadiveloo, T, Ireland, S, Hopkinson, P, Struthers, AD, et al. Cardiovascular effects of switching from tobacco cigarettes to electronic cigarettes. J Am Coll Cardiol . (2019) 74:3112–20. doi: 10.1016/j.jacc.2019.09.067

34. Veldheer, S, Yingst, J, Midya, V, Hummer, B, Lester, C, Krebs, N, et al. Pulmonary and other health effects of electronic cigarette use among adult smokers participating in a randomized controlled smoking reduction trial. Addict Behav . (2019) 91:95–101. doi: 10.1016/j.addbeh.2018.10.041

35. Farsalinos, K, Cibella, F, Caponnetto, P, Campagna, D, Morjaria, JB, Battaglia, E, et al. Effect of continuous smoking reduction and abstinence on blood pressure and heart rate in smokers switching to electronic cigarettes. Intern Emerg Med . (2016) 11:85–94. doi: 10.1007/s11739-015-1361-y

36. Klonizakis, M, Gumber, A, McIntosh, E, and Brose, LS. Medium- and longer-term cardiovascular effects of e-cigarettes in adults making a stop-smoking attempt: a randomized controlled trial. BMC Med . (2022) 20:276. doi: 10.1186/s12916-022-02451-9

37. Polosa, R, Cibella, F, Caponnetto, P, Maglia, M, Prosperini, U, Russo, C, et al. Health impact of E-cigarettes: a prospective 3.5-year study of regular daily users who have never smoked. Sci Rep . (2017) 7:13825. doi: 10.1038/s41598-017-14043-2

38. Flouris, AD, Poulianiti, KP, Chorti, MS, Jamurtas, AZ, Kouretas, D, Owolabi, EO, et al. Acute effects of electronic and tobacco cigarette smoking on complete blood count. Food Chem Toxicol Int J Publ Br Ind Biol Res Assoc . (2012) 50:3600–3. doi: 10.1016/j.fct.2012.07.025

39. Mobarrez, F, Antoniewicz, L, Hedman, L, Bosson, JA, and Lundbäck, M. Electronic cigarettes containing nicotine increase endothelial and platelet derived extracellular vesicles in healthy volunteers. Atherosclerosis . (2020) 301:93–100. doi: 10.1016/j.atherosclerosis.2020.02.010

40. Nocella, C, Biondi-Zoccai, G, Sciarretta, S, Peruzzi, M, Pagano, F, Loffredo, L, et al. Impact of tobacco versus electronic cigarette smoking on platelet function. Am J Cardiol . (2018) 122:1477–81. doi: 10.1016/j.amjcard.2018.07.029

41. Antoniewicz, L, Bosson, JA, Kuhl, J, Abdel-Halim, SM, Kiessling, A, Mobarrez, F, et al. Electronic cigarettes increase endothelial progenitor cells in the blood of healthy volunteers. Atherosclerosis . (2016) 255:179–85. doi: 10.1016/j.atherosclerosis.2016.09.064

42. Farsalinos, KE, Tsiapras, D, Kyrzopoulos, S, Savvopoulou, M, and Voudris, V. Acute effects of using an electronic nicotine-delivery device (electronic cigarette) on myocardial function: comparison with the effects of regular cigarettes. BMC Cardiovasc Disord . (2014) 14:78. doi: 10.1186/1471-2261-14-78

43. Alzahrani, T, Pena, I, Temesgen, N, and Glantz, SA. Association between electronic cigarette use and myocardial infarction. Am J Prev Med . (2018) 55:455–61. doi: 10.1016/j.amepre.2018.05.004

44. Vansickel, AR, Cobb, CO, Weaver, MF, and Eissenberg, TE. A clinical laboratory model for evaluating the acute effects of electronic “cigarettes”: nicotine delivery profile and cardiovascular and subjective effects. Cancer Epidemiol Biomarkers Prev . (2010) 19:1945–53. doi: 10.1158/1055-9965.EPI-10-0288

45. Song, M-A, Reisinger, SA, Freudenheim, JL, Brasky, TM, Mathé, EA, McElroy, JP, et al. Effects of electronic cigarette constituents on the human lung: a pilot clinical trial. Cancer Prev Res (Phila) . (2020) 13:145–52. doi: 10.1158/1940-6207.CAPR-19-0400

46. Staudt, MR, Salit, J, Kaner, RJ, Hollmann, C, and Crystal, RG. Altered lung biology of healthy never smokers following acute inhalation of E-cigarettes. Respir Res . (2018) 19:78. doi: 10.1186/s12931-018-0778-z

47. Ferrari, M, Zanasi, A, Nardi, E, Morselli Labate, AM, Ceriana, P, Balestrino, A, et al. Short-term effects of a nicotine-free e-cigarette compared to a traditional cigarette in smokers and non-smokers. BMC Pulm Med . (2015) 15:120. doi: 10.1186/s12890-015-0106-z

48. Chaumont, M, Tagliatti, V, Channan, EM, Colet, J-M, Bernard, A, Morra, S, et al. Short halt in vaping modifies cardiorespiratory parameters and urine metabolome: a randomized trial. Am J Phys Lung Cell Mol Phys . (2020) 318:L331–44. doi: 10.1152/ajplung.00268.2019

49. Flouris, AD, Chorti, MS, Poulianiti, KP, Jamurtas, AZ, Kostikas, K, Tzatzarakis, MN, et al. Acute impact of active and passive electronic cigarette smoking on serum cotinine and lung function. Inhal Toxicol . (2013) 25:91–101. doi: 10.3109/08958378.2012.758197

50. Dicpinigaitis, PV, Lee Chang, A, Dicpinigaitis, AJ, and Negassa, A. Effect of electronic cigarette use on the urge-to-cough sensation. Nicotine Tob Res Off J Soc Res Nicotine Tob . (2016) 18:1763–5. doi: 10.1093/ntr/ntw021

51. Cioe, PA, Mercurio, AN, Lechner, W, Costantino, CC, Tidey, JW, Eissenberg, T, et al. A pilot study to examine the acceptability and health effects of electronic cigarettes in HIV-positive smokers. Drug Alcohol Depend . (2020) 206:107678. doi: 10.1016/j.drugalcdep.2019.107678

52. O’Connell, G, Graff, DW, and D’Ruiz, CD. Reductions in biomarkers of exposure (BoE) to harmful or potentially harmful constituents (HPHCs) following partial or complete substitution of cigarettes with electronic cigarettes in adult smokers. Toxicol Mech Methods . (2016) 26:453–64. doi: 10.1080/15376516.2016.1196282

53. Campagna, D, Cibella, F, Caponnetto, P, Amaradio, MD, Caruso, M, Morjaria, JB, et al. Changes in breathomics from a 1-year randomized smoking cessation trial of electronic cigarettes. Eur J Clin Investig . (2016) 46:698–706. doi: 10.1111/eci.12651

54. Harlow, AF, Hatch, EE, Wesselink, AK, Rothman, KJ, and Wise, LA. Electronic cigarettes and Fecundability: results from a prospective preconception cohort study. Am J Epidemiol . (2020) 190:353–61. doi: 10.1093/aje/kwaa067

55. Venkatesan, P . UK tobacco and vapes bill one step closer. Lancet Oncol . (2024) 25:541. doi: 10.1016/S1470-2045(24)00185-2

56. Levy, DT, Yuan, Z, Luo, Y, and Abrams, DB. The relationship of E-cigarette use to cigarette quit attempts and cessation: insights from a large, nationally representative U.S. survey. Nicotine Tob Res . (2018) 20:931–9. doi: 10.1093/ntr/ntx166

57. Kalkhoran, S, and Glantz, SA. E-cigarettes and smoking cessation in real-world and clinical settings: a systematic review and meta-analysis. Lancet Respir Med . (2016) 4:116–28. doi: 10.1016/S2213-2600(15)00521-4

58. Zhang, Y-Y, Bu, F-L, Dong, F, Wang, J-H, Zhu, S-J, Zhang, X-W, et al. The effect of e-cigarettes on smoking cessation and cigarette smoking initiation: an evidence-based rapid review and meta-analysis. Tob Induc Dis . (2021) 19:1–15. doi: 10.18332/tid/131624

59. Lin, H-C, Buu, A, and Su, W-C. Disposable E-cigarettes and associated health risks: an experimental study. Int J Environ Res Public Health . (2022) 19:10633. doi: 10.3390/ijerph191710633

60. Patnode, CD, Henderson, JT, Thompson, JH, Senger, CA, Fortmann, SP, and Whitlock, EP. Behavioral counseling and pharmacotherapy interventions for tobacco cessation in adults, including pregnant women: a review of reviews for the U.S. preventive services task force. Ann Intern Med . (2015) 163:608–21. doi: 10.7326/M15-0171

61. Travis, N, Knoll, M, Cadham, CJ, Cook, S, Warner, KE, Fleischer, NL, et al. Health effects of electronic cigarettes: an umbrella review and methodological considerations. Int J Environ Res Public Health . (2022) 19:9054. doi: 10.3390/ijerph19159054

62. Peruzzi, M, Biondi-Zoccai, G, Carnevale, R, Cavarretta, E, Frati, G, and Versaci, F. Vaping cardiovascular health risks: an updated umbrella review. Curr Emerg Hosp Med Rep . (2020) 8:103–9. doi: 10.1007/s40138-020-00219-0

63. National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Population Health and Public Health Practice (2018). Committee on the Review of the Health Effects of Electronic Nicotine Delivery Systems. Public Health Consequences of E-Cigarettes . Eaton DL, Kwan LY, Stratton K, editors. Washington (DC): National Academies Press (US). Available at: http://www.ncbi.nlm.nih.gov/books/NBK507171/ (Accessed April 11, 2024)

64. Larue, F, Tasbih, T, Ribeiro, PAB, Lavoie, KL, Dolan, E, and Bacon, SL. Immediate physiological effects of acute electronic cigarette use in humans: a systematic review and meta-analysis. Respir Med . (2021) 190:106684. doi: 10.1016/j.rmed.2021.106684

65. Mishra, A, Chaturvedi, P, Datta, S, Sinukumar, S, Joshi, P, and Garg, A. Harmful effects of nicotine. Ind J Med Paediatr Oncol Off J Ind Soc Med Pediatr Oncol . (2015) 36:24–31. doi: 10.4103/0971-5851.151771

66. Pisinger, C, and Døssing, M. A systematic review of health effects of electronic cigarettes. Prev Med . (2014) 69:248–60. doi: 10.1016/j.ypmed.2014.10.009

67. Harrell, PT, Simmons, VN, Correa, JB, Padhya, TA, and Brandon, TH. Electronic nicotine delivery systems (“e-cigarettes”): review of safety and smoking cessation efficacy. Otolaryngol Head Neck Surg Off J Am Acad . (2014) 151:381–93. doi: 10.1177/0194599814536847

68. Gotts, JE, Jordt, S-E, McConnell, R, and Tarran, R. What are the respiratory effects of e-cigarettes? BMJ . (2019) 366:l 5275. doi: 10.1136/bmj.l5275

Keywords: E-cigarettes, health effects, tobacco harm reduction, chronic effects of vaping, systematic review

Citation: Izquierdo-Condoy JS, Naranjo-Lara P, Morales-Lapo E, Hidalgo MR, Tello-De-la-Torre A, Vásconez-Gonzáles E, Salazar-Santoliva C, Loaiza-Guevara V, Rincón Hernández W, Becerra DA, González MBD, López-Cortés A and Ortiz-Prado E (2024) Direct health implications of e-cigarette use: a systematic scoping review with evidence assessment. Front. Public Health . 12:1427752. doi: 10.3389/fpubh.2024.1427752

Received: 04 May 2024; Accepted: 15 July 2024; Published: 29 July 2024.

Reviewed by:

Copyright © 2024 Izquierdo-Condoy, Naranjo-Lara, Morales-Lapo, Hidalgo, Tello-De-la-Torre, Vásconez-Gonzáles, Salazar-Santoliva, Loaiza-Guevara, Rincón Hernández, Becerra, González, López-Cortés and Ortiz-Prado. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Esteban Ortiz-Prado, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• BMJ NPH Collections
• BMJ Journals

You are here

• Online First
• Food environment and obesity: a systematic review and meta-analysis
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

other Versions

• You are currently viewing an earlier version of this article (April 22, 2024).
• View the most recent version of this article

• http://orcid.org/0000-0002-5210-1538 Elisa Pineda 1 , 2 ,
• Jemima Stockton 3 ,
• Shaun Scholes 3 ,
• Camille Lassale 4 , 5 and
• Jennifer S Mindell 3
• 1 The George Institute for Global Health UK , Imperial College London , London , UK
• 2 School of Public Health , Imperial College London , London , UK
• 3 Research Department of Epidemiology and Public Health , University College London , London , UK
• 4 Barcelona Institute for Global Health (ISGlobal) , Barcelona , Spain
• 5 CIBER Physiopathology of Obesity and Nutrition (CIBEROBN) , Carlos III Health Institute (ISCIII) , Madrid , Spain
• Correspondence to Dr Elisa Pineda; e.pineda{at}imperial.ac.uk

Background Obesity is influenced by a complex, multifaceted system of determinants, including the food environment. Governments need evidence to act on improving the food environment. The aim of this study was to review the evidence from spatial environmental analyses and to conduct the first series of meta-analyses to assess the impact of the retail food environment on obesity.

Methods We performed a systematic review and random-effects meta-analyses, focusing on geographical–statistical methods to assess the associations between food outlet availability and obesity. We searched OvidSP-Medline, Scielo, Scopus and Google Scholar databases up to January 2022. The search terms included spatial analysis, obesity and the retail food environment. Effect sizes were pooled by random-effects meta-analyses separately according to food outlet type and geographical and statistical measures.

Findings Of the 4118 retrieved papers, we included 103 studies. Density (n=52, 50%) and linear and logistic regressions (n=68, 66%) were the main measures used to assess the association of the food environment with obesity. Multilevel or autocorrelation analyses were used in 35 (34%) studies. Fast-food outlet proximity was positively and significantly associated with obesity (OR: 1.15, 95% CI: 1.02 to 1.30, p=0.02). Fresh fruit and vegetable outlet density and supermarket proximity were inversely associated with obesity (OR: 0.93, 95% CI: 0.90 to 0.96, p<0.001; OR: 0.90, 95% CI: 0.82 to 0.98, p=0.02). No significant associations were found for restaurants, convenience stores or any of the body mass index measures.

Conclusions Food outlets which sell mostly unhealthy and ultra-processed foods were associated with higher levels of obesity, while fruit and vegetable availability and supermarket accessibility, which enable healthier food access, were related to lower levels of obesity. The regulation of food outlets through zoning laws may not be enough to tackle the burden of obesity. Regulations that focus on increasing the availability of healthy food within stores and ensure overall healthy food environments require further attention.

PROSPERO registration number CRD42018111652.

• Malnutrition

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:  https://creativecommons.org/licenses/by/4.0/ .

https://doi.org/10.1136/bmjnph-2023-000663

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

The food environment is a recognised key determinant for the prevention of obesity and other diet-related non-communicable diseases (NCDs). Multiple studies have identified inconsistent findings regarding the association between elements of the retail food environment and obesity. Variability in geographical and analytical methods has been pointed out as a potential cause for these discrepancies.

WHAT THIS STUDY ADDS

This systematic literature review and meta-analyses consolidates all the evidence and effect sizes to determine which elements of the retail food environment have the greatest impact on obesity. It stratigically considers elements of the retail food environment, along with geographical and statistical methods to provide increased statistical power, accuracy, and a comprehensive summary of findings regarding the association of the food environment with obesity.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

The evidence generated from this systematic review and meta-analyses can serve as a foundational tool for policymakers and researchers in developing programmes and interventions for the prevention of obesity and other diet-related NCDs. This study offers a quantitative and visual guide for identifying the retail food environment elements that require greater focus in strategies aimed at tackling obesity.

Introduction

The retail food environment and obesity.

Obesity, a critical risk factor for non-communicable diseases (NCDs), is prevalent in countries across all income levels, including low-, middle- and high-income nations. 1 2 Its prevalence is shaped by a complex array of determinants, notably the retail food environment and advertising landscapes. 3 Modern food environments are marked by the widespread availability and promotion of energy-dense, nutrient-poor foods. 4 For instance, the increase in food retailers has contributed to a significant rise in calorie availability, facilitating greater access to a wide array of food choices. 5 To combat structural overconsumption and curb the obesity epidemic, policy interventions must be enacted, even in the face of commercial interests. However, the specific influence of food environments on obesity, as distinct from individual behaviour, remains poorly defined. 6 7 There is a scarcity of evidence identifying the exact elements of food environments that contribute to obesity and could be targeted for change. 3 4 8 This review aims to enhance understanding of the analytical methods required to dissect the various components of the modern retail food environment in relation to obesity and to assess the impact of retail food environments on obesity levels.

Analysing the retail food environment

Spatial analysis, leveraging Geographic Information Systems (GIS), has become instrumental in exploring the interplay between the environment and health outcomes. It particularly aids in investigating the food environment by mapping the locations of food stores, examining their spatial distribution and assessing their impact on obesity and population health. This approach enables the study of how the proximity and density of food outlets relative to residential areas influence access to healthy versus unhealthy food options, thereby identifying key environmental factors and protective measures against obesity through spatial patterns. 9–12

Previous literature reviews

Previous literature reviews on the relationship between the retail food environment and obesity have underscored methodological issues that may affect the analysis and interpretation of how food environments influence health and dietary outcomes. There is a recognised need for precise, comprehensive evaluations, including standardised and validated measurement techniques and diverse approaches to assessing the retail food environment, as current methods exhibit considerable variability. 12–14 Essential aspects of retail food environment research involve confirming the location and type of food outlets through store audits (ground truthing), 13 considering the confounding effects of socioeconomic status 14 15 and using longitudinal studies to observe changes in the retail food environment and dietary choices over time. 15 16

Despite numerous studies investigating the retail food environment’s impact on obesity, systematic reviews and meta-analyses are scarce. 17–20 Previous analyses have often been restricted to specific regions or populations, with limited attention to the methodologies for measuring the retail food environment. 17–20 This paper undertakes a systematic review and meta-analyses to synthesise available evidence on the retail food environment’s role in obesity and diet-related NCDs, aiming to pinpoint elements that could be targeted by policy interventions. Furthermore, it critically assesses the methodological strategies used to study the global impact of the retail food environment on obesity.

Obesity and the food environment

The food environment encompasses physical, economic, political and sociocultural factors affecting dietary choices. 21 Glanz et al. ’s 22 model suggests that dietary intake is shaped by policy, environment, individual and behavioural factors. This includes the community nutrition environment (types of food stores, locations, and availability), which in this study we refer to as the 'retail food environment'; organisational settings (neighbourhood, school, workplace); and consumer aspects (food availability, placement, pricing, promotions, nutrition labelling). Key attributes defining the food environment are geographical access, availability, affordability and advertising. 23–25 While various factors contribute to obesity, environmental and policy measures can significantly improve the food environment, leading to widespread dietary changes and reduced obesity and disease rates. 26

We performed a systematic review and meta-analyses to assess the association of the retail food environment with adult obesity and to evaluate the geographical and statistical methods used. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed ( online supplemental figure S1 ). Search results were screened by two reviewers for eligibility. The review was registered in PROSPERO as CRD42018111652.

Supplemental material

Literature search strategy.

We conducted a literature search on 31 January 2022, spanning papers published from 1946 onwards, to identify studies focusing on the impact of the retail food environment on obesity through spatial analysis. Using OvidSP-Medline, Scopus and Google Scholar databases, we structured the search around three primary themes: the retail food environment, obesity and spatial analysis. Initially, each theme was explored individually, and subsequently, we employed the ‘AND’ operator to search them concurrently. Using the Population, Intervention, Control, Outcome (PICO) framework ( online supplemental table S1 ) for eligibility assessment, 27 we considered publications examining the influence of the retail food environment on adult obesity or body mass index (BMI) for inclusion in our systematic literature review and meta-analyses.

Our literature search strategy involved MeSH words, Boolean search terms and proximity searching characters ($, *, W, #) on Medline (OvidSP, 1946–current: 31 January 2022). The terms covered diverse aspects such as buffer, chain, convenience, density variations (denoted by densit*), desert, distance, eating habits (indicated by eat$), environmental factors, farmers’ markets, fast food, geography, geolocation, geospatial analysis, GIS (geographic information systems), global, grocery stores, increase, index, location, markets, access, provision, proximity, restaurants, retail, spatial considerations, stores, supermarkets, supply, BMI (body mass index), body mass, nutrition, obesity, overweight, positional factors, weight gain and overeating. Additionally, the search extended to Scopus and Google Scholar using the query “(ALL (obesity) AND ALL (food environment OR convenience store OR food retail) AND ALL (GIS OR spatial analysis OR geographic information systems))” as of 31 January 2022.

Risk of bias and quality assessment criteria

Risk of bias and quality were evaluated using a weighted quality score derived from the Cochrane risk-of-bias tool, the systematic review data collection procedures from The Guide to Community Preventive Services 28 and the food environment quality assessment by Williams et al . 29 Nine criteria were assessed: population representativeness, outcome validity, exposure representativeness, exposure source, retail food environment assessment method, physical activity assessment, study design, statistical methods and data temporality. Studies received one point for each criterion met ( online supplemental table S2 ).

Spatial and statistical methods and study design appraisal

Study design, statistical methods and models were explored and assessed according to their consideration of spatial clustering, 30 and according to their inclusion of confounders.

Meta-analysis

We performed random-effect meta-analyses to explore the link between the retail food environment and obesity, analysing data from various outlets including fast-food restaurants, convenience stores, supermarkets and farmers’ markets. We evaluated the retail food environment using density, proximity and the Retail Food Environment Index (RFEI)—the ratio of unhealthy to healthy food outlets. Our analyses focused on ORs for categorical outcomes and beta-coefficients (β) for continuous variables, combining similar measures for meta-analyses. We assessed the impact of the retail food environment on adult BMI (β) and obesity prevalence (ORs), selecting the most relevant estimate from studies providing multiple results to ensure observations remained independent. Only models adjusted for confounders were included. For comparability, we considered data within 1 mile buffers or equivalent, representing walkable distances. In longitudinal studies, the most recent data were used. When results were stratified by sex and socioeconomic position (SEP), we chose observations based on the largest sample size or prioritised women and low-income groups if sizes were equal. We reported effect sizes and 95% CIs for each study, using Stata V.16.0 for all statistical analyses. 31

We retrieved 4118 studies, and after applying inclusion and exclusion criteria, retained 103 articles yielding 526 data points ( online supplemental figure S1 ). These were categorised by statistical measure, geographical measure and food outlet type, with 437 data points used in meta-analyses and meta-regression. The analysis covered 16 countries, with 90% of the studies from high-income countries: 1 from Africa, 5 each from Asia, Latin America and Australia, 14 from Europe and 74 from North America, spanning from 2004 to 2021, predominantly between 2011 and 2017 (n=54, 52%) ( online supplemental table S3 ).

In terms of retail food environment measures, 52 (50%) studies evaluated density, 21 (20%) proximity, 3 (3%) both, 4 (4%) the RFEI or variants and 15 (15%) other measures like ratio and diversity. Most studies (n=77, 75%) assessed one geographical measure, 20 (19%) evaluated two and six (6%) assessed up to three. From the 526 data points that were extracted from all studies, fast-food outlets were the most examined (n=166, 32%), followed by supermarkets (n=102, 19%), restaurants (n=101, 19%) and convenience stores (n=61, 12%), fresh fruit and vegetable stores (n=17, 3%), grocery stores (n=14, 3%), specialty stores (n=8, 2%), supercentres (n=5, 1%), and farmers’ markets (n=4, 1%). A majority of the studies, 61% (n=63), accounted for walkability or physical activity as a confounder ( online supplemental table S4 ).

Associations varied by geographical area, underscoring the need for representative geographical selection. For example, Fan et al 32 found different associations between restaurants and obesity for men at the census tract level and for women at the block level. However, 64% (n=66) of studies did not perform ground truthing or verify retail food environment data ( online supplemental table S4 ).

Statistical and geographical methods

Of the studies analysed, 68 (66%) applied linear or logistic regression, while 35 (34%) used multilevel modelling or methods accounting for spatial factors and clustering ( online supplemental table S3 ). In terms of data sources for food outlet locations, 39 (38%) used government databases, 27 (26%) commercial databases, 14 (14%) conducted ground truthing, 23 (22%) employed various methods and 1 (1%) did not disclose their source. Among the studies employing multilevel modelling or spatial considerations, 26 (74%) identified positive correlations between the presence of food retailers selling foods high in fat, sugar and salt (HFSS) and obesity rates ( online supplemental table S3 ).

Study design

Of the 89 cross-sectional studies analysed, 59 (66%) discovered a correlation between obesity and food retailers specialising in unhealthy foods and beverages, such as convenience stores and fast-food outlets. Among the 14 longitudinal studies, half revealed a significant link between the presence of unhealthy food outlets and obesity (refer to online supplemental tables S3 and S4 for detailed findings).

Quality and bias assessment of studies

The mean quality score of the studies was low, at 4 out of 9 points, with the highest being 7. 33 34 Key limitations included the reliance on cross-sectional designs, the failure to account for clustering or to apply spatial methods in 30 (29%) studies, reliance on self-reported height and weight data in 34 (33%) studies and the use of inappropriate statistical methods in 43 (42%) studies ( online supplemental table S5 ). Studies deemed to have a high risk of bias were excluded from the meta-analyses.

In the meta-analyses conducted, significant heterogeneity was observed across the studies, stemming from variations in statistical methods, study designs, stratification by gender and ethnicity, geographical measures of the retail food environment, classifications of food outlets and the definitions used to measure or define obesity, thereby limiting the robustness of the pooled analyses. Despite these variances, the majority of the studies used BMI, derived from measured height and weight, as a primary indicator, reporting it either as a continuous variable (kg/m 2 ) or in categorical terms (overweight or obesity). However, there was a notable scarcity of studies disaggregating outcome data by critical demographic factors such as age group, gender, ethnicity or SEP, which is pivotal considering the diverse exposure to retail food environments experienced by these groups. 35 Results of the meta-analyses are presented below by measure of the retail food environment (ie, density and proximity) and statistical measures (ORs and Beta-coefficients─in the supplemental material).

The findings revealed that the density of fast-food outlets did not significantly influence obesity rates (OR: 1.01, 95% CI: 0.99 to 1.04, p=0.18), in contrast to proximity to fast-food outlets, which showed a significant association with obesity (OR: 1.15, 95% CI: 1.02 to 1.30, p=0.02) ( figure 1 ). Restaurant density’s correlation with obesity was marginally significant (OR: 0.92, 95% CI: 0.85 to 1.00, p=0.05), yet the literature lacked sufficient data to evaluate the impact of restaurant proximity ( figure 2 ). No significant relationship was identified between the density of convenience stores and obesity (OR: 1.02, 95% CI: 0.95 to 1.10, p=0.64), and a similar non-significant trend was observed for proximity to convenience stores (OR: 1.04, 95% CI: 0.97 to 1.11, p=0.31) ( figure 3 ).

• Download figure
• Open in new tab
• Download powerpoint

Fast-food outlet density and proximity and its association with obesity. REML, Restricted Maximum Likelihood.

Restaurant density and its association with obesity. REML, Restricted Maximum Likelihood.

Convenience store density and proximity and its association with obesity. REML, Restricted Maximum Likelihood.

Furthermore, supermarket density did not show a significant relationship with obesity (OR: 0.98, 95% CI: 0.92 to 1.05, p=0.53), whereas a significant inverse relationship was evident between supermarket proximity and obesity (OR: 0.90, 95% CI: 0.82 to 0.98, p=0.02) ( figure 4 ). An inverse association was also noted between the density of fresh fruit and vegetable stores and obesity (OR: 0.93, 95% CI: 0.90 to 0.96, p<0.001) ( figure 5 ), though data were insufficient to assess the impact of proximity to these outlets. The RFEI did not reveal any significant associations with obesity (OR: 1.00, 95% CI: 0.99 to 1.01, p=0.99) ( figure 6 ), and BMI as a continuous variable showed no association with any type of food outlet, indicating a nuanced and complex relationship between the retail food environment and obesity ( online supplemental figures S2–S7 ).

Supermarket density and proximity and its association with obesity. REML, Restricted Maximum Likelihood.

Fruit and vegetable store density and its association with obesity. REML, Restricted Maximum Likelihood.

Retail Food Environment Index (RFEI) and its association with obesity. REML, Restricted Maximum Likelihood.

The results of our systematic review and meta-analyses indicate a nuanced relationship between the retail food environment and obesity. Results for the association between the retail food environment and obesity varied significantly by type of food outlet, statistical measure and geographical measure. However, the pooled effect sizes show that proximity of fast-food outlets was associated with a higher risk of obesity, while proximity of supermarkets and fresh fruit and vegetable stores was associated with a lower risk of obesity.

Previous research highlights the crucial role of fruit and vegetable availability and affordability in fostering healthy eating habits and preventing obesity and chronic diseases. 36 Conversely, fast-food outlets predominantly offer ultra-processed foods—industrially processed items rich in fat, salt and/or sugar—whose consumption is associated with increased risks of obesity and chronic conditions. 37

The observed phenomenon can be attributed to the ease of access to different types of food outlets and their impact on dietary choices. Fast-food outlets, often closer to residential areas or on the pathways from school or the office to home, provide convenient access to high-calorie, processed foods, which can contribute to higher obesity rates among nearby residents. 14 Conversely, supermarkets, which are sometimes located further from residential areas, offer a broader range of healthier food options. When supermarkets are closer, it encourages the purchase and consumption of healthier foods, potentially reducing obesity risk. 38 This highlights the significant role of the retail food environment accessibility in influencing dietary behaviours and obesity prevalence.

In addition, socioeconomic area level may play a critical role in this context by influencing both access to and choices within the retail food environment. 39 Individuals living in lower socioeconomic areas may have more limited access to supermarkets offering a variety of healthy options due to cost or proximity, leading to a reliance on closer, often less expensive fast-food outlets. 39 This disparity can result in dietary patterns that contribute to higher obesity rates in these populations, underscoring the need for targeted interventions to improve access to healthy food options across all socioeconomic groups.

Importantly, while geographical measures such as proximity and density provide insights into the retail food environment or built food environment, they do not capture the complexities within food outlets that influence consumer choices. The 'in-store food environment', encompassing product placement, promotion strategies and food layout, plays a pivotal role in shaping dietary habits. Studies have demonstrated that strategic placement of healthy food options at eye level or in prominent store locations can significantly influence consumer purchases towards healthier choices. 40–43

A comprehensive approach, addressing both the proximity of various food outlet types and the intricate details of the in-store food environment, is essential for devising effective public health interventions aimed at reducing obesity. Future research and policy efforts should consider these dimensions of the food environment to develop more nuanced and impactful strategies for obesity prevention.

The UK is a pioneer in regulating the food environment, having introduced legislation to restrict the promotion and placement of HFSS foods within retail settings, both online and physical. 44 This legislation targets the influence of food retailers on consumer choices, particularly aiming to reduce the impact of price promotions on children’s food preferences by limiting promotions and strategic placement of HFSS products. This is a crucial step in promoting healthier eating habits and combating obesity and related health issues.

Additionally, in high-income countries, zoning powers allow local authorities to regulate food outlets’ location, and healthy food carts have been effectively deployed in urban areas to increase access to nutritious food. 18

Studies on the food environment can inform the creation of improved land use and public health policies, mitigating the negative effects of local food and nutrition environments on population health 45 Effective obesity reduction efforts should include policies or regulations to limit the availability of low-quality food in neighbourhoods, schools and other sensitive areas. However, the relationship between food outlets and obesity has shown inconsistent results, underscoring the need for solid evidence to guide government actions on enhancing the food environment.

This research significantly advances the evidence 18–20 by integrating a systematic review with meta-analyses to explore the retail food environment’s influence on obesity and BMI. This dual approach, not previously used for this topic, integrates geographical and statistical analyses and offers a comprehensive analysis of the relationship between food outlet types, BMI and obesity. Furthermore, this study is distinct as it includes analyses that employ spatial methodologies to explore the retail food environment’s components and their correlation with obesity, providing a comprehensive evidence base for policy formulation aimed at enhancing public health.

Implications for policymakers and urban planners

The observed association between fast-food outlet proximity and increased obesity risk emphasises the need for zoning regulations to manage their density in residential areas, schools and communal spaces. This strategic intervention becomes crucial in mitigating the obesity crisis. Our study discerns variations in associations among different food outlet types. While proximity of fast-food outlets correlates positively with obesity, proximity of supermarkets and fresh produce stores demonstrates an inverse relationship. Urban planners can influence health outcomes by strategically placing health-promoting outlets in residential areas, aligning with the concept of fostering a ‘healthy food environment’.

Beyond reaffirming existing knowledge, our study introduces novel insights into nuanced relationships between specific food outlets and obesity risk. Policymakers and urban planners can leverage this information to refine existing zoning laws based on prevalent food outlet types.

Our analysis also reveals a gap in the assessment of in-store food environments. Policymakers should focus on internal dynamics, implementing regulations targeting the arrangement and promotion of food items within stores to encourage healthier choices. Moreover, they should engage with town planners, health professionals and community representatives to develop comprehensive strategies. Collaborative efforts can lead to urban spaces that limit the impact of detrimental food outlets and food choices while promoting health and well-being. This aligns with the broader goal of fostering healthier communities, emphasising the importance of continued research and dialogue between academia and policymakers.

Strengths and limitations

This study’s primary strength lies in its comprehensive systematic search strategy, which involved querying multiple databases, imposing no publication date restrictions and conducting searches in two languages. Additionally, it uniquely explored and assessed geographical measures and statistical methods within a systematic literature review context and conducted a risk-of-bias assessment to objectively evaluate the reviewed literature.

By incorporating spatial analysis, this study addressed gaps in previous literature by elucidating the impact of food outlets’ geographical distribution on obesity rates. This approach enabled the identification of spatial patterns and correlations potentially overlooked in traditional epidemiological studies, thereby providing insight into the obesogenic environment.

Spatial analysis also enhanced the meta-analyses by facilitating the integration and comparison of findings from studies across different geographical scales and settings, thereby bolstering the robustness of our conclusions. This rigour in methodology supported evidence synthesis, offering a detailed overview of the retail food environment’s role in obesity.

Through a detailed spatial analysis, our study not only corroborates the significance of geographical factors in obesity prevalence but also underscores the need for targeted public health interventions. By pinpointing areas with high concentrations of unhealthy food outlets relative to healthy ones, policymakers and urban planners can devise more effective strategies aimed at improving the food environment and, subsequently, public health.

However, the study has limitations. The review focused on obesity in the adult population because of the diverse reviews already focused on children, and because of the important role that adults play in food outlet selection within a family setting. Focusing on adult populations is critical for chronic disease prevention and successful ageing. Only studies based on neighbourhood, rural or urban environments were considered. Studies that did not include an objective measure of obesity such as BMI via measured height and weight were excluded. However, many studies that used BMI and other measures of diet and obesity were considered. The identified exposures, measures and outcomes included in this study were the most reported in the literature. Although this may exclude other important obesity-related outcomes (eg, adiposity, fat mass, diet), focusing on BMI and obesity allowed a wider comparison between studies and could facilitate translation into policies and actions to regulate and improve the food environment.

Despite significant methodological diversity among the studies reviewed, the literature consistently identifies the food environment as a crucial factor in preventing obesity. Regions characterised by abundant fast-food outlets, limited supermarket access and scarce fresh fruit and vegetable stores tend to have higher obesity rates. While regulating access to healthier food options is necessary, it may not suffice to combat obesity on its own. Comprehensive strategies are also needed, including regulation of the in-store availability of unhealthy foods and the promotion of a food environment that supports healthy and affordable diets.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

Acknowledgments.

The authors wish to acknowledge Dr Clare Llewelyn and Professor Eric Brunner for their guidance and support on this study.

• Boutayeb A ,
• Butland B ,
• Kopelman P , et al
• Geiger BM , et al
• Ritchie H ,
• Dubowitz T ,
• Bird CE , et al
• Jones NRV ,
• Conklin AI ,
• Suhrcke M , et al
• Cromley EK ,
• McLafferty SL
• Penney TL ,
• Rainham DGC ,
• Dummer TJB , et al
• Thornton LE ,
• Pearce JR ,
• Kavanagh AM
• Charreire H ,
• Salze P , et al
• Sorensen G ,
• Subramanian SV , et al
• Fraser LK ,
• Edwards KL ,
• Cade J , et al
• Dzewaltowski DA
• Rangan A , et al
• Westbury S ,
• Jones HM , et al
• Fuentes Pacheco A ,
• Carrillo Balam G ,
• Archibald D , et al
• Osei-Assibey G ,
• Macdiarmid J , et al
• Swinburn B ,
• Vandevijvere S , et al
• Sallis JF ,
• Saelens BE , et al
• Office of Nutrition Policy and Promotion HC
• Hommel B , et al
• Excellence NIfHaC
• Wright-De Agüero LK ,
• Briss PA , et al
• Williams J ,
• Scarborough P ,
• Matthews A , et al
• Gleditsch KS
• Hanson HA ,
• Zick CD , et al
• Carroll SJ ,
• Taylor AW , et al
• Brunner EJ ,
• Llewellyn CH , et al
• Patton GC ,
• Sawyer SM ,
• Santelli JS , et al
• Alae-Carew C , et al
• Monteiro C ,
• Lawrence M , et al
• Michimi A ,
• Wimberly MC
• Ohri-Vachaspati P ,
• DeWeese RS ,
• Acciai F , et al
• Liberato SC ,
• Brimblecombe J
• McGowan CC ,
• Zenk SN , et al
• Morales KH , et al
• Department of Health & Social care
• Cardinal BJ , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1

X @elisap_ana

Contributors EP designed the study, collected and analysed the data, drafted the manuscript, and was responsible for the overall content as the guarantor. JS, SS, CL and JSM drafted and revised the draft and provided statistical advice.

Funding This study was funded by CONACYT, the National Council on Science and Technology in Mexico.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: