sources for literature review

Purdue Online Writing Lab Purdue OWL® College of Liberal Arts

Writing a Literature Review

Welcome to the Purdue OWL

This page is brought to you by the OWL at Purdue University. When printing this page, you must include the entire legal notice.

Copyright ©1995-2018 by The Writing Lab & The OWL at Purdue and Purdue University. All rights reserved. This material may not be published, reproduced, broadcast, rewritten, or redistributed without permission. Use of this site constitutes acceptance of our terms and conditions of fair use.

A literature review is a document or section of a document that collects key sources on a topic and discusses those sources in conversation with each other (also called synthesis ). The lit review is an important genre in many disciplines, not just literature (i.e., the study of works of literature such as novels and plays). When we say “literature review” or refer to “the literature,” we are talking about the research ( scholarship ) in a given field. You will often see the terms “the research,” “the scholarship,” and “the literature” used mostly interchangeably.

Where, when, and why would I write a lit review?

There are a number of different situations where you might write a literature review, each with slightly different expectations; different disciplines, too, have field-specific expectations for what a literature review is and does. For instance, in the humanities, authors might include more overt argumentation and interpretation of source material in their literature reviews, whereas in the sciences, authors are more likely to report study designs and results in their literature reviews; these differences reflect these disciplines’ purposes and conventions in scholarship. You should always look at examples from your own discipline and talk to professors or mentors in your field to be sure you understand your discipline’s conventions, for literature reviews as well as for any other genre.

A literature review can be a part of a research paper or scholarly article, usually falling after the introduction and before the research methods sections. In these cases, the lit review just needs to cover scholarship that is important to the issue you are writing about; sometimes it will also cover key sources that informed your research methodology.

Lit reviews can also be standalone pieces, either as assignments in a class or as publications. In a class, a lit review may be assigned to help students familiarize themselves with a topic and with scholarship in their field, get an idea of the other researchers working on the topic they’re interested in, find gaps in existing research in order to propose new projects, and/or develop a theoretical framework and methodology for later research. As a publication, a lit review usually is meant to help make other scholars’ lives easier by collecting and summarizing, synthesizing, and analyzing existing research on a topic. This can be especially helpful for students or scholars getting into a new research area, or for directing an entire community of scholars toward questions that have not yet been answered.

What are the parts of a lit review?

Most lit reviews use a basic introduction-body-conclusion structure; if your lit review is part of a larger paper, the introduction and conclusion pieces may be just a few sentences while you focus most of your attention on the body. If your lit review is a standalone piece, the introduction and conclusion take up more space and give you a place to discuss your goals, research methods, and conclusions separately from where you discuss the literature itself.

Introduction:

• An introductory paragraph that explains what your working topic and thesis is
• A forecast of key topics or texts that will appear in the review
• Potentially, a description of how you found sources and how you analyzed them for inclusion and discussion in the review (more often found in published, standalone literature reviews than in lit review sections in an article or research paper)
• Summarize and synthesize: Give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: Don’t just paraphrase other researchers – add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically Evaluate: Mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: Use transition words and topic sentence to draw connections, comparisons, and contrasts.

Conclusion:

• Summarize the key findings you have taken from the literature and emphasize their significance
• Connect it back to your primary research question

How should I organize my lit review?

Lit reviews can take many different organizational patterns depending on what you are trying to accomplish with the review. Here are some examples:

• Chronological : The simplest approach is to trace the development of the topic over time, which helps familiarize the audience with the topic (for instance if you are introducing something that is not commonly known in your field). If you choose this strategy, be careful to avoid simply listing and summarizing sources in order. Try to analyze the patterns, turning points, and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred (as mentioned previously, this may not be appropriate in your discipline — check with a teacher or mentor if you’re unsure).
• Thematic : If you have found some recurring central themes that you will continue working with throughout your piece, you can organize your literature review into subsections that address different aspects of the topic. For example, if you are reviewing literature about women and religion, key themes can include the role of women in churches and the religious attitude towards women.
• Qualitative versus quantitative research
• Empirical versus theoretical scholarship
• Divide the research by sociological, historical, or cultural sources
• Theoretical : In many humanities articles, the literature review is the foundation for the theoretical framework. You can use it to discuss various theories, models, and definitions of key concepts. You can argue for the relevance of a specific theoretical approach or combine various theorical concepts to create a framework for your research.

What are some strategies or tips I can use while writing my lit review?

Any lit review is only as good as the research it discusses; make sure your sources are well-chosen and your research is thorough. Don’t be afraid to do more research if you discover a new thread as you’re writing. More info on the research process is available in our "Conducting Research" resources .

As you’re doing your research, create an annotated bibliography ( see our page on the this type of document ). Much of the information used in an annotated bibliography can be used also in a literature review, so you’ll be not only partially drafting your lit review as you research, but also developing your sense of the larger conversation going on among scholars, professionals, and any other stakeholders in your topic.

Usually you will need to synthesize research rather than just summarizing it. This means drawing connections between sources to create a picture of the scholarly conversation on a topic over time. Many student writers struggle to synthesize because they feel they don’t have anything to add to the scholars they are citing; here are some strategies to help you:

• It often helps to remember that the point of these kinds of syntheses is to show your readers how you understand your research, to help them read the rest of your paper.
• Writing teachers often say synthesis is like hosting a dinner party: imagine all your sources are together in a room, discussing your topic. What are they saying to each other?
• Look at the in-text citations in each paragraph. Are you citing just one source for each paragraph? This usually indicates summary only. When you have multiple sources cited in a paragraph, you are more likely to be synthesizing them (not always, but often
• Read more about synthesis here.

The most interesting literature reviews are often written as arguments (again, as mentioned at the beginning of the page, this is discipline-specific and doesn’t work for all situations). Often, the literature review is where you can establish your research as filling a particular gap or as relevant in a particular way. You have some chance to do this in your introduction in an article, but the literature review section gives a more extended opportunity to establish the conversation in the way you would like your readers to see it. You can choose the intellectual lineage you would like to be part of and whose definitions matter most to your thinking (mostly humanities-specific, but this goes for sciences as well). In addressing these points, you argue for your place in the conversation, which tends to make the lit review more compelling than a simple reporting of other sources.

• Library Guides

The Literature Review

Primary and secondary sources, the literature review: primary and secondary sources.

• Searching the literature
• Grey literature
• Organising and analysing
• Systematic Reviews
• The Literature Review Toolbox

On this page

• Primary vs secondary sources: The differences explained 

Can something be both a primary and secondary source?

Research for your literature review can be categorised as either primary or secondary in nature. The simplest definition of primary sources is either original information (such as survey data) or a first person account of an event (such as an interview transcript). Whereas secondary sources are any publshed or unpublished works that describe, summarise, analyse, evaluate, interpret or review primary source materials. Secondary sources can incorporate primary sources to support their arguments.

Ideally, good research should use a combination of both primary and secondary sources. For example, if a researcher were to investigate the introduction of a law and the impacts it had on a community, he/she might look at the transcripts of the parliamentary debates as well as the parliamentary commentary and news reporting surrounding the laws at the time. 

Examples of primary and secondary sources

Primary vs secondary sources: The differences explained

Finding primary sources

• VU Special Collections  - The Special Collections at Victoria University Library are a valuable research resource. The Collections have strong threads of radical literature, particularly Australian Communist literature, much of which is rare or unique. Women and urban planning also feature across the Collections. There are collections that give you a picture of the people who donated them like Ray Verrills, John McLaren, Sir Zelman Cowen, and Ruth & Maurie Crow. Other collections focus on Australia's neighbours – PNG and Timor-Leste.
• POLICY - Sharing the latest in policy knowledge and evidence, this database supports enhanced learning, collaboration and contribution.
• Indigenous Australia  -  The Indigenous Australia database represents the collections of the Aboriginal and Torres Strait Islander Commission Library.
• Australian Heritage Bibliography - Aboriginal and Torres Strait Islander Subset (AHB-ATSIS)  - AHB is a bibliographic database that indexes and abstracts articles from published and unpublished material on Australia's natural and cultural environment. The AHB-ATSIS subset contains records that specifically relate to the Aboriginal and Torres Strait Islander peoples.include journal articles, unpublished reports, books, videos and conference proceedings from many different sources around Australia. Emphasis is placed on reports written or commissioned by government and non-government heritage agencies throughout the country.
• ATSIhealth  - The Aboriginal and Torres Strait Islander Health Bibliography (ATSIhealth), compiled by Neil Thomson and Natalie Weissofner at the School of Indigenous Australian Studies, Kurongkurl Katitjin, Edith Cowan University, is a bibliographic database that indexes published and unpublished material on Australian Indigenous health. Source documents include theses, unpublished articles, government reports, conference papers, abstracts, book chapters, books, discussion and working papers, and statistical documents. 
• National Archive of Australia  - The National Archives of Australia holds the memory of our nation and keeps vital Australian Government records safe. 
• National Library of Australia: Manuscripts  - Manuscripts collection that is wide ranging and provides rich evidence of the lives and activities of Australians who have shaped our society.
• National Library of Australia: Printed ephemera  - The National Library has been selectively collecting Australian printed ephemera since the early 1960s as a record of Australian life and social customs, popular culture, national events, and issues of national concern.
• National Library of Australia: Oral history and folklore - The Library’s Oral History and Folklore Collection dates back to the 1950’s and includes a rich and diverse collection of interviews and recordings with Australians from all walks of life.
• Historic Hansard - Commonwealth of Australia parliamentary debates presented in an easy-to-read format for historians and other lovers of political speech.
• The Old Bailey Online - A fully searchable edition of the largest body of texts detailing the lives of non-elite people ever published, containing 197,745 criminal trials held at London's central criminal court.

Whether or not a source can be considered both primary and  secondary, depends on the context. In some instances, material may act as a secondary source for one research area, and as a primary source for another. For example, Niccolò Machiavelli’s The Prince , published in 1513, is an important secondary source for any study of the various Renaissance princes in the Medici family; but the same book is also a primary source for the political thought that was characteristic of the sixteenth century because it reflects the attitudes of a person living in the 1500s.

Source: Craver, 1999, as cited in University of South Australia Library. (2021, Oct 6).  Can something be a primary and secondary source?.  University of South Australia Library. https://guides.library.unisa.edu.au/historycultural/sourcetypes

• << Previous: Overview
• Next: Searching the literature >>
• Last Updated: Mar 27, 2024 2:06 PM
• URL: https://libraryguides.vu.edu.au/the-literature-review

Literature review sources

Sources for literature review can be divided into three categories as illustrated in table below. In your dissertation you will need to use all three categories of literature review sources:

Sources for literature review and examples

Generally, your literature review should integrate a wide range of sources such as:

• Books . Textbooks remain as the most important source to find models and theories related to the research area. Research the most respected authorities in your selected research area and find the latest editions of books authored by them. For example, in the area of marketing the most notable authors include Philip Kotler, Seth Godin, Malcolm Gladwell, Emanuel Rosen and others.
• Magazines . Industry-specific magazines are usually rich in scholarly articles and they can be effective source to learn about the latest trends and developments in the research area. Reading industry magazines can be the most enjoyable part of the literature review, assuming that your selected research area represents an area of your personal and professional interests, which should be the case anyways.
• Newspapers can be referred to as the main source of up-to-date news about the latest events related to the research area. However, the proportion of the use of newspapers in literature review is recommended to be less compared to alternative sources of secondary data such as books and magazines. This is due to the fact that newspaper articles mainly lack depth of analyses and discussions.
• Online articles . You can find online versions of all of the above sources. However, note that the levels of reliability of online articles can be highly compromised depending on the source due to the high levels of ease with which articles can be published online. Opinions offered in a wide range of online discussion blogs cannot be usually used in literature review. Similarly, dissertation assessors are not keen to appreciate references to a wide range of blogs, unless articles in these blogs are authored by respected authorities in the research area.

Your secondary data sources may comprise certain amount of grey literature as well. The term grey literature refers to type of literature produced by government, academics, business and industry in print and electronic formats, which is not controlled by commercial publishers. It is called ‘grey’ because the status of the information in grey literature is not certain. In other words, any publication that has not been peer reviewed for publication is grey literature.

The necessity to use grey literature arises when there is no enough peer reviewed publications are available for the subject of your study.

John Dudovskiy

• Subject guides
• Researching for your literature review
• Literature sources

Researching for your literature review: Literature sources

• Literature reviews
• Before you start
• Develop a search strategy
• Keyword search activity
• Subject search activity
• Combined keyword and subject searching
• Online tutorials
• Apply search limits
• Run a search in different databases
• Supplementary searching
• Save your searches
• Manage results

Scholarly databases

It's important to make a considered decision as to where to search for your review of the literature. It's uncommon for a disciplinary area to be covered by a single publisher, so searching a single publisher platform or database is unlikely to give you sufficient coverage of studies for a review. A good quality literature review involves searching a number of databases individually.

The most common method is to search a combination of large inter-disciplinary databases such as Scopus & Web of Science Core Collection, and some subject-specific databases (such as PsycInfo or EconLit etc.). The Library databases are an excellent place to start for sources of peer-reviewed journal articles.

Depending on disciplinary expectations, or the topic of our review, you may also need to consider sources or search methods other than database searching. There is general information below on searching grey literature. However, due to the wide varieties of grey literature available, you may need to spend some time investigating sources relevant for your specific need.

Grey literature

Grey literature is information which has been published informally or non-commercially (where the main purpose of the producing body is not commercial publishing) or remains unpublished. One example may be Government publications.

Grey literature may be included in a literature review to minimise publication bias . The quality of grey literature can vary greatly - some may be peer reviewed whereas some may not have been through a traditional editorial process.

See the Grey Literature guide for further information on finding and evaluating grey sources.

See the Moodle book MNHS: Systematically searching the grey literature for a comprehensive module on grey literature for systematic reviews.

In certain disciplines (such as physics) there can be a culture of preprints being made available prior to submissions to journals. There has also been a noticeable rise in preprints in medical and health areas in the wake of Covid-19.

If preprints are relevant for you, you can search preprint servers directly. A workaround might be to utilise a search engine such as Google Scholar to search specifically for preprints, as Google Scholar has timely coverage of most preprint servers including ArXiv, RePec, SSRN, BioRxiv, and MedRxiv.

Articles in Press are not preprints, but are accepted manuscripts that are not yet formally published. Articles in Press have been made available as an early access online version of a paper that may not yet have received its final formatting or an allocation of a volume/issue number. As well as being available on a journal's website, Articles in Press are available in databases such as Scopus and Web of Science, and so (unlike preprints) don't necessarily require a separate search.

Conference papers

Conference papers are typically published in conference proceedings (the collection of papers presented at a conference), and may be found on an organisation or Society's website, as a journal, or as a special issue of journal.

In certain disciplines (such as computer science), conference papers may be highly regarded as a form of scholarly communication; the conferences are highly selective, the papers are generally peer reviewed, and papers are published in proceedings affiliated with high-quality publishing houses.

Conference papers may be indexed in a range of scholarly databases. If you only want to see conference papers, database limits can be used to filter results, or try a specific index such as the examples below:

• Conference proceedings citation index. Social science & humanities (CPCI-SSH)
• Conference proceedings citation index. Science (CPCI-S)
• ASME digital library conference proceedings

Honours students and postgraduates may request conference papers through Interlibrary Loans . However, conference paper requests may take longer than traditional article requests as they can be difficult to locate; they may have been only supplied to attendees or not formally published. Sometimes only the abstract is available.

If you are specifically looking for statistical data, try searching for the keyword statistics in a Google Advanced Search and limiting by a relevant site or domain. Below are some examples of sites, or you can try a domain such as .gov for government websites.

Statistical data can be found in the following selected sources:

• Australian Bureau of Statistics
• World Health Organization: Health Data and statistics
• Higher Education Statistics
• UNESCO Institute for Statistics
• Tourism Australia Statistics

For a list of databases that include statistics see: Databases by Subject: Statistics .

If you are specifically looking for information found in newspapers, the library has a large collection of Australian and overseas newspapers, both current and historical.

To search the full-text of newspapers in electronic format use a database such as  Newsbank.

Alternatively, see the Newspapers subject guide for comprehensive information on newspaper sources available via Monash University library and open source databases, as well as searching tips, online videos and more.

Dissertations and theses

The Monash University Library Theses subject guide provides resources and guidelines for locating and accessing theses (dissertations) produced by Monash University as well as other universities in Australia and internationally.  

International theses:

There are a number of theses databases and repositories.

A popular source is:

• ProQuest Dissertations & Theses Global  which predominantly, covers North American masters and doctoral theses. Full text is available for theses added since 1997. 

Australia and New Zealand theses:

Theses that are available in the library can be found using the  Search catalogue.

These include:

• Monash doctoral, masters and a small number of honours theses 
• other Australian and overseas theses that have been purchased for the collection.

Formats include print (not available for loan), microfiche and online (some may have access restrictions).

Trove includes doctoral, masters and some honours theses from all Australian and New Zealand universities, as well as theses awarded elsewhere but held by Australian institutions.

Tips:  

• Type in the title, author surname and/or keywords. Then on the results page refine your search to 'thesis'.
• Alternatively, use the Advanced search and include 'thesis' as a keyword or limi t your result to format = thesis
• << Previous: Literature reviews
• Next: Before you start >>

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jun 20, 2024 9:08 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Working with sources
• Primary vs. Secondary Sources | Difference & Examples

Primary vs. Secondary Sources | Difference & Examples

Published on June 20, 2018 by Raimo Streefkerk . Revised on May 31, 2023.

When you do research, you have to gather information and evidence from a variety of sources.

Primary sources provide raw information and first-hand evidence. Examples include interview transcripts, statistical data, and works of art. Primary research gives you direct access to the subject of your research.

Secondary sources provide second-hand information and commentary from other researchers. Examples include journal articles, reviews, and academic books . Thus, secondary research describes, interprets, or synthesizes primary sources.

Primary sources are more credible as evidence, but good research uses both primary and secondary sources.

Table of contents

What is a primary source, what is a secondary source, primary and secondary source examples, how to tell if a source is primary or secondary, primary vs secondary sources: which is better, other interesting articles, frequently asked questions about primary and secondary sources.

A primary source is anything that gives you direct evidence about the people, events, or phenomena that you are researching. Primary sources will usually be the main objects of your analysis.

If you are researching the past, you cannot directly access it yourself, so you need primary sources that were produced at the time by participants or witnesses (e.g. letters, photographs, newspapers ).

If you are researching something current, your primary sources can either be qualitative or quantitative data that you collect yourself (e.g. through interviews , surveys , experiments ) or sources produced by people directly involved in the topic (e.g. official documents or media texts).

Scribbr Citation Checker New

The AI-powered Citation Checker helps you avoid common mistakes such as:

• Missing commas and periods
• Incorrect usage of “et al.”
• Ampersands (&) in narrative citations
• Missing reference entries

A secondary source is anything that describes, interprets, evaluates, or analyzes information from primary sources. Common examples include:

• Books , articles and documentaries that synthesize information on a topic
• Synopses and descriptions of artistic works
• Encyclopedias and textbooks that summarize information and ideas
• Reviews and essays that evaluate or interpret something

When you cite a secondary source, it’s usually not to analyze it directly. Instead, you’ll probably test its arguments against new evidence or use its ideas to help formulate your own.

Examples of sources that can be primary or secondary

A secondary source can become a primary source depending on your research question . If the person, context, or technique that produced the source is the main focus of your research, it becomes a primary source.

Documentaries

If you are researching the causes of World War II, a recent documentary about the war is a secondary source . But if you are researching the filmmaking techniques used in historical documentaries, the documentary is a primary source .

Reviews and essays

If your paper is about the novels of Toni Morrison, a magazine review of one of her novels is a secondary source . But if your paper is about the critical reception of Toni Morrison’s work, the review is a primary source .

Newspaper articles

If your aim is to analyze the government’s economic policy, a newspaper article about a new policy is a secondary source . But if your aim is to analyze media coverage of economic issues, the newspaper article is a primary source .

To determine if something can be used as a primary or secondary source in your research, there are some simple questions you can ask yourself:

• Does this source come from someone directly involved in the events I’m studying (primary) or from another researcher (secondary)?
• Am I interested in evaluating the source itself (primary) or only using it for background information (secondary)?
• Does the source provide original information (primary) or does it comment upon information from other sources (secondary)?

Prevent plagiarism. Run a free check.

Most research uses both primary and secondary sources. They complement each other to help you build a convincing argument. Primary sources are more credible as evidence, but secondary sources show how your work relates to existing research. Tertiary sources are often used in the first, exploratory stage of research.

What do you use primary sources for?

Primary sources are the foundation of original research. They allow you to:

• Make new discoveries
• Provide credible evidence for your arguments
• Give authoritative information about your topic

If you don’t use any primary sources, your research may be considered unoriginal or unreliable.

What do you use secondary sources for?

Secondary sources are good for gaining a full overview of your topic and understanding how other researchers have approached it. They often synthesize a large number of primary sources that would be difficult and time-consuming to gather by yourself. They allow you to:

• Gain background information on the topic
• Support or contrast your arguments with other researchers’ ideas
• Gather information from primary sources that you can’t access directly (e.g. private letters or physical documents located elsewhere)

When you conduct a literature review or meta analysis, you can consult secondary sources to gain a thorough overview of your topic. If you want to mention a paper or study that you find cited in a secondary source, seek out the original source and cite it directly.

Remember that all primary and secondary sources must be cited to avoid plagiarism . You can use Scribbr’s free citation generator to do so!

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

• ChatGPT vs human editor
• ChatGPT citations
• Is ChatGPT trustworthy?
• Using ChatGPT for your studies
• What is ChatGPT?
• Chicago style
• Paraphrasing

 Plagiarism

• Types of plagiarism
• Self-plagiarism
• Avoiding plagiarism
• Academic integrity
• Consequences of plagiarism
• Common knowledge

Common examples of primary sources include interview transcripts , photographs, novels, paintings, films, historical documents, and official statistics.

Anything you directly analyze or use as first-hand evidence can be a primary source, including qualitative or quantitative data that you collected yourself.

Common examples of secondary sources include academic books, journal articles , reviews, essays , and textbooks.

Anything that summarizes, evaluates or interprets primary sources can be a secondary source. If a source gives you an overview of background information or presents another researcher’s ideas on your topic, it is probably a secondary source.

To determine if a source is primary or secondary, ask yourself:

• Was the source created by someone directly involved in the events you’re studying (primary), or by another researcher (secondary)?
• Does the source provide original information (primary), or does it summarize information from other sources (secondary)?
• Are you directly analyzing the source itself (primary), or only using it for background information (secondary)?

Some types of source are nearly always primary: works of art and literature, raw statistical data, official documents and records, and personal communications (e.g. letters, interviews ). If you use one of these in your research, it is probably a primary source.

Primary sources are often considered the most credible in terms of providing evidence for your argument, as they give you direct evidence of what you are researching. However, it’s up to you to ensure the information they provide is reliable and accurate.

Always make sure to properly cite your sources to avoid plagiarism .

A fictional movie is usually a primary source. A documentary can be either primary or secondary depending on the context.

If you are directly analyzing some aspect of the movie itself – for example, the cinematography, narrative techniques, or social context – the movie is a primary source.

If you use the movie for background information or analysis about your topic – for example, to learn about a historical event or a scientific discovery – the movie is a secondary source.

Whether it’s primary or secondary, always properly cite the movie in the citation style you are using. Learn how to create an MLA movie citation or an APA movie citation .

Articles in newspapers and magazines can be primary or secondary depending on the focus of your research.

In historical studies, old articles are used as primary sources that give direct evidence about the time period. In social and communication studies, articles are used as primary sources to analyze language and social relations (for example, by conducting content analysis or discourse analysis ).

If you are not analyzing the article itself, but only using it for background information or facts about your topic, then the article is a secondary source.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Streefkerk, R. (2023, May 31). Primary vs. Secondary Sources | Difference & Examples. Scribbr. Retrieved August 21, 2024, from https://www.scribbr.com/working-with-sources/primary-and-secondary-sources/

Is this article helpful?

Raimo Streefkerk

Other students also liked, how to avoid plagiarism | tips on citing sources, the basics of in-text citation | apa & mla examples, how to quote | citing quotes in apa, mla & chicago, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Aug 21, 2024 8:54 AM
• URL: https://libguides.usc.edu/writingguide

• University of La Verne
• Subject Guides

Literature Review Basics

• Primary & Secondary Sources
• Literature Review Introduction
• Writing Literature Reviews
• Tutorials & Samples

The Literature

The Literature refers to the collection of scholarly writings on a topic. This includes peer-reviewed articles, books, dissertations and conference papers.

• When reviewing the literature, be sure to include major works as well as studies that respond to major works. You will want to focus on primary sources, though secondary sources can be valuable as well.

Primary Sources

The term primary source is used broadly to embody all sources that are original. P rimary sources provide first-hand information that is closest to the object of study. Primary sources vary by discipline.

• In the natural and social sciences, original reports of research found in academic journals detailing the methodology used in the research, in-depth descriptions, and discussions of the findings are considered primary sources of information.
• Other common examples of primary sources include speeches, letters, diaries, autobiographies, interviews, official reports, court records, artifacts, photographs, and drawings.  

Galvan, J. L. (2013). Writing literature reviews: A guide for students of the social and behavioral sciences . Glendale, CA: Pyrczak.

Secondary Sources

A secondary source is a source that provides non-original or secondhand data or information. 

• Secondary sources are written about primary sources.
• Research summaries reported in textbooks, magazines, and newspapers are considered secondary sources. They typically provide global descriptions of results with few details on the methodology. Other examples of secondary sources include biographies and critical studies of an author's work.

Secondary Source. (2005). In W. Paul Vogt (Ed.), Dictionary of Statistics & Methodology. (3 rd ed., p. 291). Thousand Oaks, CA: SAGE Publications, Inc.

Weidenborner, S., & Caruso, D. (1997). Writing research papers: A guide to the process . New York: St. Martin's Press.

More Examples of Primary and Secondary Sources

• << Previous: Writing Literature Reviews
• Next: Tutorials & Samples >>
• Last Updated: Jul 30, 2024 9:59 AM
• URL: https://laverne.libguides.com/litreviews

Library Guides

Comprehensive Literature Review: A Guide

What is a literature review, components of a literature review, literature review structures, sample literature reviews, additional resources.

• Advanced Searching for the Lit Review
• Managing What You Find
• Student Feedback

Virtual Research Help

A literature review is a collection of selected articles, books and other sources about a specific subject. The purpose is to summarize the existing research that has been done on the subject in order to put your research in context and to highlight what your research will add to the existing body of knowledge. Literature reviews are typically organized chronologically, thematically, or based on their methods.

Despite their various structures (see the descriptions below),  literature reviews consist of the following elements :

• Citations for the referenced materials
• A discussion of the materials' research purpose, methods, and findings
• A discussion of how those findings relate to your research
• A discussion of the similarities and differences between cited materials 
• A discussion of the gaps created by the material referenced and how your research can close those gaps
• Chronological
• Methodological

Chronological order creates paragraph/sections that review the material in sequential order . This structure is useful when tracing the history of a research area. Remember, your materials should be discussed in chronological order regardless of your overarching review structure.

Literature Reviews that are organized methodologically consist of paragraphs/sections that are based on the methods used in the literature found . This approach is most appropriate when you are using new methods on a research question that has already been explored . Since literature review structures are not mutually exclusive, you can organize the use of these methods in chronological order .

Thematic literature review structure organizes paragraphs/sections of the review based on the themes in the literature . This approach may be useful when you are studying a new research problem but would like to contextualize your research with similar literature .

• Sample Literature Review - Political Science A brief literature review within a political scientists’ National Science Foundation Graduate Research Fellowship grant. Annotated by the University of Wisconsin Madison Writing Center.
• Sample Literature Review - Philosophy A several-page literature review at the beginning of a published, academic article about philosophy. Annotated by the University of Wisconsin Madison Writing Center.
• Sample Literature Review - Chemistry A brief literature review at the beginning of a published, academic article about photochemistry. Annotated by the University of Wisconsin Madison Writing Center.
• Sample Literature Review - Librarianship Extract of literature review from article on collaboration between university libraries and writing centers. Click here for article in full.

These resources provide an overview of the literature review: the purpose of a lit review, what to include in one and how to organize one.

• University of North Carolina-Chapel Hill Writing Center A thorough description of literature reviews, their purpose, and how to go about writing one.
• Learn how to write a review of literature (UW-Madison) Outlines the key components of a literature review.
• University of California-Santa Cruz Library Outlines how to write a literature review. Includes citations for examples of published literature reviews.
• Literature Reviews: An Overview for Graduate Students What is a literature review? What purpose does it serve in research? What should you expect when writing one?
• Purdue Owl (Purdue University Online Writing Lab): Writing a Literature Review Breaks down parts and types of literature reviews along with strategies and tips.

• Next: Advanced Searching for the Lit Review >>
• Last Updated: May 29, 2024 4:03 PM
• URL: https://libguides.tulane.edu/litreview

The Sheridan Libraries

• Write a Literature Review
• Sheridan Libraries
• Evaluate This link opens in a new window

What Will You Do Differently?

Please help your librarians by filling out this two-minute survey of today's class session..

Professor, this one's for you .

Introduction

Literature reviews take time. here is some general information to know before you start.  .

•  VIDEO -- This video is a great overview of the entire process.  (2020; North Carolina State University Libraries) --The transcript is included --This is for everyone; ignore the mention of "graduate students" --9.5 minutes, and every second is important  
• OVERVIEW -- Read this page from Purdue's OWL. It's not long, and gives some tips to fill in what you just learned from the video.  
• NOT A RESEARCH ARTICLE -- A literature review follows a different style, format, and structure from a research article.  

• Next: Evaluate >>
• Last Updated: Jul 30, 2024 1:42 PM
• URL: https://guides.library.jhu.edu/lit-review

Literature Review: Lit Review Sources

• Lit Review Types
• GRADE System
• Do a Lit Review
• Citation Justice
• Lit Review Sources
• AI for Research This link opens in a new window

Where do I find information for a literature review?

Research is done by...

...by way of...

...communicated through...

...and organized in...

Types of sources for a review...

• Primary source: Usually a report by the original researchers of a study (unfiltered sources)
• Secondary source: Description or summary by somebody other than the original researcher, e.g. a review article (filtered sources)
• Conceptual/theoretical: Papers concerned with description or analysis of theories or concepts associated with the topic
• Anecdotal/opinion/clinical: Views or opinions about the subject that are not research, review or theoretical (case studies or reports from clinical settings)

A Heirarchy of research information:

Source: SUNY Downstate Medical Center. Medical Research Library of Brooklyn. Evidence Based Medicine Course. A Guide to Research Methods: The Evidence Pyramid: http://library.downstate.edu/EBM2/2100.htm

Life Cycle of Publication

Click image to enlarge

Scientific information has a ‘life cycle’ of its own… it is born as an idea, and then matures and becomes more available to the public. First it appears within the so-called ‘invisible college’ of experts in the field, discussed at conferences and symposia or posted as pre-prints for comments and corrections. Then it appears in the published literature (the primary literature), often as a journal article in a peer-reviewed journal.

Researchers can use the indexing and alerting services of the secondary literature to find out what has been published in a field. Depending on how much information is added by the indexer or abstracter, this may take a few months (though electronic publication has sped up this process). Finally, the information may appear in more popular or reference sources, sometimes called the tertiary literature.

The person beginning a literature search may take this process in reverse: using tertiary sources for general background, then going to the secondary literature to survey what has been published, following up by finding the original (primary) sources, and generating their own research Idea.

(Original content by Wade Lee-Smith)

• << Previous: Citation Justice
• Next: Readings >>
• Last Updated: Jul 1, 2024 11:03 AM
• URL: https://libguides.utoledo.edu/litreview

The Literature Review

• Publications: A World of Information

Description

Primary sources, secondary sources, tertiary sources.

• Types of Reviews and Their Differences
• Beginning Steps and Finishing a Review
• Information Sources: Where to Find Them
• Webinar Recording (20 Minutes, Slides and Quiz)
• Webinar Recording (50 Minutes, Slides and Quiz)

Scholarly, professional literature falls under 3 categories, primary, secondary, and tertiary.  Published works (also known as a publication) may fall into one or more of these categories, depending on the discipline.  See definitions and linked examples of primary, secondary, and tertiary sources.

Differences in Publishing Norms by Broader Discipline  

Scholarly and professional communication norms can be different among various disciplines.  For instance, scholars in political science or law will generally publish their knowledge and research differently than those in chemistry or physics.  To show these distinctions, links to examples are provided for primary, secondary and tertiary sources.  The broader disciplines of business, education, and social science tend to use publications from both the sciences and humanities.

If you are in STEM or Nursing, see these examples of primary, secondary and tertiary literature within your disciplines:

• Nursing - Primary, Secondary and Tertiary Sources (Using Library Resources for Information and Research)
• STEM: Primary, Secondary and Tertiary Sources (Guide to Science Information Sources)

Image:  Typewriter, by DanielMcCullough , Permission by Unsplash.com license .

A primary source is a document or work where its author had a direct interaction or was involved with what was studied or created.  These sources are recommended when you need to get information or findings that are a direct result or finding from a study, research, or creation.   A primary source can also be an actual creative work or original material.

Humanities:

• Magazine or newspaper articles
• Creative works ( literature , poetry , fiction books, film, works of art or design, performances)
• Autobiographies and memoirs
• Interviews and oral histories
• Laws, statutes and official documents

Science / Health Sciences:

• Reports of  original research :  journal articles, poster presentations, conference papers
• Theses & dissertations
• Technical reports

Image:   Library stacks.  Permission by Pixabay.com  license .

A secondary source is a document or work where its author had an indirect part in a study or creation; an author is usually writing about or reporting the work or research done by someone else.  Secondary sources can be used for additional or supporting information; they are not the direct product of research or the making of a creative work.

• Books (monographs) written about a topic
• Articles:  Criticism or interpretation of creative works
• Biographies
• Reviews of creative works
• Bibliographies
• Textbooks and books (monographs)  
• Reviews: Literature or systematic
• Articles in trade journals (not based on research)
• Opinion pieces or commentaries

Image:  Encyclopedias, by StockSnap, Permission by Pixabay.com  license .

A tertiary source provides agreed-upon facts like measurements, dates, and definitions.  They are usually known as reference works and include the following:

• Encyclopedias
• Standards :  a document with specifications that create rules, guidelines or characteristics to ensure that materials, processes, or services are fit for their purpose.  They are established by a professional organization to provide a baseline of acceptable quality.
• Handbooks and manuals :  a resource that summarizes major topics or processes within a field.  These often provide established measurements, definitions or research methods.
• Pathfinders ( Research Guides ): a list of recommended information sources on a topic or discipline.
• << Previous: Publications: A World of Information
• Next: Types of Reviews and Their Differences >>
• Last Updated: Jul 30, 2024 3:09 PM
• URL: https://libguides.fau.edu/literature-review

Florida Atlantic University Libraries 777 Glades Road Boca Raton, FL 33431 (561) 297-6911

Literature Review - what is a Literature Review, why it is important and how it is done

• Strategies to Find Sources

Evaluating Literature Reviews and Sources

Reading critically, tips to evaluate sources.

• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

• A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips. First part focus on Introductions and in page 10 in the PDF, 37 in the text, it focus on "literature reviews".
• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

To be able to write a good Literature Review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

• Who is the author? What is his/her standing in the field.
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved overtime?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

• How to Read a Paper (University of Waterloo, Canada) This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.

Criteria to evaluate sources:

• Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful site - Critically Analyzing Information Sources (Cornell University Library)

• << Previous: Strategies to Find Sources
• Next: Tips for Writing Literature Reviews >>
• Last Updated: Jul 3, 2024 10:56 AM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

• Libraries & Collections
• News & Events
• Process: Literature Reviews
• Literature Review
• Managing Sources

Ask a Librarian

Does your assignment or publication require that you write a literature review? This guide is intended to help you understand what a literature is, why it is worth doing, and some quick tips composing one.

Understanding Literature Reviews

What is a literature review  .

Typically, a literature review is a written discussion that examines publications about  a particular subject area or topic. Depending on disciplines, publications, or authors a literature review may be: 

A summary of sources An organized presentation of sources A synthesis or interpretation of sources An evaluative analysis of sources

A Literature Review may be part of a process or a product. It may be:

A part of your research process A part of your final research publication An independent publication

Why do a literature review?

The Literature Review will place your research in context. It will help you and your readers:  

Locate patterns, relationships, connections, agreements, disagreements, & gaps in understanding Identify methodological and theoretical foundations Identify landmark and exemplary works Situate your voice in a broader conversation with other writers, thinkers, and scholars

The Literature Review will aid your research process. It will help you to:

Establish your knowledge Understand what has been said Define your questions Establish a relevant methodology Refine your voice Situate your voice in the conversation

What does a literature review look like?

The Literature Review structure and organization may include sections such as:  

An introduction or overview A body or organizational sub-divisions A conclusion or an explanation of significance

The body of a literature review may be organized in several ways, including:

Chronologically: organized by date of publication Methodologically: organized by type of research method used Thematically: organized by concept, trend, or theme Ideologically: organized by belief, ideology, or school of thought

• Find a focus
• Find models
• Review your target publication
• Track citations
• Read critically
• Manage your citations
• Ask friends, faculty, and librarians

Additional Sources

• Reviewing the literature. Project Planner.
• Literature Review: By UNC Writing Center
• PhD on Track
• CU Graduate Students Thesis & Dissertation Guidance
• CU Honors Thesis Guidance

• Next: Managing Sources >>
• University of Colorado Boulder Libraries
• Research Guides
• Research Strategies
• Last Updated: Jun 20, 2024 3:23 PM
• URL: https://libguides.colorado.edu/strategies/litreview
• © Regents of the University of Colorado

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PLoS Comput Biol
• v.9(7); 2013 Jul

Ten Simple Rules for Writing a Literature Review

Marco pautasso.

1 Centre for Functional and Evolutionary Ecology (CEFE), CNRS, Montpellier, France

2 Centre for Biodiversity Synthesis and Analysis (CESAB), FRB, Aix-en-Provence, France

Literature reviews are in great demand in most scientific fields. Their need stems from the ever-increasing output of scientific publications [1] . For example, compared to 1991, in 2008 three, eight, and forty times more papers were indexed in Web of Science on malaria, obesity, and biodiversity, respectively [2] . Given such mountains of papers, scientists cannot be expected to examine in detail every single new paper relevant to their interests [3] . Thus, it is both advantageous and necessary to rely on regular summaries of the recent literature. Although recognition for scientists mainly comes from primary research, timely literature reviews can lead to new synthetic insights and are often widely read [4] . For such summaries to be useful, however, they need to be compiled in a professional way [5] .

When starting from scratch, reviewing the literature can require a titanic amount of work. That is why researchers who have spent their career working on a certain research issue are in a perfect position to review that literature. Some graduate schools are now offering courses in reviewing the literature, given that most research students start their project by producing an overview of what has already been done on their research issue [6] . However, it is likely that most scientists have not thought in detail about how to approach and carry out a literature review.

Reviewing the literature requires the ability to juggle multiple tasks, from finding and evaluating relevant material to synthesising information from various sources, from critical thinking to paraphrasing, evaluating, and citation skills [7] . In this contribution, I share ten simple rules I learned working on about 25 literature reviews as a PhD and postdoctoral student. Ideas and insights also come from discussions with coauthors and colleagues, as well as feedback from reviewers and editors.

Rule 1: Define a Topic and Audience

How to choose which topic to review? There are so many issues in contemporary science that you could spend a lifetime of attending conferences and reading the literature just pondering what to review. On the one hand, if you take several years to choose, several other people may have had the same idea in the meantime. On the other hand, only a well-considered topic is likely to lead to a brilliant literature review [8] . The topic must at least be:

• interesting to you (ideally, you should have come across a series of recent papers related to your line of work that call for a critical summary),
• an important aspect of the field (so that many readers will be interested in the review and there will be enough material to write it), and
• a well-defined issue (otherwise you could potentially include thousands of publications, which would make the review unhelpful).

Ideas for potential reviews may come from papers providing lists of key research questions to be answered [9] , but also from serendipitous moments during desultory reading and discussions. In addition to choosing your topic, you should also select a target audience. In many cases, the topic (e.g., web services in computational biology) will automatically define an audience (e.g., computational biologists), but that same topic may also be of interest to neighbouring fields (e.g., computer science, biology, etc.).

Rule 2: Search and Re-search the Literature

After having chosen your topic and audience, start by checking the literature and downloading relevant papers. Five pieces of advice here:

• keep track of the search items you use (so that your search can be replicated [10] ),
• keep a list of papers whose pdfs you cannot access immediately (so as to retrieve them later with alternative strategies),
• use a paper management system (e.g., Mendeley, Papers, Qiqqa, Sente),
• define early in the process some criteria for exclusion of irrelevant papers (these criteria can then be described in the review to help define its scope), and
• do not just look for research papers in the area you wish to review, but also seek previous reviews.

The chances are high that someone will already have published a literature review ( Figure 1 ), if not exactly on the issue you are planning to tackle, at least on a related topic. If there are already a few or several reviews of the literature on your issue, my advice is not to give up, but to carry on with your own literature review,

The bottom-right situation (many literature reviews but few research papers) is not just a theoretical situation; it applies, for example, to the study of the impacts of climate change on plant diseases, where there appear to be more literature reviews than research studies [33] .

• discussing in your review the approaches, limitations, and conclusions of past reviews,
• trying to find a new angle that has not been covered adequately in the previous reviews, and
• incorporating new material that has inevitably accumulated since their appearance.

When searching the literature for pertinent papers and reviews, the usual rules apply:

• be thorough,
• use different keywords and database sources (e.g., DBLP, Google Scholar, ISI Proceedings, JSTOR Search, Medline, Scopus, Web of Science), and
• look at who has cited past relevant papers and book chapters.

Rule 3: Take Notes While Reading

If you read the papers first, and only afterwards start writing the review, you will need a very good memory to remember who wrote what, and what your impressions and associations were while reading each single paper. My advice is, while reading, to start writing down interesting pieces of information, insights about how to organize the review, and thoughts on what to write. This way, by the time you have read the literature you selected, you will already have a rough draft of the review.

Of course, this draft will still need much rewriting, restructuring, and rethinking to obtain a text with a coherent argument [11] , but you will have avoided the danger posed by staring at a blank document. Be careful when taking notes to use quotation marks if you are provisionally copying verbatim from the literature. It is advisable then to reformulate such quotes with your own words in the final draft. It is important to be careful in noting the references already at this stage, so as to avoid misattributions. Using referencing software from the very beginning of your endeavour will save you time.

Rule 4: Choose the Type of Review You Wish to Write

After having taken notes while reading the literature, you will have a rough idea of the amount of material available for the review. This is probably a good time to decide whether to go for a mini- or a full review. Some journals are now favouring the publication of rather short reviews focusing on the last few years, with a limit on the number of words and citations. A mini-review is not necessarily a minor review: it may well attract more attention from busy readers, although it will inevitably simplify some issues and leave out some relevant material due to space limitations. A full review will have the advantage of more freedom to cover in detail the complexities of a particular scientific development, but may then be left in the pile of the very important papers “to be read” by readers with little time to spare for major monographs.

There is probably a continuum between mini- and full reviews. The same point applies to the dichotomy of descriptive vs. integrative reviews. While descriptive reviews focus on the methodology, findings, and interpretation of each reviewed study, integrative reviews attempt to find common ideas and concepts from the reviewed material [12] . A similar distinction exists between narrative and systematic reviews: while narrative reviews are qualitative, systematic reviews attempt to test a hypothesis based on the published evidence, which is gathered using a predefined protocol to reduce bias [13] , [14] . When systematic reviews analyse quantitative results in a quantitative way, they become meta-analyses. The choice between different review types will have to be made on a case-by-case basis, depending not just on the nature of the material found and the preferences of the target journal(s), but also on the time available to write the review and the number of coauthors [15] .

Rule 5: Keep the Review Focused, but Make It of Broad Interest

Whether your plan is to write a mini- or a full review, it is good advice to keep it focused 16 , 17 . Including material just for the sake of it can easily lead to reviews that are trying to do too many things at once. The need to keep a review focused can be problematic for interdisciplinary reviews, where the aim is to bridge the gap between fields [18] . If you are writing a review on, for example, how epidemiological approaches are used in modelling the spread of ideas, you may be inclined to include material from both parent fields, epidemiology and the study of cultural diffusion. This may be necessary to some extent, but in this case a focused review would only deal in detail with those studies at the interface between epidemiology and the spread of ideas.

While focus is an important feature of a successful review, this requirement has to be balanced with the need to make the review relevant to a broad audience. This square may be circled by discussing the wider implications of the reviewed topic for other disciplines.

Rule 6: Be Critical and Consistent

Reviewing the literature is not stamp collecting. A good review does not just summarize the literature, but discusses it critically, identifies methodological problems, and points out research gaps [19] . After having read a review of the literature, a reader should have a rough idea of:

• the major achievements in the reviewed field,
• the main areas of debate, and
• the outstanding research questions.

It is challenging to achieve a successful review on all these fronts. A solution can be to involve a set of complementary coauthors: some people are excellent at mapping what has been achieved, some others are very good at identifying dark clouds on the horizon, and some have instead a knack at predicting where solutions are going to come from. If your journal club has exactly this sort of team, then you should definitely write a review of the literature! In addition to critical thinking, a literature review needs consistency, for example in the choice of passive vs. active voice and present vs. past tense.

Rule 7: Find a Logical Structure

Like a well-baked cake, a good review has a number of telling features: it is worth the reader's time, timely, systematic, well written, focused, and critical. It also needs a good structure. With reviews, the usual subdivision of research papers into introduction, methods, results, and discussion does not work or is rarely used. However, a general introduction of the context and, toward the end, a recapitulation of the main points covered and take-home messages make sense also in the case of reviews. For systematic reviews, there is a trend towards including information about how the literature was searched (database, keywords, time limits) [20] .

How can you organize the flow of the main body of the review so that the reader will be drawn into and guided through it? It is generally helpful to draw a conceptual scheme of the review, e.g., with mind-mapping techniques. Such diagrams can help recognize a logical way to order and link the various sections of a review [21] . This is the case not just at the writing stage, but also for readers if the diagram is included in the review as a figure. A careful selection of diagrams and figures relevant to the reviewed topic can be very helpful to structure the text too [22] .

Rule 8: Make Use of Feedback

Reviews of the literature are normally peer-reviewed in the same way as research papers, and rightly so [23] . As a rule, incorporating feedback from reviewers greatly helps improve a review draft. Having read the review with a fresh mind, reviewers may spot inaccuracies, inconsistencies, and ambiguities that had not been noticed by the writers due to rereading the typescript too many times. It is however advisable to reread the draft one more time before submission, as a last-minute correction of typos, leaps, and muddled sentences may enable the reviewers to focus on providing advice on the content rather than the form.

Feedback is vital to writing a good review, and should be sought from a variety of colleagues, so as to obtain a diversity of views on the draft. This may lead in some cases to conflicting views on the merits of the paper, and on how to improve it, but such a situation is better than the absence of feedback. A diversity of feedback perspectives on a literature review can help identify where the consensus view stands in the landscape of the current scientific understanding of an issue [24] .

Rule 9: Include Your Own Relevant Research, but Be Objective

In many cases, reviewers of the literature will have published studies relevant to the review they are writing. This could create a conflict of interest: how can reviewers report objectively on their own work [25] ? Some scientists may be overly enthusiastic about what they have published, and thus risk giving too much importance to their own findings in the review. However, bias could also occur in the other direction: some scientists may be unduly dismissive of their own achievements, so that they will tend to downplay their contribution (if any) to a field when reviewing it.

In general, a review of the literature should neither be a public relations brochure nor an exercise in competitive self-denial. If a reviewer is up to the job of producing a well-organized and methodical review, which flows well and provides a service to the readership, then it should be possible to be objective in reviewing one's own relevant findings. In reviews written by multiple authors, this may be achieved by assigning the review of the results of a coauthor to different coauthors.

Rule 10: Be Up-to-Date, but Do Not Forget Older Studies

Given the progressive acceleration in the publication of scientific papers, today's reviews of the literature need awareness not just of the overall direction and achievements of a field of inquiry, but also of the latest studies, so as not to become out-of-date before they have been published. Ideally, a literature review should not identify as a major research gap an issue that has just been addressed in a series of papers in press (the same applies, of course, to older, overlooked studies (“sleeping beauties” [26] )). This implies that literature reviewers would do well to keep an eye on electronic lists of papers in press, given that it can take months before these appear in scientific databases. Some reviews declare that they have scanned the literature up to a certain point in time, but given that peer review can be a rather lengthy process, a full search for newly appeared literature at the revision stage may be worthwhile. Assessing the contribution of papers that have just appeared is particularly challenging, because there is little perspective with which to gauge their significance and impact on further research and society.

Inevitably, new papers on the reviewed topic (including independently written literature reviews) will appear from all quarters after the review has been published, so that there may soon be the need for an updated review. But this is the nature of science [27] – [32] . I wish everybody good luck with writing a review of the literature.

Acknowledgments

Many thanks to M. Barbosa, K. Dehnen-Schmutz, T. Döring, D. Fontaneto, M. Garbelotto, O. Holdenrieder, M. Jeger, D. Lonsdale, A. MacLeod, P. Mills, M. Moslonka-Lefebvre, G. Stancanelli, P. Weisberg, and X. Xu for insights and discussions, and to P. Bourne, T. Matoni, and D. Smith for helpful comments on a previous draft.

Funding Statement

This work was funded by the French Foundation for Research on Biodiversity (FRB) through its Centre for Synthesis and Analysis of Biodiversity data (CESAB), as part of the NETSEED research project. The funders had no role in the preparation of the manuscript.

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Evaluating Sources & Lit. Reviews

Literature Review: The What, Why and How-to Guide — Evaluating Sources & Lit. Reviews

• Getting Started
• Introduction
• How to Pick a Topic
• Strategies to Find Sources
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

Reading Critically

To be able to write a good literature review, you need to be able to read critically. Below are some tips that will help you evaluate the sources for your paper.

Reading critically (summary from How to Read Academic Texts Critically)

• Who is the author? What is his/her standing in the field?
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, to critique, or clarify?
• Note the experts in the field. Are there specific names/labs that are frequently cited?
• Pay attention to methodology. Is it sound? What testing procedures, subjects, and materials were used?
• Note conflicting theories, methodologies and results. Are there any assumptions being made by most/some researchers?
• Theories: Have they evolved overtime?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?

Useful links:

Tips to Evaluate Sources

Criteria to evaluate sources:

Authority : Who is the author? what is his/her credentials--what university he/she is affliliated? Is his/her area of expertise?

Usefulness : How this source related to your topic? How current or relevant it is to your topic?

Reliability : Does the information comes from a reliable, trusted source such as an academic journal?

Useful sites

• Critically Analyzing Information Sources (Cornell University Library)

Evaluating Literature Reviews and Sources

A good literature review evaluates a wide variety of sources (academic articles, scholarly books, government/NGO reports). It also evaluates literature reviews that study similar topics. This page offers you a list of resources and tips on how to evaluate the sources that you may use to write your review.

• A Closer Look at Evaluating Literature Reviews Excerpt from the book chapter, “Evaluating Introductions and Literature Reviews” in Fred Pyrczak’s Evaluating Research in Academic Journals: A Practical Guide to Realistic Evaluation , (Chapter 4 and 5). This PDF discusses and offers great advice on how to evaluate "Introductions" and "Literature Reviews" by listing questions and tips.

• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.
• << Previous: Strategies to Find Sources
• Next: Tips for Writing Literature Reviews >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Jul 15, 2024 10:34 AM
• URL: https://guides.lib.udel.edu/researchmethods

• Library Homepage

Literature Review: The What, Why and How-to Guide: Evaluating Sources & Literature Reviews

• Literature Reviews?
• Strategies to Finding Sources
• Keeping up with Research!
• Evaluating Sources & Literature Reviews
• Organizing for Writing
• Writing Literature Review
• Other Academic Writings

Evaluating Literature Reviews and Sources

• Tips for Evaluating Sources (Print vs. Internet Sources) Excellent page that will guide you on what to ask to determine if your source is a reliable one. Check the other topics in the guide: Evaluating Bibliographic Citations and Evaluation During Reading on the left side menu.

Criteria to evaluate sources:

• Authority : Who is the author? What are the author's credentials and areas of expertise? Is he or she affiliated with a university?
• Usefulness : How this source related to your topic? How current or relevant it is to your topic?
• Reliability : Does the information comes from a reliable, trusted source such as an academic journal?
• Critically Analyzing Information Sources: Critical Appraisal and Analysis (Cornell University Library) Ten things to look for when you evaluate an information source.

Reading Critically

Reading critically (summary from how to read academic texts critically).

• Who is the author? What is his/her standing in the field?
• What is the author’s purpose? To offer advice, make practical suggestions, solve a specific problem, critique or clarify?
• Note the experts in the field: are there specific names/labs that are frequently cited?
• Pay attention to methodology: is it sound? what testing procedures, subjects, materials were used?
• Note conflicting theories, methodologies, and results. Are there any assumptions being made by most/some researchers?
• Theories: have they evolved over time?
• Evaluate and synthesize the findings and conclusions. How does this study contribute to your project?
• How to Read Academic Texts Critically Excellent document about how best to read critically academic articles and other texts.
• How to Read an Academic Article This is an excellent paper that teach you how to read an academic paper, how to determine if it is something to set aside, or something to read deeply. Good advice to organize your literature for the Literature Review or just reading for classes.
• << Previous: Keeping up with Research!
• Next: Organizing for Writing >>
• Last Updated: Mar 5, 2024 11:44 AM
• URL: https://guides.library.ucsb.edu/litreview

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Systematic literature review of heuristic-optimized microgrids and energy-flexible factories.

1. Introduction

• the optimal system sizing of microgrids (OSS);
• the optimization of electrical energy distribution to storage systems and consumers (EED); and
• the energy flexibilization of factories (EF)?
• What is the scope of the functionality of the models and optimization algorithms of the respective research approaches?
• To what extent do research approaches that integrate these three areas into a common use case already exist?

2. Foundations of Microgrids, Energy-Flexible Factories, and Optimization Concepts

2.1. introduction to microgrids, 2.2. electrical energy distribution strategies, 2.3. economic and sustainable optimization objectives, 2.4. energy flexible factories, 2.5. basics of simulation models, 2.6. introduction to metaheuristic optimization, 3. implementation of the systematic literature review, 3.1. phase a—comprehensive collection of literature.

• For the SLR OSS search path, synonyms were chosen in the areas of hydrogen storage, decentralized energy supply, PV, BESS, and P2P systems, and methods for optimizing the MG component dimensions.
• For the SLR EED search path, functionalities related to load shifting, demand management, flexibility, dynamic pricing, and optimization were included.
• For the SLR EF search path, keywords were formulated in the areas of flexibility, resource scheduling, production planning, energy management, and factory decarbonization.

3.2. Phase B—Selection of Sources

4.1. descriptive analysis, 4.2. individual analysis, 4.2.1. analysis of the research area of the optimal system sizing of microgrids.

4.2.2. Analysis of the Research Area for the Optimization of Electrical Energy Distribution to Storage Systems and Consumers

4.2.3. Analysis of the Research Area of the Energy Flexibilization of Factories

4.3. research gap analysis.

• Although energy pricing incorporated granular dynamic ToU prices, emissions were not considered in similar detail. A ToU consideration of emissions per kWh of electricity could shift focus from predominantly economic to sustainability considerations. Databases, such as [ 69 ], have already provided time- and location-dependent emissions of electricity.
• A holistic multi-agent simulation and optimization model covering all three research areas could not be identified. It is advisable to develop an overarching optimization algorithm that integrates various metaheuristics, thereby combining their respective advantages.
• The increasing complexity of optimization models, arising from the integration of numerous sub-models and functionalities into the optimization calculations, should not be underestimated. There is a risk of becoming trapped in local optima during the optimization process. However, the accumulation of experiential knowledge and the precise tuning of hyperparameters for various metaheuristics in relation to the specific application can enhance the results and improve the robustness of the methodology and models.
• Various functionalities have been introduced; however, an approach that implements all functionalities cannot be found. It is crucial to emphasize that not all functionalities can be applied within a single simulation and optimization model because of the potential complexity that could outweigh the benefits.
• Studies analyzing all functionalities through sensitivity analyses to determine their criticality are lacking, particularly for EED and EF strategies. Furthermore, an analysis should be conducted to determine which functionalities and model characteristics are better suited for the design and operation stages.
• A user-friendly modeling approach specifically tailored for factories, particularly a holistic implementation methodology with template-based generic sub-models designed for SMEs, is currently lacking.

5. Discussion and Conclusions

Author contributions, data availability statement, acknowledgments, conflicts of interest.

• IPCC. Climate Change 2023: Synthesis Report. Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change ; Lee, H., Romero, J., Eds.; IPCC: Geneva, Switzerland, 2023. [ Google Scholar ] [ CrossRef ]
• Deutsche Bundesregierung. Klimaschutzgesetz. 2023. Available online: https://www.bundesregierung.de/breg-de/themen/klimaschutz/klimaschutzgesetz-2021-1913672 (accessed on 3 May 2024).
• Deutsche Bundesregierung. Wo Steht Deutschland Bei Der Energiewende|Anteil der Erneuerbaren Energien Steigt Weiter. 2022. Available online: https://www.bundesregierung.de/breg-de/themen/klimaschutz/faq-energiewende-2067498 (accessed on 5 May 2024).
• Umweltbundesamt. Emissionsvermeidung Durch Erneuerbare Energieträger. 2022. Available online: https://www.umweltbundesamt.de/themen/klima-energie/erneuerbare-energien/emissionsvermeidung-durch-erneuerbare#Emissionsbilanz (accessed on 5 May 2024).
• Bundesnetzagentur. Bericht zu Stand und Entwicklung der Versorgungssicherheit im Bereich der Versorgung mit Elektrizität. 2023. Available online: https://www.bmwk.de/Redaktion/DE/Downloads/V/versorgungssicherheitsbericht-strom.pdf?__blob=publicationFile&v=4 (accessed on 3 May 2024).
• Interconnector GmbH. Direktvermarktung Strom: Zuverlässig & Transparent. 2023. Available online: https://www.interconnector.de/direktvermarktung-strom/ (accessed on 3 May 2024).
• Tischkov, S. Windparkbetreiber Kämpfen mit Strompreis und Überkapazitäten. 2023. Available online: https://www.tagesschau.de/wirtschaft/windkraft-probleme-101.html (accessed on 3 May 2024).
• Claußner, M.; Huneke, F.; Brinkhaus, M.; Peper, D.; Kost, C.; Fluri, V. Potentiale und Rahmenbedingungen Für den Ausbau des Prosuming ; Bundesverband der Energie- und Wasserwirtschaft (BDEW): Berlin/Freiburg, Germany, 2022; Available online: https://www.bdew.de/service/publikationen/potentiale-und-rahmenbedingungen-fuer-den-ausbau-des-prosuming/ (accessed on 3 May 2024).
• BDEW. Industriestrompreise (Inklusive Stromsteuer) in Deutschland in den Jahren 1998 bis 2024. 2024. Available online: https://de.statista.com/statistik/daten/studie/252029/umfrage/industriestrompreise-inkl-stromsteuer-in-deutschland/ (accessed on 3 May 2024).
• Bundesministerium Für Wirtschaft und Klimaschutz. Aktionsplan, Mittelstand, Klimaschutz und Transformation. 2023. Available online: https://www.bmwk.de/Redaktion/DE/Dossier/politik-fuer-den-mittelstand.html (accessed on 3 May 2024).
• Peiseler, F.; Runkel, M.; Wettingfeld, M.; Mensinger, L. Nachhaltige Soziale Marktwirtschaft. 2023. Available online: https://www.bertelsmann-stiftung.de/de/publikationen/publikation/did/steuerpolitik-fuer-eine-nachhaltige-transformation-des-mittelstands (accessed on 10 October 2023).
• Amoussou, I.; Tanyi, E.; Fatma, L.; Agajie, T.F.; Boulkaibet, I.; Khezami, N.; Ali, A.; Khan, B. The Optimal Design of a Hybrid Solar PV/Wind/Hydrogen/Lithium Battery for the Replacement of a Heavy Fuel Oil Thermal Power Plant. Sustainability 2023 , 15 , 11510. [ Google Scholar ] [ CrossRef ]
• Sinnemann, J.; Prior, J.; Kuhlenkötter, B. Skalierbare Elektrolyseurmontage. Z. Für Wirtsch. Fabr. 2021 , 116 , 913–916. [ Google Scholar ] [ CrossRef ]
• Quaschning, V. Regenerative Energiesysteme: Technologie—Berechnung—Klimaschutz , 11th ed.; Hanser: München, Germany, 2021; ISBN 9783446472068. [ Google Scholar ]
• Eiselt, H.A. Operations Research: A Model-Based Approach ; Springer: Berlin/Heidelberg, Germany, 2022; ISBN 978-3-030-97161-8. [ Google Scholar ]
• Schulz, J.; Scharmer, V.M.; Zaeh, M.F. Energy self-sufficient manufacturing systems—Integration of renewable and decentralized energy generation systems. Procedia Manuf. 2020 , 43 , 40–47. [ Google Scholar ] [ CrossRef ]
• Popp, R.S.; Zaeh, M.F. Determination of the Technical Energy Flexibility of Production Systems. Adv. Mater. Res. 2014 , 1018 , 365–372. [ Google Scholar ] [ CrossRef ]
• Shahbazitabar, M.; Abdi, H.; Nourianfar, H.; Anvari-Moghaddam, A.; Mohammadi-Ivatloo, B.; Hatziargyriou, N. An Introduction to Microgrids, Concepts, Definition, and Classifications. In Microgrids: Advances in Operation, Control, and Protection ; Hatziargyriou, N., Mohammadi-Ivatloo, B., Abdi, H., Anvari-Moghaddam, A., Eds.; Springer: Cham, Switzerland, 2021; pp. 3–16. ISBN 978-3-030-59749-8. [ Google Scholar ]
• Tafarte, P.; Geiger, C.; Lehmann, P. Quantifying the opportunity cost of land use restrictions and their impact on the energy transition—A case study for Germany’s onshore wind power. In Proceedings of the IEEE 18th International Conference on the European Energy Market, Ljubljana, Slovenia, 13–15 September 2022; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Schleipen, M.; Drath, R. Three-view-concept for modeling process or manufacturing plants with AutomationML. In Proceedings of the 2009 IEEE Conference on Emerging Technologies & Factory Automation, Palma de Mallorca, Spain, 22–25 September 2009; pp. 1–4. [ Google Scholar ]
• Zhang, Y. Integration of Distributed Renewable Energy and Energy Storages in Buildings ; KTH Royal Institute of Technology: Stockholm, Sweden, 2019; ISBN 978-91-7873-193-0. [ Google Scholar ]
• Tiemann, P.H.; Bensmann, A.; Stuke, V.; Hanke-Rauschenbach, R. Electrical energy storage for industrial grid fee reduction—A large scale analysis. Energy Convers. Manag. 2020 , 208 , 112539. [ Google Scholar ] [ CrossRef ]
• Hopkinson, M. Net Present Value and Risk Modelling for Projects ; Routledge: London, UK, 2016; ISBN 9781315248172. [ Google Scholar ]
• Bundesnetzagentur. SMARD. 2023. Available online: https://www.smard.de/home/downloadcenter/download-marktdaten/ (accessed on 15 December 2023).
• Umweltbundesamt. Strom- und Wärmeversorgung in Zahlen. 2024. Available online: https://www.umweltbundesamt.de/themen/klima-energie/energieversorgung/strom-waermeversorgung-in-zahlen#Strommix (accessed on 15 March 2024).
• electricitymaps. 2024. Available online: https://app.electricitymaps.com/map (accessed on 15 March 2024).
• Siemon, L.; Blume, C.; Mennenga, M.; Herrmann, C. Multi-scale Simulation for Energy Flexible Factories and Factory Networks: A System of Systems Perspective. Procedia CIRP 2022 , 105 , 404–409. [ Google Scholar ] [ CrossRef ]
• Bortolato, A.; Faria, P.; Vale, Z. Probabilistic Determination of Consumers Response and Consumption Management Strategies in Demand Response Programs. In Proceedings of the 2020 IEEE/PES Transmission and Distribution Conference and Exposition (T&D), Chicago, IL, USA, 12–15 October 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Binder, C.; Hünecke, P.; Neureiter, C.; Lüder, A. Towards flexible production systems engineering according to RAMI 4.0 by utilizing PPR notation. In Proceedings of the 2023 IEEE 21st International Conference on Industrial Informatics (INDIN), Lemgo, Germany, 18–20 July 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Bundesministerium für Wirtschaft und Klimaschutz. RAMI 4.0—Ein Orientierungsrahmen für die Digitalisierung. 2018. Available online: https://www.plattform-i40.de/IP/Redaktion/DE/Downloads/Publikation/rami40-eine-einfuehrung.html (accessed on 14 January 2024).
• Fazli Khalaf, A.; Wang, Y. Energy-cost-aware flow shop scheduling considering intermittent renewables, energy storage, and real-time electricity pricing. Int. J. Energy Res. 2018 , 42 , 3928–3942. [ Google Scholar ] [ CrossRef ]
• Alp, E.; Ercan, F.; Kuhlenkötter, B.; Herzog, M. Modeling and Simulation for the Operative Service Delivery Planning in the Context of Product-Service Systems. In Proceedings of the IEEE 2023 Winter Simulation Conference (WSC), San Antonio, TX, USA, 10–13 December 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1936–1947. [ Google Scholar ] [ CrossRef ]
• Coakley, D.; Raftery, P.; Keane, M. A review of methods to match building energy simulation models to measured data. Renew. Sustain. Energy Rev. 2014 , 37 , 123–141. [ Google Scholar ] [ CrossRef ]
• Prior, J.; Hypki, A.; Kuhlenkötter, B. Heuristische Optimierung von P2G2P Speichern: Ein Template basierter Ansatz zur Evaluierung von Speicherstrategien und Optimierungsalgorithmen. In Erfolg Durch Nachhaltiges Energie- und Ressourcenmanagement ; Posch, W., Vorbach, S., Zsifkovits, H., Feichtinger, G., Eds.; Nomos: Baden-Baden, Germany, 2023; pp. 193–207. ISBN 978-3-98542-056-8. [ Google Scholar ]
• Spong, M.W. Introduction to Modeling and Simulation: A Systems Approach , 1st ed.; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2023; ISBN 9781119982906. [ Google Scholar ]
• Dutta, P.; Mahanand, B.S. Chapter 9—Affordable energy-intensive routing using metaheuristics. In Cognitive Big Data Intelligence with a Metaheuristic Approach ; Mishra, S., Tripathy, H.K., Mallick, P.K., Sangaiah, A.K., Chae, G.-S., Eds.; Academic Press: Cambridge, MA, USA, 2022; pp. 193–210. ISBN 978-0-323-85117-6. [ Google Scholar ]
• Lee, K.Y.; Vale, Z.A. (Eds.) Applications of Modern Heuristic Optimization Methods in Power and Energy Systems ; Wiley: Hoboken, NJ, USA, 2020; ISBN 9781119602293. [ Google Scholar ]
• Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Computat. 2002 , 6 , 182–197. [ Google Scholar ] [ CrossRef ]
• Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021 , 372 , n71. [ Google Scholar ] [ CrossRef ]
• Elsevier. Scopus: Umfassende, Multidisziplinäre Abstract- und Quellenangaben-Datenbank. 2023. Available online: https://www.elsevier.com/de-de/products/scopus (accessed on 12 December 2023).
• Catalogue of the German National Library. Available online: https://www.dnb.de/DE/Home/home_node.html (accessed on 25 May 2024).
• Fraunhofer-Gesellschaft. Landing Page. 2024. Available online: https://www.fraunhofer.de/en.html (accessed on 13 February 2024).
• Akhavan Shams, S.; Ahmadi, R. Dynamic optimization of solar-wind hybrid system connected to electrical battery or hydrogen as an energy storage system. Int J. Energy Res. 2021 , 45 , 10630–10654. [ Google Scholar ] [ CrossRef ]
• Singh, S.; Singh, N.; Gupta, A. System sizing of hybrid solar-fuel cell battery energy system using artificial bee colony algorithm with predator effect. Int J. Energy Res. 2022 , 46 , 5847–5863. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Sun, H.; Tan, J.; Li, Z.; Hou, W.; Guo, Y. Capacity configuration optimization of multi-energy system integrating wind turbine/photovoltaic/hydrogen/battery. Energy 2022 , 252 , 124046. [ Google Scholar ] [ CrossRef ]
• Chen, Z.; Yan, X.; Yuan, Y. Capacity configuration optimization of port multi-energy system integrating wind turbine/photovoltaic/battery/hydrogen. In Proceedings of the 2023 7th International Conference on Transportation Information and Safety (ICTIS), Xi’an, China, 4–6 August 2023; pp. 1–8. [ Google Scholar ] [ CrossRef ]
• Xing, L.; Liu, Y. An Optimization Capacity Design Method of Wind/Photovoltaic/Hydrogen Storage Power System Based on PSO-NSGA-II. Energy Eng. 2023 , 120 , 1023–1043. [ Google Scholar ] [ CrossRef ]
• Zhang, Y.; Campana, P.E.; Lundblad, A.; Yan, J. Comparative study of hydrogen storage and battery storage in grid connected photovoltaic system: Storage sizing and rule-based operation. Appl. Energy 2017 , 201 , 397–411. [ Google Scholar ] [ CrossRef ]
• Crespi, E.; Colbertaldo, P.; Guandalini, G.; Campanari, S. Energy storage with Power-to-Power systems relying on photovoltaic and hydrogen: Modelling the operation with secondary reserve provision. J. Energy Storage 2022 , 55 , 105613. [ Google Scholar ] [ CrossRef ]
• Le, T.S.; Nguyen, T.N.; Bui, D.-K.; Ngo, T.D. Optimal sizing of renewable energy storage: A techno-economic analysis of hydrogen, battery and hybrid systems considering degradation and seasonal storage. Appl. Energy 2023 , 336 , 120817. [ Google Scholar ] [ CrossRef ]
• Crespi, E.; Colbertaldo, P.; Guandalini, G.; Campanari, S. Design of hybrid power-to-power systems for continuous clean PV-based energy supply. Int. J. Hydrogen Energy 2021 , 46 , 13691–13708. [ Google Scholar ] [ CrossRef ]
• YALMIP. GUROBI. 2016. Available online: https://yalmip.github.io/solver/gurobi/ (accessed on 31 May 2024).
• HOMER. Hybrid Renewable and Distributed Generation System Design Software. 2024. Available online: https://homerenergy.com/ (accessed on 31 May 2024).
• Jaramillo, L.B.; Weidlich, A. Optimal microgrid scheduling with peak load reduction involving an electrolyzer and flexible loads. Appl. Energy 2016 , 169 , 857–865. [ Google Scholar ] [ CrossRef ]
• Shahryari, E.; Shayeghi, H.; Mohammadi-ivatloo, B.; Moradzadeh, M. A copula-based method to consider uncertainties for multi-objective energy management of microgrid in presence of demand response. Energy 2019 , 175 , 879–890. [ Google Scholar ] [ CrossRef ]
• Mosa, M.A.; Ali, A.A. Energy management system of low voltage dc microgrid using mixed-integer nonlinear programing and a global optimization technique. Electr. Power Syst. Res. 2021 , 192 , 106971. [ Google Scholar ] [ CrossRef ]
• Ruiming, F. Multi-objective optimized operation of integrated energy system with hydrogen storage. Int. J. Hydrogen Energy 2019 , 44 , 29409–29417. [ Google Scholar ] [ CrossRef ]
• Guo, Z.; Wei, W.; Bai, J.; Mei, S. Long-term operation of isolated microgrids with renewables and hybrid seasonal-battery storage. Appl. Energy 2023 , 349 , 121628. [ Google Scholar ] [ CrossRef ]
• Khan, M.W.; Wang, J.; Xiong, L.; Ma, M. Modelling and optimal management of distributed microgrid using multi-agent systems. Sustain. Cities Soc. 2018 , 41 , 154–169. [ Google Scholar ] [ CrossRef ]
• Vaish, J.; Tiwari, A.K.; Siddiqui, K.M. Optimization of micro grid with distributed energy resources using physics based meta heuristic techniques. IET Renew. Power Gener. 2023 , 1–17. [ Google Scholar ] [ CrossRef ]
• Cambambi, C.A.C.; Lautert, R.R.; Rangel, C.A.S.; Milani, I.; Canha, L.N. Model for Cooperative Operation of Li-ion Batteries and Hydrogen Storage Systems in Microgrids Considering Cycling Costs and Dynamic Prices. In Proceedings of the 2023 IEEE Power & Energy Society General Meeting (PESGM), Orlando, FL, USA, 16–20 July 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• VDEW-Lastprofile. 1999. Available online: https://www.bdew.de/media/documents/1999_Repraesentative-VDEW-Lastprofile.pdf (accessed on 15 May 2024).
• Küster, T.; Rayling, P.; Wiersig, R.; Pardo, F.D.P. Multi-objective optimization of energy-efficient production schedules using genetic algorithms. Optim. Eng. 2021 , 24 , 447–468. [ Google Scholar ] [ CrossRef ]
• Caro-Ruiz, C.; Lombardi, P.; Richter, M.; Pelzer, A.; Komarnicki, P.; Pavas, A.; Mojica-Nava, E. Coordination of optimal sizing of energy storage systems and production buffer stocks in a net zero energy factory. Appl. Energy 2019 , 238 , 851–862. [ Google Scholar ] [ CrossRef ]
• Lombardi, P.; Komarnicki, P.; Zhu, R.; Liserre, M. Flexibility options identification within Net Zero Energy Factories. In Proceedings of the 2019 IEEE Milan PowerTech, Milan, Italy, 23–27 June 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Wanapinit, N.; Thomsen, J.; Kost, C.; Weidlich, A. An MILP model for evaluating the optimal operation and flexibility potential of end-users. Appl. Energy 2021 , 282 , 116183. [ Google Scholar ] [ CrossRef ]
• Beier, J.; Thiede, S.; Herrmann, C. Energy flexibility of manufacturing systems for variable renewable energy supply integration: Real-time control method and simulation. J. Clean. Prod. 2017 , 141 , 648–661. [ Google Scholar ] [ CrossRef ]
• Erlach, K.; Westkämper, E. Energiewertstrom: Der Weg zur Energieeffizienten Fabrik ; Fraunhofer Verl.: Stuttgart, Germany, 2009; ISBN 9783839600108. [ Google Scholar ]
• Open Weather. Current Weather and Forecast. 2024. Available online: https://openweathermap.org/ (accessed on 16 February 2024).
• Agajie, T.F.; Ali, A.; Fopah-Lele, A.; Amoussou, I.; Khan, B.; Velasco, C.L.R.; Tanyi, E. A Comprehensive Review on Techno-Economic Analysis and Optimal Sizing of Hybrid Renewable Energy Sources with Energy Storage Systems. Energies 2023 , 16 , 642. [ Google Scholar ] [ CrossRef ]
• Arsad, A.Z.; Hannan, M.A.; Al-Shetwi, A.Q.; Mansur, M.; Muttaqi, K.M.; Dong, Z.Y.; Blaabjerg, F. Hydrogen energy storage integrated hybrid renewable energy systems: A review analysis for future research directions. Int. J. Hydrogen Energy 2022 , 47 , 17285–17312. [ Google Scholar ] [ CrossRef ]
• Fernandes, J.M.R.C.; Homayouni, S.M.; Fontes, D.B.M.M. Energy-Efficient Scheduling in Job Shop Manufacturing Systems: A Literature Review. Sustainability 2022 , 14 , 6264. [ Google Scholar ] [ CrossRef ]
• Khan, T.; Yu, M.; Waseem, M. Review on recent optimization strategies for hybrid renewable energy system with hydrogen technologies: State of the art, trends and future directions. Int. J. Hydrogen Energy 2022 , 47 , 25155–25201. [ Google Scholar ] [ CrossRef ]
• Rathod, A.A.; Subramanian, B. Scrutiny of Hybrid Renewable Energy Systems for Control, Power Management, Optimization and Sizing: Challenges and Future Possibilities. Sustainability 2022 , 14 , 16814. [ Google Scholar ] [ CrossRef ]
• Bänsch, K.; Busse, J.; Meisel, F.; Rieck, J.; Scholz, S.; Volling, T.; Wichmann, M.G. Energy-aware decision support models in production environments: A systematic literature review. Comput. Ind. Eng. 2021 , 159 , 107456. [ Google Scholar ] [ CrossRef ]
• Klemm, C.; Vennemann, P. Modeling and optimization of multi-energy systems in mixed-use districts: A review of existing methods and approaches. Renew. Sustain. Energy Rev. 2021 , 135 , 110206. [ Google Scholar ] [ CrossRef ]
• Rangu, S.K.; Lolla, P.R.; Dhenuvakonda, K.R.; Singh, A.R. Recent trends in power management strategies for optimal operation of distributed energy resources in microgrids: A comprehensive review. Int J. Energy Res. 2020 , 44 , 9889–9911. [ Google Scholar ] [ CrossRef ]
• Weitzel, T.; Glock, C.H. Energy management for stationary electric energy storage systems: A systematic literature review. Eur. J. Oper. Res. 2018 , 264 , 582–606. [ Google Scholar ] [ CrossRef ]
• Zia, M.F.; Elbouchikhi, E.; Benbouzid, M. Microgrids energy management systems: A critical review on methods, solutions, and prospects. Appl. Energy 2018 , 222 , 1033–1055. [ Google Scholar ] [ CrossRef ]
• Büsch, L.; Jakschik, M.; Syniawa, D.; Masuhr, C.; Christ, L.; Schachtsiek, J.; Haalck, K.; Nerlich, L.; Frömsdorf, E.; Schirmack, N.; et al. HyPLANT100: Industrialization from Assembly to the Construction Site for Gigawatt Electrolysis. Hydrogen 2024 , 5 , 185–208. [ Google Scholar ] [ CrossRef ]
• dena. Studie: Geschäftsmodelle für Dezentrale Wasserstoffkonzepte—Zeit zum Nachsteuern. 2023. Available online: https://www.dena.de/fileadmin/dena/Publikationen/PDFs/2023/STUDIE_Geschaeftsmodelle_fuer_dezentrale_Wasserstoffkonzepte_-_Zeit_zum_Nachsteuern.pdf (accessed on 18 August 2024).
• eur-lex.europa.eu. Green Deal. 2024. Available online: https://eur-lex.europa.eu/legal-content/DE/TXT/HTML/?uri=PI_COM:C(2021)2800&from=EN (accessed on 13 March 2024).

Click here to enlarge figure

Share and Cite

Prior, J.; Drees, T.; Miro, M.; Kuhlenkötter, B. Systematic Literature Review of Heuristic-Optimized Microgrids and Energy-Flexible Factories. Clean Technol. 2024 , 6 , 1114-1141. https://doi.org/10.3390/cleantechnol6030055

Prior J, Drees T, Miro M, Kuhlenkötter B. Systematic Literature Review of Heuristic-Optimized Microgrids and Energy-Flexible Factories. Clean Technologies . 2024; 6(3):1114-1141. https://doi.org/10.3390/cleantechnol6030055

Prior, Johannes, Tobias Drees, Michael Miro, and Bernd Kuhlenkötter. 2024. "Systematic Literature Review of Heuristic-Optimized Microgrids and Energy-Flexible Factories" Clean Technologies 6, no. 3: 1114-1141. https://doi.org/10.3390/cleantechnol6030055

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

This paper is in the following e-collection/theme issue:

Published on 19.8.2024 in Vol 26 (2024)

This is a member publication of Open University

Prevalence of Health Misinformation on Social Media—Challenges and Mitigation Before, During, and Beyond the COVID-19 Pandemic: Scoping Literature Review

Authors of this article:

• Dhouha Kbaier 1 , PhD   ; 
• Annemarie Kane 2 , PhD   ; 
• Mark McJury 3   ; 
• Ian Kenny 1 , PhD  

1 School of Computing and Communications, The Open University, Milton Keynes, United Kingdom

2 Faculty of Arts and Social Sciences, The Open University, Milton Keynes, United Kingdom

3 School of Physical Sciences, The Open University, Milton Keynes, United Kingdom

Corresponding Author:

Dhouha Kbaier, PhD

School of Computing and Communications

The Open University

Walton Hall

Milton Keynes, MK7 6AA

United Kingdom

Email: [email protected]

Background: This scoping review accompanies our research study “The Experience of Health Professionals With Misinformation and Its Impact on Their Job Practice: Qualitative Interview Study.” It surveys online health misinformation and is intended to provide an understanding of the communication context in which health professionals must operate.

Objective: Our objective was to illustrate the impact of social media in introducing additional sources of misinformation that impact health practitioners’ ability to communicate effectively with their patients. In addition, we considered how the level of knowledge of practitioners mitigated the effect of misinformation and additional stress factors associated with dealing with outbreaks, such as the COVID-19 pandemic, that affect communication with patients.

Methods: This study used a 5-step scoping review methodology following Arksey and O’Malley’s methodology to map relevant literature published in English between January 2012 and March 2024, focusing on health misinformation on social media platforms. We defined health misinformation as a false or misleading health-related claim that is not based on valid evidence or scientific knowledge. Electronic searches were performed on PubMed, Scopus, Web of Science, and Google Scholar. We included studies on the extent and impact of health misinformation in social media, mitigation strategies, and health practitioners’ experiences of confronting health misinformation. Our independent reviewers identified relevant articles for data extraction.

Results: Our review synthesized findings from 70 sources on online health misinformation. It revealed a consensus regarding the significant problem of health misinformation disseminated on social network platforms. While users seek trustworthy sources of health information, they often lack adequate health and digital literacies, which is exacerbated by social and economic inequalities. Cultural contexts influence the reception of such misinformation, and health practitioners may be vulnerable, too. The effectiveness of online mitigation strategies like user correction and automatic detection are complicated by malicious actors and politicization. The role of health practitioners in this context is a challenging one. Although they are still best placed to combat health misinformation, this review identified stressors that create barriers to their abilities to do this well. Investment in health information management at local and global levels could enhance their capacity for effective communication with patients.

Conclusions: This scoping review underscores the significance of addressing online health misinformation, particularly in the postpandemic era. It highlights the necessity for a collaborative global interdisciplinary effort to ensure equitable access to accurate health information, thereby empowering health practitioners to effectively combat the impact of online health misinformation. Academic research will need to be disseminated into the public domain in a way that is accessible to the public. Without equipping populations with health and digital literacies, the prevalence of online health misinformation will continue to pose a threat to global public health efforts.

Introduction

The global adoption of the internet has made health information more accessible, and the development of digital technology has enabled its rapid dissemination. However, the internet has also made possible the dissemination of false and misleading health misinformation and disinformation, with negative consequences, including the potential to exacerbate health inequalities. Health practitioners have found themselves at the forefront of communicating with patients who have taken on board health misinformation in the context of its proliferation on the web. This paper (associated with the study by Ismail et al [ 1 ]) surveyed the current literature concerning online health misinformation to establish the extent and scope of the problem, with special reference to the needs of health practitioners in their efforts to mitigate its impact. Several studies have established useful definitions of the terms misinformation and disinformation and distinctions between them. Misinformation has been defined as information that is not supported by evidence and contradicts the best-supported evidence available [ 2 , 3 ]. Wang et al [ 4 ] made a further distinction between online misinformation and disinformation, in particular on social media platforms. For Wang et al [ 4 ], misinformation is information that is not known to be false and is shared without malice. By contrast, disinformation involves the knowing and sharing of false information with the purpose of causing harm. This paper follows the distinctions of Wang et al [ 4 ] to use the terms misinformation and disinformation as appropriate.

It is important to acknowledge at the outset that digital technology in health and social contexts presents both risks and opportunities for equity among different information audiences [ 5 ]. However, there has recently been a change in the reception and assessment of the role of the internet, social media in particular, among researchers, even predating the COVID-19 pandemic. In the early days of social media, researchers largely identified prosocial and altruistic uses of social media platforms such as Facebook and Twitter by the public. However, considerable disquiet about the impact of social media and its potential for the spread of “fake news” and the amplification of conspiracy theories has displaced the more positive evaluation that was apparent when social media was in its infancy [ 6 ]. In the majority of the current research, there is a view that digital technology, particularly social media, has amplified the problem of health misinformation. The risk most frequently identified, either explicitly or implicitly, is the susceptibility of ordinary users, who may be lacking sophisticated levels of health and digital literacies, to health misinformation. Further risks noted in the literature include disinformation disseminated by organized trolling networks and bots that can be hard to distinguish from human users. The recognition of these risks underpins an emerging policy discourse about the threat of health misinformation, particularly the role of social media in its spread, in which health information and misinformation has become a politicized issue. From one policy perspective, there is an assumption that social media users are vulnerable, even passive, recipients of health misinformation rather than reflective interpreters of the available information. The corollary of this is that correcting misinformation with authoritative knowledge will solve the problem. However, a survey of the literature suggested that neither assumption fully expresses the complexity of how health misinformation is disseminated, received, and used via the internet. This may be because although there is a growing body of evidence on the extent of online health misinformation, there is much less research about what users do with health misinformation, why users consume health misinformation, and why (and which) people believe health misinformation [ 7 - 9 ].

In this scoping review, we reviewed the current state of knowledge regarding the prevalence of online misinformation before and during the COVID-19 pandemic and the impact that has on users’ understanding of health information. We considered this context with special reference to patients’ understanding, health practitioners’ practice in response to that, and policy makers’ concerns. The pressures and distractions that health professionals face in attempting to mitigate the impacts of online health misinformation are discussed in relation to patients’ health and digital literacies and the politicization of health information and misinformation.

Information Sources

We conducted a comprehensive literature search to identify relevant studies that explored health misinformation on social media platforms. The search was conducted across multiple electronic databases, including PubMed, Scopus, Web of Science, and Google Scholar.

The search terms included a combination of relevant keywords and phrases, including “health misinformation,” “social media,” “online health communities,” and “COVID-19 pandemic.” The search was not limited by publication date. Detailed search strategies are provided in Multimedia Appendix 1 .

Study Selection

Our study selection process followed a scoping approach, where we aimed to identify and include studies that provided insights into the prevalence and challenges of health misinformation on social media platforms. Initially, 2 researchers independently screened titles and abstracts of the identified articles to determine their relevance. Articles that did not meet the inclusion criteria were excluded at this stage.

Inclusion Criteria

Articles were included if they discussed health misinformation on social media, addressed the challenges posed by health misinformation, or were relevant to the period before, during, and after the COVID-19 pandemic.

Any disagreements between the 2 researchers were resolved through discussion and consensus. Full-text articles were then retrieved for the remaining studies, and a further assessment of eligibility was conducted based on the same inclusion criteria.

Data Extraction

We gathered information on (1) study objectives, (2) research methods, (3) findings, and (4) key themes related to health misinformation. This process was performed independently by 2 researchers, and any discrepancies were resolved through discussion.

Data Synthesis and Analysis

We adopted a scoping review content analysis approach to analyze the data extracted from the selected articles. The analysis process involved identifying key themes and patterns related to health misinformation on social media. The content analysis allowed us to gain a deeper understanding of the challenges posed by health misinformation and the strategies for its mitigation, both before and during the COVID-19 pandemic.

Results of Search

In our article selection process ( Figure 1 ), we initiated our search by identifying a total of 4563 articles from various databases. Following the removal of duplicates, 1295 articles were excluded, leaving us with 3268 unique articles. Subsequently, these articles underwent an initial screening, which involved evaluating their abstracts and titles, resulting in the exclusion of 2635 articles that did not align with our inclusion criteria. Further scrutiny was applied during full-text screening, which was conducted on 633 articles. Among these, 563 articles were found ineligible due to reasons such as not meeting the inclusion criteria (n=378 articles), being classified as literature reviews, editorials, or letters (n=174 articles), or the full texts being inaccessible (n=11 articles). A total of 70 articles were ultimately included in this scoping review.

Characteristics of Included Documents (n=70)

The majority (65/70, 93%) of documents were published in the last 10 years and originated predominantly in North America (42/70, 60%), Europe (19/70, 27%), and Asia (11/70, 16%). The funding sources were mainly public (61/70, 87%). The documents were classified as original research papers (38/70, 54%), subjective “opinion” articles (editorials, viewpoints, commentaries, and letters to the journal; 11/70, 16%), and knowledge syntheses or reviews (9/70, 13%) which included systematic reviews (n=6), descriptive reviews (n=2), and 1 integrated theoretic review.

Extent and Impact of Health Misinformation Disseminated Across a Range of Outlets

This section will review the literature concerning the extent and impact of the problem of health misinformation, including the spread of antivaccination discourse. In a study by Wood et al [ 10 ] among health practitioners in North Carolina, 94.2% of the respondents reported encounters with patient health misinformation within the previous year. While the sources of this misinformation were not broken down and identified by Wood et al [ 10 ], several other studies linked patient health misinformation to the prevalence of health misinformation on social media sites, identifying the latter as a significant problem [ 4 , 11 - 15 ]. There is a growing consensus among researchers, health professionals, and policy makers about the need to confront, challenge, and even prevent the online dissemination of health misinformation [ 16 ]. Since the emergence of online social networks, users have increasingly sought and shared health information on social media sites. It is estimated that around 70% of adult internet users search health matters on the web. With the emergence of social media platforms, there has been a rise in “peer-to-peer health care,” through which individuals seek and share health information, forming online health communities with others who have similar health concerns [ 3 ]. In addition, health organizations and health professionals are increasingly using social media to disseminate and promote health information and advice. The opportunities for sharing and promoting good health information via the internet are evident, and it is important to acknowledge that in online health communities, users share experiences and receive and give different kinds of support, including emotional support, to cope with specific health conditions. However, the medium has also enabled the dissemination of health misinformation, and the prosocial aspects of sharing are also likely to involve the sharing of misinformation, putting the health of users at risk [ 3 ].

Misinformation Spreads on Social Media

There is a high degree of consensus among researchers that mainly because of the increasing popularity of social media, the internet has become a space for the dissemination and amplification of “fake news,” misleading information, and rumor, including health misinformation and antivaccine conspiracy theories [ 17 ]. The COVID-19 pandemic has heightened these concerns, resulting in a proliferation of recent studies and rapid reviews focusing on the online spread of misinformation. Lee et al [ 18 ] proposed that the proliferation of health misinformation during the COVID-19 pandemic became a major public health issue. At the earliest signs of the emerging COVID-19 pandemic, the director-general of the World Health Organization, Tedros Adhanom Ghebreyesus, speaking at the February 2020 Munich Security Conference, expressed concern about the risk of an infodemic of health misinformation disseminated via social media, identifying “vaccine hesitancy” as 1 of the top 10 global health threats [ 19 ]. Bapaye and Bapaye [ 20 ] agreed that the risks of misinformation on social networking sites constitute a global issue, referring specifically to the COVID-19 infodemic.

However, this is not in itself a new problem; longstanding concerns about “fake news” and misinformation in traditional media have been evident since the early decades of the 20th century [ 21 ], and the prevalence of misinformation on internet platforms certainly predates the COVID-19 pandemic. Therefore, because the COVID-19 pandemic has only intensified the concern regarding health misinformation, it might be more appropriate to see the pandemic as symptomatic of, and crystallizing, the challenges of countering health misinformation in the digital age, as the development of digital technology and the internet have brought about profound changes in the capacity of both misinformation and disinformation to spread globally and amplify rapidly [ 4 ].

Suarez-Lledo and Alvarez-Galvez [ 16 ] undertook a review of 69 studies of health misinformation on social media to identify the main health misinformation topics and their frequency on different social media platforms. The studies surveyed used a variety of research methods, including social network analysis (28%), evaluation of content (26%), evaluation of quality (24%), content/text analysis (16%), and sentiment analysis (6%). Suarez-Lledo and Alvarez-Galvez [ 16 ] concluded that the incidence of health misinformation was highest on Twitter, in particular, regarding the use of tobacco and other drugs, with some studies citing 87% of such posts containing misinformation. However, health misinformation about vaccines was also prevalent, with around 43% of posts containing misinformation, with the human papillomavirus vaccine being the most affected. This review by Suarez-Lledo and Alvarez-Galvez [ 16 ] confirmed many of the findings from earlier surveys. For example, in their survey of 57 articles, Wang et al [ 4 ] found that the most frequently discussed topics were regarding vaccination and infectious diseases, including Ebola and the Zika virus. Other topics such as nutrition, cancer, water fluoridation, and smoking were also prevalent. The studies they surveyed had tended to find that a high degree of misinformation on these topics was being shared and liked on social media.

Lee et al [ 18 ] conducted a cross-sectional online survey in South Korea to examine the prevalence of COVID-19 misinformation and the impact of exposure to COVID-19 misinformation on beliefs and behaviors. They found that exposure to COVID-19 misinformation was associated with misinformation belief, which then resulted in fewer preventive behaviors. Therefore, they highlighted the potential of misinformation to undermine global efforts in disease control and argued that public health strategies are needed to combat the proliferation of misinformation. Bapaye and Bapaye [ 20 ] conducted a cross-sectional online questionnaire survey of 1137 WhatsApp users in India. They noted that most research on the prevalence of misinformation in social media has focused on Twitter and Facebook and on the Global North. Measured by age, researchers found that users aged >65 years were the most vulnerable to accepting the veracity of messages containing health misinformation (K=0.38, 95% CI 0.341-0.419) Respondents aged 19 to 25 years displayed much lower vulnerability (K=0.31, 95% CI 0.301-0.319) than those aged >25 years ( P <.05). Measured by occupational category, users employed in nonprofessional occupations had the highest vulnerability (K=0.38, 95% CI 0.356-0.404); this was significantly higher than those of professionals and students ( P <.05). Notably, the vulnerability of health professionals was not significantly different from those of other occupation groups ( P >.05).

The authors concluded that in a developing country, WhatsApp users aged >65 years and those involved in nonprofessional occupations are the most vulnerable to false information disseminated via WhatsApp. Crucially, they noted that health care workers, who might be expected by laypersons to have expert knowledge, were as likely to be vulnerable to health misinformation as other occupation groups.

Antivaxxer Spread Before, During, and Beyond the COVID-19 Pandemic

Much of the current unease from researchers, understandably, centers on health misinformation about vaccines in the wake of the COVID-19 pandemic. In particular, there is concern about the growth and spread of so-called antivaxxer misinformation and beliefs. In 2019, the United States had its biggest measles outbreak in 30 years, with most cases involving people who had not been vaccinated. Hotez [ 22 ] claimed that much of the reason for the growth of antivaccine beliefs is because of a campaign of misinformation. He argued that social media sites are meeting places for the sharing of antivaccine views. To evade social media platforms’ automated moderation tools, which tend to focus on words, several antivaxxer groups, including one with around 250,000 members, began using visual codes, such as the carrot emoji, to hide antivaxxer content.

However, some of the misinformation has gained credibility because it has come from sources that laypersons would expect to be trustworthy. For example, in 1998, the British medical journal The Lancet published a paper by Dr Andrew Wakefield claiming a link between the measles, mumps, rubella vaccine and the onset of autism spectrum disorder. Wakefield’s paper was later rebutted, and an overwhelming body of evidence now refutes its conclusions [ 23 ]. However, despite long being discredited, Wakefield’s claims have remained a part of the antivaccine discourse. The persistence of the antivaccination narrative demonstrates the power of such discourses even in the face of evidence to challenge them.

Although strong antivaccine beliefs, and the more ambivalent attitude of vaccine hesitancy, have been around as long as there have been vaccines, until recent decades, they were on the margins. However, evidence supports the claim that they have been gaining momentum in the United States and Europe.

A survey by Skafle et al [ 24 ] aimed to synthesize the results from 19 studies in which the effect of social media misinformation on vaccine hesitancy was measured or discussed. The authors noted that the “vast majority” of studies were from industrialized Western countries. Only 1 study contained misinformation about autism as a side effect of COVID-19 vaccines. Nevertheless, the studies implied that information spread on social media had a negative effect on vaccine hesitancy and uptake. The conclusions from Skafle et al [ 24 ] were supported by data from online polling agencies. For example, a US YouGov poll from May 2020 found that only 55% of respondents would definitely take a COVID-19 vaccine if one were to become available, whereas 19% of respondents said that they would refuse and 26% were still undecided [ 25 ].

While much of the research about online vaccine discourse comes from the United States, there is also evidence that vaccine hesitancy has risen elsewhere. For example, in an Ipsos-MORI survey taken in December 2020, only 40% of respondents in France said they would take a COVID-19 vaccine, a figure symptomatic of a steep and swift decline in vaccine confidence in France [ 26 ]. However, interestingly, the same Ipsos-MORI poll indicated a rise in vaccine confidence among respondents in the United States since the earlier YouGov poll, cited earlier, by approximately 10% to 65%, and respondents in the United Kingdom expressed a still higher willingness to take a COVID-19 vaccine at approximately 77%. It is notable that in the United States and United Kingdom, the Ipsos-MORI results came after a period of intermittent lockdowns. The contrast with the results from France is, nevertheless, striking.

Understanding the Challenges Surrounding Health Misinformation

Here, we consider the challenges created by health misinformation on the web: (1) the role played by malicious actors on social media in spreading vaccine disinformation and misinformation and (2) how contextual and cultural issues have different effects on patients’ understanding of what is considered genuine, valid, and authentic health information.

Spread of Health Misinformation on Social Media by Malicious Actors

One strand of research presents the issue of health misinformation as a contest between trolls and bots on the one hand and the voices of trustworthy public health agencies on the other [ 6 ]. This view was supported by Hotez [ 22 ] and Broniatowski et al [ 11 ]. The latter investigated the role of bots and trolls as malicious actors mobilizing vaccination discourse on the web. Their study focused specifically on vaccine-related health messaging on Twitter. Comparing the rates of vaccine-related messages, they found that sophisticated bots and Russian trolls tweeted at higher rates than “average users.” However, the respective content from bots and trolls differed. Whereas bots communicated antivaccine messages, Russian troll accounts provided a seemingly balanced discussion of both provaccination and antivaccination arguments, implying an equivalence between them. The authors argued that amplifying and normalizing a debate is done with the purpose of sowing discord and may lead to undermining public confidence in scientific consensus about the effectiveness of vaccines. Wang et al [ 4 ] acknowledged that it is a challenge to readily distinguish between misinformation and disinformation on the web. They noted that disinformation, such as antivaccine propaganda, can unknowingly be spread by users with genuine concerns [ 4 ], as individuals increasingly seek health and healthy lifestyle information via the internet.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Politicization of the Problem of Health Misinformation

The identification of online trolls, bots, and orchestrated networks as major contributors to the spread of health disinformation and misinformation is now part of mainstream political discourse in the United States. On July 16, 2021, a quarrel broke out between the president of the United States, Joe Biden, and Facebook over the spread of health misinformation on the company’s social media platforms. Speaking to journalists, Biden blamed social media companies for a rise in the number of deaths from COVID-19 among the unvaccinated in the United States. Referring explicitly to Facebook, the president claimed that by allowing the proliferation of health misinformation on its platforms, the company was “killing people” [ 27 ]. Discursive interventions from politicians are never neutral; nevertheless, Biden’s claim about the impact of health misinformation on social media is backed up by many of the studies surveyed for this paper. Facebook immediately rebutted Biden’s accusation by citing their rules, introduced in February 2021, which banned posts that make identifiably false claims about vaccines. Furthermore, Facebook challenged Biden’s claim by asserting that not only has Facebook provided more authoritative information about COVID-19 and vaccines than any other internet site, reaching 2 billion people with such posts, but also that the platform’s vaccine finder tool had been used by more than 3 million Americans.

These figures suggest that although antivaxxer groups find ways to evade detection, their reach may be countered by that of information grounded in current science. A spokesperson for the company said that, far from killing people, “The facts show that Facebook is helping save lives. Period” [ 27 ]. The argument between Biden and Facebook may indeed signal more lay awareness of the problem and echo the concerns of the recent academic research about the dissemination of health misinformation by organized bot and troll networks. Framed as it is, in terms of apportioning the blame for the spread of health misinformation, Biden’s intervention mirrors much of the academic discourse in the United States on the subject. However, it is also symptomatic of the politicization of health misinformation, arguably accelerated by the COVID-19 pandemic, which may thwart evidence-based decision-making. This point was emphasized strongly by Kyabaggu et al [ 5 ]. They framed the problem of pervasive misinformation and disinformation in terms of prime movers and beneficiaries who use it to advance sociopolitical agendas and entrench asymmetrical power, especially in times of uncertainty and threat, such as the COVID-19 pandemic.

Kyabaggu et al [ 5 ] identified government failures to adopt evidence-informed decision-making. They noted that such failures have costs that not only are economic but, crucially, result in poorer health outcomes. They cited as an example the United Kingdom government’s initial prevaccine herd immunity strategy. The intention of this strategy was to allow SARS-CoV-2 to indiscriminately spread to a critical mass to build up population immunity. The authors noted that this was “a particularly concerning example of evidence framing by a government.” Kyabaggu et al [ 5 ] argued that public acceptance of health risk messages and adoption of health-protecting behaviors is highly contingent on the degree to which governments engage in evidence-informed decision-making and communicate this basis effectively. The authors cited several instances of government actors failing to recognize misinformation, disseminating inconsistent or inaccurate information, and not using evidence- and information-based decision-making processes. In recent years, the public policy discourse in the United Kingdom has been veering away from evidence- and information-based decision-making, as politicians have denounced “experts” and their “influence” on policy [ 28 , 29 ].

Finally, Gruzd et al [ 30 ] reported on the impact of coordinated link-sharing behavior to spread and amplify conspiracy-related misinformation. They found a coalition of Facebook accounts that engaged in coordinated link sharing behavior to promote COVID-19 related misinformation. This coalition included US-based pro-Trump, QAnon, and antivaccination accounts.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Health Literacies and Inequality

While the approach of Broniatowski et al [ 11 ], for example, provided a persuasive account of ways in which online health misinformation can be disseminated, there are limitations to this approach, as it did not provide an account of how users respond to the misinformation they encounter. The responses of ordinary users were assumed rather than investigated. Research by Vosoughi et al [ 31 ] provided a caveat to the claim that it is bots that accelerate the spread of misinformation. Their work supported that of Broniatowski et al [ 11 ] in suggesting that bots spread accurate and false information at the same rate. However, Vosoughi et al [ 31 ] also explained that misinformation spreads more rapidly than accurate information because humans, rather than bots, are more likely to spread misinformation [ 31 ]. This claim was further supported by Wang [ 32 ], who suggested that in democracies, where ideas compete for attention in a marketplace, accurate scientific information, which, for the layperson, may be boring or difficult to understand, is easily crowded out by information that is more easily grasped or sensational. Mokhtari and Mirzaei [ 12 ] located this problem specifically in the context of the COVID-19 pandemic. They considered that high mortality from COVID-19, its complexity, and its unknown features resulted in fear, anxiety, and mental pressure among people worldwide. To allay anxiety, people needed health information literacy, defined by the American Library Association as a set of abilities individuals require to recognize when information is needed and to locate, evaluate, and use it effectively [ 33 ]. In addition, Wang [ 32 ] noted that individuals are differentially vulnerable to health misinformation depending on their level of health literacy and that models need to account for this. Mokhtari and Mirzaei [ 12 ] argued that not only information and health literacies but also media literacy are needed. However, studies in the field of health literacy suggest that significant inequalities in health and digital literacies exist.

Researchers have argued that “vastly undervalued and unrecognized” health literacy ought to be considered the best “social vaccine” for preventing COVID-19 in populations [ 5 ]. However, inequalities in health literacy persist. Kyabaggu et al [ 5 ] defined health literacy as encompassing cognitive and social skills that determine individuals’ motivation and ability to access, understand, and use information, including quantitative health risk information, in ways that promote and maintain good health across the life course. They asserted that health literacy is an essential self-management skill and community resource for health, noting that health literacy is positively associated with patients’ involvement in clinical decision-making, willingness to express health concerns, and compliance with clinical guidance. However, despite research demonstrating the importance of health literacy, evidence, even from high-income countries, suggested relatively low levels of health literacy.

Kyabaggu et al [ 5 ] drew a link between health literacy and digital literacy. They suggested that the latter can be understood as health literacy in digital information and technology spaces. They argued that inequalities in health outcomes are exacerbated by a widening digital divide. While digital technology in health and social contexts presents both new risks and opportunities for equity in different information audiences, the ways in which power and privilege operated in the COVID-19 misinformation discourse have not been sufficiently examined. Although socially and economically disadvantaged groups were at a greater risk of exposure to COVID-19, their voices and experiences were often marginalized. In addition, inequalities in access to accurate information are not only related to issues of digital access and literacy but are also situational. For example, disadvantaged individuals may have fewer social connections, and low pay may necessitate longer working hours, militating against individuals having the resources of time and energy to seek out accurate health information and enhance their level of health literacy.

The experiences of specific groups may also go unreported. Quraishi [ 34 ] addressed the impact of misinformation on South Asian students—a fast-growing group in the United States, but one that often receives little media attention. Quraishi [ 34 ] concluded that there is a relationship between the COVID-19 pandemic and students’ academic performance and mental health, as well as an increase in the spread of misinformation regarding COVID-19 public safety guidelines.

Older adults can be a vulnerable group in relation to their comparatively poor digital literacy. Zhou et al [ 35 ] reported on the accuracy of older adults in judging health information credibility. They found that on average, participants only successfully judged 41.38% of health articles. Attractive headlines increased participant credibility judgments on the content, and of the articles shared with others, 62.5% contained falsehoods.

Contextual Factors Influencing the Reception of and Responses to Misinformation: Cultures and Values

Larson and Broniatowski [ 19 ] argued that developing the kinds of literacy advocated by Mokhtari and Mirzaei [ 12 ] and Tully et al [ 2 ] will not address the deep-seated problems they identified. The work by Kyabaggu et al [ 5 ] supported this, and noted that the infodemic crisis is not merely a health and digital literacy issue. Some demographics may be more vulnerable to persuasive communication from broader sociocultural forces. Kyabaggu et al [ 5 ] argued that in considering the social determinants of health, attention must be paid not only to digital and health literacies but also to the ways in which these literacies coexist and interact with other influences. Larson and Broniatowski [ 19 ] suggested that one of the strongest determinants of vaccine confidence or vaccine hesitancy is the level of trust or distrust in the institutions that produce vaccines. A higher level of trust encourages the willingness to accept a high level of risk for a greater benefit. A lower level of trust militates against the acceptance of even a low level of perceived risk. For Larson and Broniatowski [ 19 ], it is not simply the presence of misinformation on social media networks but the social and cultural context of users’ reception of that information that influences responses. Health information operates in a complex and contentious social world. Individuals and communities respond to new information in terms of already developed political, cultural, and social values that influence whether they trust or distrust authority. Populations may be characterized by trust or mistrust of scientific institutions and government. Trust has been eroded through the exposure of fraud, research scandals, and misconduct by major multinational pharmaceutical companies, for example. Communities may be predisposed to distrust the government and its agents depending on their own status or identity. According to Goldenberg [ 36 ], these contexts can make misinformation and health conspiracy theories compelling.

Strategies to Correct Online Misinformation

We address the additional pressures on health professionals in communicating accurate information to mitigate the effects of misinformation, particularly with regard to the additional requirements imposed as a result of the precautions being taken during the pandemic. One area of disagreement in the literature concerns the usefulness of user correction response.

Research Into User Correction Strategies

There is some disagreement as to whether engagement with misinformation by users spreads and reinforces it or even whether extended debates over health misinformation cause users to doubt the possibility of knowable facts. For example, Broniatowski et al [ 11 ] argued that when ordinary users directly confront vaccine-skeptic messages from bots, it only serves to legitimize the “debate.” By contrast, Tully et al [ 2 ] argued that social media users have a role to play in either spreading or stopping the spread of misinformation across platforms. Their research aimed to uncover what factors influenced users’ responses. Tully et al [ 2 ] acknowledged that a range of factors can influence the spread or prevention of misinformation, including the behavior of malicious actors such as bots and trolls; the platform’s terms of service; and content moderation policies. As already noted, while most users are not creators of misinformation, they may spread and amplify it by liking, sharing, or replying. In opposition to the work of Broniatowski et al [ 11 ], Tully et al [ 2 ] argued that the content of engagement is particularly important, as their research suggested that multiple corrections by social media users may be required to reduce misperceptions. However, they claimed that most people simply ignore misinformation when they see it on social media.

Tully et al [ 2 ] noted the promise in mobilizing users to engage in such correction, given the vast numbers of users on these sites, in comparison with professional fact-checkers and health authorities.

They considered whether the tone of a correction would influence perceptions of the credibility of the message. However, despite some mixed evidence, they concluded that overall, the tone was not a significant factor and that neutral, affirmative, and uncivil corrections were all effective at reducing misperceptions. They found that participants were generally unlikely to reply to the misinformation tweet. However, their content analysis of hypothetical replies suggested that when users did reply, they mainly provided correct information, particularly after seeing other corrections. Tully et al [ 2 ] concluded that user corrections offer “untapped potential” in responding to misinformation on social media, but further work is needed to consider how users can be mobilized to provide corrections, given their overall unwillingness to reply. However, a limitation of the experimental approach of Tully et al [ 2 ], acknowledged by the researchers, is that in asking individuals what they would hypothetically do, this may not reflect what they actually do in a real social media setting, especially in relation to an issue they care more strongly about. Although the experiment gauged attitudes, it did not delve into how strongly these attitudes were held. It is also not clear to what degree corrections were effective at reducing misperceptions and how reductions were measured.

By contrast, the results of experimental studies by Ittefaq [ 37 ] and Mourali and Drake [ 38 ] suggested that correcting misinformation is by no means a straightforward proposition. They noted the previous research on rebuttal, which suggested that properly designed corrections can mitigate the effects of misinformation. However, such studies have tended to compare responses to misinformation followed by correction with responses of a control group that receives no correction or receives an alternative correction. Mourali and Drake [ 38 ] argued that this static approach misses the dynamic nature of social media debate. They noted that the correction of misinformation is generally followed up with a rebuke by the original poster, inciting further correction and prolonged back-and-forth debate. Mourali and Drake [ 38 ] cited previous studies showing that exposure to conflicting information about health topics, including mammography, nutrition, and the human papillomavirus vaccine, may increase confusion and negative attitudes toward that particular health topic. The researchers found that initial exposure to misinformation had a negative impact on attitudes and intentions toward masking, consistent with previous studies that concluded that exposure to misinformation negatively impacts attitudes and intentions toward behaviors favored by science. Also consistent with previous research, they found that the first correction of the false claim improved attitudes and intentions toward masking. The authors suggested that this effect is partially explained by a decrease in the perceived strength of the argument underlying the false claim. However, this initial improvement diminished on further exposure to false claims and refutation attempts. This finding confirmed their hypothesis that extended exposure to false claims and refutation attempts appears to weaken belief in the possibility of objective knowledge, leading to less positive reactions toward masking as a science-based behavior. They concluded that the level of exposure to contradictory information needs to reach a certain threshold before it affects perceived truth objectivity. However, although people are more likely to share misinformation when its content is consistent with their existing beliefs or when its message is simple, direct, or sensational, correcting misinformation does reduce its likelihood of being shared on social media, an effect that persists even after multiple exposures.

Mourali and Drake [ 38 ] noted that each social media platform exhibits particular interaction norms, which may impact how users interpret the conversation. As their study was limited to a single platform, Reddit, and the debate was restricted to 4 exchanges between only 2 protagonists, the researchers acknowledged that these aspects limit the generalizability of the results. They suggested that future research could attempt to replicate their findings on different social media platforms, and to include more than 2 protagonists and more than 4 exchanges. They noted further that although extended debates are common on social media, it is not known how frequently they occur, echoing the comments by Suarez-Lledo and Alvarez-Galvez [ 16 ] that the extent of misinformation is not clear.

In contrast to the fairly sanguine view of Tully et al [ 2 ] about the potential of users to spread corrective information, Mourali and Drake [ 38 ] problematized the position, pointing to the potential for more complex and uncertain outcomes, whereas Larson and Broniatowski [ 19 ] argued that although the importance of correcting misinformation, item by item, should not be diminished, only if underlying issues driving misinformation are addressed can, for example, long-term vaccine confidence in populations be sustained. They argue that simply responding to misinformation with factual corrections is not likely to reverse the dissent that has been evident among antivaxxers or to necessarily persuade the more ambivalent vaccine-hesitant individuals. They identified deeper social and cultural issues at play, which have been discussed in this paper in the previous sections.

Research Into Effective Models to Accomplish the Automatic Detection of Health Misinformation in Online Health Communities

Here, we consider examples of research into the automatic detection of health misinformation in online health communities. Zhao et al [ 3 ] began from the premise that there is a vast amount of health misinformation, creating a challenge for health communities in identifying misinformation. Rather than relying on users’ ability to correct misinformation, they proposed that there is a need for an effective model to achieve automatic detection of health misinformation in online health communities. This view was also put forward by Weinzierl and Harabagiu [ 39 ]. Focusing specifically on COVID-19 vaccine misinformation, they argued that automatic detection of misinformation on social media is an essential first step in delivering interventions designed to address vaccine hesitancy.

Zhao et al [ 3 ] identified much of the existing analysis as concentrating on the linguistic features of communications only. They wanted to examine the underresearched area of whether integrating user behavioral features with linguistic features, sentiment features, and topic features could effectively distinguish misinformation from accurate information in online health communities. Their study combined the aforementioned features to build a detection model targeting misinformation in online health communities’ contexts. The behavioral features targeted were discussion initiation, interaction engagement, influential scope, relational mediation, and informational independence. Descriptions of these behavioral features are reproduced in Table 1 .

The authors tested their detection model on a data set collected from a real online health community, selecting as their data source Zibizheng Ba, an autism forum on the Baidu Tieba online health community site hosted by the Chinese web service Baidu. Baidu Tieba claims to be one of the largest interest-based discussion platforms in China. Users can generate topic-based discussion forums on the platform, share information, and make friends with other users. Posts on Baidu Tieba are indexed by Baidu, China’s most popular search engine, so users can readily find misinformation when searching for health-related information through the search engine. The authors developed a python-based web crawler to collect data from the forum. To train the health misinformation detection model, 5000 records were sampled from the whole data set by stratification according to 3 types of records (ie, thread, reply, and reply to reply) using stratified sampling methods. Therefore, the constituent types of the records (ie, thread, reply, and reply to reply) in the sample data set were consistent with the composition of the whole data set.

The researchers applied the elaboration likelihood model (ELM). The model, originally developed by Petty and Cacioppo [ 40 ] to explain attitude change, has been used extensively in advertising to try to influence consumers.

Overall, 4 types of misinformation were identified through their coding analysis, and the model correctly detected about 85% of the health misinformation. Their results also indicated that behavioral features were more informative than linguistic features in detecting misinformation. The authors concluded that their results not only demonstrated the efficacy of behavioral features in health misinformation detection but also offered both methodological and theoretical contributions to misinformation detection by integrating the features of messages as well as the features of message creators. Others have also highlighted the problems posed by misleading visual information [ 41 ].

It is worth noting that during the pandemic, the UK National Health Service (NHS) began using Twitter to promote provaccine messaging, which closely follows a combination of the features suggested by Zhao et al [ 3 ]. When users searched for the term “vaccine” or related terms, the top post was a message prominently displaying the NHS logo, identifying it as reputable and trustworthy. The tweets contained links to NHS websites providing information about vaccines and COVID-19. The posts differed in linguistic content and visual design. For example, one featured only written text on a white background and stated in bold, “Know the facts.” Another featured a large image of a happy minority ethnic family, washing dishes together, with the message that the COVID-19 vaccine decreases household transmission by up to half. The contrasting designs suggest that the message was targeted specifically to users’ timelines. It was also apparent that elements of ELM were being applied, combining the features identified by Zhao et al [ 3 ] in different ways.

Weinzierl and Harabagiu [ 39 ] adopted a different method than Zhao et al [ 3 ], reversing the more commonly used classification approach. The authors of each study claimed strong results in identifying health misinformation on social media platforms. However, Nabożny et al [ 42 ] argued that the current automatic systems for assessing the credibility of health information are not sufficiently precise to be used without supervision by human medical expert annotators.

Barve and Saini [ 43 ] have reported on their use of automated fact-checking using a coded content similarity measure (CSM). In this approach, the CSM showed improved accuracy (91.06%) compared to the accuracy of the Jaccard similarity measure (74.26%). Further, the algorithmic approach outperformed the feature-based method.

Neither Zhao et al [ 3 ] nor Weinzierl and Harabagiu [ 39 ] recorded what happens when misinformation is detected. Research from Broniatowksi et al [ 44 ] suggested that once detected, steps taken by social media platforms such as content removal or deplatforming may not be effective in stemming the spread of misinformation and may even be counterproductive. Social media platforms use a combination of “hard” and “soft” content remedies to reduce the spread of health misinformation. Soft remedies include warning labels attached to content and downranking of some content in web searches, whereas hard remedies include content removal and deplatforming of accounts. Hard remedies are controversial and have given rise to accusations of censorship. For the authors, short-term evidence for the effectiveness of hard remedies is in any case mixed, and long-term evidence is yet to be examined. Their study focused on Facebook and found that while hard remedies did reduce the number of antivaccine posts, they also produced unintended consequences. Provaccine content was removed, and engagement with the remaining antivaccine content repeatedly recovered to prepolicy levels. Worryingly, this content became more misinformative, more politically polarized, and more likely to be seen in users’ news feeds. The authors explain these results as a product of Facebook’s architecture, which is designed to promote community formation. Members of communities dedicated to vaccine refusal seek out misinformation. To meet this demand, and to circumvent content moderation efforts, antivaccine content producers post links to external sources of misinformative content, such as Bitchute, Rumble, Gab, and Telegram, in lieu of more mainstream platforms that had implemented similar content removal policies (eg, YouTube and Twitter). Broniatowski et al [ 44 ] argued that Facebook’s policy reduced the number of posts in antivaccine venues but was not successful in inducing a sustained reduction in engagement with antivaccine content, including misinformation. The authors noted that alternative platforms often host politically extreme right-wing content. Therefore, they argued that Facebook’s content removal policies may have the unintended consequence of radicalizing their audiences, and their findings suggested the need to address how social media platform architecture enables community formation and mobilization around misinformative topics when managing the spread of online content.

These studies advocate for the automatic detection of health misinformation. However, work that calls into question the ability of automatic detection to operate without human intervention has also been discussed. In addition, there are questions raised in the literature about what should be done when misinformation is detected and concerns about whether content removal or deplatforming of accounts are the most effective ways to reduce the spread of health misinformation or may even be counterproductive.

The Roles of Health Practitioners

The discussion so far has highlighted the complex and multifaceted dimensions of the context of online health misinformation in which health practitioners must operate. As noted in our introduction, a study of health practitioners in North Carolina found that nearly 95% had encountered patient health misinformation within the previous year [ 10 ]. There is very little research on the amount or effectiveness of training received by health professionals to prepare them for engaging with patients about health misinformation. Wood et al [ 10 ] found that most respondents had not received relevant training despite overwhelmingly reporting encountering health misinformation.

Nevertheless, within the literature, there is no shortage of advice from researchers and health professionals addressed to health practitioners on how to approach and correct health misinformation. This advice stems from both original research studies and reviews of best practices featured in peer-reviewed medical and health journals. Such advice centers on the need for health practitioners to understand misinformation and how to address it. Health practitioners are advised of the need to be aware of health myths and urged to dismantle them in providing accurate health guidance [ 45 , 46 ]. Practitioners are further advised that misinformation and pseudoscience are appealing to those seeking certainty because they present information in absolutes, whereas medical science is often ambiguous and contingent. Health practitioners are also encouraged to learn how to message more clearly and to mimic the strategies of misinformation [ 45 ]. One study recommends that “practitioners familiarize themselves with the tools of scientific enquiry and consider the pros and cons of various conspiracy evaluation guidelines” [ 47 ]. Thompson [ 48 ] reports on the activity of health professional influencers and pedagogues in combating misinformation. However, the effectiveness of such social media influencers who are also health professionals remains unclear. At the same time, there is some acknowledgment in this body of literature that misinformation cannot simply be offset with facts, confirming the challenges, discussed earlier, of simply engaging in online refutation. Addressing misinformation also depends on meeting patients’ emotional needs [ 45 , 49 ].

In this context, the one-to-one patient-provider relationship in the practice setting is perceived as paramount [ 45 ]. As suggested by much of the research, source credibility, or trust, is understood to be the strongest driver of effective correction strategies [ 50 ]. It is argued that health care practitioners have the unique opportunity to guide patients toward high-quality, evidence-based medical information [ 10 ]. However, it is also noted that practitioners will need patience in their efforts to persuade patients to abandon strongly held self-beliefs, however harmful. Doing so may mean patients relinquishing membership of online communities that have become integral in their lives and even their identities. As noted earlier, belief in misinformation is often persistent in the face of evidence. Success is more likely when individuals are encouraged to reexamine their information sources, alongside new information providing additional context, rather than simply characterizing the individual’s beliefs as wrong [ 51 ]. Kyabaggu et al [ 5 ] commented that good health communication needs to be tailored to the underlying cause of the misinformation problem, and efforts should be made to take on board inequalities within populations to create accurate, low-barrier, targeted health risk messaging. Skafle et al [ 24 ] contended that to challenge misconceptions, false claims need to be openly addressed and discussed with both cultural and religious awareness in mind. Guidance for practitioners noted that while responding to patient questions about alternative or unproven therapies may become laborious, a strong bond of trust between health practitioner and patient gives a patient a feeling of being supported and increases their adherence to treatment [ 52 ]. Rather than waiting for patients to raise misinformation issues, health care practitioners are advised to anticipate and proactively address potential misinformation and myths with patients. For example, the mortality rate for pediatric cancer has risen during the COVID-19 pandemic because of delayed access to medical care, but misinformation related to COVID-19 may also be a contributing factor [ 53 ]. The literature highlights the challenge of navigating the information and misinformation and the need for health practitioners to communicate with their patients more effectively. However, such efforts are not always successful. Some of the factors that may prevent effective communication of good health information have already been raised in this paper. They are revisited and discussed in the next section, along with other stressors for health practitioners.

Stressors for Health Practitioners

Challenges for health practitioners include time pressures and the additional burdens placed on them during the COVID-19 pandemic. These additional pressures add to the issues health practitioners face in trying to mitigate the impact of misinformation. The following is a brief overview of these issues.

On the one hand, administrative burdens placed on practitioners frequently deny them time for dialogue with their patients [ 52 ]. On the other, in different contexts, practitioners may be coping with a lack of proper facilities; poor infrastructure for patient care; insufficient or ineffective personal protective equipment; lack of awareness among the general population; poor compliance with preventive methods; and the fear of being infected with the virus, as they too are exposed to misinformation. During the COVID-19 pandemic, health practitioners were considered more vulnerable than other workers to developing psychological problems and other stress-related disorders, as they treated patients confirmed with COVID-19 while also dealing with misinformation [ 54 ].

As noted above, practitioners are recommended to invest in developing high levels of patient trust and to proactively correct health misinformation. However, recommendations presuppose that health practitioners necessarily have the resources to do these things well. Some of the materials produced to educate patients are not always reliable or evidence based, resulting ultimately in a loss of trust on the part of patients [ 52 ]. In addition, as noted previously, health practitioners themselves are not necessarily immune from accepting health misinformation as credible. Evidence about the level of knowledge and understanding of COVID-19 among practitioners reveals its unevenness. A study of dentists and oral health practitioners’ knowledge about COVID-19 suggested that their knowledge was at a relatively high level [ 55 ]. By contrast, a study of 310 eye care professionals in Nepal revealed some knowledge but also some acceptance of misinformation. Symptoms of COVID-19 were known to 94% of participants, but only 49% of participants were aware of how the disease is transmitted. More significantly, 41% of participants believed that the consumption of hot drinks helps to destroy the virus, in contradiction to World Health Organization information. The mean overall “knowledge” performance score, as measured by the benchmarks set by the researchers, was 69.65% [ 56 ].

A qualitative study to investigate primary health care practitioners’ perceptions and understanding of the COVID-19 pandemic was conducted in KwaZulu-Natal, South Africa. The study collected data from 15 participants at 2 different clinics situated in rural KwaZulu-Natal. Participants comprised nurses, physiotherapists, pharmacists, community caregivers, social workers, and clinical associates. Data were collected through individual, in-depth face-to-face interviews using a semistructured interview guide. The participants reported prepandemic and pandemic experiences of fear or denial. There was a perception of poor preparation for the COVID-19 outbreak. The findings also revealed participants’ misperceptions regarding the nature of the COVID-19 pandemic. Researchers concluded that respondents’ misunderstandings regarding the pandemic were primarily a result of misinformation found on social media [ 57 ].

The discussion in this section so far has highlighted the significant potential of health practitioners in mitigating the impact of online health misinformation. However, it has also underlined factors that may militate against health practitioners’ ability to do so effectively. Not least of these is the issue of health practitioners’ own knowledge, which coexists with other stressors for health practitioners in combating misinformation. The discussion will now consider health information management (HIM) as a tool for supporting health practitioners’ knowledge base as one element in a multifaceted strategy for combating misinformation on the web.

HIM as a Mitigation Strategy

We have seen there is a need for health practitioners to be supported with evidence-based knowledge that they can share with patients. Kyabaggu et al [ 5 ] argued that the COVID-19 pandemic has demonstrated that in an infectious health crisis, the gathering of accurate and reliable data to assist with the public health response is essential. They highlighted the importance of HIM professionals in supporting contact tracing and syndromic surveillance, as well as in mapping and forecasting health data. They noted that the generation of health information supports the continuum of care and the setting of targets and indicators and aids the planning, monitoring, and evaluation of health programs locally and globally. The health information produced also underpins the development of equitable, efficient, and accessible health care systems, contributing to improving public health initiatives and outcomes. Kyabaggu et al [ 5 ] emphasized the importance of an area of HIM, currently in its early stages, that deals with gathering and identifying evidence about the structural inequalities that underlie the disparities in vulnerability to health misinformation discussed in this paper. The collection of rich, high-quality information, including patient-reported experience, outcome measures, and culturally appropriate identity data, can enable health practitioners and public health advisers serving the most disadvantaged and underrepresented communities to use more tools of advocacy for patients.

The authors noted that advances in technology, including artificial intelligence, have the potential to relieve some of the pressures and constraints on health practitioners working on the front line during crises such as the COVID-19 pandemic, allowing more time for one-to-one engagement with patients. Kyabaggu et al [ 5 ] advocated for the content expertise of health information managers to serve health practitioners by delivering patient-facing information triaging services; constructing user-friendly knowledge representations, such as data visualizations; and developing information interpretation tools, such as decision aids, plain language summaries, and supplementary explanatory information and metadata. Kyabaggu et al [ 5 ] identified the interdisciplinary underpinnings of HIM as essential in contributing to the educational, informational, and decision-making support for addressing current and future infodemic management crises.

Summary of Results

Within the literature, there is a consensus that there exists a significant problem of online health misinformation disseminated via the internet on social network platforms, often by online health communities. It is apparent that while users seek trustworthy sources of health information, they are unequally equipped to assess its credibility. This is partly because some groups lack sufficient levels of health and digital literacies, which may be exacerbated by concomitant social and economic inequalities. Reception of, and response to, online health misinformation is also shaped by users’ cultural contexts, values, and experiences, which may hinder trust in scientific institutions and governments. Evidence suggests that some demographics are more vulnerable to accepting health misinformation as credible and that health practitioners are unevenly prepared in the context of new global health crises, such as the COVID-19 pandemic. Furthermore, the evidence of disparities in positive and negative attitudes toward vaccination highlights a need to pay specific attention to regional and national settings, even in the current global context. Preexisting levels of local trust in vaccine providers may be a significant factor to consider. While the validity and reliability of YouGov polls are limited, nevertheless, the data from an admittedly narrow range of sources suggests that vaccine confidence may have become more fluctuating and potentially vulnerable to destabilization in the digital era.

While online mitigation strategies such as user correction and automatic detection may have their uses, their effectiveness is contested, and some studies suggest they may even be counterproductive. Our analysis of the available literature indicates that the effectiveness of these strategies varies and needs further evaluation [ 42 , 58 ]. The issue of online health misinformation is further complicated by the operation of malicious actors and politicization of the issue, particularly during the COVID-19 pandemic, militating against the equitable and trusted dissemination of evidence-based knowledge. The role of health practitioners in this context is a challenging one. Research suggests that on the one hand, they are still best placed, at the front line of care, to combat health misinformation with science-based knowledge and advice. On the other hand, the stressors identified in this review create barriers to their abilities to do this well. Constraints of time and lack of supporting infrastructure add to the knowledge deficit noted earlier. Our review underlines the complexity of the environment in which health practitioners operate and calls for greater support and resources to enable effective mitigation of health misinformation [ 59 ]. Investment in HIM at local and global levels could address all 3 deficits, creating the potential for health practitioners to enhance their capacity to build trust via knowledgeable one-to-one communication with patients.

Limitations

The limitations of this study are the following: First, the constraints of time and space have necessarily limited the scale and scope of the survey. Second, the study of online health misinformation is a growing field, and inevitably, the nature of the issue means that new evidence is emerging at a rapid rate. In particular, new knowledge and further reflection in the wake of the COVID-19 pandemic will continue to shed new light on the subject. Our study acknowledges these limitations and emphasizes the dynamic nature of the field.

Conclusions

Our survey of the literature on online health misinformation has revealed a complex and multifaceted context in which health practitioners must operate. As the world renormalizes following the pandemic, a collaborative global interdisciplinary effort to provide equitable access to timely, accurate, and complete health information will be needed to support health practitioners in combating the impact of online health misinformation. Academic research will need to be disseminated into the public domain in a way that is accessible to the public to counter misinformation and educate populations concerning how science is carried out. Our conclusions drawn from this review stress the urgency of effective strategies and collaborative efforts to mitigate the prevalence and impact of health misinformation on a global scale. Without strategies for equipping populations with the health and digital literacies required to interpret and use information appropriately, the prevalence of online health misinformation will continue to pose a threat to global public health efforts, disproportionately affecting vulnerable and resource-limited populations. Although social media platforms have a responsibility to correct misinformation, governments will need to engage in evidence-informed decision-making and invest in HIM to support frontline health practitioners in their work, enhance population health literacy, and strengthen evidence-informed decision-making at all levels.

Several issues for further investigation arise from the findings of this review. These include the following:

• The long-term impact of COVID-19 vaccine hesitancy
• Whether the COVID-19 pandemic has intensified or diminished information literacy, and the related question of whether the pandemic will incentivize health information literacy
• The effects of social and cultural differences on the long-term traction of future health misinformation
• Whether social and economic inequalities will become less or more pronounced in the face of a global pandemic
• The comparative effectiveness of strategies to enhance populations’ media and digital literacies to facilitate the mitigation of health misinformation and its effects
• The influence of state actors on the propagation of health misinformation on the web
• The extent to which academic research has been disseminated into the public domain in a way that is accessible to the public, and the effectiveness of strategies to do so to counter misinformation and educate populations concerning how science is carried out

Acknowledgments

This research was funded by the School of Computing and Communications at the Open University. It allowed researchers across several faculties to collaborate and build a research team that focused on the experience of health practitioners with misinformation and its impact on their job practice. The authors would also like to thank Tracie Farrell and Nashwa Ismail for their invaluable suggestions and recommendations, as well as their assistance in the article screening process.

Data Availability

The data analyzed in this study are derived from published articles available on Google Scholar. All articles included in the review are cited in the reference list. No additional data or code were collected or generated as part of this study.

Authors' Contributions

The study was conceptualized by DK; funding acquisition was managed by DK; data were curated by DK, AK, MM, and IK; formal analysis was conducted by DK and MM; the investigation was carried out by AK and MM; the methodology was designed by DK and MM; project administration was overseen by DK; resources were provided by DK; supervision was carried out by DK; validation was conducted by DK, AK, MM, and IK; visualization was handled by DK and MM; writing (original draft preparation) was done by DK; and writing (review and editing) was carried out by DK, AK, and MM. All authors reviewed and approved the final version.

Conflicts of Interest

None declared.

Detailed search strategy.

• Ismail N, Kbaier D, Farrell T, Kane A. The experience of health professionals with misinformation and its impact on their job practice: qualitative interview study. JMIR Form Res. Nov 02, 2022;6(11):e38794. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tully M, Bode L, Vraga EK. Mobilizing users: does exposure to misinformation and its correction affect users’ responses to a health misinformation post? Soc Media Soc. Dec 10, 2020;6(4):205630512097837. [ CrossRef ]
• Zhao Y, Da J, Yan J. Detecting health misinformation in online health communities: Incorporating behavioral features into machine learning based approaches. Inf Process Manage. Jan 2021;58(1):102390. [ CrossRef ]
• Wang Y, McKee M, Torbica A, Stuckler D. Systematic literature review on the spread of health-related misinformation on social media. Soc Sci Med. Nov 2019;240:112552. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kyabaggu R, Marshall D, Ebuwei P, Ikenyei U. Health literacy, equity, and communication in the COVID-19 era of misinformation: emergence of health information professionals in infodemic management. JMIR Infodemiology. 2022;2(1):e35014. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sutton J. Health communication trolls and bots versus public health agencies' trusted voices. Am J Public Health. Oct 2018;108(10):1281-1282. [ CrossRef ] [ Medline ]
• Kim S, Capasso A, Ali SH, Headley T, DiClemente RJ, Tozan Y. What predicts people's belief in COVID-19 misinformation? A retrospective study using a nationwide online survey among adults residing in the United States. BMC Public Health. Nov 18, 2022;22(1):2114. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Morgan JC, Cappella JN. The effect of repetition on the perceived truth of tobacco-related health misinformation among U.S. adults. J Health Commun. Mar 04, 2023;28(3):182-189. [ CrossRef ] [ Medline ]
• Nan X, Wang Y, Thier K. Why do people believe health misinformation and who is at risk? A systematic review of individual differences in susceptibility to health misinformation. Soc Sci Med. Dec 2022;314:115398. [ CrossRef ] [ Medline ]
• Wood JL, Lee GY, Stinnett SS, Southwell BG. A pilot study of medical misinformation perceptions and training among practitioners in North Carolina (USA). Inquiry. 2021;58:469580211035742. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broniatowski DA, Jamison AM, Qi S, AlKulaib L, Chen T, Benton A, et al. Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health. Oct 2018;108(10):1378-1384. [ CrossRef ] [ Medline ]
• Mokhtari H, Mirzaei A. The tsunami of misinformation on COVID-19 challenged the health information literacy of the general public and the readability of educational material: a commentary. Public Health. Oct 2020;187:109-110. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Schmid P, Altay S, Scherer LD. The psychological impacts and message features of health misinformation - a systematic review of randomized controlled trials. Eur Psychol. Jul 2023;28(3):162-172. [ CrossRef ]
• Southwell BG, Otero Machuca J, Cherry ST, Burnside M, Barrett NJ. Health misinformation exposure and health disparities: observations and opportunities. Annu Rev Public Health. Apr 03, 2023;44(1):113-130. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Westberry C, Palmer XL, Potter L. Social media and health misinformation: a literature review. In: Arai K, editor. Proceedings of the Future Technologies Conference (FTC) 2023, Volume 3. FTC 2023. Lecture Notes in Networks and Systems, vol 815. Cham, Switzerland. Springer; 2023.
• Suarez-Lledo V, Alvarez-Galvez J. Prevalence of health misinformation on social media: systematic review. J Med Internet Res. Jan 20, 2021;23(1):e17187. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Borges do Nascimento IJ, Pizarro AB, Almeida J, Azzopardi-Muscat N, Gonçalves MA, Björklund M, et al. Infodemics and health misinformation: a systematic review of reviews. Bull World Health Organ. Sep 01, 2022;100(9):544-561. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lee JJ, Kang K, Wang MP, Zhao SZ, Wong JYH, O'Connor S, et al. Associations between COVID-19 misinformation exposure and belief with COVID-19 knowledge and preventive behaviors: cross-sectional online study. J Med Internet Res. Nov 13, 2020;22(11):e22205. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Larson HJ, Broniatowski DA. Why debunking misinformation is not enough to change people's minds about vaccines. Am J Public Health. Jun 2021;111(6):1058-1060. [ CrossRef ] [ Medline ]
• Bapaye JA, Bapaye HA. Demographic factors influencing the impact of coronavirus-related misinformation on WhatsApp: cross-sectional questionnaire study. JMIR Public Health Surveill. Jan 30, 2021;7(1):e19858. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McKernon E. Fake news and the public. Harper's. Oct 1925. URL: https://harpers.org/archive/1925/10/fake-news-and-the-public/ [accessed 2024-07-30]
• Hotez P. The physician-scientist: defending vaccines and combating antiscience. J Clin Invest. Apr 29, 2019;129(6):2169-2171. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Dyer C. Lancet retracts Wakefield's MMR paper. BMJ. Mar 02, 2010;340:c696. [ CrossRef ] [ Medline ]
• Skafle I, Nordahl-Hansen A, Quintana DS, Wynn R, Gabarron E. Misinformation about COVID-19 vaccines on social media: rapid review. J Med Internet Res. Aug 04, 2022;24(8):e37367. [ FREE Full text ] [ CrossRef ] [ Medline ]
• New Yahoo News/YouGov coronavirus poll: almost 1 in 5 say they won't get vaccinated. Yahoo News. URL: https:/​/www.​yahoo.com/​news/​new-yahoo-news-you-gov-coronavirus-poll-almost-one-in-five-say-they-wont-get-vaccinated-143852222.​html [accessed 2024-08-06]
• Peretti-Watel P, Verger P, Raude J, Constant A, Gautier A, Jestin C, et al. Dramatic change in public attitudes towards vaccination during the 2009 influenza A(H1N1) pandemic in France. Euro Surveill. Oct 31, 2013;18(44):20623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Reuters. ‘They’re killing people’: Biden slams Facebook for Covid disinformation. The Guardian. Jul 21, 2021. URL: https:/​/www.​theguardian.com/​media/​2021/​jul/​17/​theyre-killing-people-biden-slams-facebook-for-covid-misinformation [accessed 2024-07-30]
• Britain has had enough of experts, says Gove. Financial Times. URL: https://www.ft.com/content/3be49734-29cb-11e6-83e4-abc22d5d108c [accessed 2024-07-30]
• US news roundup: 19-25 August. Research Professional News. Aug 25, 2022. URL: https://www.researchprofessionalnews.com/rr-news-usa-2022-8-us-news-roundup-19-25-august/ [accessed 2024-07-30]
• Gruzd A, Mai P, Soares FB. How coordinated link sharing behavior and partisans' narrative framing fan the spread of COVID-19 misinformation and conspiracy theories. Soc Netw Anal Min. 2022;12(1):118. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vosoughi S, Roy D, Aral S. The spread of true and false news online. Science. Mar 09, 2018;359(6380):1146-1151. [ CrossRef ] [ Medline ]
• Wang Y. Systematic review on the social mechanism of health misinformation dissemination in the internet era. Eur J Public Health. Nov 2018;28(suppl_4):cky213.194. [ CrossRef ]
• Framework for information literacy for higher education. American Library Association. 2016. URL: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/infolit/framework1.pdf [accessed 2024-07-30]
• Quraishi Z. Addressing mental health, misinformation, and religious tensions among South Asian students across California higher education during the COVID-19 pandemic: A qualitative research study. Heliyon. Jun 2023;9(6):e16396. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhou J, Xiang H, Xie B. Better safe than sorry: a study on older adults' credibility judgments and spreading of health misinformation. Univers Access Inf Soc. Aug 04, 2022;22(3):1-10. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Goldenberg MJ. Vaccine Hesitancy: Public Trust, Expertise, and the War on Science. Pittsburgh, PA. University of Pittsburgh Press; 2021.
• Ittefaq M. "It frustrates me beyond words that I can't fix that": health misinformation correction on Facebook during COVID-19. Health Commun. Nov 12, 2023;12:1-11. [ CrossRef ] [ Medline ]
• Mourali M, Drake C. The challenge of debunking health misinformation in dynamic social media conversations: online randomized study of public masking during COVID-19. J Med Internet Res. Mar 02, 2022;24(3):e34831. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Weinzierl MA, Harabagiu SM. Automatic detection of COVID-19 vaccine misinformation with graph link prediction. J Biomed Inform. Dec 2021;124:103955. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Petty RE, Cacioppo JT. Elaboration likelihood model of persuasion. In: Communication and Persuasion. Springer Series in Social Psychology. New York, NY. Springer; 1986.
• Cowles K, Miller R, Suppok R. When seeing isn't believing: navigating visual health misinformation through library instruction. Med Ref Serv Q. 2024;43(1):44-58. [ CrossRef ] [ Medline ]
• Nabożny A, Balcerzak B, Morzy M, Wierzbicki A, Savov P, Warpechowski K. Improving medical experts' efficiency of misinformation detection: an exploratory study. World Wide Web. 2023;26(2):773-798. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barve Y, Saini JR. Detecting and classifying online health misinformation with 'Content Similarity Measure (CSM)' algorithm: an automated fact-checking-based approach. J Supercomput. 2023;79(8):9127-9156. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Broniatowski D, Gu J, Jamison AM, Simons JR. Facebook's architecture undermines vaccine misinformation removal efforts. arXiv. Preprint posted online Feb 4, 2022. [ FREE Full text ] [ CrossRef ]
• Russell N. Misinformation during COVID: how should nurse practitioners respond? J Nurse Pract. Jun 2021;17(6):763-764. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Wilner T, Holton A. Breast cancer prevention and treatment: misinformation on Pinterest, 2018. Am J Public Health. Oct 2020;110(S3):S300-S304. [ CrossRef ] [ Medline ]
• MacFarlane D, Hurlstone MJ, Ecker UK. Protecting consumers from fraudulent health claims: A taxonomy of psychological drivers, interventions, barriers, and treatments. Soc Sci Med. Aug 2020;259:112790. [ CrossRef ] [ Medline ]
• Thompson JD. Public health pedagogy and digital misinformation: health professional influencers and the politics of expertise. J Sociol. Sep 28, 2022;59(3):646-663. [ CrossRef ]
• Gunter J. Medical misinformation and the internet: a call to arms. Lancet. Jun 08, 2019;393(10188):2294-2295. [ CrossRef ] [ Medline ]
• Sui Y, Zhang B. Determinants of the perceived credibility of rebuttals concerning health misinformation. Int J Environ Res Public Health. Mar 02, 2021;18(3):1345. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Collier R. Containing health myths in the age of viral misinformation. CMAJ. May 14, 2018;190(19):E578. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peterson JS, Swire-Thompson B, Johnson SB. What is the alternative? Responding strategically to cancer misinformation. Future Oncol. Sep 2020;16(25):1883-1888. [ CrossRef ] [ Medline ]
• Guidry JP, Miller CA, Ksinan AJ, Rohan JM, Winter MA, Carlyle KE, et al. COVID-19-related misinformation among parents of patients with pediatric cancer. Emerg Infect Dis. Mar 2021;27(2):650-652. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ali S, Khalid A, Zahid E. Is COVID-19 immune to misinformation? A brief overview. Asian Bioeth Rev. Jun 2021;13(2):255-277. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jafari A, Mohammadpour M, Ghanbarzadegan A, Rossi-Fedele G, Bastani P. Oral health practitioners' knowledge, attitude, and awareness about coronavirus: A systematic review and meta-analysis. J Educ Health Promot. 2021;10:39. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sanyam SD, Sah SK, Chaudhary P, Burton MJ, Hoffman JJ. Knowledge and awareness-based survey of COVID-19 within the eye care profession in Nepal: Misinformation is hiding the truth. PLoS One. 2021;16(7):e0254761. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Nxumalo CT, Mchunu GG. A qualitative study to explore primary health care practitioners' perceptions and understanding regarding the COVID-19 pandemic in KwaZulu-Natal, South Africa. Afr J Prim Health Care Fam Med. Nov 26, 2021;13(1):e1-e11. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Abbasi K. Where now in the danse macabre of COVID-19 and misinformation? BMJ. Aug 17, 2023;382:1884. [ CrossRef ]
• Pesko MF, Cummings KM, Douglas CE, Foulds J, Miller T, Rigotti NA, et al. United States public health officials need to correct e-cigarette health misinformation. Addiction. May 2023;118(5):785-788. [ CrossRef ] [ Medline ]

Abbreviations

Edited by G Eysenbach, T Leung; submitted 15.04.22; peer-reviewed by G Nneji, S-F Tsao; comments to author 07.06.22; revised version received 29.09.22; accepted 12.07.24; published 19.08.24.

©Dhouha Kbaier, Annemarie Kane, Mark McJury, Ian Kenny. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 19.08.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Introduction
• Conclusions
• Article Information

LMIC indicates low- and- middle-income country; SR, systematic review.

a This review included distinct conclusions about separate conditions and comparators, and so it appears in this map more than once.

eAppendix 1. Search Strategies

eAppendix 2. Excluded Studies

eAppendix 3. Evidence Table

eAppendix 4. Conditions in Previously Published Map in 2018 and Current Map

eReferences.

Data Sharing Statement

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Mak S , Allen J , Begashaw M, et al. Use of Massage Therapy for Pain, 2018-2023 : A Systematic Review . JAMA Netw Open. 2024;7(7):e2422259. doi:10.1001/jamanetworkopen.2024.22259

Manage citations:

© 2024

• Permissions

Use of Massage Therapy for Pain, 2018-2023 : A Systematic Review

• 1 Veterans Health Administration, Greater Los Angeles Healthcare System, Los Angeles, California
• 2 UCLA Fielding School of Public Health, University of California, Los Angeles
• 3 RAND Corporation, Santa Monica, California

Question   What is the certainty or quality of evidence in recent systematic reviews for use of massage therapy for painful adult health conditions?

Findings   This systematic review identified 129 systematic reviews in a search of the literature published since 2018; of these, 41 assessed the certainty or quality of evidence of their conclusions. Overall, 17 systematic reviews regarding 13 health conditions were mapped, and most reviews concluded that the certainty of evidence was low or very low.

Meaning   This study found that despite massage therapy having been the subject of hundreds of randomized clinical trials and dozens of systematic reviews about adult health conditions since 2018, there were few conclusions that had greater than low certainty of evidence.

Importance   Massage therapy is a popular treatment that has been advocated for dozens of painful adult health conditions and has a large evidence base.

Objective   To map systematic reviews, conclusions, and certainty or quality of evidence for outcomes of massage therapy for painful adult health conditions.

Evidence Review   In this systematic review, a computerized search was conducted of PubMed, the Allied and Complementary Medicine Database, the Cumulated Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews, and Web of Science from 2018 to 2023. Included studies were systematic reviews of massage therapy for pain in adult health conditions that formally rated the certainty, quality, or strength of evidence for conclusions. Studies of sports massage therapy, osteopathy, dry cupping or dry needling, and internal massage therapy (eg, for pelvic floor pain) were ineligible, as were self-administered massage therapy techniques, such as foam rolling. Reviews were categorized as those with at least 1 conclusion rated as high-certainty evidence, at least 1 conclusion rated as moderate-certainty evidence, and all conclusions rated as low- or very low–certainty evidence; a full list of conclusions and certainty of evidence was collected.

Findings   A total of 129 systematic reviews of massage therapy for painful adult health conditions were found; of these, 41 reviews used a formal method to rate certainty or quality of evidence of their conclusions and 17 reviews were mapped, covering 13 health conditions. Across these reviews, no conclusions were rated as high certainty of evidence. There were 7 conclusions that were rated as moderate-certainty evidence; all remaining conclusions were rated as low- or very low–certainty evidence. All conclusions rated as moderate certainty were that massage therapy had a beneficial associations with pain.

Conclusions and Relevance   This study found that despite a large number of randomized clinical trials, systematic reviews of massage therapy for painful adult health conditions rated a minority of conclusions as moderate-certainty evidence and that conclusions with moderate- or high-certainty evidence that massage therapy was superior to other active therapies were rare.

Massage therapy is a popular and widely accepted complementary and integrative health modality for individuals seeking relief from pain. 1 This therapy is the practice of manual assessment and manipulation of the superficial soft tissues of skin, muscle, tendon, ligament, and fascia and the structures that lie within the superficial tissues for therapeutic purpose. 2 Individuals may seek massage therapy to address pain where conventional treatments may not always provide complete relief or may come with potential adverse effects. Massage therapy encompasses a range of techniques, styles, and durations and is intended to be delivered by uniquely trained and credentialed therapists. 3 Original research studies have reported on massage therapy delivered by a wide variety of health care professionals, such as physical therapists, physiotherapists, and nurses. 4 , 5 Despite massage therapy’s popularity and long history in practice, evidence of beneficial outcomes associated with massage therapy remains limited.

The Department of Veterans Affairs (VA) previously produced an evidence map of massage therapy for pain, which included systematic reviews published through 2018. 6 An evidence map is a form of systemic review that assesses a broad field to identify the state of the evidence, gaps in knowledge, and future research needs and that presents results in a user-friendly format, often a visual figure or graph. 7 To categorize this evidence base for use in decision-making by policymakers and practitioners, VA policymakers requested a new evidence map of reviews published since 2018 to answer the question “What is the certainty of evidence in systematic reviews of massage therapy for pain?”

This systematic review is an extension of a study commissioned by the VA. While not a full systematic review, this study nevertheless reports methods and results using the Preferred Reporting Items for Systematic Reviews and Meta-analyses ( PRISMA ) reporting guideline where applicable and filed the a priori protocol with the VA Evidence Synthesis Program Coordinating Center. Requirements for review and informed consent were waived because the study was designated as not human participants research.

Literature searches were based on searches used for the evidence map of massage therapy completed in 2018. 8 We searched 5 databases for relevant records published from July 2018 to April 2023 using the search terms “massage,” “acupressure,” “shiatsu,” “myofascial release therapy,” “systematic*,” “metaanaly*,” and similar terms. The databases were PubMed, the Allied and Complementary Medicine Database, the Cumulated Index to Nursing and Allied Health Literature, the Cochrane Database of Systematic Reviews, and Web of Science. See eAppendix 1 in Supplement 1 for full search strategies.

Each title was screened independently by 2 authors for relevance (S.M., J.A., and P.G.S.). Abstracts were then reviewed in duplicate, with any discrepancies resolved by group discussion. To be included, abstracts or titles needed to be about efficacy or effectiveness of massage therapy for a painful adult health condition and be a systematic review with more than 1 study about massage therapy. A systematic review was defined as a review that had a documented systematic method for identifying and critically appraising evidence. In general, any therapist-delivered modality described as massage therapy by review authors was considered eligible (eg, tuina, acupressure, auricular acupressure, reflexology, and myofascial release). Sports massage therapy, osteopathy, dry cupping or dry needling, and internal massage therapy (eg, for pelvic floor pain) were ineligible, as were self-administered massage therapy techniques, like foam rolling. Reviews had to be about a painful condition for adults, and we excluded publications in low- and middle-income countries because of differences in resources for usual care or other active treatments for included conditions. Publications were required to compare massage therapy with sham or placebo massage, usual care, or other active therapies. Systematic reviews that covered other interventions were eligible if results for massage therapy were reported separately.

We next restricted eligibility to reviews that used formal methods to assess the certainty (sometimes called strength or quality) of the evidence for conclusions. In general, this meant using Grading of Recommendations, Assessment, Development, and Evaluations (GRADE). 9 However, other formal methods were also included, such as the approach used by the US Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) program. To be included, a review had to state or cite the method used and report the certainty (or strength or quality) of evidence for each conclusion. After we applied this restriction, most health conditions had only 1 systematic review meeting the eligibility criteria, and we used this review for the map. Among conditions for which we identified more than 1 review meeting the eligibility criteria, we first assessed whether reviews differed in some other feature used to classify reviews on our map (eg, different comparators or type of massage therapy), which we would label with the appropriate designation (such as vs usual care or reflexology ). If there were multiple reviews about the same condition and they did not differ in some other feature, we selected the systematic review we judged as being most informative for readers. In general, this was the most recent review or the review with the greatest number of included studies.

Data on study condition, number of articles in a review, intervention characteristics, comparators, conclusions, and certainty, quality, or strength of evidence were extracted by 1 reviewer and then verified by a second reviewer (S.M., J.A., and P.G.S.). Our evidence mapping process produced a visual depiction of the evidence for massage therapy, as well as an accompanying narrative with an ancillary figure and table.

The visual depiction or evidence map uses a bubble plot format to display information on 4 dimensions: bubble size, bubble label, x-axis, and y-axis. This allowed us to provide the following types of information about each included systematic review:

Number of articles in systematic review (bubble size): The size of each bubble corresponds to the number of relevant primary research studies included in a systematic review.

Condition (bubble label): Each bubble is labeled with the condition discussed by that systematic review.

Shapes and colors: Intervention characteristics for each condition are presented in the form of colors (type of intervention) and shapes (comparators). For type of intervention, we included nonspecified massage therapy, tuina, myofascial release, reflexology, acupressure, and auricular acupressure. For comparators, we included mixed comparators with subgroups, mixed comparators with no subgroups, sham or placebo, and active therapy or usual care. A condition can appear more than once if multiple systematic reviews included different type of massage therapy or different comparators.

Strength of findings (rows): Each condition is plotted on the map based on the ratings of certainty of evidence statement as reported in the systematic reviews: high, moderate, low, or very low.

Outcome associated with massage therapy (columns): Each condition is plotted in potential benefit or no benefit as the outcome associated with massage therapy. Columns are not mutually exclusive. A review could have more than 1 conclusion, and conclusions could differ in the benefit associated with massage therapy. Both conclusions are included on the map.

Risk of bias is not part of the method of an evidence map. We assessed the quality of included reviews using criteria developed by the U S Preventive Services Task Force (USPSTF). Certainty of evidence as determined by the original authors of the systematic review was abstracted for each conclusion in each systematic review and tabulated.

The search identified 1164 potentially relevant citations. Among 129 full-text articles screened, 41 publications were retained for further review. Of these, 24 reviews were excluded from the map for the following reasons: only 1 primary study about interventions of interest (11 studies), outcomes associated with massage therapy could not be distinguished from other included interventions (5 studies), not an intervention of interest (3 studies), not a comparison of interest (2 studies), overlap with a more recent or larger review that was already included on the map (2 studies), and self-delivered therapy (1 study). We included 17 publications in this map covering 13 health conditions. 4 , 10 - 25 The literature flowchart ( Figure 1 ) summarizes results of the study selection process, and eAppendix 2 in Supplement 1 presents citations for all excluded reviews at full-text screening.

The total number of primary studies about massage therapy for pain in the included reviews ranged from 2 studies to 23 studies. There were 12 reviews that included fewer than 10 primary studies 4 , 11 - 17 , 20 - 23 and 5 reviews that included 10 to 25 studies about massage therapy for pain. 10 , 18 , 19 , 24 , 25 Of included reviews, 3 reviews were completed by the Cochrane Collaboration 4 , 19 , 23 and 2 reviews were completed by the AHRQ EPC program. 11 , 18

We categorized the included 17 reviews by health condition. These categories were cancer-related pain, 15 , 24 back pain (including chronic back pain, 25 chronic low back pain, 18 , 22 and low back pain 17 ), chronic neck pain, 18 fibromyalgia, 21 labor pain, 4 , 19 mechanical neck pain, 13 myofascial pain, 14 palliative care needs, 10 plantar fasciitis, 12 post–breast cancer surgery pain, 16 postcesarean pain, 23 postpartum pain, 20 and postoperative pain. 11

Of 17 included reviews, 3 reviews included more than 1 type of massage therapy and 14 reviews included 1 type of massage therapy. Reviews by Chou et al 11 and Smith et al 16 included acupressure and nonspecified massage therapy as interventions. The review by Candy et al 7 included reflexology and nonspecified massage therapy as interventions. Of the 14 reviews with 1 type of massage therapy, there were 5 reviews describing nonspecified massage therapy, 10 , 14 , 17 , 20 1 review about tuina, 22 5 reviews about myofascial release, 8 , 9 , 12 , 18 , 19 and 3 reviews about acupressure. 13 , 15 , 21

A variety of comparators were included in reviews. Of 9 reviews that included more than 1 comparator in analyses, 4 , 11 , 13 , 14 , 18 - 22 2 reviews did not conduct separate analyses by comparator (labeled mixed with no subgroups ) 13 , 14 and 3 reviews conducted separate analyses by comparator (labeled mixed with subgroups ). 4 , 21 , 22 The other 4 reviews included a mix of comparators with separate conclusions: sham or placebo and active therapy or usual care, 11 mixed with no subgroups and active therapy or usual care, 18 mixed with subgroups and active therapy or usual care, 20 and mixed with no subgroups, sham, and active therapy or usual care. 19 There were 8 reviews that included 1 comparator only in their analyses, 10 , 12 , 15 - 17 , 23 - 25 with 7 reviews that described interventions compared with active therapy or usual care only, 10 , 12 , 15 , 17 , 23 - 25 while 1 review limited inclusion to primary studies with a sham or placebo comparator. 16

There was substantial variation in the reporting of other details from primary studies in included reviews. Any study that did not specify the mode of delivery was included; studies that explicitly stated that massage therapy was self-delivered were excluded. Of the 17 included reviews, 5 reviews provided details of personnel who administered the therapy, including massage therapist, nurse, aromatherapist, physiotherapist, and reflexologist. 4 , 10 , 19 - 21 A total of 7 reviews presented length of sessions (eg, 5-minute or 90-minute sessions for massage therapy studies and 30-second or 5-minute sessions for acupressure studies). 10 , 16 , 18 , 20 - 23 With the exception of the review by He et al, 15 all reviews reported details about frequency, duration, or both when available. A total of 9 reviews included information about frequency of sessions (eg, 1 session or once every 3 weeks for massage therapy studies and 4 times per day or daily for acupressure studies), 10 , 12 , 16 - 18 , 20 - 23 and 9 reviews reported duration of sessions (eg, single session or 3 months). 10 - 12 , 16 - 18 , 20 , 22 , 23 There were 7 reviews that included details about follow-up (eg, 1 week or 12 months). 10 , 13 , 17 , 18 , 21 , 23 , 25

Using USPSTF criteria to rate the quality of included reviews, 10 reviews were rated good 4 , 10 , 11 , 14 - 16 , 18 , 19 , 21 , 23 and 7 reviews were rated fair. 12 , 13 , 17 , 20 , 22 , 24 , 25 See eAppendix 3 in Supplement 1 for each review’s rating.

Figure 2 is a visual depiction of the following types of information about each included systematic review: condition, types of comparison treatments (shapes), types of massage therapy (color), number of articles included for each conclusion (bubble size), outcomes associated with massage therapy for pain (columns), and certainty of evidence rating (rows). There were 6 reviews mapped more than once, reflecting primary studies describing more than 1 health condition, 18 more than 1 type of massage therapy, 10 , 20 or outcomes associated with massage therapy compared with different comparators. 11 , 17 - 19 There were 7 conditions from reviews 14 , 16 - 19 , 21 , 22 that reported 1 conclusion rated as moderate-certainty evidence, all of which concluded that massage therapy was associated with beneficial outcomes for pain ( Table 1 ). However, most other conditions had conclusions rated as low- or very low–certainty evidence (12 reviews about 10 conditions 4 , 10 - 13 , 15 , 17 - 20 , 23 - 25 ). This rating means “Our confidence in the effect estimate is limited. The true effect may be substantially different from the estimate of effect,” or “We have very little confidence in the effect estimate.” See eAppendix 3 in Supplement 1 for conclusions in all reviews. This map included 4 conditions that did not appear in the 2018 map, 12 , 16 , 20 , 23 and there were 8 conditions in the 2018 map that did not have new reviews meeting eligibility criteria (mainly a formal grading of the certainty of evidence); 7 health conditions 10 , 11 , 13 - 15 , 17 , 18 , 21 , 22 , 24 , 25 were included in the 2018 map and the new map (see details in eAppendix 4 in Supplement 1 ).

Evidence about adverse events was collected by approximately half of included reviews, and no serious adverse events were reported. While 11 of 17 reviews 10 , 11 , 13 , 15 , 17 - 19 , 22 - 25 described adverse events, 2 reviews 18 , 23 included certainty of evidence conclusions for adverse events for 3 health conditions ( Table 2 ).

There is a large literature of original randomized clinical trials and systematic reviews of randomized clinical trials of massage therapy as a treatment for pain. Our systematic review found that despite this literature, there were only a few conditions for which authors of systematic reviews concluded that there was at least moderate-certainty evidence regarding health outcomes associated with massage therapy and pain. Most reviews reported low- or very low–certainty evidence. Although adverse events associated with massage therapy for pain were rare, the evidence was limited. For reviews that had conclusions about adverse events, authors were uncertain if there was a difference between groups or did not find a difference between groups and rated the evidence low to very low certainty of evidence.

Massage therapy is a broad term that is inclusive of many styles and techniques. We applied exclusion criteria determined a priori to help identify publications for inclusion in the evidence map. Despite that procedure, there was still a lack of clarity in determining what massage therapy is. For instance, acupressure was sometimes considered acupuncture and other times considered massage therapy, depending on author definition. In this case, we reviewed and included only publications that were explicitly labeled acupressure and did not review publications about acupuncture only. This highlights a fundamental issue with examining the evidence base of massage therapy for pain when there is ambiguity in defining what is considered massage therapy.

Unlike a pharmaceutical placebo, sham massage therapy may not be truly inactive. It is conceivable that even the light touch or touch with no clear criterion 26 used in sham massage therapy may be associated with some positive outcomes, meaning that patients who receive the massage therapy intervention and those who receive a sham massage therapy could both demonstrate some degree of symptom improvement. Limitations of sham comparators raise the question of whether sham or placebo treatment is an appropriate comparison group in massage therapy trials. It may be more informative to compare massage therapy with other treatments that are accessible and whose benefits are known so that any added beneficial outcomes associated with massage therapy could be better isolated and understood.

Compared with the 2018 map, our map included 4 new conditions not on the 2018 map, while 8 conditions from the 2018 map had no new reviews meeting eligibility criteria and 7 health conditions appeared in both maps. Despite identifying new conditions and conclusions with higher certainty of evidence in several reviews in our updated search, most included reviews reported low or very low certainty of evidence, suggesting that the most critical research need is for better evidence to increase certainty of evidence for massage therapy for pain. This is a challenge given that massage, like other complementary and integrative health interventions, does not have the historical research infrastructure that most health professions have. 27 Nevertheless, it is only when systematic reviews and meta-analyses are conducted with high-quality primary studies that the association or lack of association of massage therapy with pain will reach higher certainties of evidence. Studies comparing massage therapy with placebo or sham are probably not the priority; rather, the priority should be studies comparing massage therapy with other recommended, accepted, and active therapies for pain. Studies comparing massage therapy with other recommended therapies should also have a sufficiently long follow-up to allow any nonspecific outcomes (eg, those associated with receiving some new treatment) to dissipate. For example, this period has been proposed to be at least 6 months for studies of chronic pain.

There are 2 main limitations to this systematic review’s evidence map. The first, common to all systematic reviews, is that we may not have identified all potentially eligible evidence. If a systematic review was published in a journal not indexed in any of 5 databases we searched and we did not identify it as part of our search of references of included publications, then we would have missed it. Nevertheless, our search strategy identified more than 200 publications about massage therapy for pain published since July 2018, so we did not lack potential reviews to evaluate. The second limitation of evidence maps is that we did not independently evaluate the source evidence; in other words, we took conclusions of authors of the systematic review at face value. That is the nature of an evidence map. Particular to this application of the mapping process, we mapped the review we deemed most informative for the 2 health conditions that had more than 1 eligible review (back pain and labor pain). This necessarily requires judgment, and others could disagree with that judgment. We included the citation for reviews excluded from the map for this overlap reason in supplemental material, and interested readers can review it for additional information. As in all evidence-based products and particularly in 1 such as this covering a large and complex evidence base, it is possible that there are errors of data extraction and compilation. We used dual review to minimize the chance of such errors, but if we are notified of errors, we will correct them.

Although this systematic review found that the number of conclusions about the effectiveness of massage therapy that were judged to have at least moderate certainty of evidence was greater now than in 2018, it was still small relative to the need. More high-quality randomized clinical trials are needed to provide a stronger evidence base to assess the effect of massage therapy on pain. For painful conditions that do not have at least moderate-certainty evidence supporting use of massage therapy, new studies that address limitations of existing research are needed. The field of massage therapy would be best advanced by educating the wider research community with clearer definitions of massage therapy and whether it is appropriate to include multiple modalities in the same systematic review.

Accepted for Publication: May 15, 2024.

Published: July 15, 2024. doi:10.1001/jamanetworkopen.2024.22259

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2024 Mak S et al. JAMA Network Open .

Corresponding Author: Selene Mak, PhD, MPH, Veterans Health Administration, Greater Los Angeles Healthcare System, 11301 Wilshire Blvd, Los Angeles, CA 90073 ( [email protected] ).

Author Contributions: Drs Mak and Shekelle had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Mak, Miake-Lye, Shekelle.

Acquisition, analysis, or interpretation of data: Mak, Allen, Begashaw, Beroes-Severin, De Vries, Lawson, Shekelle.

Drafting of the manuscript: Mak, Allen, Begashaw, Beroes-Severin, De Vries, Lawson, Shekelle.

Critical review of the manuscript for important intellectual content: Mak, Miake-Lye, Shekelle.

Statistical analysis: Allen.

Obtained funding: Shekelle.

Administrative, technical, or material support: Begashaw, Miake-Lye, Beroes-Severin, De Vries, Lawson.

Supervision: Mak, Shekelle.

Conflict of Interest Disclosures: None reported.

Funding/Support: Funding was provided by the Department of Veterans Affairs Health Services Research and Development.

Role of the Funder/Sponsor: The funders had no role in the collection, management, analysis, and interpretation of the data and preparation of the manuscript. The funders participated in the design and conduct of the study, the review and approval of the manuscript, and the decision to submit the manuscript for publication.

Data Sharing Statement: See Supplement 2 .

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Behind the Intent of Extract Method Refactoring: A Systematic Literature Review

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, view options, recommendations, uml model refactoring: a systematic literature review.

Model-driven engineering, an emerging trend in software engineering, has enabled the use of refactoring to UML models. Numerous approaches to model refactoring, problem detection, consistency management, and tool support to automate and verify ...

Automatic software refactoring: a systematic literature review

Refactoring a software artifact is an embedded task in the maintenance phase of the software life cycle. To reduce the time and effort required for this task, researchers proposed methods to automate the software refactoring process at the design ...

Proactive clone recommendation system for extract method refactoring

"Extract Method" refactoring is commonly used for merging code clones into a single new method. In this position paper, we propose a proactive clone recommendation system for "Extract Method" refactoring. The proposed system that has been implemented as ...

Information

Published in, publication history.

• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Share this publication link.

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

Does Migration for Domestic Work Reduce Poverty? A Review of the Literature and an Agenda for Research

Ids item types, copyright holder, external publisher.

• https://www.migratingoutofpoverty.org/files/file.php?name=wp-15---deshingkar-zeitlyn-holtom-2014-does-migration-for-domestic-work-reduce-poverty.pdf&site=354

Usage metrics