medical article review essay

• Search Menu

Sign in through your institution

• Advance Articles
• Editor's Choice
• CME Reviews
• Best of 2021 collection
• Abbreviated Breast MRI Virtual Collection
• Contrast-enhanced Mammography Collection
• Author Guidelines
• Submission Site
• Open Access
• Self-Archiving Policy
• Accepted Papers Resource Guide
• About Journal of Breast Imaging
• About the Society of Breast Imaging
• Guidelines for Reviewers
• Resources for Reviewers and Authors
• Editorial Board
• Advertising Disclaimer
• Advertising and Corporate Services
• Journals on Oxford Academic
• Books on Oxford Academic

• < Previous

A Step-by-Step Guide to Writing a Scientific Review Article

• Article contents
• Figures & tables
• Supplementary Data

Manisha Bahl, A Step-by-Step Guide to Writing a Scientific Review Article, Journal of Breast Imaging , Volume 5, Issue 4, July/August 2023, Pages 480–485, https://doi.org/10.1093/jbi/wbad028

• Permissions Icon Permissions

Scientific review articles are comprehensive, focused reviews of the scientific literature written by subject matter experts. The task of writing a scientific review article can seem overwhelming; however, it can be managed by using an organized approach and devoting sufficient time to the process. The process involves selecting a topic about which the authors are knowledgeable and enthusiastic, conducting a literature search and critical analysis of the literature, and writing the article, which is composed of an abstract, introduction, body, and conclusion, with accompanying tables and figures. This article, which focuses on the narrative or traditional literature review, is intended to serve as a guide with practical steps for new writers. Tips for success are also discussed, including selecting a focused topic, maintaining objectivity and balance while writing, avoiding tedious data presentation in a laundry list format, moving from descriptions of the literature to critical analysis, avoiding simplistic conclusions, and budgeting time for the overall process.

• narrative discourse

Society of Breast Imaging members

Personal account.

• Sign in with email/username & password
• Get email alerts
• Save searches
• Purchase content
• Activate your purchase/trial code
• Add your ORCID iD

Institutional access

Sign in with a library card.

• Sign in with username/password
• Recommend to your librarian
• Institutional account management
• Get help with access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institution’s website and Oxford Academic.

• Click Sign in through your institution.
• Select your institution from the list provided, which will take you to your institution's website to sign in.
• When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
• Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institution’s website, please contact your librarian or administrator.

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see ‘Sign in through society site’ in the sign in pane within a journal:

• Click Sign in through society site.
• When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

• View your signed in personal account and access account management features.
• View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Short-term Access

To purchase short-term access, please sign in to your personal account above.

Don't already have a personal account? Register

Email alerts

Citing articles via.

• Recommend to your Librarian
• Journals Career Network

Affiliations

• Online ISSN 2631-6129
• Print ISSN 2631-6110
• Copyright © 2024 Society of Breast Imaging
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Rights and permissions
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

How to write a good scientific review article

Affiliation.

• 1 The FEBS Journal Editorial Office, Cambridge, UK.
• PMID: 35792782
• DOI: 10.1111/febs.16565

Literature reviews are valuable resources for the scientific community. With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and highlights key gaps and challenges to address with future research. Writing a review article also helps to expand the writer's knowledge of their specialist area and to develop their analytical and communication skills, amongst other benefits. Thus, the importance of building review-writing into a scientific career cannot be overstated. In this instalment of The FEBS Journal's Words of Advice series, I provide detailed guidance on planning and writing an informative and engaging literature review.

© 2022 Federation of European Biochemical Societies.

PubMed Disclaimer

Similar articles

• Rules to be adopted for publishing a scientific paper. Picardi N. Picardi N. Ann Ital Chir. 2016;87:1-3. Ann Ital Chir. 2016. PMID: 28474609
• How to write an original article. Mateu Arrom L, Huguet J, Errando C, Breda A, Palou J. Mateu Arrom L, et al. Actas Urol Esp (Engl Ed). 2018 Nov;42(9):545-550. doi: 10.1016/j.acuro.2018.02.011. Epub 2018 May 18. Actas Urol Esp (Engl Ed). 2018. PMID: 29779648 Review. English, Spanish.
• [Writing a scientific review, advice and recommendations]. Turale S. Turale S. Soins. 2013 Dec;(781):39-43. Soins. 2013. PMID: 24558688 French.
• How to write a research paper. Alexandrov AV. Alexandrov AV. Cerebrovasc Dis. 2004;18(2):135-8. doi: 10.1159/000079266. Epub 2004 Jun 23. Cerebrovasc Dis. 2004. PMID: 15218279 Review.
• How to write a review article. Williamson RC. Williamson RC. Hosp Med. 2001 Dec;62(12):780-2. doi: 10.12968/hosp.2001.62.12.2389. Hosp Med. 2001. PMID: 11810740 Review.
• A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data. Connolly A, Kirwan M, Matthews A. Connolly A, et al. Int J Qual Health Care. 2024 May 10;36(2):mzae037. doi: 10.1093/intqhc/mzae037. Int J Qual Health Care. 2024. PMID: 38662407 Free PMC article. Review.
• Ado-tratuzumab emtansine beyond breast cancer: therapeutic role of targeting other HER2-positive cancers. Zheng Y, Zou J, Sun C, Peng F, Peng C. Zheng Y, et al. Front Mol Biosci. 2023 May 11;10:1165781. doi: 10.3389/fmolb.2023.1165781. eCollection 2023. Front Mol Biosci. 2023. PMID: 37251081 Free PMC article. Review.
• Connecting authors with readers: what makes a good review for the Korean Journal of Women Health Nursing. Kim HK. Kim HK. Korean J Women Health Nurs. 2023 Mar;29(1):1-4. doi: 10.4069/kjwhn.2023.02.23. Epub 2023 Mar 31. Korean J Women Health Nurs. 2023. PMID: 37037445 Free PMC article. No abstract available.
• Ketcham C, Crawford J. The impact of review articles. Lab Invest. 2007;87:1174-85. https://doi.org/10.1038/labinvest.3700688
• Muka T, Glisic M, Milic J, Verhoog S, Bohlius J, Bramer W, et al. A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research. Eur J Epidemiol. 2020;35:49-60. https://doi.org/10.1007/s10654-019-00576-5
• Tawfik GM, Dila KAS, Mohamed MYF, Tam DNH, Kien ND, Ahmed AM, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47:46. https://doi.org/10.1186/s41182-019-0165-6
• Zimba O, Gasparyan AY. Scientific authorship: a primer for researchers. Reumatologia. 2020;58(6):345-9. https://doi.org/10.5114/reum.2020.101999
• Gasparyan AY, Yessirkepov M, Voronov AA, Maksaev AA, Kitas GD. Article-level metrics. J Korean Med Sci. 2021;36(11):e74.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Website maintenance is scheduled for Saturday, September 7, and Sunday, September 8. Short disruptions may occur during these days.

JAY SIWEK, M.D., MARGARET L. GOURLAY, M.D., DAVID C. SLAWSON, M.D., AND ALLEN F. SHAUGHNESSY, PHARM.D.

Am Fam Physician. 2002;65(2):251-258

Traditional clinical review articles, also known as updates, differ from systematic reviews and meta-analyses. Updates selectively review the medical literature while discussing a topic broadly. Nonquantitative systematic reviews comprehensively examine the medical literature, seeking to identify and synthesize all relevant information to formulate the best approach to diagnosis or treatment. Meta-analyses (quantitative systematic reviews) seek to answer a focused clinical question, using rigorous statistical analysis of pooled research studies. This article presents guidelines for writing an evidence-based clinical review article for American Family Physician . First, the topic should be of common interest and relevance to family practice. Include a table of the continuing medical education objectives of the review. State how the literature search was done and include several sources of evidence-based reviews, such as the Cochrane Collaboration, BMJ's Clinical Evidence , or the InfoRetriever Web site. Where possible, use evidence based on clinical outcomes relating to morbidity, mortality, or quality of life, and studies of primary care populations. In articles submitted to American Family Physician , rate the level of evidence for key recommendations according to the following scale: level A (randomized controlled trial [RCT], meta-analysis); level B (other evidence); level C (consensus/expert opinion). Finally, provide a table of key summary points.

American Family Physician is particularly interested in receiving clinical review articles that follow an evidence-based format. Clinical review articles, also known as updates, differ from systematic reviews and meta-analyses in important ways. 1 Updates selectively review the medical literature while discussing a topic broadly. An example of such a topic is, “The diagnosis and treatment of myocardial ischemia.” Systematic reviews comprehensively examine the medical literature, seeking to identify and synthesize all relevant information to formulate the best approach to diagnosis or treatment. Examples are many of the systematic reviews of the Cochrane Collaboration or BMJ's Clinical Evidence compendium. Meta-analyses are a special type of systematic review. They use quantitative methods to analyze the literature and seek to answer a focused clinical question, using rigorous statistical analysis of pooled research studies. An example is, “Do beta blockers reduce mortality following myocardial infarction?”

The best clinical review articles base the discussion on existing systematic reviews and meta-analyses, and incorporate all relevant research findings about the management of a given disorder. Such evidence-based updates provide readers with powerful summaries and sound clinical guidance.

In this article, we present guidelines for writing an evidence-based clinical review article, especially one designed for continuing medical education (CME) and incorporating CME objectives into its format. This article may be read as a companion piece to a previous article and accompanying editorial about reading and evaluating clinical review articles. 1 , 2 Some articles may not be appropriate for an evidence-based format because of the nature of the topic, the slant of the article, a lack of sufficient supporting evidence, or other factors. We encourage authors to review the literature and, wherever possible, rate key points of evidence. This process will help emphasize the summary points of the article and strengthen its teaching value.

Topic Selection

Choose a common clinical problem and avoid topics that are rarities or unusual manifestations of disease or that have curiosity value only. Whenever possible, choose common problems for which there is new information about diagnosis or treatment. Emphasize new information that, if valid, should prompt a change in clinical practice, such as the recent evidence that spironolactone therapy improves survival in patients who have severe congestive heart failure. 3 Similarly, new evidence showing that a standard treatment is no longer helpful, but may be harmful, would also be important to report. For example, patching most traumatic corneal abrasions may actually cause more symptoms and delay healing compared with no patching. 4

Searching the Literature

When searching the literature on your topic, please consult several sources of evidence-based reviews ( Table 1 ) . Look for pertinent guidelines on the diagnosis, treatment, or prevention of the disorder being discussed. Incorporate all high-quality recommendations that are relevant to the topic. When reviewing the first draft, look for all key recommendations about diagnosis and, especially, treatment. Try to ensure that all recommendations are based on the highest level of evidence available. If you are not sure about the source or strength of the recommendation, return to the literature, seeking out the basis for the recommendation.

In particular, try to find the answer in an authoritative compendium of evidence-based reviews, or at least try to find a meta-analysis or well-designed randomized controlled trial (RCT) to support it. If none appears to be available, try to cite an authoritative consensus statement or clinical guideline, such as a National Institutes of Health Consensus Development Conference statement or a clinical guideline published by a major medical organization. If no strong evidence exists to support the conventional approach to managing a given clinical situation, point this out in the text, especially for key recommendations. Keep in mind that much of traditional medical practice has not yet undergone rigorous scientific study, and high-quality evidence may not exist to support conventional knowledge or practice.

Patient-Oriented vs. Disease-Oriented Evidence

With regard to types of evidence, Shaughnessy and Slawson 5 – 7 developed the concept of Patient-Oriented Evidence that Matters (POEM), in distinction to Disease-Oriented Evidence (DOE). POEM deals with outcomes of importance to patients, such as changes in morbidity, mortality, or quality of life. DOE deals with surrogate end points, such as changes in laboratory values or other measures of response. Although the results of DOE sometimes parallel the results of POEM, they do not always correspond ( Table 2 ) . 2 When possible, use POEM-type evidence rather than DOE. When DOE is the only guidance available, indicate that key clinical recommendations lack the support of outcomes evidence. Here is an example of how the latter situation might appear in the text: “Although prostate-specific antigen (PSA) testing identifies prostate cancer at an early stage, it has not yet been proved that PSA screening improves patient survival.” (Note: PSA testing is an example of DOE, a surrogate marker for the true outcomes of importance—improved survival, decreased morbidity, and improved quality of life.)

Evaluating the Literature

Evaluate the strength and validity of the literature that supports the discussion (see the following section, Levels of Evidence). Look for meta-analyses, high-quality, randomized clinical trials with important outcomes (POEM), or well-designed, nonrandomized clinical trials, clinical cohort studies, or case-controlled studies with consistent findings. In some cases, high-quality, historical, uncontrolled studies are appropriate (e.g., the evidence supporting the efficacy of Papanicolaou smear screening). Avoid anecdotal reports or repeating the hearsay of conventional wisdom, which may not stand up to the scrutiny of scientific study (e.g., prescribing prolonged bed rest for low back pain).

Look for studies that describe patient populations that are likely to be seen in primary care rather than subspecialty referral populations. Shaughnessy and Slawson's guide for writers of clinical review articles includes a section on information and validity traps to avoid. 2

Levels of Evidence

Readers need to know the strength of the evidence supporting the key clinical recommendations on diagnosis and treatment. Many different rating systems of varying complexity and clinical relevance are described in the medical literature. Recently, the third U.S. Preventive Services Task Force (USPSTF) emphasized the importance of rating not only the study type (RCT, cohort study, case-control study, etc.), but also the study quality as measured by internal validity and the quality of the entire body of evidence on a topic. 8

While it is important to appreciate these evolving concepts, we find that a simplified grading system is more useful in AFP . We have adopted the following convention, using an ABC rating scale. Criteria for high-quality studies are discussed in several sources. 8 , 9 See the AFP Web site ( www.aafp.org/afp/authors ) for additional information about levels of evidence and see the accompanying editorial in this issue discussing the potential pitfalls and limitations of any rating system.

Level A (randomized controlled trial/meta-analysis): High-quality randomized controlled trial (RCT) that considers all important outcomes. High-quality meta-analysis (quantitative systematic review) using comprehensive search strategies.

Level B (other evidence): A well-designed, nonrandomized clinical trial. A nonquantitative systematic review with appropriate search strategies and well-substantiated conclusions. Includes lower quality RCTs, clinical cohort studies, and case-controlled studies with non-biased selection of study participants and consistent findings. Other evidence, such as high-quality, historical, uncontrolled studies, or well-designed epidemiologic studies with compelling findings, is also included.

Level C (consensus/expert opinion): Consensus viewpoint or expert opinion.

Each rating is applied to a single reference in the article, not to the entire body of evidence that exists on a topic. Each label should include the letter rating (A, B, C), followed by the specific type of study for that reference. For example, following a level B rating, include one of these descriptors: (1) nonrandomized clinical trial; (2) nonquantitative systematic review; (3) lower quality RCT; (4) clinical cohort study; (5) case-controlled study; (6) historical uncontrolled study; (7) epidemiologic study.

Here are some examples of the way evidence ratings should appear in the text:

“To improve morbidity and mortality, most patients in congestive heart failure should be treated with an angiotensin-converting enzyme inhibitor. [Evidence level A, RCT]”

“The USPSTF recommends that clinicians routinely screen asymptomatic pregnant women 25 years and younger for chlamydial infection. [Evidence level B, non-randomized clinical trial]”

“The American Diabetes Association recommends screening for diabetes every three years in all patients at high risk of the disease, including all adults 45 years and older. [Evidence level C, expert opinion]”

When scientifically strong evidence does not exist to support a given clinical recommendation, you can point this out in the following way:

“Physical therapy is traditionally prescribed for the treatment of adhesive capsulitis (frozen shoulder), although there are no randomized outcomes studies of this approach.”

Format of the Review

Introduction.

The introduction should define the topic and purpose of the review and describe its relevance to family practice. The traditional way of doing this is to discuss the epidemiology of the condition, stating how many people have it at one point in time (prevalence) or what percentage of the population is expected to develop it over a given period of time (incidence). A more engaging way of doing this is to indicate how often a typical family physician is likely to encounter this problem during a week, month, year, or career. Emphasize the key CME objectives of the review and summarize them in a separate table entitled “CME Objectives.”

The methods section should briefly indicate how the literature search was conducted and what major sources of evidence were used. Ideally, indicate what predetermined criteria were used to include or exclude studies (e.g., studies had to be independently rated as being high quality by an established evaluation process, such as the Cochrane Collaboration). Be comprehensive in trying to identify all major relevant research. Critically evaluate the quality of research reviewed. Avoid selective referencing of only information that supports your conclusions. If there is controversy on a topic, address the full scope of the controversy.

The discussion can then follow the typical format of a clinical review article. It should touch on one or more of the following subtopics: etiology, pathophysiology, clinical presentation (signs and symptoms), diagnostic evaluation (history, physical examination, laboratory evaluation, and diagnostic imaging), differential diagnosis, treatment (goals, medical/surgical therapy, laboratory testing, patient education, and follow-up), prognosis, prevention, and future directions.

The review will be comprehensive and balanced if it acknowledges controversies, unresolved questions, recent developments, other viewpoints, and any apparent conflicts of interest or instances of bias that might affect the strength of the evidence presented. Emphasize an evidence-supported approach or, where little evidence exists, a consensus viewpoint. In the absence of a consensus viewpoint, you may describe generally accepted practices or discuss one or more reasoned approaches, but acknowledge that solid support for these recommendations is lacking.

In some cases, cost-effectiveness analyses may be important in deciding how to implement health care services, especially preventive services. 10 When relevant, mention high-quality cost-effectiveness analyses to help clarify the costs and health benefits associated with alternative interventions to achieve a given health outcome. Highlight key points about diagnosis and treatment in the discussion and include a summary table of the key take-home points. These points are not necessarily the same as the key recommendations, whose level of evidence is rated, although some of them will be.

Use tables, figures, and illustrations to highlight key points, and present a step-wise, algorithmic approach to diagnosis or treatment when possible.

Rate the evidence for key statements, especially treatment recommendations. We expect that most articles will have at most two to four key statements; some will have none. Rate only those statements that have corresponding references and base the rating on the quality and level of evidence presented in the supporting citations. Use primary sources (original research, RCTs, meta-analyses, and systematic reviews) as the basis for determining the level of evidence. In other words, the supporting citation should be a primary research source of the information, not a secondary source (such as a nonsystematic review article or a textbook) that simply cites the original source. Systematic reviews that analyze multiple RCTs are good sources for determining ratings of evidence.

The references should include the most current and important sources of support for key statements (i.e., studies referred to, new information, controversial material, specific quantitative data, and information that would not usually be found in most general reference textbooks). Generally, these references will be key evidence-based recommendations, meta-analyses, or landmark articles. Although some journals publish exhaustive lists of reference citations, AFP prefers to include a succinct list of key references. (We will make more extensive reference lists available on our Web site or provide links to your personal reference list.)

You may use the following checklist to ensure the completeness of your evidence-based review article; use the source list of reviews to identify important sources of evidence-based medicine materials.

Checklist for an Evidence-Based Clinical Review Article

The topic is common in family practice, especially topics in which there is new, important information about diagnosis or treatment.

The introduction defines the topic and the purpose of the review, and describes its relevance to family practice.

A table of CME objectives for the review is included.

The review states how you did your literature search and indicates what sources you checked to ensure a comprehensive assessment of relevant studies (e.g., MEDLINE, the Cochrane Collaboration Database, the Center for Research Support, TRIP Database).

Several sources of evidence-based reviews on the topic are evaluated ( Table 1 ) .

Where possible, POEM (dealing with changes in morbidity, mortality, or quality of life) rather than DOE (dealing with mechanistic explanations or surrogate end points, such as changes in laboratory tests) is used to support key clinical recommendations ( Table 2 ) .

Studies of patients likely to be representative of those in primary care practices, rather than subspecialty referral centers, are emphasized.

Studies that are not only statistically significant but also clinically significant are emphasized; e.g., interventions with meaningful changes in absolute risk reduction and low numbers needed to treat. (See http://www.cebm.net/index.aspx?o=1116 .) 11

The level of evidence for key clinical recommendations is labeled using the following rating scale: level A (RCT/meta-analysis), level B (other evidence), and level C (consensus/expert opinion).

Acknowledge controversies, recent developments, other viewpoints, and any apparent conflicts of interest or instances of bias that might affect the strength of the evidence presented.

Highlight key points about diagnosis and treatment in the discussion and include a summary table of key take-home points.

Use tables, figures, and illustrations to highlight key points and present a step-wise, algorithmic approach to diagnosis or treatment when possible.

Emphasize evidence-based guidelines and primary research studies, rather than other review articles, unless they are systematic reviews.

The essential elements of this checklist are summarized in Table 3 .

Siwek J. Reading and evaluating clinical review articles. Am Fam Physician. 1997;55:2064-2069.

Shaughnessy AF, Slawson DC. Getting the most from review articles: a guide for readers and writers. Am Fam Physician. 1997;55:2155-60.

Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709-17.

Flynn CA, D'Amico F, Smith G. Should we patch corneal abrasions? A meta-analysis. J Fam Pract. 1998;47:264-70.

Slawson DC, Shaughnessy AF, Bennett JH. Becoming a medical information master: feeling good about not knowing everything. J Fam Pract. 1994;38:505-13.

Shaughnessy AF, Slawson DC, Bennett JH. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract. 1994;39:489-99.

Slawson DC, Shaughnessy AF. Becoming an information master: using POEMs to change practice with confidence. Patient-oriented evidence that matters. J Fam Pract. 2000;49:63-7.

Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, et al. Methods Work Group, Third U.S. Preventive Services Task Force. Current methods of the U.S. Preventive Services Task Force. A review of the process. Am J Prev Med. 2001;20(3 suppl):21-35.

CATbank topics: levels of evidence and grades of recommendations. Retrieved November 2001, from: http://www.cebm.net/ .

Saha S, Hoerger TJ, Pignone MP, Teutsch SM, Helfand M, Mandelblatt JS. for the Cost Work Group of the Third U.S. Preventive Services Task Force. The art and science of incorporating cost effectiveness into evidence-based recommendations for clinical preventive services. Am J Prev Med. 2001;20(3 suppl):36-43.

Evidence-based medicine glossary. Retrieved November 2001, from: http://www.cebm.net/index.aspx?o=1116 .

Continue Reading

More in afp, more in pubmed.

Copyright © 2002 by the American Academy of Family Physicians.

This content is owned by the AAFP. A person viewing it online may make one printout of the material and may use that printout only for his or her personal, non-commercial reference. This material may not otherwise be downloaded, copied, printed, stored, transmitted or reproduced in any medium, whether now known or later invented, except as authorized in writing by the AAFP.  See permissions  for copyright questions and/or permission requests.

Copyright © 2024 American Academy of Family Physicians. All Rights Reserved.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

Massive Attack’s science-led drive to lower music’s carbon footprint

Career Feature 04 SEP 24

Tales of a migratory marine biologist

Career Feature 28 AUG 24

Nail your tech-industry interviews with these six techniques

Career Column 28 AUG 24

Why I’m committed to breaking the bias in large language models

Career Guide 04 SEP 24

Binning out-of-date chemicals? Somebody think about the carbon!

Correspondence 27 AUG 24

No more hunting for replication studies: crowdsourced database makes them easy to find

Nature Index 27 AUG 24

Publishing nightmare: a researcher’s quest to keep his own work from being plagiarized

News 04 SEP 24

Intellectual property and data privacy: the hidden risks of AI

How can I publish open access when I can’t afford the fees?

Career Feature 02 SEP 24

Director, Clinical Translational Program, Center for Alzheimer’s and Related Dementias

The Intramural Research Programs (IRPs) of the National Institute on Aging (NIA)

Bethesda, Maryland (US)

National Institutes of Health

Assistant Professor - Plant Single Cell Biology

The Department of Cell & Systems Biology in the Faculty of Arts & Science at the University of Toronto invites applications for a full-time tenure ...

Toronto (City), Ontario (CA)

University of Toronto

Assistant Professor, Tenure Track

Scripps Research Institute is seeking outstanding applicants for a tenure track Assistant Professor position in La Jolla, CA.

La Jolla, California

Scripps Research Institute

Faculty Recruitment, Westlake University School of Medicine

Faculty positions are open at four distinct ranks: Assistant Professor, Associate Professor, Full Professor, and Chair Professor.

Hangzhou, Zhejiang, China

Westlake University

Postdoctoral Researcher - Neural Circuits Genetics and Physiology for Learning and Memory

A postdoctoral position is available to study molecular mechanisms, neural circuits and neurophysiology of learning and memory.

Dallas, Texas (US)

The University of Texas Southwestern Medical Center

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• YJBM Updates
• Editorial Board
• Colloquium Series
• Podcast Series
• Open Access
• Call for Papers
• PubMed Publications
• News & Views
• Why Publish in YJBM?
• Types of Articles Published
• Manuscript Submission Guidelines
• YJBM Ethical Guidelines

Points to Consider When Reviewing Articles

• Writing and Submitting a Review
• Join Our Peer Reviewer Database

INFORMATION FOR

• Residents & Fellows
• Researchers

General questions that Reviewers should keep in mind when reviewing articles are the following:

• Is the article of interest to the readers of YJBM ?
• What are the strengths and weaknesses of the manuscript?
• How can the Editors work with the Authors to improve the submitted manuscripts, if the topic and scope of the manuscript is of interest to YJBM readers?

The following contains detailed descriptions as to what should be included in each particular type of article as well as points that Reviewers should keep in mind when specifically reviewing each type of article.

YJBM will ask Reviewers to Peer Review the following types of submissions:

Download PDF

Frequently asked questions.

These manuscripts should present well-rounded studies reporting innovative advances that further knowledge about a topic of importance to the fields of biology or medicine. The conclusions of the Original Research Article should clearly be supported by the results. These can be submitted as either a full-length article (no more than 6,000 words, 8 figures, and 4 tables) or a brief communication (no more than 2,500 words, 3 figures, and 2 tables). Original Research Articles contain five sections: abstract, introduction, materials and methods, results and discussion.

Reviewers should consider the following questions:

• What is the overall aim of the research being presented? Is this clearly stated?
• Have the Authors clearly stated what they have identified in their research?
• Are the aims of the manuscript and the results of the data clearly and concisely stated in the abstract?
• Does the introduction provide sufficient background information to enable readers to better understand the problem being identified by the Authors?
• Have the Authors provided sufficient evidence for the claims they are making? If not, what further experiments or data needs to be included?
• Are similar claims published elsewhere? Have the Authors acknowledged these other publications? Have the Authors made it clear how the data presented in the Author’s manuscript is different or builds upon previously published data?
• Is the data presented of high quality and has it been analyzed correctly? If the analysis is incorrect, what should the Authors do to correct this?
• Do all the figures and tables help the reader better understand the manuscript? If not, which figures or tables should be removed and should anything be presented in their place?
• Is the methodology used presented in a clear and concise manner so that someone else can repeat the same experiments? If not, what further information needs to be provided?
• Do the conclusions match the data being presented?
• Have the Authors discussed the implications of their research in the discussion? Have they presented a balanced survey of the literature and information so their data is put into context?
• Is the manuscript accessible to readers who are not familiar with the topic? If not, what further information should the Authors include to improve the accessibility of their manuscript?
• Are all abbreviations used explained? Does the author use standard scientific abbreviations?

Case reports describe an unusual disease presentation, a new treatment, an unexpected drug interaction, a new diagnostic method, or a difficult diagnosis. Case reports should include relevant positive and negative findings from history, examination and investigation, and can include clinical photographs. Additionally, the Author must make it clear what the case adds to the field of medicine and include an up-to-date review of all previous cases. These articles should be no more than 5,000 words, with no more than 6 figures and 3 tables. Case Reports contain five sections: abstract; introduction; case presentation that includes clinical presentation, observations, test results, and accompanying figures; discussion; and conclusions.

• Does the abstract clearly and concisely state the aim of the case report, the findings of the report, and its implications?
• Does the introduction provide enough details for readers who are not familiar with a particular disease/treatment/drug/diagnostic method to make the report accessible to them?
• Does the manuscript clearly state what the case presentation is and what was observed so that someone can use this description to identify similar symptoms or presentations in another patient?
• Are the figures and tables presented clearly explained and annotated? Do they provide useful information to the reader or can specific figures/tables be omitted and/or replaced by another figure/table?
• Are the data presented accurately analyzed and reported in the text? If not, how can the Author improve on this?
• Do the conclusions match the data presented?
• Does the discussion include information of similar case reports and how this current report will help with treatment of a disease/presentation/use of a particular drug?

Reviews provide a reasoned survey and examination of a particular subject of research in biology or medicine. These can be submitted as a mini-review (less than 2,500 words, 3 figures, and 1 table) or a long review (no more than 6,000 words, 6 figures, and 3 tables). They should include critical assessment of the works cited, explanations of conflicts in the literature, and analysis of the field. The conclusion must discuss in detail the limitations of current knowledge, future directions to be pursued in research, and the overall importance of the topic in medicine or biology. Reviews contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Is the review accessible to readers of YJBM who are not familiar with the topic presented?
• Does the abstract accurately summarize the contents of the review?
• Does the introduction clearly state what the focus of the review will be?
• Are the facts reported in the review accurate?
• Does the Author use the most recent literature available to put together this review?
• Is the review split up under relevant subheadings to make it easier for the readers to access the article?
• Does the Author provide balanced viewpoints on a specific topic if there is debate over the topic in the literature?
• Are the figures or tables included relevant to the review and enable the readers to better understand the manuscript? Are there further figures/tables that could be included?
• Do the conclusions and outlooks outline where further research can be done on the topic?

Perspectives provide a personal view on medical or biomedical topics in a clear narrative voice. Articles can relate personal experiences, historical perspective, or profile people or topics important to medicine and biology. Long perspectives should be no more than 6,000 words and contain no more than 2 tables. Brief opinion pieces should be no more than 2,500 words and contain no more than 2 tables. Perspectives contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately and concisely summarize the main points provided in the manuscript?
• Does the introduction provide enough information so that the reader can understand the article if he or she were not familiar with the topic?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the author has provided too much detail that detracts from the main point?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the Author break up the article under specific topics?
• Do the conclusions follow from the information provided by the Author?
• Does the Author reflect and provide lessons learned from a specific personal experience/historical event/work of a specific person?

Analyses provide an in-depth prospective and informed analysis of a policy, major advance, or historical description of a topic related to biology or medicine. These articles should be no more than 6,000 words with no more than 3 figures and 1 table. Analyses contain four sections: abstract, introduction, topics (with headings and subheadings), and conclusions and outlook.

• Does the abstract accurately summarize the contents of the manuscript?
• Does the introduction provide enough information if the readers are not familiar with the topic being addressed?
• Are there specific areas in which the Author can provide more detail to help the reader better understand the manuscript? Or are there places where the Author has provided too much detail that detracts from the main point?

Profiles describe a notable person in the fields of science or medicine. These articles should contextualize the individual’s contributions to the field at large as well as provide some personal and historical background on the person being described. More specifically, this should be done by describing what was known at the time of the individual’s discovery/contribution and how that finding contributes to the field as it stands today. These pieces should be no more than 5,000 words, with up to 6 figures, and 3 tables. The article should include the following: abstract, introduction, topics (with headings and subheadings), and conclusions.

• Does the Author provide information about the person of interest’s background, i.e., where they are from, where they were educated, etc.?
• Does the Author indicate how the person focused on became interested or involved in the subject that he or she became famous for?
• Does the Author provide information on other people who may have helped the person in his or her achievements?
• Does the Author provide information on the history of the topic before the person became involved?
• Does the Author provide information on how the person’s findings affected the field being discussed?
• Does the introduction provide enough information to the readers, should they not be familiar with the topic being addressed?

Interviews may be presented as either a transcript of an interview with questions and answers or as a personal reflection. If the latter, the Author must indicate that the article is based on an interview given. These pieces should be no more than 5,000 words and contain no more than 3 figures and 2 tables. The articles should include: abstract, introduction, questions and answers clearly indicated by subheadings or topics (with heading and subheadings), and conclusions.

• Does the Author provide relevant information to describe who the person is whom they have chosen to interview?
• Does the Author explain why he or she has chosen the person being interviewed?
• Does the Author explain why he or she has decided to focus on a specific topic in the interview?
• Are the questions relevant? Are there more questions that the Author should have asked? Are there questions that the Author has asked that are not necessary?
• If necessary, does the Author divide the article into specific topics to help the reader better access the article? If not, how should the author break up the article under specific topics?
• Does the Author accurately summarize the contents of the interview as well as specific lesson learned, if relevant, in the conclusions?

Page Content

Overview of the review report format, the first read-through, first read considerations, spotting potential major flaws, concluding the first reading, rejection after the first reading, before starting the second read-through, doing the second read-through, the second read-through: section by section guidance, how to structure your report, on presentation and style, criticisms & confidential comments to editors, the recommendation, when recommending rejection, additional resources, step by step guide to reviewing a manuscript.

When you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. Try to respond to invitations promptly - it will prevent delays. It is also important at this stage to declare any potential Conflict of Interest.

The structure of the review report varies between journals. Some follow an informal structure, while others have a more formal approach.

" Number your comments!!! " (Jonathon Halbesleben, former Editor of Journal of Occupational and Organizational Psychology)

Informal Structure

Many journals don't provide criteria for reviews beyond asking for your 'analysis of merits'. In this case, you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, as you gain experience, rely on your own evolving style.

Formal Structure

Other journals require a more formal approach. Sometimes they will ask you to address specific questions in your review via a questionnaire. Or they might want you to rate the manuscript on various attributes using a scorecard. Often you can't see these until you log in to submit your review. So when you agree to the work, it's worth checking for any journal-specific guidelines and requirements. If there are formal guidelines, let them direct the structure of your review.

In Both Cases

Whether specifically required by the reporting format or not, you should expect to compile comments to authors and possibly confidential ones to editors only.

Following the invitation to review, when you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you need to feedback on how to improve those sections.

The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

Keep a pen and paper handy when skim-reading.

Try to bear in mind the following questions - they'll help you form your overall impression:

• What is the main question addressed by the research? Is it relevant and interesting?
• How original is the topic? What does it add to the subject area compared with other published material?
• Is the paper well written? Is the text clear and easy to read?
• Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed?
• If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible?
• If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, " Specific recommendations for remedying flaws are VERY welcome ."

Examples of possibly major flaws include:

• Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence
• The use of a discredited method
• Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

• The sampling in analytical papers
• The sufficient use of control experiments
• The precision of process data
• The regularity of sampling in time-dependent studies
• The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
• That qualitative research extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. If there are critical flaws in this, it's very likely the manuscript will need to be rejected. Such issues include:

• Insufficient data
• Unclear data tables
• Contradictory data that either are not self-consistent or disagree with the conclusions
• Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

• Help the editor properly contextualize the research and add weight to your judgement
• Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
• Focus on successful aspects of the paper so the author gets a sense of what they've done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

• Is the paper's premise interesting and important?
• Are the methods used appropriate?
• Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section "When recommending rejection."

Once the paper has passed your first read and you've decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to recommend rejection following a second reading.

" Offer clear suggestions for how the authors can address the concerns raised. In other words, if you're going to raise a problem, provide a solution ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

• Don't rely solely upon inserting comments on the manuscript document - make separate notes
• Try to group similar concerns or praise together
• If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later
• Note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

Now that you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

As you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

• Any places where the meaning is unclear or ambiguous
• Any factual errors
• Any invalid arguments

You may also wish to consider:

• Does the title properly reflect the subject of the paper?
• Does the abstract provide an accessible summary of the paper?
• Do the keywords accurately reflect the content?
• Is the paper an appropriate length?
• Are the key messages short, accurate and clear?

Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, " If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited ."

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

• Are there certain aspects that could be communicated better, such as parts of the discussion?
• Should the authors consider resubmitting to the same journal after language improvements?
• Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don't spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it's important to highlight these. Expect to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

1. The Introduction

A well-written introduction:

• Sets out the argument
• Summarizes recent research related to the topic
• Highlights gaps in current understanding or conflicts in current knowledge
• Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
• Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

Originality and Topicality

Originality and topicality can only be established in the light of recent authoritative research. For example, it's impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

Authors may make the case that a topic hasn't been investigated in several years and that new research is required. This point is only valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Clearly, authors can only do this by referencing recent literature. Obviously, where older research is seminal or where aspects of the methodology rely upon it, then it is perfectly appropriate for authors to cite some older papers.

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes ?"

It's common for the introduction to end by stating the research aims. By this point you should already have a good impression of them - if the explicit aims come as a surprise, then the introduction needs improvement.

2. Materials and Methods

Academic research should be replicable, repeatable and robust - and follow best practice.

Replicable Research

This makes sufficient use of:

• Control experiments
• Repeated analyses
• Repeated experiments

These are used to make sure observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. Statistical analyses will not be sound if methods are not replicable. Where research is not replicable, the paper should be recommended for rejection.

Repeatable Methods

These give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.

Robust Research

This has enough data points to make sure the data are reliable. If there are insufficient data, it might be appropriate to recommend revision. You should also consider whether there is any in-built bias not nullified by the control experiments.

Best Practice

During these checks you should keep in mind best practice:

• Standard guidelines were followed (e.g. the CONSORT Statement for reporting randomized trials)
• The health and safety of all participants in the study was not compromised
• Ethical standards were maintained

If the research fails to reach relevant best practice standards, it's usual to recommend rejection. What's more, you don't then need to read any further.

3. Results and Discussion

This section should tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting need to be followed by the author:

• They should start by describing in simple terms what the data show
• They should make reference to statistical analyses, such as significance or goodness of fit
• Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research
• The outcome should be a critical analysis of the data collected

Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.

4. Conclusions

This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.

5. Information Gathered: Images, Graphs and Data Tables

If you find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. This could be an issue with titles, labels, statistical notation or image quality.

Where information is clear, you should check that:

• The results seem plausible, in case there is an error in data gathering
• The trends you can see support the paper's discussion and conclusions
• There are sufficient data. For example, in studies carried out over time are there sufficient data points to support the trends described by the author?

You should also check whether images have been edited or manipulated to emphasize the story they tell. This may be appropriate but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). Where you feel that an image has been edited or manipulated without explanation, you should highlight this in a confidential comment to the editor in your report.

6. List of References

You will need to check referencing for accuracy, adequacy and balance.

Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. Otherwise, it's the editor’s role to exhaustively check the reference section for accuracy and format.

You should consider if the referencing is adequate:

• Are important parts of the argument poorly supported?
• Are there published studies that show similar or dissimilar trends that should be discussed?
• If a manuscript only uses half the citations typical in its field, this may be an indicator that referencing should be improved - but don't be guided solely by quantity
• References should be relevant, recent and readily retrievable

Check for a well-balanced list of references that is:

• Helpful to the reader
• Fair to competing authors
• Not over-reliant on self-citation
• Gives due recognition to the initial discoveries and related work that led to the work under assessment

You should be able to evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

By now you will have a deep understanding of the paper's content - and you may have some concerns about plagiarism.

Identified Concern

If you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or it may be because it is very recent or published in a journal slightly outside their usual field.

You may feel you can advise the author how to emphasize the novel aspects of their own study, so as to better differentiate it from similar research. If so, you may ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. Of course, the research similarities may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

"It's very helpful when a reviewer can point out recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere ." (Editor feedback)

Suspected Concern

If you suspect plagiarism, including self-plagiarism, but cannot recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

Most editors have access to software that can check for plagiarism.

Editors are not out to police every paper, but when plagiarism is discovered during peer review it can be properly addressed ahead of publication. If plagiarism is discovered only after publication, the consequences are worse for both authors and readers, because a retraction may be necessary.

For detailed guidelines see COPE's Ethical guidelines for reviewers and Wiley's Best Practice Guidelines on Publishing Ethics .

8. Search Engine Optimization (SEO)

After the detailed read-through, you will be in a position to advise whether the title, abstract and key words are optimized for search purposes. In order to be effective, good SEO terms will reflect the aims of the research.

A clear title and abstract will improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. The title should contain the relevant SEO terms early on. This has a major effect on the impact of a paper, since it helps it appear in search results. A poor abstract can then lose the reader's interest and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may go no further.

So ask yourself, while the abstract may have seemed adequate during earlier checks, does it:

• Do justice to the manuscript in this context?
• Highlight important findings sufficiently?
• Present the most interesting data?

Editors say, " Does the Abstract highlight the important findings of the study ?"

If there is a formal report format, remember to follow it. This will often comprise a range of questions followed by comment sections. Try to answer all the questions. They are there because the editor felt that they are important. If you're following an informal report format you could structure your report in three sections: summary, major issues, minor issues.

• Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection
• Briefly summarize what the paper is about and what the findings are
• Try to put the findings of the paper into the context of the existing literature and current knowledge
• Indicate the significance of the work and if it is novel or mainly confirmatory
• Indicate the work's strengths, its quality and completeness
• State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

• Are there any major flaws? State what they are and what the severity of their impact is on the paper
• Has similar work already been published without the authors acknowledging this?
• Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
• If major revisions are required, try to indicate clearly what they are
• Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
• Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

• Are there places where meaning is ambiguous? How can this be corrected?
• Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?
• Are there any factual, numerical or unit errors? If so, what are they?
• Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

Your review should ultimately help the author improve their article. So be polite, honest and clear. You should also try to be objective and constructive, not subjective and destructive.

You should also:

• Write clearly and so you can be understood by people whose first language is not English
• Avoid complex or unusual words, especially ones that would even confuse native speakers
• Number your points and refer to page and line numbers in the manuscript when making specific comments
• If you have been asked to only comment on specific parts or aspects of the manuscript, you should indicate clearly which these are
• Treat the author's work the way you would like your own to be treated

Most journals give reviewers the option to provide some confidential comments to editors. Often this is where editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice such as suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

However, this doesn't give reviewers permission to 'backstab' the author. Authors can't see this feedback and are unable to give their side of the story unless the editor asks them to. So in the spirit of fairness, write comments to editors as though authors might read them too.

Reviewers should check the preferences of individual journals as to where they want review decisions to be stated. In particular, bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

You will normally be asked to indicate your recommendation (e.g. accept, reject, revise and resubmit, etc.) from a fixed-choice list and then to enter your comments into a separate text box.

Recommending Acceptance

If you're recommending acceptance, give details outlining why, and if there are any areas that could be improved. Don't just give a short, cursory remark such as 'great, accept'. See Improving the Manuscript

Recommending Revision

Where improvements are needed, a recommendation for major or minor revision is typical. You may also choose to state whether you opt in or out of the post-revision review too. If recommending revision, state specific changes you feel need to be made. The author can then reply to each point in turn.

Some journals offer the option to recommend rejection with the possibility of resubmission – this is most relevant where substantial, major revision is necessary.

What can reviewers do to help? " Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revisio n." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Recommending Rejection

If recommending rejection or major revision, state this clearly in your review (and see the next section, 'When recommending rejection').

Where manuscripts have serious flaws you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

Editors say, " If a reviewer suggests a rejection, but her/his comments are not detailed or helpful, it does not help the editor in making a decision ."

In your recommendations for the author, you should:

• Give constructive feedback describing ways that they could improve the research
• Keep the focus on the research and not the author. This is an extremely important part of your job as a reviewer
• Avoid making critical confidential comments to the editor while being polite and encouraging to the author - the latter may not understand why their manuscript has been rejected. Also, they won't get feedback on how to improve their research and it could trigger an appeal

Remember to give constructive criticism even if recommending rejection. This helps developing researchers improve their work and explains to the editor why you felt the manuscript should not be published.

" When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Visit our Wiley Author Learning and Training Channel for expert advice on peer review.

Watch the video, Ethical considerations of Peer Review

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• News & Views
• Patricia Elliott:...

Patricia Elliott: doctor, activist, and general secretary of the Medical Practitioners’ Union

• Related content
• Peer review
• Rebecca Wallersteiner
• wallersteiner{at}hotmail.com

Patricia Elliott was a doctor, activist, and socialist who became general secretary of the Medical Practitioners’ Union and drew on her expertise in industrial health to speak out against the expansion of Stansted airport in the early 2000s.

Born in Southampton in March 1921 to Cyrus Lloyd, a merchant seaman, and Clemency Wharton, Patricia was only 9 years old when her mother died from tuberculosis. She was educated at a local primary school and then Southampton Girls Grammar School, where she excelled at science.

Left wing activism was the seam that ran throughout her life—she got an early taste as a teenager in Southampton when she joined a demonstration against Oswald Mosley, the leader of the British Union of Fascists.

Her father later married a wealthy widow who encouraged Patricia to become a debutante and find a husband, but, spurred on by her mother’s early death, she decided to study medicine at University College London (UCL), one of the few female medical students in her cohort.

During the second world war much of the UCL …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

• Need to activate
• Log in via institution
• Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

• Open access
• Published: 03 September 2024

Reliability of ChatGPT in automated essay scoring for dental undergraduate examinations

• Bernadette Quah 1 , 2 ,
• Lei Zheng 1 , 2   na1 ,
• Timothy Jie Han Sng 1 , 2   na1 ,
• Chee Weng Yong 1 , 2   na1 &
• Intekhab Islam   ORCID: orcid.org/0000-0002-7754-0609 1 , 2  

BMC Medical Education volume  24 , Article number:  962 ( 2024 ) Cite this article

1 Altmetric

Metrics details

This study aimed to answer the research question: How reliable is ChatGPT in automated essay scoring (AES) for oral and maxillofacial surgery (OMS) examinations for dental undergraduate students compared to human assessors?

Sixty-nine undergraduate dental students participated in a closed-book examination comprising two essays at the National University of Singapore. Using pre-created assessment rubrics, three assessors independently performed manual essay scoring, while one separate assessor performed AES using ChatGPT (GPT-4). Data analyses were performed using the intraclass correlation coefficient and Cronbach's α to evaluate the reliability and inter-rater agreement of the test scores among all assessors. The mean scores of manual versus automated scoring were evaluated for similarity and correlations.

A strong correlation was observed for Question 1 ( r  = 0.752–0.848, p  < 0.001) and a moderate correlation was observed between AES and all manual scorers for Question 2 ( r  = 0.527–0.571, p  < 0.001). Intraclass correlation coefficients of 0.794–0.858 indicated excellent inter-rater agreement, and Cronbach’s α of 0.881–0.932 indicated high reliability. For Question 1, the mean AES scores were similar to those for manual scoring ( p  > 0.05), and there was a strong correlation between AES and manual scores ( r  = 0.829, p  < 0.001). For Question 2, AES scores were significantly lower than manual scores ( p  < 0.001), and there was a moderate correlation between AES and manual scores ( r  = 0.599, p  < 0.001).

This study shows the potential of ChatGPT for essay marking. However, an appropriate rubric design is essential for optimal reliability. With further validation, the ChatGPT has the potential to aid students in self-assessment or large-scale marking automated processes.

Peer Review reports

Large Language Models (LLMs), such as OpenAI’s GPT-4, LLaMA by META, and Google’s LaMDA (Language Models for Dialogue Applications), have demonstrated tremendous potential in generating outputs based on user-specified instructions or prompts. These models are trained using large amounts of data and are capable of natural language processing tasks. Owing to their ability to comprehend, interpret, and generate natural language text, LLMs allow human-like conversations with coherent contextual responses to prompts. The capability of LLMs to summarize and generate texts that resemble human language allows the creation of task-focused systems that can ease the demands of human labor and improve efficiency.

OpenAI uses a closed application programming interface (API) to process data. Chat Generative Pre-trained Transformer (OpenAI Inc., California, USA, https://chat.openai.com/ ) was introduced globally in 2020 as ChatGPT3, a generative language model with 175 billion parameters [ 1 ]. It is based on a generative AI model that can generate new content based on the data on which they have been trained. The latest version, ChatGPT-4, was introduced in 2023 and has demonstrated improved creativity, reasoning, and the ability to process even more complicated tasks [ 2 ].

Since its release in the public domain, ChatGPT has been actively explored by both healthcare professionals and educators in an effort to attain human-like performance in the form of clinical reasoning, image recognition, diagnosis, and learning from medical databases. ChatGPT has proven to be a powerful tool with immense potential to provide students with an interactive platform to deepen their understanding of any given topic [ 3 ]. In addition, it is also capable of aiding in both lesson planning and student assessments [ 4 , 5 ].

The potential of ChatGPT for assessments

Automated Essay Scoring (AES) is not a new concept, and interest in AES has been increasing since the advent of AI. Three main categories of AES programs have been described, utilizing regression, classification, or neural network models [ 6 ]. A known problem of current AES systems is their unreliability in evaluating the content relevance and coherence of essays [ 6 ]. Newer language models such as ChatGPT, however, are potential game changers; they are simpler to learn than current deep learning programs and can therefore improve the accessibility of AES to educators. Mizumoto and Eguchi recently pioneered the potential use of ChatGPT (GPT-3.5 and 4) for AES in the field of linguistics and reported an accuracy level sufficient for use as a supportive tool even when fine-tuning of the model was not performed [ 7 ].

The use of these AI-powered tools may potentially ease the burden on educators in marking large numbers of essay scripts, while providing personalized feedback to students [ 8 , 9 ]. This is especially crucial with larger class sizes and increasing student-to-teacher ratios, where it can be more difficult for educators to actively engage individual students. Additionally, manual scoring by humans can be subjective and susceptible to fatigue, which may put the scoring at risk of being unreliable [ 7 , 10 ]. The use of AI for essay scoring may thus help reduce intra- and inter-rater variability associated with manual scoring by providing a more standardized and reliable scoring process that eases the time- and labor-intensive scoring workload of human assessors [ 10 , 11 ].

The current role of AI in healthcare education

Generative AI has permeated the healthcare industry and provided a diverse range of health enhancements. An example is how AI facilitates radiographic evaluation and clinical diagnosis to improve the quality of patient care [ 12 , 13 ]. In medical and dental education, virtual or augmented reality and haptic simulations are some of the exciting technological tools already implemented to improve student competence and confidence in patient assessment and execution of procedures [ 14 , 15 , 16 ]. The incorporation of ChatGPT into the dental curriculum would thus be the next step in enhancing student learning. The performance of ChatGPT in the United States Medical Licensing Examination (USMLE) was recently validated, with ChatGPT achieving a score equivalent to that of a third-year medical student [ 17 ]. However, no data are available on the performance of ChatGPT in the field of dentistry or oral and maxillofacial surgery (OMS). Furthermore, the reliability of AI-powered language models for the grading of essays in the medical field has not yet been evaluated; in addition to essay structure and language, the evaluation of essay scripts in the field of OMS would require a level of understanding of dentistry, medicine and surgery.

Therefore, this study aimed to evaluate the reliability of ChatGPT for AES in OMS examinations for final-year dental undergraduate students compared to human assessors. Our null hypothesis was that there would be no difference in the scores between the ChatGPT and human assessors. The research question for the study was as follows: How reliable is ChatGPT when used for AES in OMS examinations compared to human assessors?

Materials and methods

This study was conducted in the Faculty of Dentistry, National University of Singapore, under the Department of Oral and Maxillofacial Surgery. The study received ethical approval from the university’s Institutional Review Board (REF: IRB-2023–1051) and was conducted and drafted with guidance from the education interventions critical appraisal worksheet introduced by BestBETs [ 18 ].

Sample size calculation for this study was based on the formula provided by Viechtbauer et al.: n  = ln (1-γ) / ln(1-π), where n, γ and π represent the sample size, significance level and level of confidence respectively [ 19 ]. Based on a 5% margin of error, a 95% confidence level and a 50% outcome response, it was calculated that a minimum sample size of 59 subjects was required. Ultimately, the study recruited 69 participants, all of whom were final-year undergraduate dental students. A closed-book OMS examination was conducted on the Examplify platform (ExamSoft Worldwide Inc., Texas, USA) as a part of the end-of-module assessment. The examination comprised two open-ended essay questions based on the topics taught in the module (Table  1 ).

Creation of standardized assessment

An assessment rubric was created for each question through discussion and collaboration of a workgroup comprising four assessors involved in the study. All members of the work group were academic staff from the faculty (I.I., B.Q., L.Z., T.J.H.S.) (Supplementary Tables S1 and S2) [ 20 ]. An analytic rubric was generated using the strategy outlined by Popham [ 21 ]. The process involved a discussion within the workgroup to agree on the learning outcomes of the essay questions. Two authors (I. I. and B. Q) independently generated the rubric criteria and descriptions for Question 1 (Infection). Similarly, for Question 2 (Trauma), the rubric criteria and descriptions were generated independently by two authors (I.I. and T.J.H.S.). The rubrics were revised until a consensus was reached between each pair. In the event of any disagreement, a third author (L.Z.) provided their opinion to aid in decision making.

Marking categories of Poor (0 marks), Satisfactory (2 marks), Good (3 marks), and Outstanding (4 marks) were allocated to each criterion, with a maximum of 4 marks attainable from each criterion. A criterion for overall essay structure and language was also included, with a maximum attainable 5 marks from this criterion. The highest score for each question was 40.

Model answers to the essays were prepared by another author (C.W.Y.), who did not participate in the creation of the rubrics. Using the rubrics as a reference, the author modified the model answer to create 5 variants of the answers such that each variant fell within different score ranges of 0–10, 11–20, 21–30, 31–40, 41–50. Subsequently, three authors (B. Q., L. Z., and T.J.H.S) graded the essays using the prepared rubrics. Revisions to the rubrics were made with consensus by all three authors, a process that also helped calibrate these three authors for manual essay scoring.

AES with ChatGPT

Essay scoring was performed using ChatGPT (GPT-4, released March 14, 2023) by one assessor who did not participate in the manual essay scoring exercise (I.I.). Prompts were generated based on a guideline by Giray, and the components of Instruction, Context, Input Data and Output Indication as discussed in the guideline were included in each prompt (Supplementary Tables 3 and 4) [ 22 ]. A prompt template was generated for each question by one assessor (I.I.) with advice from two experts in prompt engineering, based on the marking rubric. The criterion and point allocation were clearly written in prose and point forms. For the fine-tuning process, the prompts were input into ChatGPT using variants of the model answers provided by C.W.Y. Minor adjustments were made to the wording of certain parts of the prompts as necessary to correct any potential misinterpretations of the prompts by the ChatGPT. Each time, the prompt was entered into a new chat in the ChatGPT in a browser where the browser history and cookies were cleared. Subsequently, finalized prompts (Supplementary Tables 3 and 4) were used to score the student essays. AES scores were not used to calculate students’ actual essay scores.

Manual essay scoring

Manual essay scoring was completed independently by three assessors (B.Q., L.Z., and T.J.H.S.) using the assessment rubrics (Supplementary Tables S1 and S2). Calibration was performed during the rubric creation stage. The essays were anonymized to prevent bias during the marking process. The assessors recorded the marks allocated to each criterion, as well as the overall score of each essay, on a pre-prepared Excel spreadsheet. Scoring was performed separately and independently by all assessors before the final collation by a research team member (I.I.) for statistical analyses. The student was considered ‘able to briefly mention’ a criterion if they did not mention any of the keywords of the points within the criterion. The student was considered ‘able to elaborate on’ a point within the criterion if they were able to mention the keywords of that point as stated in the rubric, and were thus awarded higher marks in accordance with the rubric (e.g. the student was given a higher mark if they were able to mention the need to check for dyspnea and dysphagia, instead of simply mentioning a need to check the patient’s airway). Grading was performed with only whole marks as specified in the rubrics, and assessors were not allowed to give half marks or subscores.

Data synthesis

The scores given out of 40 per essay by each assessor were compiled. Data analyses were subsequently performed using SPSS® version 29.0.1.0(171) (IBM Corporation, New York, United States). For each essay question, correlations between the essay scores given by each assessor were analyzed and displayed using the inter-item correlation matrix. A correlation coefficient value ( r ) of 0.90–1.00 was indicative of a very strong, 0.70–0.89 indicative of strong, 0.40–0.69 moderate, 0.10–0.39 weak and < 0.10 negligible positive correlation between the scorers [ 23 ]. The cutoff p -value for the significance level was set at p  < 0.05. The intraclass correlation coefficient (ICC) and Cronbach's α were then calculated between all assessors to assess the inter-rater agreement and reliability, respectively [ 24 ]. The ICC was interpreted on a scale of 0 to 1.00, with a higher value indicating a higher level of agreement in scores given by the scorers to each student. A value less than 0.40 was indicative of poor, 0.40–0.59 fair, 0.60–0.74 good, and 0.75–1.00 excellent agreement [ 25 ]. Using Cronbach’s α, reliability was expressed on a range from 0 to 1.00, with a higher number indicating a higher level of consistency between the scorers in their scores given across the students. The reliability was considered ‘Less Reliable’ if the score was less 0.20, ‘Rather Reliable’ if the score was 0.20–0.40, ‘Quite Reliable’ if 0.40–0.60, ‘Reliable’ if 0.60–0.80 and ‘Very Reliable’ if 0.80–1.00 [ 26 ].

Similarly, the mean scores of the three manual scorers were calculated for each question. The mean manual scores were then analyzed for correlations with AES scores by using Pearson’s correlation coefficient. Student’s t-test was also used to analyze any significant differences in mean scores between manual scoring and AES. A p -value of < 0.05 was required to conclude the presence of a statistically different score between the groups.

All final-year dental undergraduate students (69/69, 100%) had their essays graded by all manual scorers and AES as part of the study. Table 2 shows the mean scores for each individual assessor as well as the mean scores for the three manual scorers (Scorers 1, 2, and 3).

Analysis of correlation and agreement between all scorers

The inter-item correlation matrices and their respective p -values are listed in Table  3 . For Question 1, there was a strong positive correlation between the scores provided by each assessor (Scorers 1, 2, 3, and AES), with r -values ranging from 0.752–0.848. All p -values were < 0.001, indicating a significant positive correlation between all assessors. For Question 2, there was a strong positive correlation between Scorers 1 and 2 ( r  = 0.829) and Scorers 1 and 3 ( r  = 0.756). There was a moderate positive correlation between Scorers 2 and 3 ( r  = 0.655), as well as between AES and all manual scores ( r -values ranging from 0.527 to 0.571). Similarly, all p -values were < 0.001, indicative of a significant positive correlation between all scorers.

For the analysis of inter-rater agreement, ICC values of 0.858 (95% CI 0.628 – 0.933) and 0.794 (95% CI 0.563 – 0.892) were obtained for Questions 1 and 2, respectively, both of which were indicative of excellent inter-rater agreement. Cronbach’s α was 0.932 for Question 1 and 0.881 for Question 2, both of which were ‘Very Reliable’.

Analysis of correlation between manual scoring versus AES

The results of the Student’s t-test comparing the test score values from manual scoring and AES are shown in Table  2 . For Question 1, the mean manual scores (14.85 ± 4.988) were slightly higher than those of the AES (14.54 ± 5.490). However, these differences were not statistically significant ( p  > 0.05). For Question 2, the mean manual scores (23.11 ± 4.241) were also higher than those of the AES (18.62 ± 4.044); this difference was statistically significant ( p  < 0.001).

The results of the Pearson’s correlation coefficient calculations are shown in Table  4 . For Question 1, there was a strong and significant positive correlation between manual scoring and AES ( r  = 0.829, p  < 0.001). For Question 2, there was a moderate and statistically significant positive correlation between the two groups ( r  = 0.599, p  < 0.001).

Qualitative feedback from AES

Figures 1 , 2 and 3 show three examples of essay feedback and scoring provided by ChatGPT. ChatGPT provided feedback in a concise and systematic manner. Scores were clearly provided for each of the criteria listed in the assessment rubric. This was followed by in-depth feedback on the points within the criterion that the student had discussed and failed to mention. ChatGPT was able to differentiate between a student who briefly mentioned a key point and a student who provided better elaboration on the same point by allocating them two or three marks, respectively.

Example #1 of a marked essay with feedback from ChatGPT for Question 1

Example #2 of a marked essay with feedback from ChatGPT for Question 1

Example #3 of a marked essay with feedback from ChatGPT for Question 1

One limitation of ChatGPT that was identified during the scoring process was its inability to identify content that was not relevant to the essay or that was factually incorrect. This was despite the assessment rubric specifying that incorrect statements should be given 0 marks for that criterion. For example, a student who included points about incision and drainage also incorrectly stated that bone scraping to induce bleeding and packing of local hemostatic agents should be performed. Although these statements were factually incorrect, ChatGPT was unable to identify this and still awarded student marks for the point. Manual assessors were able to spot this and subsequently penalized the student for the mistake.

Since its recent rise in popularity, many people have been eager to tap into the potential of large language models, such as ChatGPT. In their review, Khan et al. discussed the growing role of ChatGPT in medical education, with promising uses for the creation of case studies and content such as quizzes and flashcards for self-directed practice [ 9 ]. As an LLM, the ability of ChatGPT to thoroughly evaluate sentence structure and clarity may allow it to confront the task of automated essay marking.

Advantages of ChatGPT in AES

This study found significant correlations and excellent inter-rater agreement between ChatGPT and manual scorers, and the mean scores between both groups showed strong to moderate correlations for both essay questions. This suggests that AES has the potential to provide a level of essay marking similar to that of the educators in our faculty. Similar positive findings were reflected in previous studies that compared manual and automated essay scoring ( r  = 0.532–0.766) [ 6 ]. However, there is still a need to further fine-tune the scoring system such that the score provided by AES deviates as little as possible from human scoring. For instance, the mean AES score was lower than that of manual scoring by 5 marks for Question 2. Although the difference may not seem large, it may potentially increase or decrease the final performance grade of students.

Apart from a decent level of reliability in manual essay scoring, there are many other benefits to using ChatGPT for AES. Compared to humans, the response time to prompts is much faster and can thus increase productivity and reduce the burden of a large workload on educational assessors [ 27 ]. In addition, ChatGPT can provide individualized feedback for each essay (Figs. 1 , 2 and 3 ). This helps provide students with comments specific to their essays, a feat that is difficult to achieve for a single educator teaching a large class size.

Similar to previous systems designed for AES, machine scoring is beneficial for removing human inconsistencies that can result from fatigue, mood swings, or bias [ 10 ]. ChatGPT is no exception. Furthermore, ChatGPT is more widely accessible than the conventional AES systems. Its software runs online instead of requiring downloads on a computer, and its user interface is simple to use. With GPT-3.5 being free to use and GPT-4 being 20 USD per month, it is also relatively inexpensive.

Marking the essay is only part of the equation, and the next step is to allow the students to know what went wrong and why. Nicol and Macfarlane described seven principles for good feedback. ChatGPT can fulfil most of these principles, namely, facilitating self-assessment, encouraging teacher and peer dialogue, clarifying what good performance is, providing opportunities to close the gap between current and desired performance, and delivering high-quality information to students [ 28 ]. In this study, the feedback given by ChatGPT was categorized based on the rubric, and elaboration was provided for each criterion on the points the student mentioned and did not mention. By highlighting the ideal answer and where the student can improve, ChatGPT can clarify performance goals and provide opportunities to close the gap between the student’s current and desired performance. This creates opportunities for selfdirected learning and the utilization of blended learning environments. Students can use ChatGPT to review their preparation on topics, self-grade their essays, and receive instant feedback. Furthermore, the simple and interactive nature of the software encourages dialogue, as it can readily respond to any clarification the student wants to make. The importance of effective feedback has been demonstrated to be an essential component in medical education, in terms of enhancing the knowledge of the student without developing negative emotions [ 29 , 30 ].

These potential advantages of engaging ChatGPT for student assessments play well into the humanistic learning theory of medical education [ 31 , 32 ]. Self-directed learning allows students the freedom to learn at their own pace, with educators simply providing a conducive environment and the goals that the student should achieve. ChatGPT has the potential to supplement the role of the educator in self-directed learning, as it can be readily available to provide constructive and tailored feedback for assignments whenever the student is ready for it. This removes the burden that assignment deadlines place on students, which can allow them a greater sense of independence and control over their learning, and lead to greater self-motivation and self-fulfillment.

Potential pitfalls of ChatGPT

Potential pitfalls associated with the use of ChatGPT were identified. First, the ability to achieve reliable scores relies heavily on a well-created marking rubric with clearly defined terms. In this study, the correlations between scorers were stronger for Question 1 compared to Question 2, and the mean scores between the AES and manual scorers were also significantly different for Question 2, but not for Question 1. The lower reliability of the AES for Question 2 may be attributed to its broader nature, use of more complex medical terms, and lengthier scoring rubrics. The broad nature of the question left more room for individual interpretation and variation between humans and AES. The ability of ChatGPT to provide accurate answers may be reduced with lengthier prompts and conversations [ 27 ]. Furthermore, with less specific instructions or complex medical jargon, both automated systems and human scorers may interpret rubrics differently, resulting in varied scores across the board [ 10 , 33 , 34 ]. The authors thus recommend that to circumvent this, the use of ChatGPT for essay scoring should be restricted to questions that are less broad (e.g. shorter essays), or by breaking the task into multiple prompts for each individual criterion to reduce variations in interpretation [ 27 , 35 ]. Furthermore, the rubrics should contain concise and explicit instructions with appropriate grammar and vocabulary to avoid misinterpretation by both ChatGPT and human scorers, and provide a brief explanation to specify what certain medical terms mean (e.g. writing ‘pulse oximetry (SpO2) monitoring’ instead of only ‘SpO2’) for better contextualization [ 35 , 36 ].

Second, prompt engineering is a critical step in producing the desired outcome from ChatGPT [ 27 ]. A prompt that is too ambiguous or lacks context can lead to a response that is incomplete, generic, or irrelevant, and a prompt that exhibits bias runs the risk of bias reinforcement in the given reply [ 22 , 34 ]. Phrasing the prompt must also be carefully checked for spelling, grammatical mistakes, or inconsistencies, since ChatGPT uses the prompt’s phrasing literally. For example, a prompt that reads ‘give 3 marks if the student covers one or more coverage points’ will result in ChatGPT only giving the marks if multiple points are covered, because of the plural nature of the word ‘points’.

Third, irrelevant content may not be penalized during the essay-marking process. Students may ‘trick’ the AES by producing a lengthier essay to hit more relevant points and increase their score. This may result in essays of lower quality with multiple incorrect or nonsensical statements still rewarded with higher scores [ 10 ]. Our assessment rubric attempts to penalize the student with 0 marks if incorrect statements on the criterion are made; however, none of the students were penalized. This issue may be resolved as ChatGPT rapidly and continuously gains more medical and dental knowledge. Although data to support the competence of AI in medical education are sparse, the quality of the medical knowledge that ChatGPT already has is sufficient to achieve a passing mark at the USMLE [ 5 , 37 ]. In dentistry, when used to disseminate information on endodontics to patients, ChatGPT was found to provide detailed answers with an overall validity of 95% [ 38 ]. Over time, LLMs such as ChatGPT may be able to identify when students are not factually correct.

Other comments

The lack of human emotion in machine scoring can be both an advantage and a disadvantage. AES can provide feedback that is entirely factual and less biased than humans, and grades are objective and final [ 39 ]. However, human empathy is an essential quality that ChatGPT does not possess. One principle of good feedback is to encourage and motivate students to provide positive learning experiences and build self-esteem [ 28 ]. While ChatGPT can provide constructive feedback, it will not be able to replace the compassion, empathy, or emotional intelligence possessed by a quality educator possesses [ 40 ]. In our study, ChatGPT awarded lower mean scores of 14.54/40 (36.4%) and 18.62/40 (46.5%) compared to manual scoring for both questions. Although objective, some may view automated scoring as harsh because it provided failing grades to an average student.

This study demonstrates the ability of GPT-4 to evaluate essays without any specialized training or prompting. One long prompt was used to score each essay. Although more technical prompting methods, such as chain of thought, could be deployed, our single prompt method makes the method scalable and easier to adopt. As discussed earlier, ChatGPT is the most reliable when prompts are short and specific [ 34 ]. Hence, each prompt should ideally task ChatGPT to score only one or two criteria, rather than the entire rubric of the 10 criteria. However, in a class of 70, the assessors are required to run 700 prompts per question, which is impractical and unnecessary. With only one prompt, a good correlation was still found between the AES and manual scoring. It is likely that further exploration and experimentation with prompting techniques can improve the output.

While LLMs have the potential to revolutionize education in healthcare, some precautions must be taken. Artificial Hallucination is a widely described phenomenon; ChatGPT may generate seemingly genuine but inaccurate information [ 41 , 42 , 43 ]. Hallucinations have been attributed to biases and limitations of training data as well as algorithmic limitations [ 2 ]. Similarly, randomness of the generated responses has been observed; while it is useful for generating creative content, this may be an issue when ChatGPT is employed for topics requiring scientific or factual content [ 44 ]. Thus, LLMs are not infallible and still require human subject matter experts to validate the generated content. Finally, it is essential that educators play an active role in driving the development of dedicated training models to ensure consistency, continuity, and accountability, as overreliance on a corporate-controlled model may place educators at the mercy of algorithm changes.

The ethical implications of using ChatGPT in medical and dental education also need to be explored. As much as LLMs can provide convenience to both students and educators, privacy and data security remain a concern [ 45 ]. Robust university privacy policies and informed consent procedures should be in place for the protection of student data prior to the use of ChatGPT as part of student assessment. Furthermore, if LLMs like ChatGPT were to be used for grading examinations in the future, issues revolving around fairness and transparency of the grading process need to be resolved [ 46 ]. GPT-4 may have provided harsh scores in this study, possibly due to some shortfall in understanding certain phrases the students have written; models used in assessments will thus require sufficient training in the field of healthcare to properly acquire the relevant medical knowledge and hence understand and grade essays fairly.

As AI continues to develop, ChatGPT may eventually replace human assessors in essay scoring for dental undergraduate examinations. However, given its current limitations and dependence on a well-formed assessment rubric, relying solely on ChatGPT for exam grading may be inappropriate when the scores can affect the student’s overall module scores, career success, and mental health [ 47 ]. While this study primarily demonstrates the use of ChatGPT to grade essays, it also points to great potential in using it as an interactive learning tool. A good start for its use is essay assignments on pre-set topics, where students can direct their learning on their own and receive objective feedback on essay structure and content that does not count towards their final scores. Students can use rubrics to practice and gain effective feedback from LLMs in an engaging and stress-free environment. This reduces the burden on educators by easing the time-consuming task of grading essay assignments and allows students the flexibility to complete and grade their assignments whenever they are ready. Furthermore, assignments repeated with new class cohorts can enable more robust feedback from ChatGPT through machine learning.

Study limitations

The limitations of this study lie in part of its methodology. The study recruited 69 dental undergraduate students; while this is above the minimum calculated sample size of 59, a larger sample size would help to increase the generalizability of the study findings to larger populations of students and a wide scope of topics. The unique field of OMS also requires knowledge of both medical and dental subjects, and hence the results obtained from the use of ChatGPT for essay marking in other medical or dental specialties may differ slightly.

The use of rubrics for manual scoring could also be a potential source of bias. While the rubrics provide a framework for objective assessment, they cannot eliminate the subjectiveness of manual scoring. Variations in the interpretation of the students’ answers, leniency errors (whereby one scorer marks more leniently than another) or rater drift (fatigue from assessing many essays may affect leniency of marking and judgment) may still occur. To minimize bias resulting from these errors, multiple assessors were recruited for the manual scoring process and the average scores were used for comparison with AES.

This study investigated the reliability of ChatGPT in essay scoring for OMS examinations, and found positive correlations between ChatGPT and manual essay scoring. However, ChatGPT tended towards stricter scoring and was not capable of penalizing irrelevant or incorrect content. In its present state, GPT-4 should not be used as a standalone tool for teaching or assessment in the field of medical and dental education but can serve as an adjunct to aid students in self-assessment. The importance of proper rubric design to achieve optimal reliability when employing ChatGPT in student assessment cannot be overemphasized.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Floridi L, Chiriatti M. GPT-3: Its nature, scope, limits, and consequences. Mind Mach. 2020;30(4):681–94.

Article   Google Scholar  

Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, Aziz S, Damseh R, Alabed Alrazak S, Sheikh J. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9:e48291.

Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, Gasser U, Groh G, Günnemann S, Hüllermeier E, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ. 2023;103:102274.

Javaid M, Haleem A, Singh RP, Khan S, Khan IH. Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. BenchCouncil Transact Benchmarks Standards Eval. 2023;3(2): 100115.

Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198.

Ramesh D, Sanampudi SK. An automated essay scoring systems: a systematic literature review. Artif Intell Rev. 2022;55(3):2495–527.

Mizumoto A, Eguchi M. Exploring the potential of using an AI language model for automated essay scoring. Res Methods Appl Linguist. 2023;2(2): 100050.

Erturk S, Tilburg W, Igou E: Off the mark: Repetitive marking undermines essay evaluations due to boredom. Motiv Emotion 2022;46.

Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT - Reshaping medical education and clinical management. Pak J Med Sci. 2023;39(2):605–7.

Hussein MA, Hassan H, Nassef M. Automated language essay scoring systems: a literature review. PeerJ Comput Sci. 2019;5:e208.

Blood I: Automated essay scoring: a literature review. Studies in Applied Linguistics and TESOL 2011, 11(2).

Menezes LDS, Silva TP, Lima Dos Santos MA, Hughes MM, Mariano Souza SDR, Leite Ribeiro PM, Freitas PHL, Takeshita WM: Assessment of landmark detection in cephalometric radiographs with different conditions of brightness and contrast using the an artificial intelligence software. Dentomaxillofac Radiol 2023:20230065.

Bennani S, Regnard NE, Ventre J, Lassalle L, Nguyen T, Ducarouge A, Dargent L, Guillo E, Gouhier E, Zaimi SH, et al. Using AI to improve radiologist performance in detection of abnormalities on chest radiographs. Radiology. 2023;309(3): e230860.

Moussa R, Alghazaly A, Althagafi N, Eshky R, Borzangy S. Effectiveness of virtual reality and interactive simulators on dental education outcomes: systematic review. Eur J Dent. 2022;16(1):14–31.

Fanizzi C, Carone G, Rocca A, Ayadi R, Petrenko V, Casali C, Rani M, Giachino M, Falsitta LV, Gambatesa E, et al. Simulation to become a better neurosurgeon An international prospective controlled trial: The Passion study. Brain Spine. 2024;4:102829.

Lovett M, Ahanonu E, Molzahn A, Biffar D, Hamilton A. Optimizing individual wound closure practice using augmented reality: a randomized controlled study. Cureus. 2024;16(4):e59296.

Google Scholar  

Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, Chartash D. How does ChatGPT perform on the United States medical licensing examination? the implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9:e45312.

Educational Intervention Worksheet, BestBets, Accessed 31/03/2024.  https://bestbets.org/ca/pdf/educational_intervention.pdf .

Viechtbauer W, Smits L, Kotz D, Budé L, Spigt M, Serroyen J, Crutzen R. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9.

Cox G, Morrison J, Brathwaite B: The Rubric: An Assessment Tool to Guide Students and Markers; 2015.

Popham J. W: “What’s Wrong—And What’s Right—With Rubrics.” Educ Leadersh. 1997;55(2):72–5.

Giray L. Prompt Engineering with ChatGPT: A Guide for Academic Writers. Ann Biomed Eng. 2023;51:3.

Schober P, Boer C, Schwarte LA. Correlation Coefficients: Appropriate Use and Interpretation. Anesth Analg. 2018;126(5):1763–8.

Liao SC, Hunt EA, Chen W. Comparison between inter-rater reliability and inter-rater agreement in performance assessment. Ann Acad Med Singap. 2010;39(8):613–8.

Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284–90.

Hair J, Black W, Babin B, Anderson R: Multivariate Data Analysis: A Global Perspective; 2010.

Nazir A, Wang Z: A Comprehensive Survey of ChatGPT: Advancements, Applications, Prospects, and Challenges. Meta Radiol 2023;1(2).

Nicol D, Macfarlane D: Rethinking Formative Assessment in HE: a theoretical model and seven principles of good feedback practice. IEEE Personal Communications - IEEE Pers Commun 2004;31.

Spooner M, Larkin J, Liew SC, Jaafar MH, McConkey S, Pawlikowska T. “Tell me what is ‘better’!” How medical students experience feedback, through the lens of self-regulatory learning. BMC Med Educ. 2023;23(1):895.

Kornegay JG, Kraut A, Manthey D, Omron R, Caretta-Weyer H, Kuhn G, Martin S, Yarris LM. Feedback in medical education: a critical appraisal. AEM Educ Train. 2017;1(2):98–109.

Mukhalalati BA, Taylor A. Adult learning theories in context: a quick guide for healthcare professional educators. J Med Educ Curric Dev. 2019;6:2382120519840332.

Taylor DC, Hamdy H. Adult learning theories: implications for learning and teaching in medical education: AMEE Guide No. 83. Med Teach. 2013;35(11):e1561-1572.

Chakraborty S, Dann C, Mandal A, Dann B, Paul M, Hafeez-Baig A: Effects of Rubric Quality on Marker Variation in Higher Education. Studies In Educational Evaluation 2021;70.

Heston T, Khun C. Prompt engineering in medical education. Int Med Educ. 2023;2:198–205.

Sun GH: Prompt Engineering for Nurse Educators. Nurse Educ 2024.

Meskó B. Prompt engineering as an important emerging skill for medical professionals: tutorial. J Med Internet Res. 2023;25:e50638.

Sun L, Yin C, Xu Q, Zhao W. Artificial intelligence for healthcare and medical education: a systematic review. Am J Transl Res. 2023;15(7):4820–8.

Mohammad-Rahimi H, Ourang SA, Pourhoseingholi MA, Dianat O, Dummer PMH, Nosrat A: Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics. Int Endodontic J 2023, n/a(n/a).

Peng X, Ke D, Xu B: Automated essay scoring based on finite state transducer: towards ASR transcription of oral English speech. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 1. Jeju Island, Korea: Association for Computational Linguistics; 2012:50–59.

Grassini S. Shaping the future of education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings. Educ Sci. 2023;13(7):692.

Limitations.  https://openai.com/blog/chatgpt .

Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1):e103.

Deng J, Lin Y. The Benefits and Challenges of ChatGPT: An Overview. Front Comput Intell Syst. 2023;2:81–3.

Choi W. Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs. BMC Med Educ. 2023;23(1):864.

Medina-Romero MÁ, Jinchuña Huallpa J, Flores-Arocutipa J, Panduro W, Chauca Huete L, Flores Limo F, Herrera E, Callacna R, Ariza Flores V, Quispe I, et al. Exploring the ethical considerations of using Chat GPT in university education. Period Eng Nat Sci (PEN). 2023;11:105–15.

Lee H. The rise of ChatGPT: Exploring its potential in medical education. Anat Sci Educ. 2024;17(5):926–31.

Steare T, Gutiérrez Muñoz C, Sullivan A, Lewis G. The association between academic pressure and adolescent mental health problems: A systematic review. J Affect Disord. 2023;339:302–17.

Download references

Acknowledgements

We would like to extend our gratitude to Mr Paul Timothy Tan Bee Xian and Mr Jonathan Sim for their invaluable advice on the process of prompt engineering for the effective execution of this study.

Author information

Lei Zheng, Timothy Jie Han Sng and Chee Weng Yong contributed equally to this work.

Authors and Affiliations

Faculty of Dentistry, National University of Singapore, Singapore, Singapore

Bernadette Quah, Lei Zheng, Timothy Jie Han Sng, Chee Weng Yong & Intekhab Islam

Discipline of Oral and Maxillofacial Surgery, National University Centre for Oral Health, 9 Lower Kent Ridge Road, Singapore, Singapore

You can also search for this author in PubMed   Google Scholar

Contributions

B.Q. contributed in the stages of conceptualization, methodology, study execution, validation, formal analysis and manuscript writing (original draft and review and editing). L.Z., T.J.H.S. and C.W.Y. contributed in the stages of methodology, study execution, and manuscript writing (review and editing). I.I. contributed in the stages of conceptualization, methodology, study execution, validation, formal analysis, manuscript writing (review and editing) and supervision. All authors provided substantial contributions to this manuscript in the following form:

Corresponding author

Correspondence to Intekhab Islam .

Ethics declarations

Ethics approval and consent to participate.

This study was approved by the Institutional Review Board of the university (REF: IRB-2023–1051). The waiver of consent from students was approved by the University’s Institutional Review Board, as the scores by ChatGPT were not used as the students’ actual grades, and all essay manuscripts were anonymized.

Consent for publication

All the authors reviewed the content of this manuscript and provided consent for publication.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Quah, B., Zheng, L., Sng, T.J.H. et al. Reliability of ChatGPT in automated essay scoring for dental undergraduate examinations. BMC Med Educ 24 , 962 (2024). https://doi.org/10.1186/s12909-024-05881-6

Download citation

Received : 04 February 2024

Accepted : 09 August 2024

Published : 03 September 2024

DOI : https://doi.org/10.1186/s12909-024-05881-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Academic performance
• Educational
• Educational needs assessment

BMC Medical Education

ISSN: 1472-6920

• General enquiries: [email protected]

A Survey on Deep Learning Techniques for Predictive Analytics in Healthcare

• Survey Article
• Published: 04 September 2024
• Volume 5 , article number  860 , ( 2024 )

Cite this article

• Mohammed Badawy   ORCID: orcid.org/0000-0001-9494-1386 1 ,
• Nagy Ramadan 1 &
• Hesham Ahmed Hefny 2  

Healthcare data is growing at more than 50% annually, making it one of the most rapidly expanding data in the digital world. Clinical problem-solving is a difficult skill that doctors must have in order to provide excellent care. This skill’s accuracy is critical to the patients’ lives and well-being. Using deep learning in the medical field may aid not only in enhancing classification accuracy but also in reducing diagnostic time and cost, as well as in disease prediction. Putting deep learning models into action requires utilizing a wide variety of programs and data sources. This study explores the existing research options and the challenges encountered in the field of deep learning in healthcare prediction. It presents a comparative and systemic study of deep learning and how it can be used for prediction in the healthcare domain. Furthermore, it provides a wide-ranging overview of the current deep learning methods used in healthcare prediction. As a result of this review, the total number of papers that were examined was 45 and covered the period from 2019 to 2023. This research compared the methodologies, strategies, datasets, and conclusions from the provided studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Deep Learning for Predictive Analytics in Healthcare

A Study on Deep Learning Predictive Models in Healthcare

Healthcare predictive analytics using machine learning and deep learning techniques: a survey

Explore related subjects.

• Artificial Intelligence
• Medical Imaging

Data Availability

Not applicable.

Hema Latha M, Ramakrishna A, Sudarsha Chakravarthi Reddy B, Venkateswarlu C, Yamini Saraswathi S (2021) Disease prediction by stacking algorithms over big data from healthcare communities. In: Intelligent manufacturing and energy sustainability: proceedings of ICIMES 2021. Singapore: Springer Singapore; 2021. pp. 355–363.

Xiong X, Cao X, Luo L. The ecology of medical care in Shanghai. BMC Health Serv Res. 2021;21:1–9.

Article   Google Scholar  

Marzorati C, Pravettoni G. Value as the key concept in the health care system: how it has influenced medical practice and clinical decision-making processes. J Multidiscip Healthcare. 2017;10:101–6.

Singh G, Al’Aref SJ, Van Assen M, Kim TS, van Rosendael A, Kolli KK, et al. Machine learning in cardiac CT: basic concepts and contemporary data. J Cardiovasc Comput Tomogr. 2018;12(3):192–201.

Lei Y, Yang B, Jiang X, Jia F, Li N, Nandi AK. Applications of machine learning to machine fault diagnosis: a review and roadmap. Mech Syst Signal Process. 2020;138: 106587.

Fatima M, Pasha M. Survey of machine learning algorithms for disease diagnostic. J Intell Learn Syst Appl. 2017;9(01):1–16.

Google Scholar  

Bengio Y, Goodfellow IJ, Courville A. Deep learning book in preparation; 2015. Draft chapters available at http://www.deeplearningbook.org

Kim KJ, Tagkopoulos I. Application of machine learning in rheumatic disease research. Korean J Intern Med. 2019;34(4):708.

Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. pp. 4700–4708.

Pandey SK, Janghel RR. Recent deep learning techniques, challenges and its applications for medical healthcare system: a review. Neural Process Lett. 2019;50:1907–35.

Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P, Nasrin MS, et al. A state-of-the-art survey on deep learning theory and architectures. Electronics. 2019;8(3):292.

Lipton ZC, Berkowitz J, Elkan C. A critical review of recurrent neural networks for sequence learning; 2015. arXiv preprint arXiv:1506.00019 .

Wang J, Hu X. Convolutional neural networks with gated recurrent connections. IEEE Trans Pattern Anal Mach Intell. 2021;44(7):3421–35.

Javaid N, Jan N, Javed MU. An adaptive synthesis to handle imbalanced big data with deep Siamese network for electricity theft detection in smart grids. J Parallel Distrib Comput. 2021;153:44–52.

Cheng B, Xu X, Zeng Y, Ren J, Jung S. Pedestrian trajectory prediction via the social-grid LSTM model. J Eng. 2018;2018(16):1468–74.

Havaei M, Guizard N, Larochelle H, Jodoin PM. Deep learning trends for focal brain pathology segmentation in MRI. In: Holzinger A, editor. Machine learning for health informatics: state-of-the-art and future challenges. Cham: Springer; 2016. p. 125–48.

Chapter   Google Scholar  

Greenspan H, Van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35(5):1153–9.

Krizhevsky A, Hinton G. Learning multiple layers of features from tiny images. 2009;7.

Lee DH. Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol. 3(2); 2013. p. 896.

Laine S, Aila T. Temporal ensembling for semi-supervised learning; 2016. arXiv preprint arXiv:1610.02242 .

Li H, Xu Z, Taylor G, Studer C, Goldstein T. Visualizing the loss landscape of neural nets. In: Advances in neural information processing systems 31; 2018.

Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft coco: common objects in context. In: Computer vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part V 13; 2014. Springer International Publishing. pp. 740–755.

Liu B, Wu Z, Hu H, Lin S. Deep metric transfer for label propagation with limited annotated data. In: Proceedings of the IEEE/CVF international conference on computer vision workshops; 2019.

Miyato T, Maeda SI, Koyama M, Ishii S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell. 2018;41(8):1979–93.

Hady MFA, Schwenker F. Semi-supervised learning. In: Bianchini M, Maggini M, Jain LC, editors. Handbook on neural information processing. Berlin, Heidelberg: Springer; 2013. p. 215–39.

Van Engelen JE, Hoos HH. A survey on semi-supervised learning. Mach Learn. 2020;109(2):373–440.

Article   MathSciNet   Google Scholar  

Berthelot D, Carlini N, Goodfellow I, Papernot N, Oliver A, Raffel CA. Mixmatch: a holistic approach to semi-supervised learning. In: Advances in neural information processing systems 32; 2019.

Rasmus A, Berglund M, Honkala M, Valpola H, Raiko T. Semi-supervised learning with ladder networks. In: Advances in neural information processing systems 28; 2015.

Tarvainen A, Valpola H. Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Advances in neural information processing systems, 30; 2017.

Zhang H, Cisse M, Dauphin YN, Lopez-Paz D. mixup: Beyond empirical risk minimization; 2017. arXiv preprint arXiv:1710.09412 .

Andreu-Perez J, Poon CC, Merrifield RD, Wong ST, Yang GZ. Big data for health. IEEE J Biomed Health Inform. 2015;19(4):1193–208.

Liang Z, Zhang G, Huang JX, Hu QV. Deep learning for healthcare decision making with EMRs. In: 2014 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE; 2014, November. pp. 556–559.

Ravì D, Wong C, Deligianni F, Berthelot M, Andreu-Perez J, Lo B, Yang GZ. Deep learning for health informatics. IEEE J Biomed Health Inform. 2016;21(1):4–21.

Leung MK, Delong A, Alipanahi B, Frey BJ. Machine learning in genomic medicine: a review of computational problems and data sets. Proc IEEE. 2015;104(1):176–97.

Maity NG, Das S. Machine learning for improved diagnosis and prognosis in healthcare. In: 2017 IEEE aerospace conference. IEEE; 2017, March. pp. 1–9.

Ayoub S, Behera NR, Raju MN, Singh P, Praveena S, Ravikiran K. Hyperparameter tuned deep learning model for healthcare monitoring system in big data. In: 2023 International conference on intelligent data communication technologies and internet of things (IDCIoT). IEEE; 2023, January. pp. 281–287.

Gupta P, Chouhan AV, Wajeed MA, Tiwari S, Bist AS, Puri SC. Prediction of health monitoring with deep learning using edge computing. Measurement Sens. 2023;25:100604.

Talukdar J, Singh TP. Early prediction of cardiovascular disease using artificial neural network. Paladyn J Behav Robot. 2023;14(1):20220107.

Baik SM, Hong KS, Park DJ. Deep learning approach for early prediction of COVID-19 mortality using chest X-ray and electronic health records. BMC Bioinform. 2023;24(1):1–13.

Alwakid G, Gouda W, Humayun M. Deep learning-based prediction of diabetic retinopathy using CLAHE and ESRGAN for enhancement. Healthcare. 2023;11(6):863.

Basak H, Yin Z. Pseudo-label guided contrastive learning for semi-supervised medical image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2023. pp. 19786–19797.

Wang H, Li X. Dhc: dual-debiased heterogeneous co-training framework for class-imbalanced semi-supervised medical image segmentation. In: Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors. International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland; 2023. p. 582–91.

Basak H, Yin Z. Semi-supervised domain adaptive medical image segmentation through consistency regularized disentangled contrastive learning. In: Greenspan H, Madabhushi A, Mousavi P, Salcudean S, Duncan J, Syeda-Mahmood T, Taylor R, editors. International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland; 2023. p. 260–70.

Chaitanya K, Erdil E, Karani N, Konukoglu E. Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation. Med Image Anal. 2023;87: 102792.

Zhang Z, Ran R, Tian C, Zhou H, Li X, Yang F, Jiao Z. Self-aware and cross-sample prototypical learning for semi-supervised medical image segmentation; 2023. arXiv preprint arXiv:2305.16214 .

Basak H, Chattopadhyay S, Kundu R, Nag S, Mallipeddi R. Ideal: improved dense local contrastive learning for semi-supervised medical image segmentation. In: ICASSP 2023–2023 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE; 2023, June. pp. 1–5.

Lou A, Tawfik K, Yao X, Liu Z, Noble J. Min–max similarity: a contrastive semi-supervised deep learning network for surgical tools segmentation. IEEE Trans Med Imaging. 2023;42(10):2832–41.

Bai Y, Chen D, Li Q, Shen W, Wang Y. Bidirectional copy-paste for semi-supervised medical image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition; 2023. pp. 11514–11524.

Yang Y, Fu H, Aviles-Rivero AI, Schönlieb CB, Zhu L. Diffmic: dual-guidance diffusion network for medical image classification. In: International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland; 2023, October. pp. 95–105.

Zhou L, Liu H, Bae J, He J, Samaras D, Prasanna P. Self pre-training with masked autoencoders for medical image classification and segmentation. In: 2023 IEEE 20th international symposium on biomedical imaging (ISBI). IEEE; 2023, April. pp. 1–6.

Wu J, Fu R, Fang H, Zhang Y, Yang Y, Xiong H, Xu Y. Medsegdiff: medical image segmentation with diffusion probabilistic model. In: Medical imaging with deep learning. PMLR; 2023, January. pp. 1623–1639.

Kamana E, Zhao J, Bai D. Predicting the impact of climate change on the re-emergence of malaria cases in China using LSTMSeq2Seq deep learning model: a modelling and prediction analysis study. BMJ Open. 2022;12(3): e053922.

Alanazi R. Identification and prediction of chronic diseases using machine learning approach. J Healthcare Eng. 2022;2022:1–9.

Gouda W, Almurafeh M, Humayun M, Jhanjhi NZ. Detection of covid-19 based on chest x-rays using deep learning. Healthcare. 2022;10(2):343.

Kumar A, Satyanarayana Reddy SS, Mahommad GB, Khan B, Sharma R. Smart healthcare: disease prediction using the cuckoo-enabled deep classifier in IoT framework. Sci Program. 2022. https://doi.org/10.1155/2022/2090681 .

Basak H, Ghosal S, Sarkar R. Addressing class imbalance in semi-supervised image segmentation: a study on cardiac MRI. In: International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland; 2022, September. pp. 224–233.

Chaitanya K, Audigier C, Balascuta LE, Mansi T. Automatic planning of liver tumor thermal ablation using deep reinforcement learning. In: International conference on medical imaging with deep learning. PMLR; 2022, December. pp. 219–230.

Basak H, Kundu R, Sarkar R. MFSNet: a multi focus segmentation network for skin lesion segmentation. Pattern Recogn. 2022;128: 108673.

Monday HN, Li JP, Nneji GU, James EC, Chikwendu IA, Ejiyi CJ et al. The capability of multi resolution analysis: a case study of COVID-19 diagnosis. In: 2021 4th International conference on pattern recognition and artificial intelligence (PRAI). IEEE; 2021, August. pp. 236–242.

Al Rahhal M, Bazi Y, Jomaa RM, Zuair M, Al Ajlan N. Deep learning approach for COVID-19 detection in computed tomography images. Comput Mater Continua. 2021;67(2):2093.

Men L, Ilk N, Tang X, Liu Y. Multi-disease prediction using LSTM recurrent neural networks. Expert Syst Appl. 2021;177: 114905.

Ahmad U, Song H, Bilal A, Mahmood S, Alazab M, Jolfaei A, et al. A novel deep learning model to secure internet of things in healthcare. In: Maleh Y, Shojafar M, Alazab M, Baddi Y, editors., et al., Machine intelligence and big data analytics for cybersecurity applications. Cham: Springer International Publishing; 2021. p. 341–53.

Irmak E. COVID-19 disease severity assessment using CNN model. IET Image Proc. 2021;15(8):1814–24.

Feng K, He F, Steinmann J, Demirkiran I. Deep-learning based approach to identify COVID-19. In: SoutheastCon 2021. IEEE; 2021, March. pp. 1–4.

Mansour RF, El Amraoui A, Nouaouri I, Díaz VG, Gupta D, Kumar S. Artificial intelligence and internet of things enabled disease diagnosis model for smart healthcare systems. IEEE Access. 2021;9:45137–46.

Kundu R, Basak H, Singh PK, Ahmadian A, Ferrara M, Sarkar R. Fuzzy rank-based fusion of CNN models using Gompertz function for screening COVID-19 CT-scans. Sci Rep. 2021;11(1):14133.

Çinar A, Yildirim M. Classification of malaria cell images with deep learning architectures. Ingén Syst Inf. 2020;25(1):35.

Rahman T, Khandakar A, Kadir MA, Islam KR, Islam KF, Mazhar R, et al. Reliable tuberculosis detection using chest X-ray with deep learning, segmentation and visualization. IEEE Access. 2020;8:191586–601.

Sevi M, Aydin İ. COVID-19 detection using deep learning methods. In: 2020 International conference on data analytics for business and industry: way towards a sustainable economy (ICDABI). IEEE; 2020, October. pp. 1–6.

Martinsson J, Schliep A, Eliasson B, Mogren O. Blood glucose prediction with variance estimation using recurrent neural networks. J Healthc Inf Res. 2020;4:1–18.

Zhang J, Xie Y, Pang G, Liao Z, Verjans J, Li W, et al. Viral pneumonia screening on chest X-rays using confidence-aware anomaly detection. IEEE Trans Med Imaging. 2020;40(3):879–90.

Hemdan EED, Shouman MA, Karar ME. Covidx-net: a framework of deep learning classifiers to diagnose covid-19 in x-ray images; 2020. arXiv preprint arXiv:2003.11055 .

Zhu T, Li K, Chen J, Herrero P, Georgiou P. Dilated recurrent neural networks for glucose forecasting in type 1 diabetes. J Healthc Inf Res. 2020;4:308–24.

Wang W, Tong M, Yu M. Blood glucose prediction with VMD and LSTM optimized by improved particle swarm optimization. IEEE Access. 2020;8:217908–16.

Rashid N, Hossain MAF, Ali M, Sukanya MI, Mahmud T, Fattah SA. Transfer learning based method for COVID-19 detection from chest X-ray images. In: 2020 IEEE region 10 conference (TENCON). IEEE; 2020, November. pp. 585–590.

Arora P, Kumar H, Panigrahi BK. Prediction and analysis of COVID-19 positive cases using deep learning models: a descriptive case study of India. Chaos Solitons Fract. 2020;139: 110017.

Zaitcev A, Eissa MR, Hui Z, Good T, Elliott J, Benaissa M. A deep neural network application for improved prediction of HbA 1c in type 1 diabetes. IEEE J Biomed Health Inform. 2020;24(10):2932–41.

Naz H, Ahuja S. Deep learning approach for diabetes prediction using PIMA Indian dataset. J Diabetes Metab Disord. 2020;19:391–403.

Wang S, Zha Y, Li W, Wu Q, Li X, Niu M, et al. A fully automatic deep learning system for COVID-19 diagnostic and prognostic analysis. Eur Respir J. 2020;56(2):2000775.

Cheon S, Kim J, Lim J. The use of deep learning to predict stroke patient mortality. Int J Environ Res Public Health. 2019;16(11):1876.

Li K, Liu C, Zhu T, Herrero P, Georgiou P. GluNet: a deep learning framework for accurate glucose forecasting. IEEE J Biomed Health Inform. 2019;24(2):414–23.

Guo Y, Ashour AS. Neutrosophic multiple deep convolutional neural network for skin dermoscopic image classification. In: Guo Y, Ashour AS, editors. Neutrosophic set in medical image analysis. New York: Academic Press; 2019. p. 269–85.

Landman BA, Lyu I, Huo Y, Asman AJ. Multiatlas segmentation. In: Handbook of medical image computing and computer assisted intervention. Amsterdam: Elsevier; 2020. p. 137–64.

Miotto R, Wang F, Wang S, Jiang X, Dudley JT. Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform. 2018;19(6):1236–46.

White G, Ariyachandra T. Big data and ethics: examining the grey areas of big data analytics. Issues Inf Syst. 2016;17(4):1–7.

Download references

Author information

Authors and affiliations.

Department of Information Systems and Technology, Faculty of Graduate Studies for Statistical Research, Cairo University, Giza, Egypt

Mohammed Badawy & Nagy Ramadan

Department of Computer Sciences, Faculty of Graduate Studies for Statistical Research, Cairo University, Giza, Egypt

Hesham Ahmed Hefny

You can also search for this author in PubMed   Google Scholar

Contributions

All authors have participated equally in this work.

Corresponding author

Correspondence to Mohammed Badawy .

Ethics declarations

Conflict of interest.

The authors declare that they have no competing interests. All authors approved the final manuscript.

Research Involving Human and/or Animals

Informed consent, additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Badawy, M., Ramadan, N. & Hefny, H.A. A Survey on Deep Learning Techniques for Predictive Analytics in Healthcare. SN COMPUT. SCI. 5 , 860 (2024). https://doi.org/10.1007/s42979-024-03188-3

Download citation

Received : 09 October 2023

Accepted : 29 July 2024

Published : 04 September 2024

DOI : https://doi.org/10.1007/s42979-024-03188-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Artificial intelligence
• Deep learning
• Healthcare prediction
• Machine learning
• Find a journal
• Publish with us
• Track your research

• Systematic Review
• Open access
• Published: 02 September 2024

Common data quality elements for health information systems: a systematic review

• Hossein Ghalavand 1 ,
• Saied Shirshahi 2 ,
• Alireza Rahimi 2 ,
• Zarrin Zarrinabadi 1 &
• Fatemeh Amani 3  

BMC Medical Informatics and Decision Making volume  24 , Article number:  243 ( 2024 ) Cite this article

Metrics details

Data quality in health information systems has a complex structure and consists of several dimensions. This research conducted for identify Common data quality elements for health information systems.

A literature review was conducted and search strategies run in Web of Knowledge, Science Direct, Emerald, PubMed, Scopus and Google Scholar search engine as an additional source for tracing references. We found 760 papers, excluded 314 duplicates, 339 on abstract review and 167 on full-text review; leaving 58 papers for critical appraisal.

Current review shown that 14 criteria are categorized as the main dimensions for data quality for health information system include: Accuracy, Consistency, Security, Timeliness, Completeness, Reliability, Accessibility, Objectivity, Relevancy, Understandability, Navigation, Reputation, Efficiency and Value- added. Accuracy, Completeness, and Timeliness, were the three most-used dimensions in literature.

Conclusions

At present, there is a lack of uniformity and potential applicability in the dimensions employed to evaluate the data quality of health information system. Typically, different approaches (qualitative, quantitative and mixed methods) were utilized to evaluate data quality for health information system in the publications that were reviewed. Consequently, due to the inconsistency in defining dimensions and assessing methods, it became imperative to categorize the dimensions of data quality into a limited set of primary dimensions.

Peer Review reports

Appropriate planning in the health sector relies on the existence of accurate data and the quality of the data must be continuously controlled. The World Health Organization has tried to ensure the quality of health data by providing a toolkit. This toolkit supports countries to assess and improve the quality of health data [ 1 , 2 ].

The existence of accurate, complete, and timely data plays an important role in health care management [ 3 , 4 , 5 ]. Data quality is often only considered a component of the effectiveness of health information systems, and hiding the value of data quality in other parts of the health field can lead to incorrect decision-making [ 6 , 7 , 8 , 9 ]. Previous studies have confirmed that data quality is a multidimensional concept. Data quality assessment requires familiarity with different subjective and objective criteria and both subjective perceptions of people and objective measurements of information must be addressed [ 10 , 11 ]. Qualitative evaluations of subjective data reflect the needs and experiences of stakeholders, and objective evaluations reflect the needs of managers and stakeholders [ 12 ].

Adverse effects on the quality of care, increasing costs, creating liability risks, and reducing the benefits of investing in health information systems can be identified as the negative effects of poor-quality data [ 13 , 14 , 15 , 16 ]. Defects in data quality can lead to incorrect diagnosis and intervention in health care [ 4 , 13 , 17 , 18 ]. The quality of healthcare depends on the existence of quality data, which ultimately leads to a significant impact on customer satisfaction [ 13 , 19 ].

Data quality in health information systems has a complex structure and consists of several dimensions and some critical factors performance such as environmental and organizational, technical and behavioral affected on data quality in health information system [ 20 , 21 , 22 ]. As we mentioned later, previous studies have sporadically reported some data quality elements in health information systems. There is no comprehensive agreement on its dimensions and there is no unique accepted definition of data quality among researchers for health information systems. However, there is still a lack of a review compiling and synthesizing all elements introduced in the literature. In this study, a more comprehensive understanding of the elements for quality of data in health information systems has been done using a systematic review method. The findings of this study can provide opportunities for health policy maker to become familiar with various data quality elements in health information. This systematic review specifically answered the following research questions:

1- What are the common data quality elements for health information systems?

2- What are the roles of common data quality elements to improve the performance of health information systems?

In this review, we used a systematic approach to retrieve the relevant research studies. Our reporting strategy follows the PRISMA guidelines [ 23 ].

Eligibility criteria

In this study the inclusion criteria were: (1) Data quality components were showcased within a health information system; (2) published from the year 2003 to 2024; (3) empirical studies that answered the research questions or tested the hypothesis and conducted on specific health system The exclusion criteria were: (1) Research that did not outline data quality dimensions in health management systems; (2) Content presented in a format other than a scientific article such as Conference papers, book sections, and …; (4) Methodologies deemed to be deficient in terms of quality; (5) Publication language not in English; and (7) The full text was unavailable.

Information sources

The literature search was conducted between September and October 2023, using the following five electronic scientific databases: Web of Knowledge, Science Direct, Emerald, PubMed, Scopus and Google Scholar search engine as an additional source for tracing references.

Search strategy

This study used a systematized review approach to identify common data quality elements for health information systems. The following keywords were used in the search strategy: Data quality, Health, clinic, Hospital, Medical, Information system. The keywords chosen were searched using various combinations and in the fields of title, abstract, subject, and keyword. We considered the search features in each database and used the Boolean operators (AND, OR) to combine and search selected keywords. An example of the search strategy was given in Table  1 .

Study selection

All the results were imported into EndNote reference management software. The duplicate and non-journal papers were removed. Next, the title and abstract of the remaining articles were screened to detect subject relevance with the research objectives. The selected articles were analyzed based on the inclusion and exclusion criteria. Finally, the reference lists of all identified articles were searched for additional studies. Two researchers undertook the screening of titles and abstracts obtained through the searches. A sample of just over 20% of articles was double screened in order to assess the level of agreement between the researchers. Disagreements were resolved through discussion or consultation with a third researcher.

Data collection process

Data extraction was completed independently by two assessors. The data were extracted from including four sections: bibliographic information, methodology, and the data quality elements investigated, and key findings. Each study was treated as a single unit of analysis and the relevant information in each study was extracted using a designated data extraction form.

Information was extracted from each included study (including first author, title, publication date, type of study, methodology, processes of knowledge management that were studied and selected results). We emphasize the results of selected papers that have reported elements for assessment data quality in health information systems.

Risk of bias in individual studies

In this study, we used the Joanna Briggs Institute (JBI) checklist [ 24 ] for quality assessment. The authors assessed the included studies with a further random examination by two independent reviewers. The results of the quality assessment were compared any disagreements between the reviewers were addressed through discussion or by involving a third reviewer.

Synthesis of results

In this review, by adopting similar identifies elements as broader themes, the results of the included studies were analyzed and categorized. Finally, the homogeneous data quality elements in health information systems were synthesized and described.

Risk of bias within studies

The JBI checklist was applied to all 58 studies; none were excluded based on quality assessment and all studies were rated as unclear or high risk of bias. In 16% of studies, we cannot find “statement locating the researcher culturally or theoretically” and in 37%, “influence of the researcher on the research” is not addressed.

The search for systematic reviews identified 734 references published between 2003 and 2024. Title and abstract review selected 167 references for full text review. In the analysis, it was found that 68 papers did not address research questions or test hypotheses, 32 papers lacked discussion on data quality dimensions in health management systems, and nine documents presented content in a format other than a scientific article.

Out of the 58 selected paper for final review, 42 were released between 2013 and 2024 [ 1 , 4 , 5 , 7 , 8 , 9 , 10 , 11 , 14 , 15 , 16 , 17 , 18 , 21 , 22 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 ]. Thirteen papers looked at information quality [ 7 , 11 , 14 , 27 , 28 , 29 , 31 , 37 , 52 , 54 , 55 , 56 ], five at content quality [ 7 , 15 , 21 , 43 , 50 ], and thirty-six at data quality [ 4 , 5 , 10 , 14 , 17 , 20 , 21 , 27 , 28 , 29 , 31 , 32 , 33 , 36 , 37 , 42 , 43 , 44 , 47 , 49 , 50 , 51 , 52 , 53 , 55 , 57 , 58 , 59 , 60 ]. None of the publications, however, made a distinction between “data” and “information,” or between “data quality” and “information quality.” As a result, “information quality” and “data quality” were used synonymously [ 21 ]. The search results and the study selection process are presented in Fig.  1 .

Flow diagram of study selection process

Evaluating the quality of the data was the primary goal of the reviewed studies [ 4 , 5 , 10 , 13 , 14 , 15 , 17 , 18 , 19 , 20 , 21 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 35 , 36 , 37 , 38 , 39 , 41 , 42 , 43 , 44 , 45 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 55 , 56 , 57 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ].Two paper focused on information quality in health systems [ 11 , 52 ]. Methods for evaluating the quality of data were presented in eight publications [ 10 , 20 , 21 , 35 , 38 , 41 , 51 , 52 ], 19 publications tended to conduct on the health information [ 5 , 8 , 10 , 11 , 16 , 17 , 20 , 21 , 22 , 26 , 31 , 37 , 42 , 47 , 49 , 50 , 51 , 55 , 57 , 60 , 66 ] and eight paper focus on health or medical records as an information system in health context [ 13 , 19 , 25 , 38 , 44 , 45 , 64 , 67 ].

To describe data quality, the studies employed a total of 57 dimensions. The first data quality attribute for health information system that was most often used was accuracy [ 4 , 5 , 15 , 17 , 19 , 28 , 29 , 32 , 33 , 34 , 37 , 41 , 43 , 45 , 46 , 49 , 51 , 53 , 59 ], second is completeness [ 4 , 5 , 20 , 28 , 29 , 30 , 41 , 44 , 45 , 46 , 48 , 49 , 51 , 52 , 53 , 56 ], and third most-frequently criterion is timeliness [ 5 , 28 , 41 , 44 , 45 , 51 ]. Table  2 displays the common dimensions of data quality in health information systems that derived from existing literature.

Data accuracy measures the extent to which information accurately represents the objects or events. The accuracy of the information that is gathered, utilized, and stored is assessed through data accuracy. It is imperative for records to serve as a dependable source of information and to facilitate the generation of valuable insights through analysis. Maintaining high data accuracy guarantees that records and datasets meet the standards for reliability and trustworthiness, allowing for their use in decision-making and various applications [ 4 , 5 , 17 , 28 , 29 , 32 , 34 ]. Correctness, precision, free of error, validity, believability and integrity are common terms that use for describe data accuracy [ 21 ]. Data believability relates to whether the data is regarded as being true, real, and credible. Data believability is based on user’s perceptions [ 1 , 36 , 40 ].

Data consistency is the state in which all copies or instances of data are identical across various information systems. This uniformity is crucial in maintaining the accuracy, currency, and coherence of data across different platforms and applications. It is essential for instilling trust in users accessing the data. Implementing data validation rules, employing data standardization techniques, and utilizing data synchronization processes are some strategies to uphold data consistency. By ensuring data consistency, organizations can provide users with reliable information for making informed decisions, streamline operations, minimize errors, and enhance efficiency [ 9 , 45 , 48 , 51 , 52 , 65 ].

Data security is the practice of protecting information from corruption, theft, or unauthorized access throughout its life cycle. This involves safeguarding hardware, software, storage devices, and user devices, as well as implementing access controls, administrative controls, and organizational policies. By utilizing tools and technologies that enhance visibility of data usage, such as data masking, encryption, and redaction, organizations can ensure the security of their data. Moreover, data security assists organizations in streamlining auditing procedures and complying with data protection regulations, ultimately reducing the risk of cyber-attacks, human error, and insider threats [ 5 , 48 , 56 ]. Secure access, safe, confidentiality and privacy are common terms that use for describe data security [ 21 ].

Data timeliness denotes the currency and availability of data at the required time for its intended use. This is critical for enabling health organizations to make swift and accurate decisions based on the most up-to-date information. The timeliness of data has an impact on data quality as it determines the reliability and usefulness of information systems. Moreover, timely data can lead to cost savings as organizations can utilize real-time data to effectively manage inventories, optimize delivery routes, and coordinate with suppliers, thus reducing the risk of stock outs, minimizing delivery delays, and ensuring smooth operations [ 5 , 25 , 28 , 41 , 44 , 45 , 51 ].

Completeness of data refers to the extent to which information includes all necessary elements and observations for a specific purpose. This factor enhances the integrity and reliability of analyses, preventing gaps in understanding and supporting more robust decision-making processes. In a complete dataset, all variables relevant to the presentation of information should be present and fully populated with valid data values. Any missing, incorrect, or incomplete entries in the dataset can compromise the quality of analyses, interpretations, and decisions based on that data [ 4 , 5 , 9 , 28 , 29 , 30 , 41 , 44 , 45 , 52 ]. Coverage, comprehensiveness, appropriate amount, adequate, appropriate amount of data and integrity are common terms that use for describe data completeness [ 21 ]. The amount of data indicates the extent of data sets obtained for analysis and processing. In present-day information systems, these sets of data are frequently observed to be escalating in size, reaching capacities such as terabytes and petabytes [ 4 , 29 , 50 , 57 ].

Data reliability pertains to the uniformity of data across various records, programs, or platforms, as well as the credibility of the data source. Reliable data remains consistently accurate, while unreliable data may not always be valid, making it challenging to ascertain its accuracy. Consequently, organizations cannot depend on unreliable data for decision-making. Data reliability, also referred to as data observability, represents the trustworthiness of data and the insights derived from it for enabling sound decision-making. Reliability is characterized by two other fundamental elements of data quality include accuracy and consistency [ 9 , 49 , 53 , 57 , 59 , 65 ].

Data accessibility refers to the ease with which users can locate, retrieve, comprehend, and utilize data within an organization’s information systems. This is crucial in the modern digital landscape, where data is valuable for decision-making, strategic planning, and operational efficiency. Ensuring data accessibility involves creating an environment where data is available, understandable, and usable by individuals with varying levels of technical expertise. This approach is closely tied to data democratization, which aims to break down silos and make data available across different levels and departments of an organization. A well-implemented data accessibility strategy ensures that data is not locked away in isolated information systems but is integrated and accessible, contributing to a more informed and agile organizational structure. The ultimate goal is to empower users to leverage data in their daily tasks and decision-making processes, thus fostering a data-driven culture [ 4 , 26 , 29 , 33 , 50 , 57 ].

Data Objectivity refers to the extent to which data is free from personal biases, emotions, and subjective interpretations. Objective data is verifiable, reliable, and accurate, meaning that it can be verified independently by multiple parties. In other words, objective data is based on facts rather than opinions or judgments. In the context of information systems, data objectivity is crucial because it enables organizations to make informed decisions based on accurate and reliable information. Objective data helps to reduce errors, inconsistencies, and uncertainties, ensuring that business processes are efficient, effective, and compliant with regulatory requirements. Data objectivity in information systems is often hindered by biases in data collection, data quality issues, information overload, and lack of standardization. Biases may arise from human error, sampling errors, or deliberate data manipulation during the collection process. Inaccuracies, inconsistencies, and incompleteness resulting from poor data quality can compromise the objectivity of the information. The overwhelming amount of data available can make it challenging to differentiate between objective and subjective information. Inconsistencies in data representation and interpretation may occur due to the use of different systems or formats [ 36 , 41 , 44 , 45 , 46 ].

Data relevancy is an aspect of data quality that determines whether the data used or generated are relevant to add to the new target system and how usable it is for users [ 9 , 29 , 45 , 48 , 51 ]. Ease of operation, Usability, applicable, utility, Usefulness, Perceived usefulness and importance are common terms that use for describe data relevancy [ 21 ]. The concept of data usability revolves around a user’s ability to obtain meaningful information from various systems. When data is stored in text files that demand prolonged and intricate processing before it can be analyzed, its usability is limited. Conversely, data that is conveniently displayed on a performance dashboard for immediate interpretation is classified as highly usable [ 4 , 25 , 29 , 45 , 48 , 50 ]. The concept of data usefulness denotes the level at which data, post-analysis, aligns with the intended purpose within a given context for its user or consumer. In most cases, data usefulness is attained when all criteria related to data quality, such as dependability, thoroughness, uniformity, and others, are fulfilled [ 43 , 50 , 52 ].

Data Understandability refer to the level at which data exhibits qualities that facilitate understanding and analysis by users, and are presented in relevant languages, symbols, and measurements within a defined context of utilization [ 22 , 34 , 37 , 46 ]. Interpretability, ease of understanding, granularity and transparency are common terms that use for describe data understandability [ 21 ].

Data navigation refers to the process of searching, locating, and extracting relevant data from a vast pool of information to support decision-making, problem-solving, or analysis. It involves the utilization of different techniques and tools to navigate through extensive data, identify patterns, trends, and correlations, and present the information in a meaningful and actionable way. The success of data navigation is contingent upon several dimensions, including technical, domain knowledge, systems, methodological, and human dimensions. The technical dimension involves mastering programming languages like SQL and Python, utilizing data visualization software such as Tableau and Power BI, and implementing data mining techniques like machine learning algorithms. Domain knowledge dimension stresses the importance of expertise in specific fields. Information system dimension highlights the role of databases, data warehouses, cloud storage platforms, and other technologies in facilitating data navigation by storing, managing, and providing access to data. Methodological dimension focuses on statistical analysis, data mining techniques, and data visualization methods as key approaches to navigating data. Lastly, human dimension recognizes the significance of communication skills, collaboration, and critical thinking in the process of data navigation [ 4 , 50 , 65 , 68 ].

Data reputation is the evaluation of the trustworthiness, reliability, and credibility of data in an information system. It signifies the extent to which stakeholders, such as users, decision-makers, and other systems, perceive the data as accurate, reliable, and complete. Within an information system, data reputation plays a crucial role in decision-making, trust, system performance, and data sharing [ 42 , 60 , 61 ].

The concept of data efficiency revolves around an organization’s effectiveness in maximizing the value obtained from its data, while simultaneously minimizing the resources essential for processing, storing, and up keeping that data. Put simply, data efficiency focuses on streamlining the collection, storage, analysis, and utilization of data to meet objectives. When considering an information system, data efficiency can be examined from various angles, such as efficiency in data acquisition, storage, processing, analysis, visualization, security, retention, and archiving [ 7 , 28 , 29 , 48 ].

Data value-added pertains to the process of refining raw data into more useful, meaningful, and valuable information that can support decision-making, drive business outcomes, and create a competitive advantage. This process involves extracting insights, patterns, or trends from large datasets and presenting them in a manner that is easy to understand and act upon. By prioritizing these dimensions of data value-added within an information system, organizations can ensure that their data is transformed into valuable insights that support informed decision-making and drive business outcomes [ 5 , 22 , 25 , 45 ].

In a few papers, the concept of “fitness for use” was applied to data quality [ 6 , 55 , 69 ]. Two viewpoints can be used to characterize data quality: (1) the inherent quality of the data elements and set, and (2) how the set satisfies the needs of the user. The definition provided by the International Standards Organization best captures the accepted meaning of data quality, which is “the totality of features and characteristics of an entity that bears on its ability to satisfy stated and implied needs” [ 4 , 15 , 28 , 33 , 53 ].

Current review study identified 14 common dimensions for data quality in health information system. In related research data quality dimensions classified on four dimensions include: intrinsic (accuracy, objectivity, reputation), contextual timeliness, completeness, and relevancy), representational (representational format, understandability, consistency), and accessibility (accessibility, security) categories [ 53 , 60 , 69 , 70 , 71 ]. There exists a certain level of intersection between the aspects of data quality recognized in this review and those research in prior classifications of data quality.

Previous literature has often discussed intrinsic data quality in terms of the absence of defects, as indicated by various dimensions such as accuracy, perfection, freshness, and uniformity [ 72 ]. and “completeness, unambiguity, meaningless and correctness” [ 54 , 73 , 74 ]. The Canadian Institute for Health Information put forth a set of 69 quality criteria, organized into 24 quality characteristics, and further classified into 6 quality dimensions: accuracy, timeliness, comparability, usability, relevance, and privacy & security [ 58 , 71 ]. Research on data quality has primarily concentrated on recognizing general quality traits like accuracy, currency, completeness, correctness, consistency, and timeliness as fundamental aspects of data quality applicable across different fields. Nevertheless, existing reviews reveal a lack of consensus regarding the conceptual framework and definition of data quality [ 70 , 73 ]. However, our pervious review shows there is a lack of consensus conceptual framework and definition for data quality [ 1 , 71 ].

In this study, the three most-frequently used dimensions of data quality were accuracy, completeness and timeliness, respectively. This arrangement is somewhat different from previous literature in which the three most-frequently used dimensions were arranged in the order of completeness, accuracy, and timeliness, respectively [ 43 , 51 , 53 ]. Furthermore, the absence of a precise definition of the data quality dimensions led to complexities in evaluating them. The definitions of dimensions and their associated metrics were occasionally based on intuition, past experiences, or the underlying goals. These results indicate that data quality is a multi-faceted phenomenon. Likewise, other scholars argue that data quality is a multi-dimensional notion [ 5 , 28 , 38 , 52 , 61 ].

The Health Information Systems heavily rely on data, as they perform essential functions like generation, compilation, analysis, synthesis, communication, and data application to support decision-making. The literature frequently evaluates the dimensions of data quality, but there is currently a lack of consistency and potential generalizability in using these dimensions and methods to assess data quality in Health Information Systems. In this review of the literature, the data quality for health information system were examined and identified 14 common dimension include: Accuracy, Consistency, Security, Timeliness, Completeness, Reliability, Accessibility, Objectivity, Relevancy, Understandability, Navigation, Reputation, Efficiency and Value- added.

The quality of data in health information systems is indispensable for healthcare institutions to make well-informed decisions and provide patients with optimal care. Accurate and timely data assists healthcare organizations and professionals in identifying patterns, predicting outcomes, and enhancing patient results. Conversely, inadequate data quality in healthcare or other data-related issues can lead to inaccurate diagnoses, inappropriate treatments, and harm to patients. To ensure data quality in healthcare, organizations must prioritize investments in data governance, data management, and data analysis tools, while also maintaining a continuous process of monitoring and improving data quality in health information systems.

It is essential to have high-quality data in order to ensure the safe and dependable delivery of healthcare services. Health facility data plays a crucial role in monitoring performance. While various organizations may prioritize different aspects of data quality, it is important to acknowledge that no health data, regardless of its source, can be deemed flawless. All data are susceptible to various limitations related to data quality, including missing values, bias, measurement error, and human errors in data entry and computation. These limitations are associated with technical, behavioral, and organizational factors [ 75 ].

This study has limitations. Firstly, the number of articles with complete data was relatively small. Secondly, assessing the quality of some studies were difficult because the quality assessment criteria were not clearly identified. We have proposed four fundamental implications to inspire future research. Firstly, it is crucial for researchers to give equal attention to all dimensions of data quality, as these dimensions can have both direct and indirect effects on data quality outcomes. Secondly, researchers should aim to evaluate the existing data quality models and frameworks through a combination of mixed methods and case study designs. Thirdly, it is important to identify the underlying causes of data quality issues in health information systems. Lastly, efforts should be made to develop interventions that can effectively address and prevent data quality issues from occurring.

Data availability

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Joanna Briggs Institute

Liaw S-T, et al. Quality assessment of real-world data repositories across the data life cycle: a literature review. J Am Med Inform Assoc. 2021;28(7):1591–9.

Article   PubMed   PubMed Central   Google Scholar  

WHO. Data Quality Assurance (DQA) . Health Service Data 2022 [cited 2022 2022]; https://www.who.int/data/data-collection-tools/health-service-data/data-quality-assurance-dqa#:~:text=WHO%20has%20produced%20the%20Data,annual%20data%20quality%20desk%20review

FMoH E. Health sector transformation plan . 2015, Addis Ababa, Ethiopia.

Rumisha SF, et al. Data quality of the routine health management information system at the primary healthcare facility and district levels in Tanzania. BMC Med Inf Decis Mak. 2020;20(1):340.

Article   Google Scholar  

Chekol A, et al. Data quality and associated factors of routine health information system among health centers of West Gojjam Zone, northwest Ethiopia, 2021. Front Health Serv. 2023;3:1059611.

Pipino LL, Lee YW, Wang RY. Data quality assessment. Commun ACM. 2002;45(4):211–8.

Ouedraogo M, et al. A quality assessment of Health Management Information System (HMIS) data for maternal and child health in Jimma Zone, Ethiopia. PLoS ONE. 2019;14(3):e0213600.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Lemma S, et al. Improving quality and use of routine health information system data in low-and middle-income countries: a scoping review. PLoS ONE. 2020;15(10):e0239683.

Bammidi TR, et al. The crucial role of Data Quality in Automated decision-making systems. Int J Manage Educ Sustainable Dev. 2024;7(7):22.

Google Scholar  

Adane A, et al. Exploring data quality and use of the routine health information system in Ethiopia: a mixed-methods study. BMJ open. 2021;11(12):e050356.

Mohammed SA, Yusof MM. Towards an evaluation framework for information quality management (IQM) practices for health information systems–evaluation criteria for effective IQM practices. J Eval Clin Pract. 2013;19(2):379–87.

Article   PubMed   Google Scholar  

Long J, Seko C. A New Method for Database Data Quality Evaluation at the Canadian Institute for Health Information (CIHI) . in ICIQ . 2002. Citeseer.

Adeleke IT, et al. Data quality assessment in healthcare: a 365-day chart review of inpatients’ health records at a Nigerian tertiary hospital. J Am Med Inform Assoc. 2012;19(6):1039–42.

Singh M, et al. Health management information system data quality under NRHM in District Sonipat, Haryana. Int J Health Sci Res (IJHSR). 2016;6(9):11–4.

CAS   Google Scholar  

Harrison K, Rahimi N. Carolina Danovaro-Holliday, factors limiting data quality in the expanded programme on immunization in low and middle-income countries: a scoping review . Vaccine. 2020;38(30):4652–63.

Shama AT, et al. Assessment of quality of routine health information system data and associated factors among departments in public health facilities of Harari region, Ethiopia. BMC Med Inf Decis Mak. 2021;21(1):1–12.

Bosch-Capblanch X, et al. Does an innovative paper-based health information system (PHISICC) improve data quality and use in primary healthcare? Protocol of a multicountry, cluster randomised controlled trial in sub-saharan African rural settings. BMJ Open. 2021;11(7):e051823.

Ehsani-Moghaddam B, Martin K, Queenan JA. Data quality in healthcare: a report of practical experience with the Canadian Primary Care Sentinel Surveillance Network data. Health Inform Manage J. 2021;50(1–2):88–92.

Brown PJB, Warmington V. Data quality probes—exploiting and improving the quality of electronic patient record data and patient care. Int J Med Informatics. 2002;68(1):91–8.

Lima CR, et al. [Review of data quality dimensions and applied methods in the evaluation of health information systems]. Cad Saude Publica. 2009;25(10):2095–109.

Alipour J, Ahmadi M. Dimensions and assessment methods of data quality in health information systems. Acta Med Mediterranea. 2017;33(2):313–20.

Tolera A et al. Barriers to healthcare data quality and recommendations in public health facilities in dire Dawa city administration, eastern Ethiopia: a qualitative study. Front Digit Health, 2024. 6.

Vrabel M. M. Preferred reporting items for systematic reviews and meta-analyses . In Oncology nursing forum . Oncology Nursing Society; 2015.

JBI QARI Critical appraisal checklist for interpretive & critical research . The Joanna Briggs Institute, Adelaide 2018; http://joannabriggs.org/research/critical-appraisal-tools.html

Fraser HSF, et al. Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the role of clinical Alerts: cross-sectional observational study. JMIR Public Health Surveill. 2024;10:e49127.

Madandola OO, et al. The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review. J Am Med Inform Assoc. 2023;31(1):240–55.

Getachew N, Erkalo B, Garedew MG. Data quality and associated factors in the health management information system at health centers in Shashogo district, Hadiya Zone, southern Ethiopia, 2021. Volume 22. BMC Medical Informatics and Decision Making; 2022. pp. 1–9. 1.

Solomon M, et al. Data quality assessment and associated factors in the health management information system among health centers of Southern Ethiopia. PLoS ONE. 2021;16(10):e0255949.

Moukénet A, et al. Health management information system (HMIS) data quality and associated factors in Massaguet district, Chad. BMC Med Inf Decis Mak. 2021;21(1):326.

do Einloft N. Data quality and arbovirus infection associated factors in pregnant and non-pregnant women of childbearing age in Brazil: a surveillance database analysis. One Health. 2021;12:100244.

Ayele W et al. Data quality and it’s correlation with routine health information system structure and input at public health centers in Addis Ababa, Ethiopia. Ethiop J Health Dev, 2021. 35(1).

Mulissa Z, et al. Effect of data quality improvement intervention on health management information system data accuracy: an interrupted time series analysis. PLoS ONE. 2020;15(8):e0237703.

Yourkavitch J, Prosnitz D, Herrera S. Data quality assessments stimulate improvements to health management information systems: evidence from five African countries. J Glob Health. 2019;9(1):010806.

Endriyas M, et al. Understanding performance data: health management information system data accuracy in Southern Nations nationalities and people’s Region, Ethiopia. BMC Health Serv Res. 2019;19(1):1–6.

Biancone P, et al. Data quality methods and applications in health care system: a systematic literature review. Int J Bus Manage. 2019;14(4):35–47.

Liu Y, et al. [Designing and implementation of the data quality control in the information system of air pollution and health impact monitoring]. Wei Sheng Yan Jiu. 2018;47(2):277–80.

PubMed   Google Scholar  

Kumar M, et al. Research gaps in routine health information system design barriers to data quality and use in low- and middle-income countries: a literature review. Int J Health Plann Manage. 2018;33(1):e1–9.

Feder SL. Data quality in electronic health records research: quality domains and assessment methods. West J Nurs Res. 2018;40(5):753–66.

Watson NL, et al. Data management and data quality in PERCH, a large international case-control study of severe childhood pneumonia. Clin Infect Dis. 2017;64(suppl3):S238–44.

Wagenaar BH, et al. Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia. BMC Health Serv Res. 2017;17:65–75.

Puttkammer N, et al. Identifying priorities for data quality improvement within Haiti׳s iSanté EMR system: comparing two methods. Health Policy Technol. 2017;6(1):93–104.

Finnegan K, et al. Barriers and facilitators of Data Quality and Use in Malawi’s Health Information System. Annals Global Health. 2017;83(1):36–7.

Chen H, et al. Data Quality of the Chinese National AIDS Information System: a critical review. Stud Health Technol Inf. 2017;245:1352.

Woinarowicz M, Howell M. The impact of electronic health record (EHR) interoperability on immunization information system (IIS) data quality. Online J Public Health Inf. 2016;8(2):e184.

Puttkammer N, et al. An assessment of data quality in a multi-site electronic medical record system in Haiti. Int J Med Informatics. 2016;86:104–16.

Article   CAS   Google Scholar  

Nicol E, Dudley L, Bradshaw D. Assessing the quality of routine data for the prevention of mother-to-child transmission of HIV: an analytical observational study in two health districts with high HIV prevalence in South Africa. Int J Med Informatics. 2016;95:60–70.

Wagenaar BH, et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009–2012. Popul Health Metr. 2015;13:9.

Taggart J, Liaw S-T, Yu H. Structured data quality reports to improve EHR data quality. Int J Med Informatics. 2015;84(12):1094–8.

Glèlè Ahanhanzo Y, et al. Data quality assessment in the routine health information system: an application of the Lot Quality Assurance Sampling in Benin. Health Policy Plan. 2015;30(7):837–43.

Glèlè Ahanhanzo Y, et al. Factors associated with data quality in the routine health information system of Benin. Arch Public Health. 2014;72(1):25.

Chen H, et al. A review of data quality assessment methods for public health information systems. Int J Environ Res Public Health. 2014;11(5):5170–207.

Hahn D, Wanjala P, Marx M. Where is information quality lost at clinical level? A mixed-method study on information systems and data quality in three urban Kenyan ANC clinics. Glob Health Action. 2013;6:21424.

Chen H, Yu P, Wang N. Do we have the reliable data? An exploration of data quality for AIDS information system in China. Stud Health Technol Inf. 2013;192:1042.

Choquet R, et al. The Information Quality Triangle: a methodology to assess clinical information quality , in MEDINFO 2010 . IOS; 2010. pp. 699–703.

Mettler T, Rohner P, Baacke L. Improving data quality of health information systems: a holistic design-oriented approach. 2008.

Sørensen HT, et al. Identification of cases of meningococcal disease: data quality in two Danish population-based information systems during a 14-year period. Int J Risk Saf Med. 1995;7(3):179–89.

Gimbel S, et al. An assessment of routine primary care health information system data quality in Sofala Province, Mozambique. Popul Health Metr. 2011;9:12.

Kerr K, Norris T, Stockdale R. Data quality information and decision making: a healthcare case study. ACIS 2007 proceedings, 2007: p. 98.

Ben Saïd M, et al. A multi-source information System via the internet for end-stage renal disease: Scalability and Data Quality. Stud Health Technol Inf. 2005;116:994–9.

Fletcher DM. Achieving data quality. How data from a pediatric health information system earns the trust of its users. J Ahima. 2004;75(10):22–6.

Bean KP. Data quality in hospital strategic information systems: a summary of survey findings. Top Health Inf Manage. 1994;15(2):13–25.

CAS   PubMed   Google Scholar  

Kelly A, Becker W. Nutrition information systems and data quality requirements. WHO Reg Publ Eur Ser. 1991;34:15–24.

Leitheiser RL. Data quality in health care data warehouse environments . in Proceedings of the 34th annual Hawaii international conference on system sciences . 2001. IEEE.

Ndira S, Rosenberger K, Wetter T. Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda). Methods Inf Med. 2008;47(06):489–98.

Article   CAS   PubMed   Google Scholar  

Silva AA, et al. [Evaluation of data quality from the information system on live births in 1997–1998]. Rev Saude Publica. 2001;35(6):508–14.

Woelk GB, Moyo IM, Ray CS. A health information system revised. Part II: improving data quality and utilization. Cent Afr J Med. 1987;33(7):170–3.

Abbasi R, Khajouei R, Sadeqi M, Jabali. Timeliness and accuracy of information sharing from hospital information systems to electronic health record in Iran. J Health Adm. 2019;22(2):28–40.

Elavsky F, Nadolskis L, Moritz D. Data navigator: an accessibility-centered data navigation toolkit. IEEE Trans Vis Comput Graph. 2023;20(1):16–25.

Wang RY. A product perspective on total data quality management. Commun ACM. 1998;41(2):58–65.

Liaw S-T et al. Data quality and fitness for purpose of routinely collected data–a general practice case study from an electronic practice-based research network (ePBRN) . in AMIA Annual Symposium Proceedings . 2011. American Medical Informatics Association.

Rahimi A, et al. Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review. Decis Analytics. 2014;1:1–31.

Redman TC. Measuring data accuracy: A framework and review. Information quality, 2014: pp. 21–36.

Orme AM, Yao H, Etzkorn LH. Indicating ontology data quality, stability, and completeness throughout ontology evolution. J Softw Maintenance Evolution: Res Pract. 2007;19(1):49–75.

Yao H, Orme AM, Etzkorn L. Cohesion metrics for ontology design and application. J Comput Sci. 2005;1(1):107–13.

Endriyas M, et al. Understanding performance data: health management information system data accuracy in Southern Nations nationalities and people’s Region, Ethiopia. BMC Health Serv Res. 2019;19(175):1–6.

Download references

Acknowledgements

Not applicable.

This study was supported by Abadan University of medical sciences, Research code: 1557.

Author information

Authors and affiliations.

Department of Medical library and Information Science, Abadan University of Medical Sciences, Abadan, Iran

Hossein Ghalavand & Zarrin Zarrinabadi

Department of Medical library and Information Science, School of Health Management and Information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran

Saied Shirshahi & Alireza Rahimi

Student Research Committee, Abadan University of Medical Sciences, Abadan, Iran

Fatemeh Amani

You can also search for this author in PubMed   Google Scholar

Contributions

Hossein Ghalavand and Saied Shirshahi Conceived the study, prepared the analysis plan, conducted the analysis, and prepared the draft manuscript. Alireza Rahimi, Zarrin Zarrinabadi and Fatemeh Amani Conceived the study, prepared the analysis plan, performed the literature search, screening for study inclusion/exclusion, and risk of bias assessment, conducted the analysis, and prepared the draft manuscript. All authors contributed to the final version of the manuscript.

Corresponding author

Correspondence to Hossein Ghalavand .

Ethics declarations

Ethics approval and consent to participate.

This research study was approved by Ethics Committee in Biomedical Research at Abadan University of Medical Sciences (Ethical code: IR.ABADANUMS.REC.1401.122).

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ghalavand, H., Shirshahi, S., Rahimi, A. et al. Common data quality elements for health information systems: a systematic review. BMC Med Inform Decis Mak 24 , 243 (2024). https://doi.org/10.1186/s12911-024-02644-7

Download citation

Received : 17 April 2024

Accepted : 21 August 2024

Published : 02 September 2024

DOI : https://doi.org/10.1186/s12911-024-02644-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Data quality
• Health Information System
• Systematic review

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

• General enquiries: [email protected]

Peer Reviewed

GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation

Article metrics.

CrossRef Citations

Altmetric Score

PDF Downloads

Academic journals, archives, and repositories are seeing an increasing number of questionable research papers clearly produced using generative AI. They are often created with widely available, general-purpose AI applications, most likely ChatGPT, and mimic scientific writing. Google Scholar easily locates and lists these questionable papers alongside reputable, quality-controlled research. Our analysis of a selection of questionable GPT-fabricated scientific papers found in Google Scholar shows that many are about applied, often controversial topics susceptible to disinformation: the environment, health, and computing. The resulting enhanced potential for malicious manipulation of society’s evidence base, particularly in politically divisive domains, is a growing concern.

Swedish School of Library and Information Science, University of Borås, Sweden

Department of Arts and Cultural Sciences, Lund University, Sweden

Division of Environmental Communication, Swedish University of Agricultural Sciences, Sweden

Research Questions

• Where are questionable publications produced with generative pre-trained transformers (GPTs) that can be found via Google Scholar published or deposited?
• What are the main characteristics of these publications in relation to predominant subject categories?
• How are these publications spread in the research infrastructure for scholarly communication?
• How is the role of the scholarly communication infrastructure challenged in maintaining public trust in science and evidence through inappropriate use of generative AI?

research note Summary

• A sample of scientific papers with signs of GPT-use found on Google Scholar was retrieved, downloaded, and analyzed using a combination of qualitative coding and descriptive statistics. All papers contained at least one of two common phrases returned by conversational agents that use large language models (LLM) like OpenAI’s ChatGPT. Google Search was then used to determine the extent to which copies of questionable, GPT-fabricated papers were available in various repositories, archives, citation databases, and social media platforms.
• Roughly two-thirds of the retrieved papers were found to have been produced, at least in part, through undisclosed, potentially deceptive use of GPT. The majority (57%) of these questionable papers dealt with policy-relevant subjects (i.e., environment, health, computing), susceptible to influence operations. Most were available in several copies on different domains (e.g., social media, archives, and repositories).
• Two main risks arise from the increasingly common use of GPT to (mass-)produce fake, scientific publications. First, the abundance of fabricated “studies” seeping into all areas of the research infrastructure threatens to overwhelm the scholarly communication system and jeopardize the integrity of the scientific record. A second risk lies in the increased possibility that convincingly scientific-looking content was in fact deceitfully created with AI tools and is also optimized to be retrieved by publicly available academic search engines, particularly Google Scholar. However small, this possibility and awareness of it risks undermining the basis for trust in scientific knowledge and poses serious societal risks.

Implications

The use of ChatGPT to generate text for academic papers has raised concerns about research integrity. Discussion of this phenomenon is ongoing in editorials, commentaries, opinion pieces, and on social media (Bom, 2023; Stokel-Walker, 2024; Thorp, 2023). There are now several lists of papers suspected of GPT misuse, and new papers are constantly being added. 1 See for example Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . While many legitimate uses of GPT for research and academic writing exist (Huang & Tan, 2023; Kitamura, 2023; Lund et al., 2023), its undeclared use—beyond proofreading—has potentially far-reaching implications for both science and society, but especially for their relationship. It, therefore, seems important to extend the discussion to one of the most accessible and well-known intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated  (Simon et al., 2023).

Google Scholar, https://scholar.google.com , is an easy-to-use academic search engine. It is available for free, and its index is extensive (Gusenbauer & Haddaway, 2020). It is also often touted as a credible source for academic literature and even recommended in library guides, by media and information literacy initiatives, and fact checkers (Tripodi et al., 2023). However, Google Scholar lacks the transparency and adherence to standards that usually characterize citation databases. Instead, Google Scholar uses automated crawlers, like Google’s web search engine (Martín-Martín et al., 2021), and the inclusion criteria are based on primarily technical standards, allowing any individual author—with or without scientific affiliation—to upload papers to be indexed (Google Scholar Help, n.d.). It has been shown that Google Scholar is susceptible to manipulation through citation exploits (Antkare, 2020) and by providing access to fake scientific papers (Dadkhah et al., 2017). A large part of Google Scholar’s index consists of publications from established scientific journals or other forms of quality-controlled, scholarly literature. However, the index also contains a large amount of gray literature, including student papers, working papers, reports, preprint servers, and academic networking sites, as well as material from so-called “questionable” academic journals, including paper mills. The search interface does not offer the possibility to filter the results meaningfully by material type, publication status, or form of quality control, such as limiting the search to peer-reviewed material.

To understand the occurrence of ChatGPT (co-)authored work in Google Scholar’s index, we scraped it for publications, including one of two common ChatGPT responses (see Appendix A) that we encountered on social media and in media reports (DeGeurin, 2024). The results of our descriptive statistical analyses showed that around 62% did not declare the use of GPTs. Most of these GPT-fabricated papers were found in non-indexed journals and working papers, but some cases included research published in mainstream scientific journals and conference proceedings. 2 Indexed journals mean scholarly journals indexed by abstract and citation databases such as Scopus and Web of Science, where the indexation implies journals with high scientific quality. Non-indexed journals are journals that fall outside of this indexation. More than half (57%) of these GPT-fabricated papers concerned policy-relevant subject areas susceptible to influence operations. To avoid increasing the visibility of these publications, we abstained from referencing them in this research note. However, we have made the data available in the Harvard Dataverse repository.

The publications were related to three issue areas—health (14.5%), environment (19.5%) and computing (23%)—with key terms such “healthcare,” “COVID-19,” or “infection”for health-related papers, and “analysis,” “sustainable,” and “global” for environment-related papers. In several cases, the papers had titles that strung together general keywords and buzzwords, thus alluding to very broad and current research. These terms included “biology,” “telehealth,” “climate policy,” “diversity,” and “disrupting,” to name just a few.  While the study’s scope and design did not include a detailed analysis of which parts of the articles included fabricated text, our dataset did contain the surrounding sentences for each occurrence of the suspicious phrases that formed the basis for our search and subsequent selection. Based on that, we can say that the phrases occurred in most sections typically found in scientific publications, including the literature review, methods, conceptual and theoretical frameworks, background, motivation or societal relevance, and even discussion. This was confirmed during the joint coding, where we read and discussed all articles. It became clear that not just the text related to the telltale phrases was created by GPT, but that almost all articles in our sample of questionable articles likely contained traces of GPT-fabricated text everywhere.

Evidence hacking and backfiring effects

Generative pre-trained transformers (GPTs) can be used to produce texts that mimic scientific writing. These texts, when made available online—as we demonstrate—leak into the databases of academic search engines and other parts of the research infrastructure for scholarly communication. This development exacerbates problems that were already present with less sophisticated text generators (Antkare, 2020; Cabanac & Labbé, 2021). Yet, the public release of ChatGPT in 2022, together with the way Google Scholar works, has increased the likelihood of lay people (e.g., media, politicians, patients, students) coming across questionable (or even entirely GPT-fabricated) papers and other problematic research findings. Previous research has emphasized that the ability to determine the value and status of scientific publications for lay people is at stake when misleading articles are passed off as reputable (Haider & Åström, 2017) and that systematic literature reviews risk being compromised (Dadkhah et al., 2017). It has also been highlighted that Google Scholar, in particular, can be and has been exploited for manipulating the evidence base for politically charged issues and to fuel conspiracy narratives (Tripodi et al., 2023). Both concerns are likely to be magnified in the future, increasing the risk of what we suggest calling evidence hacking —the strategic and coordinated malicious manipulation of society’s evidence base.

The authority of quality-controlled research as evidence to support legislation, policy, politics, and other forms of decision-making is undermined by the presence of undeclared GPT-fabricated content in publications professing to be scientific. Due to the large number of archives, repositories, mirror sites, and shadow libraries to which they spread, there is a clear risk that GPT-fabricated, questionable papers will reach audiences even after a possible retraction. There are considerable technical difficulties involved in identifying and tracing computer-fabricated papers (Cabanac & Labbé, 2021; Dadkhah et al., 2023; Jones, 2024), not to mention preventing and curbing their spread and uptake.

However, as the rise of the so-called anti-vaxx movement during the COVID-19 pandemic and the ongoing obstruction and denial of climate change show, retracting erroneous publications often fuels conspiracies and increases the following of these movements rather than stopping them. To illustrate this mechanism, climate deniers frequently question established scientific consensus by pointing to other, supposedly scientific, studies that support their claims. Usually, these are poorly executed, not peer-reviewed, based on obsolete data, or even fraudulent (Dunlap & Brulle, 2020). A similar strategy is successful in the alternative epistemic world of the global anti-vaccination movement (Carrion, 2018) and the persistence of flawed and questionable publications in the scientific record already poses significant problems for health research, policy, and lawmakers, and thus for society as a whole (Littell et al., 2024). Considering that a person’s support for “doing your own research” is associated with increased mistrust in scientific institutions (Chinn & Hasell, 2023), it will be of utmost importance to anticipate and consider such backfiring effects already when designing a technical solution, when suggesting industry or legal regulation, and in the planning of educational measures.

Recommendations

Solutions should be based on simultaneous considerations of technical, educational, and regulatory approaches, as well as incentives, including social ones, across the entire research infrastructure. Paying attention to how these approaches and incentives relate to each other can help identify points and mechanisms for disruption. Recognizing fraudulent academic papers must happen alongside understanding how they reach their audiences and what reasons there might be for some of these papers successfully “sticking around.” A possible way to mitigate some of the risks associated with GPT-fabricated scholarly texts finding their way into academic search engine results would be to provide filtering options for facets such as indexed journals, gray literature, peer-review, and similar on the interface of publicly available academic search engines. Furthermore, evaluation tools for indexed journals 3 Such as LiU Journal CheckUp, https://ep.liu.se/JournalCheckup/default.aspx?lang=eng . could be integrated into the graphical user interfaces and the crawlers of these academic search engines. To enable accountability, it is important that the index (database) of such a search engine is populated according to criteria that are transparent, open to scrutiny, and appropriate to the workings of  science and other forms of academic research. Moreover, considering that Google Scholar has no real competitor, there is a strong case for establishing a freely accessible, non-specialized academic search engine that is not run for commercial reasons but for reasons of public interest. Such measures, together with educational initiatives aimed particularly at policymakers, science communicators, journalists, and other media workers, will be crucial to reducing the possibilities for and effects of malicious manipulation or evidence hacking. It is important not to present this as a technical problem that exists only because of AI text generators but to relate it to the wider concerns in which it is embedded. These range from a largely dysfunctional scholarly publishing system (Haider & Åström, 2017) and academia’s “publish or perish” paradigm to Google’s near-monopoly and ideological battles over the control of information and ultimately knowledge. Any intervention is likely to have systemic effects; these effects need to be considered and assessed in advance and, ideally, followed up on.

Our study focused on a selection of papers that were easily recognizable as fraudulent. We used this relatively small sample as a magnifying glass to examine, delineate, and understand a problem that goes beyond the scope of the sample itself, which however points towards larger concerns that require further investigation. The work of ongoing whistleblowing initiatives 4 Such as Academ-AI, https://www.academ-ai.info/ , and Retraction Watch, https://retractionwatch.com/papers-and-peer-reviews-with-evidence-of-chatgpt-writing/ . , recent media reports of journal closures (Subbaraman, 2024), or GPT-related changes in word use and writing style (Cabanac et al., 2021; Stokel-Walker, 2024) suggest that we only see the tip of the iceberg. There are already more sophisticated cases (Dadkhah et al., 2023) as well as cases involving fabricated images (Gu et al., 2022). Our analysis shows that questionable and potentially manipulative GPT-fabricated papers permeate the research infrastructure and are likely to become a widespread phenomenon. Our findings underline that the risk of fake scientific papers being used to maliciously manipulate evidence (see Dadkhah et al., 2017) must be taken seriously. Manipulation may involve undeclared automatic summaries of texts, inclusion in literature reviews, explicit scientific claims, or the concealment of errors in studies so that they are difficult to detect in peer review. However, the mere possibility of these things happening is a significant risk in its own right that can be strategically exploited and will have ramifications for trust in and perception of science. Society’s methods of evaluating sources and the foundations of media and information literacy are under threat and public trust in science is at risk of further erosion, with far-reaching consequences for society in dealing with information disorders. To address this multifaceted problem, we first need to understand why it exists and proliferates.

Finding 1: 139 GPT-fabricated, questionable papers were found and listed as regular results on the Google Scholar results page. Non-indexed journals dominate.

Most questionable papers we found were in non-indexed journals or were working papers, but we did also find some in established journals, publications, conferences, and repositories. We found a total of 139 papers with a suspected deceptive use of ChatGPT or similar LLM applications (see Table 1). Out of these, 19 were in indexed journals, 89 were in non-indexed journals, 19 were student papers found in university databases, and 12 were working papers (mostly in preprint databases). Table 1 divides these papers into categories. Health and environment papers made up around 34% (47) of the sample. Of these, 66% were present in non-indexed journals.

Finding 2: GPT-fabricated, questionable papers are disseminated online, permeating the research infrastructure for scholarly communication, often in multiple copies. Applied topics with practical implications dominate.

The 20 papers concerning health-related issues are distributed across 20 unique domains, accounting for 46 URLs. The 27 papers dealing with environmental issues can be found across 26 unique domains, accounting for 56 URLs.  Most of the identified papers exist in multiple copies and have already spread to several archives, repositories, and social media. It would be difficult, or impossible, to remove them from the scientific record.

As apparent from Table 2, GPT-fabricated, questionable papers are seeping into most parts of the online research infrastructure for scholarly communication. Platforms on which identified papers have appeared include ResearchGate, ORCiD, Journal of Population Therapeutics and Clinical Pharmacology (JPTCP), Easychair, Frontiers, the Institute of Electrical and Electronics Engineer (IEEE), and X/Twitter. Thus, even if they are retracted from their original source, it will prove very difficult to track, remove, or even just mark them up on other platforms. Moreover, unless regulated, Google Scholar will enable their continued and most likely unlabeled discoverability.

A word rain visualization (Centre for Digital Humanities Uppsala, 2023), which combines word prominences through TF-IDF 5 Term frequency–inverse document frequency , a method for measuring the significance of a word in a document compared to its frequency across all documents in a collection. scores with semantic similarity of the full texts of our sample of GPT-generated articles that fall into the “Environment” and “Health” categories, reflects the two categories in question. However, as can be seen in Figure 1, it also reveals overlap and sub-areas. The y-axis shows word prominences through word positions and font sizes, while the x-axis indicates semantic similarity. In addition to a certain amount of overlap, this reveals sub-areas, which are best described as two distinct events within the word rain. The event on the left bundles terms related to the development and management of health and healthcare with “challenges,” “impact,” and “potential of artificial intelligence”emerging as semantically related terms. Terms related to research infrastructures, environmental, epistemic, and technological concepts are arranged further down in the same event (e.g., “system,” “climate,” “understanding,” “knowledge,” “learning,” “education,” “sustainable”). A second distinct event further to the right bundles terms associated with fish farming and aquatic medicinal plants, highlighting the presence of an aquaculture cluster.  Here, the prominence of groups of terms such as “used,” “model,” “-based,” and “traditional” suggests the presence of applied research on these topics. The two events making up the word rain visualization, are linked by a less dominant but overlapping cluster of terms related to “energy” and “water.”

The bar chart of the terms in the paper subset (see Figure 2) complements the word rain visualization by depicting the most prominent terms in the full texts along the y-axis. Here, word prominences across health and environment papers are arranged descendingly, where values outside parentheses are TF-IDF values (relative frequencies) and values inside parentheses are raw term frequencies (absolute frequencies).

Finding 3: Google Scholar presents results from quality-controlled and non-controlled citation databases on the same interface, providing unfiltered access to GPT-fabricated questionable papers.

Google Scholar’s central position in the publicly accessible scholarly communication infrastructure, as well as its lack of standards, transparency, and accountability in terms of inclusion criteria, has potentially serious implications for public trust in science. This is likely to exacerbate the already-known potential to exploit Google Scholar for evidence hacking (Tripodi et al., 2023) and will have implications for any attempts to retract or remove fraudulent papers from their original publication venues. Any solution must consider the entirety of the research infrastructure for scholarly communication and the interplay of different actors, interests, and incentives.

We searched and scraped Google Scholar using the Python library Scholarly (Cholewiak et al., 2023) for papers that included specific phrases known to be common responses from ChatGPT and similar applications with the same underlying model (GPT3.5 or GPT4): “as of my last knowledge update” and/or “I don’t have access to real-time data” (see Appendix A). This facilitated the identification of papers that likely used generative AI to produce text, resulting in 227 retrieved papers. The papers’ bibliographic information was automatically added to a spreadsheet and downloaded into Zotero. 6 An open-source reference manager, https://zotero.org .

We employed multiple coding (Barbour, 2001) to classify the papers based on their content. First, we jointly assessed whether the paper was suspected of fraudulent use of ChatGPT (or similar) based on how the text was integrated into the papers and whether the paper was presented as original research output or the AI tool’s role was acknowledged. Second, in analyzing the content of the papers, we continued the multiple coding by classifying the fraudulent papers into four categories identified during an initial round of analysis—health, environment, computing, and others—and then determining which subjects were most affected by this issue (see Table 1). Out of the 227 retrieved papers, 88 papers were written with legitimate and/or declared use of GPTs (i.e., false positives, which were excluded from further analysis), and 139 papers were written with undeclared and/or fraudulent use (i.e., true positives, which were included in further analysis). The multiple coding was conducted jointly by all authors of the present article, who collaboratively coded and cross-checked each other’s interpretation of the data simultaneously in a shared spreadsheet file. This was done to single out coding discrepancies and settle coding disagreements, which in turn ensured methodological thoroughness and analytical consensus (see Barbour, 2001). Redoing the category coding later based on our established coding schedule, we achieved an intercoder reliability (Cohen’s kappa) of 0.806 after eradicating obvious differences.

The ranking algorithm of Google Scholar prioritizes highly cited and older publications (Martín-Martín et al., 2016). Therefore, the position of the articles on the search engine results pages was not particularly informative, considering the relatively small number of results in combination with the recency of the publications. Only the query “as of my last knowledge update” had more than two search engine result pages. On those, questionable articles with undeclared use of GPTs were evenly distributed across all result pages (min: 4, max: 9, mode: 8), with the proportion of undeclared use being slightly higher on average on later search result pages.

To understand how the papers making fraudulent use of generative AI were disseminated online, we programmatically searched for the paper titles (with exact string matching) in Google Search from our local IP address (see Appendix B) using the googlesearch – python library(Vikramaditya, 2020). We manually verified each search result to filter out false positives—results that were not related to the paper—and then compiled the most prominent URLs by field. This enabled the identification of other platforms through which the papers had been spread. We did not, however, investigate whether copies had spread into SciHub or other shadow libraries, or if they were referenced in Wikipedia.

We used descriptive statistics to count the prevalence of the number of GPT-fabricated papers across topics and venues and top domains by subject. The pandas software library for the Python programming language (The pandas development team, 2024) was used for this part of the analysis. Based on the multiple coding, paper occurrences were counted in relation to their categories, divided into indexed journals, non-indexed journals, student papers, and working papers. The schemes, subdomains, and subdirectories of the URL strings were filtered out while top-level domains and second-level domains were kept, which led to normalizing domain names. This, in turn, allowed the counting of domain frequencies in the environment and health categories. To distinguish word prominences and meanings in the environment and health-related GPT-fabricated questionable papers, a semantically-aware word cloud visualization was produced through the use of a word rain (Centre for Digital Humanities Uppsala, 2023) for full-text versions of the papers. Font size and y-axis positions indicate word prominences through TF-IDF scores for the environment and health papers (also visualized in a separate bar chart with raw term frequencies in parentheses), and words are positioned along the x-axis to reflect semantic similarity (Skeppstedt et al., 2024), with an English Word2vec skip gram model space (Fares et al., 2017). An English stop word list was used, along with a manually produced list including terms such as “https,” “volume,” or “years.”

• Artificial Intelligence
• / Search engines

Cite this Essay

Haider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024). GPT-fabricated scientific papers on Google Scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School (HKS) Misinformation Review . https://doi.org/10.37016/mr-2020-156

• / Appendix B

Bibliography

Antkare, I. (2020). Ike Antkare, his publications, and those of his disciples. In M. Biagioli & A. Lippman (Eds.), Gaming the metrics (pp. 177–200). The MIT Press. https://doi.org/10.7551/mitpress/11087.003.0018

Barbour, R. S. (2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? BMJ , 322 (7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Bom, H.-S. H. (2023). Exploring the opportunities and challenges of ChatGPT in academic writing: A roundtable discussion. Nuclear Medicine and Molecular Imaging , 57 (4), 165–167. https://doi.org/10.1007/s13139-023-00809-2

Cabanac, G., & Labbé, C. (2021). Prevalence of nonsensical algorithmically generated papers in the scientific literature. Journal of the Association for Information Science and Technology , 72 (12), 1461–1476. https://doi.org/10.1002/asi.24495

Cabanac, G., Labbé, C., & Magazinov, A. (2021). Tortured phrases: A dubious writing style emerging in science. Evidence of critical issues affecting established journals . arXiv. https://doi.org/10.48550/arXiv.2107.06751

Carrion, M. L. (2018). “You need to do your research”: Vaccines, contestable science, and maternal epistemology. Public Understanding of Science , 27 (3), 310–324. https://doi.org/10.1177/0963662517728024

Centre for Digital Humanities Uppsala (2023). CDHUppsala/word-rain [Computer software]. https://github.com/CDHUppsala/word-rain

Chinn, S., & Hasell, A. (2023). Support for “doing your own research” is associated with COVID-19 misperceptions and scientific mistrust. Harvard Kennedy School (HSK) Misinformation Review, 4 (3). https://doi.org/10.37016/mr-2020-117

Cholewiak, S. A., Ipeirotis, P., Silva, V., & Kannawadi, A. (2023). SCHOLARLY: Simple access to Google Scholar authors and citation using Python (1.5.0) [Computer software]. https://doi.org/10.5281/zenodo.5764801

Dadkhah, M., Lagzian, M., & Borchardt, G. (2017). Questionable papers in citation databases as an issue for literature review. Journal of Cell Communication and Signaling , 11 (2), 181–185. https://doi.org/10.1007/s12079-016-0370-6

Dadkhah, M., Oermann, M. H., Hegedüs, M., Raman, R., & Dávid, L. D. (2023). Detection of fake papers in the era of artificial intelligence. Diagnosis , 10 (4), 390–397. https://doi.org/10.1515/dx-2023-0090

DeGeurin, M. (2024, March 19). AI-generated nonsense is leaking into scientific journals. Popular Science. https://www.popsci.com/technology/ai-generated-text-scientific-journals/

Dunlap, R. E., & Brulle, R. J. (2020). Sources and amplifiers of climate change denial. In D.C. Holmes & L. M. Richardson (Eds.), Research handbook on communicating climate change (pp. 49–61). Edward Elgar Publishing. https://doi.org/10.4337/9781789900408.00013

Fares, M., Kutuzov, A., Oepen, S., & Velldal, E. (2017). Word vectors, reuse, and replicability: Towards a community repository of large-text resources. In J. Tiedemann & N. Tahmasebi (Eds.), Proceedings of the 21st Nordic Conference on Computational Linguistics (pp. 271–276). Association for Computational Linguistics. https://aclanthology.org/W17-0237

Google Scholar Help. (n.d.). Inclusion guidelines for webmasters . https://scholar.google.com/intl/en/scholar/inclusion.html

Gu, J., Wang, X., Li, C., Zhao, J., Fu, W., Liang, G., & Qiu, J. (2022). AI-enabled image fraud in scientific publications. Patterns , 3 (7), 100511. https://doi.org/10.1016/j.patter.2022.100511

Gusenbauer, M., & Haddaway, N. R. (2020). Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research Synthesis Methods , 11 (2), 181–217.   https://doi.org/10.1002/jrsm.1378

Haider, J., & Åström, F. (2017). Dimensions of trust in scholarly communication: Problematizing peer review in the aftermath of John Bohannon’s “Sting” in science. Journal of the Association for Information Science and Technology , 68 (2), 450–467. https://doi.org/10.1002/asi.23669

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research , 13 (4), 1148–1154. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10164801/

Jones, N. (2024). How journals are fighting back against a wave of questionable images. Nature , 626 (8000), 697–698. https://doi.org/10.1038/d41586-024-00372-6

Kitamura, F. C. (2023). ChatGPT is shaping the future of medical writing but still requires human judgment. Radiology , 307 (2), e230171. https://doi.org/10.1148/radiol.230171

Littell, J. H., Abel, K. M., Biggs, M. A., Blum, R. W., Foster, D. G., Haddad, L. B., Major, B., Munk-Olsen, T., Polis, C. B., Robinson, G. E., Rocca, C. H., Russo, N. F., Steinberg, J. R., Stewart, D. E., Stotland, N. L., Upadhyay, U. D., & Ditzhuijzen, J. van. (2024). Correcting the scientific record on abortion and mental health outcomes. BMJ , 384 , e076518. https://doi.org/10.1136/bmj-2023-076518

Lund, B. D., Wang, T., Mannuru, N. R., Nie, B., Shimray, S., & Wang, Z. (2023). ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing. Journal of the Association for Information Science and Technology, 74 (5), 570–581. https://doi.org/10.1002/asi.24750

Martín-Martín, A., Orduna-Malea, E., Ayllón, J. M., & Delgado López-Cózar, E. (2016). Back to the past: On the shoulders of an academic search engine giant. Scientometrics , 107 , 1477–1487. https://doi.org/10.1007/s11192-016-1917-2

Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E. (2021). Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A multidisciplinary comparison of coverage via citations. Scientometrics , 126 (1), 871–906. https://doi.org/10.1007/s11192-020-03690-4

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School (HKS) Misinformation Review, 4 (5). https://doi.org/10.37016/mr-2020-127

Skeppstedt, M., Ahltorp, M., Kucher, K., & Lindström, M. (2024). From word clouds to Word Rain: Revisiting the classic word cloud to visualize climate change texts. Information Visualization , 23 (3), 217–238. https://doi.org/10.1177/14738716241236188

Swedish Research Council. (2017). Good research practice. Vetenskapsrådet.

Stokel-Walker, C. (2024, May 1.). AI Chatbots Have Thoroughly Infiltrated Scientific Publishing . Scientific American. https://www.scientificamerican.com/article/chatbots-have-thoroughly-infiltrated-scientific-publishing/

Subbaraman, N. (2024, May 14). Flood of fake science forces multiple journal closures: Wiley to shutter 19 more journals, some tainted by fraud. The Wall Street Journal . https://www.wsj.com/science/academic-studies-research-paper-mills-journals-publishing-f5a3d4bc

The pandas development team. (2024). pandas-dev/pandas: Pandas (v2.2.2) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.10957263

Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science , 379 (6630), 313–313. https://doi.org/10.1126/science.adg7879

Tripodi, F. B., Garcia, L. C., & Marwick, A. E. (2023). ‘Do your own research’: Affordance activation and disinformation spread. Information, Communication & Society , 27 (6), 1212–1228. https://doi.org/10.1080/1369118X.2023.2245869

Vikramaditya, N. (2020). Nv7-GitHub/googlesearch [Computer software]. https://github.com/Nv7-GitHub/googlesearch

This research has been supported by Mistra, the Swedish Foundation for Strategic Environmental Research, through the research program Mistra Environmental Communication (Haider, Ekström, Rödl) and the Marcus and Amalia Wallenberg Foundation [2020.0004] (Söderström).

Competing Interests

The authors declare no competing interests.

The research described in this article was carried out under Swedish legislation. According to the relevant EU and Swedish legislation (2003:460) on the ethical review of research involving humans (“Ethical Review Act”), the research reported on here is not subject to authorization by the Swedish Ethical Review Authority (“etikprövningsmyndigheten”) (SRC, 2017).

This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided that the original author and source are properly credited.

Data Availability

All data needed to replicate this study are available at the Harvard Dataverse: https://doi.org/10.7910/DVN/WUVD8X

Acknowledgements

The authors wish to thank two anonymous reviewers for their valuable comments on the article manuscript as well as the editorial group of Harvard Kennedy School (HKS) Misinformation Review for their thoughtful feedback and input.

Here’s how you know

• U.S. Department of Health and Human Services
• National Institutes of Health

Acupuncture: Effectiveness and Safety

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What is acupuncture?

Acupuncture is a technique in which practitioners insert fine needles into the skin to treat health problems. The needles may be manipulated manually or stimulated with small electrical currents (electroacupuncture). Acupuncture has been in use in some form for at least 2,500 years. It originated from  traditional Chinese medicine but has gained popularity worldwide since the 1970s.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} How widely is acupuncture used?

According to the World Health Organization, acupuncture is used in 103 of 129 countries that reported data.

In the United States, data from the National Health Interview Survey show that the use of acupuncture by U.S. adults more than doubled between 2002 and 2022. In 2002, 1.0 percent of U.S. adults used acupuncture; in 2022, 2.2 percent used it. 

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What is acupuncture used for?

National survey data indicate that in the United States, acupuncture is most commonly used for pain, such as back, joint, or neck pain.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} How does acupuncture work scientifically?

How acupuncture works is not fully understood. However, there’s evidence that acupuncture may have effects on the nervous system, effects on other body tissues, and nonspecific (placebo) effects. 

• Studies in animals and people, including studies that used imaging methods to see what’s happening in the brain, have shown that acupuncture may affect nervous system function.
• Acupuncture may have direct effects on the tissues where the needles are inserted. This type of effect has been seen in connective tissue.
• Acupuncture has nonspecific effects (effects due to incidental aspects of a treatment rather than its main mechanism of action). Nonspecific effects may be due to the patient’s belief in the treatment, the relationship between the practitioner and the patient, or other factors not directly caused by the insertion of needles. In many studies, the benefit of acupuncture has been greater when it was compared with no treatment than when it was compared with sham (simulated or fake) acupuncture procedures, such as the use of a device that pokes the skin but does not penetrate it. These findings suggest that nonspecific effects contribute to the beneficial effect of acupuncture on pain or other symptoms. 
• In recent research, a nonspecific effect was demonstrated in a unique way: Patients who had experienced pain relief during a previous acupuncture session were shown a video of that session and asked to imagine the treatment happening again. This video-guided imagery technique had a significant pain-relieving effect.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What does research show about the effectiveness of acupuncture for pain?

Research has shown that acupuncture may be helpful for several pain conditions, including back or neck pain, knee pain associated with osteoarthritis, and postoperative pain. It may also help relieve joint pain associated with the use of aromatase inhibitors, which are drugs used in people with breast cancer. 

An analysis of data from 20 studies (6,376 participants) of people with painful conditions (back pain, osteoarthritis, neck pain, or headaches) showed that the beneficial effects of acupuncture continued for a year after the end of treatment for all conditions except neck pain.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Back or Neck Pain

• In a 2018 review, data from 12 studies (8,003 participants) showed acupuncture was more effective than no treatment for back or neck pain, and data from 10 studies (1,963 participants) showed acupuncture was more effective than sham acupuncture. The difference between acupuncture and no treatment was greater than the difference between acupuncture and sham acupuncture. The pain-relieving effect of acupuncture was comparable to that of nonsteroidal anti-inflammatory drugs (NSAIDs).
• A 2017 clinical practice guideline from the American College of Physicians included acupuncture among the nondrug options recommended as first-line treatment for chronic low-back pain. Acupuncture is also one of the treatment options recommended for acute low-back pain. The evidence favoring acupuncture for acute low-back pain was judged to be of low quality, and the evidence for chronic low-back pain was judged to be of moderate quality.

For more information, see the  NCCIH webpage on low-back pain .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Osteoarthritis

• In a 2018 review, data from 10 studies (2,413 participants) showed acupuncture was more effective than no treatment for osteoarthritis pain, and data from 9 studies (2,376 participants) showed acupuncture was more effective than sham acupuncture. The difference between acupuncture and no treatment was greater than the difference between acupuncture and sham acupuncture. Most of the participants in these studies had knee osteoarthritis, but some had hip osteoarthritis. The pain-relieving effect of acupuncture was comparable to that of NSAIDs.
• A 2018 review evaluated 6 studies (413 participants) of acupuncture for hip osteoarthritis. Two of the studies compared acupuncture with sham acupuncture and found little or no difference between them in terms of effects on pain. The other four studies compared acupuncture with a variety of other treatments and could not easily be compared with one another. However, one of the trials indicated that the addition of acupuncture to routine care by a physician may improve pain and function in patients with hip osteoarthritis.
• A 2019 clinical practice guideline from the American College of Rheumatology and the Arthritis Foundation conditionally recommends acupuncture for osteoarthritis of the knee, hip, or hand. The guideline states that the greatest number of studies showing benefits have been for knee osteoarthritis.

For more information, see the  NCCIH webpage on osteoarthritis .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Headache and Migraine

• A 2020   review of nine studies that compared acupuncture with various drugs for preventing migraine found that acupuncture was slightly more effective, and study participants who received acupuncture were much less likely than those receiving drugs to drop out of studies because of side effects.
• There’s moderate-quality evidence that acupuncture may reduce the frequency of migraines (from a 2016 evaluation of 22 studies with almost 5,000 people). The evidence from these studies also suggests that acupuncture may be better than sham acupuncture, but the difference is small. There is moderate- to low-quality evidence that acupuncture may reduce the frequency of tension headaches (from a 2016 evaluation of 12 studies with about 2,350 people).

For more information, see the  NCCIH webpage on headache .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Myofascial Pain Syndrome

• Myofascial pain syndrome is a common form of pain derived from muscles and their related connective tissue (fascia). It involves tender nodules called “trigger points.” Pressing on these nodules reproduces the patient’s pattern of pain.
• A combined analysis of a small number of studies of acupuncture for myofascial pain syndrome showed that acupuncture applied to trigger points had a favorable effect on pain intensity (5 studies, 215 participants), but acupuncture applied to traditional acupuncture points did not (4 studies, 80 participants).  

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Sciatica

• Sciatica involves pain, weakness, numbness, or tingling in the leg, usually on one side of the body, caused by damage to or pressure on the sciatic nerve—a nerve that starts in the lower back and runs down the back of each leg.
• Two 2015 evaluations of the evidence, one including 12 studies with 1,842 total participants and the other including 11 studies with 962 total participants, concluded that acupuncture may be helpful for sciatica pain, but the quality of the research is not good enough to allow definite conclusions to be reached.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Postoperative Pain

• A 2016 evaluation of 11 studies of pain after surgery (with a total of 682 participants) found that patients treated with acupuncture or related techniques 1 day after surgery had less pain and used less opioid pain medicine after the operation.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Cancer Pain

• A 2016 review of 20 studies (1,639 participants) indicated that acupuncture was not more effective in relieving cancer pain than conventional drug therapy. However, there was some evidence that acupuncture plus drug therapy might be better than drug therapy alone.
• A 2017 review of 5 studies (181 participants) of acupuncture for aromatase inhibitor-induced joint pain in breast cancer patients concluded that 6 to 8 weeks of acupuncture treatment may help reduce the pain. However, the individual studies only included small numbers of women and used a variety of acupuncture techniques and measurement methods, so they were difficult to compare.
• A larger 2018 study included 226 women with early-stage breast cancer who were taking aromatase inhibitors. The study found that the women who received 6 weeks of acupuncture treatment, given twice each week, reported less joint pain than the participants who received sham or no acupuncture.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Chronic Prostatitis/Chronic Pelvic Pain Syndrome

• Chronic prostatitis/chronic pelvic pain syndrome is a condition in men that involves inflammation of or near the prostate gland; its cause is uncertain.
• A review of 3 studies (204 total participants) suggested that acupuncture may reduce prostatitis symptoms, compared with a sham procedure. Because follow-up of the study participants was relatively brief and the numbers of studies and participants were small, a definite conclusion cannot be reached about acupuncture’s effects.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Irritable Bowel Syndrome

• A 2019 review of 41 studies (3,440 participants) showed that acupuncture was no more effective than sham acupuncture for symptoms of irritable bowel syndrome, but there was some evidence that acupuncture could be helpful when used in addition to other forms of treatment.

For more information, see the  NCCIH webpage on irritable bowel syndrome .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Fibromyalgia

• A 2019 review of 12 studies (824 participants) of people with fibromyalgia indicated that acupuncture was significantly better than sham acupuncture for relieving pain, but the evidence was of low-to-moderate quality.

For more information, see the  NCCIH webpage on fibromyalgia . 

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What does research show about acupuncture for conditions other than pain?

In addition to pain conditions, acupuncture has also been studied for at least 50 other health problems. There is evidence that acupuncture may help relieve seasonal allergy symptoms, stress incontinence in women, and nausea and vomiting associated with cancer treatment. It may also help relieve symptoms and improve the quality of life in people with asthma, but it has not been shown to improve lung function.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Seasonal Allergies (Allergic Rhinitis or Hay Fever)

• A 2015 evaluation of 13 studies of acupuncture for allergic rhinitis, involving a total of 2,365 participants, found evidence that acupuncture may help relieve nasal symptoms. The study participants who received acupuncture also had lower medication scores (meaning that they used less medication to treat their symptoms) and lower blood levels of immunoglobulin E (IgE), a type of antibody associated with allergies.
• A 2014 clinical practice guideline from the American Academy of Otolaryngology–Head and Neck Surgery included acupuncture among the options health care providers may offer to patients with allergic rhinitis.

For more information, see the  NCCIH webpage on seasonal allergies .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Urinary Incontinence

• Stress incontinence is a bladder control problem in which movement—coughing, sneezing, laughing, or physical activity—puts pressure on the bladder and causes urine to leak.
• In a 2017 study of about 500 women with stress incontinence, participants who received electroacupuncture treatment (18 sessions over 6 weeks) had reduced urine leakage, with about two-thirds of the women having a decrease in leakage of 50 percent or more. This was a rigorous study that met current standards for avoiding bias.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Treatment-Related Nausea and Vomiting in Cancer Patients

• Experts generally agree that acupuncture is helpful for treatment-related nausea and vomiting in cancer patients, but this conclusion is based primarily on research conducted before current guidelines for treating these symptoms were adopted. It’s uncertain whether acupuncture is beneficial when used in combination with current standard treatments for nausea and vomiting.

For more information, see the  NCCIH webpage on cancer .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Asthma

• In a study conducted in Germany in 2017, 357 participants receiving routine asthma care were randomly assigned to receive or not receive acupuncture, and an additional 1,088 people who received acupuncture for asthma were also studied. Adding acupuncture to routine care was associated with better quality of life compared to routine care alone.
• A review of 9 earlier studies (777 participants) showed that adding acupuncture to conventional asthma treatment improved symptoms but not lung function.

For more information, see the  NCCIH webpage on asthma .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Depression

• A 2018 review of 64 studies (7,104 participants) of acupuncture for depression indicated that acupuncture may result in a moderate reduction in the severity of depression when compared with treatment as usual or no treatment. However, these findings should be interpreted with caution because most of the studies were of low or very low quality.

For more information, see the  NCCIH webpage on depression .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Quitting Smoking

• In recommendations on smoking cessation treatment issued in 2021, the U.S. Preventive Services Task Force, a panel of experts that makes evidence-based recommendations about disease prevention, did not make a recommendation about the use of acupuncture as a stop-smoking treatment because only limited evidence was available. This decision was based on a 2014 review of 9 studies (1,892 participants) that looked at the effect of acupuncture on smoking cessation results for 6 months or more and found no significant benefit. Some studies included in that review showed evidence of a possible small benefit of acupuncture on quitting smoking for shorter periods of time.

For more information, see the  NCCIH webpage on quitting smoking .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Infertility

• A 2021 review evaluated 6 studies (2,507 participants) that compared the effects of acupuncture versus sham acupuncture on the success of in vitro fertilization as a treatment for infertility. No difference was found between the acupuncture and sham acupuncture groups in rates of pregnancy or live birth.
• A 2020 review evaluated 12 studies (1,088 participants) on the use of acupuncture to improve sperm quality in men who had low sperm numbers and low sperm motility. The reviewers concluded that the evidence was inadequate for firm conclusions to be drawn because of the varied design of the studies and the poor quality of some of them. 

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Carpal Tunnel Syndrome

• A 2018 review of 12 studies with 869 participants concluded that acupuncture and laser acupuncture (a treatment that uses lasers instead of needles) may have little or no effect on carpal tunnel syndrome symptoms in comparison with sham acupuncture. It’s uncertain how the effects of acupuncture compare with those of other treatments for this condition.    
• In a 2017 study not included in the review described above, 80 participants with carpal tunnel syndrome were randomly assigned to one of three interventions: (1) electroacupuncture to the more affected hand; (2) electroacupuncture at “distal” body sites, near the ankle opposite to the more affected hand; and (3) local sham electroacupuncture using nonpenetrating placebo needles. All three interventions reduced symptom severity, but local and distal acupuncture were better than sham acupuncture at producing desirable changes in the wrist and the brain.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Hot Flashes Associated With Menopause

• A 2018 review of studies of acupuncture for vasomotor symptoms associated with menopause (hot flashes and related symptoms such as night sweats) analyzed combined evidence from an earlier review of 15 studies (1,127 participants) and 4 newer studies (696 additional participants). The analysis showed that acupuncture was better than no acupuncture at reducing the frequency and severity of symptoms. However, acupuncture was not shown to be better than sham acupuncture.

For more information, see the  NCCIH webpage on menopause .

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} What is auricular acupuncture good for?

• Auricular acupuncture is a type of acupuncture that involves stimulating specific areas of the ear. 
• In a 2019 review of 15 studies (930 participants) of auricular acupuncture or auricular acupressure (a form of auricular therapy that does not involve penetration with needles), the treatment significantly reduced pain intensity, and 80 percent of the individual studies showed favorable effects on various measures related to pain.
• A 2020 review of 9 studies (783 participants) of auricular acupuncture for cancer pain showed that auricular acupuncture produced better pain relief than sham auricular acupuncture. Also, pain relief was better with a combination of auricular acupuncture and drug therapy than with drug therapy alone.
• An inexpensive, easily learned form of auricular acupuncture called “battlefield acupuncture” has been used by the U.S. Department of Defense and Department of Veterans Affairs to treat pain. However, a 2021 review of 9 studies (692 participants) of battlefield acupuncture for pain in adults did not find any significant improvement in pain when this technique was compared with no treatment, usual care, delayed treatment, or sham battlefield acupuncture.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Is acupuncture safe?

• Relatively few complications from using acupuncture have been reported. However, complications have resulted from use of nonsterile needles and improper delivery of treatments.
• When not delivered properly, acupuncture can cause serious adverse effects, including infections, punctured organs, and injury to the central nervous system.
• The U.S. Food and Drug Administration (FDA) regulates acupuncture needles as medical devices and requires that they be sterile and labeled for single use only.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Is acupuncture covered by health insurance?

• Some health insurance policies cover acupuncture, but others don’t. Coverage is often limited based on the condition being treated.
• An analysis of data from the Medical Expenditure Panel Survey, a nationally representative U.S. survey, showed that the share of adult acupuncturist visits with any insurance coverage increased from 41.1 percent in 2010–2011 to 50.2 percent in 2018–2019.
• Medicare covers acupuncture only for the treatment of chronic low-back pain. Coverage began in 2020. Up to 12 acupuncture visits are covered, with an additional 8 visits available if the first 12 result in improvement. Medicaid coverage of acupuncture varies from state to state.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Do acupuncturists need to be licensed?

• Most states license acupuncturists, but the requirements for licensing vary from state to state. To find out more about licensing of acupuncturists and other complementary health practitioners, visit the NCCIH webpage  Credentialing, Licensing, and Education . 

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} NCCIH-Funded Research

NCCIH funds research to evaluate acupuncture’s effectiveness for various kinds of pain and other conditions and to further understand how the body responds to acupuncture and how acupuncture might work. Some recent NCCIH-supported studies involve:

• Evaluating the feasibility of using acupuncture in hospital emergency departments.
• Testing whether the effect of acupuncture on chronic low-back pain can be enhanced by combining it with transcranial direct current stimulation.
• Evaluating a portable acupuncture-based nerve stimulation treatment for anxiety disorders.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} More To Consider

• Don’t use acupuncture to postpone seeing a health care provider about a health problem.
• Take charge of your health—talk with your health care providers about any complementary health approaches you use. Together, you can make shared, well-informed decisions.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} For More Information

Nccih clearinghouse.

The NCCIH Clearinghouse provides information on NCCIH and complementary and integrative health approaches, including publications and searches of Federal databases of scientific and medical literature. The Clearinghouse does not provide medical advice, treatment recommendations, or referrals to practitioners.

Toll-free in the U.S.: 1-888-644-6226

Telecommunications relay service (TRS): 7-1-1

Website: https://www.nccih.nih.gov

Email: [email protected] (link sends email)

Know the Science

NCCIH and the National Institutes of Health (NIH) provide tools to help you understand the basics and terminology of scientific research so you can make well-informed decisions about your health. Know the Science features a variety of materials, including interactive modules, quizzes, and videos, as well as links to informative content from Federal resources designed to help consumers make sense of health information.

Explaining How Research Works (NIH)

Know the Science: How To Make Sense of a Scientific Journal Article

Understanding Clinical Studies (NIH)

A service of the National Library of Medicine, PubMed® contains publication information and (in most cases) brief summaries of articles from scientific and medical journals. For guidance from NCCIH on using PubMed, see How To Find Information About Complementary Health Approaches on PubMed .

Website: https://pubmed.ncbi.nlm.nih.gov/

NIH Clinical Research Trials and You

The National Institutes of Health (NIH) has created a website, NIH Clinical Research Trials and You, to help people learn about clinical trials, why they matter, and how to participate. The site includes questions and answers about clinical trials, guidance on how to find clinical trials through ClinicalTrials.gov and other resources, and stories about the personal experiences of clinical trial participants. Clinical trials are necessary to find better ways to prevent, diagnose, and treat diseases.

Website: https://www.nih.gov/health-information/nih-clinical-research-trials-you

Research Portfolio Online Reporting Tools Expenditures & Results (RePORTER)

RePORTER is a database of information on federally funded scientific and medical research projects being conducted at research institutions.

Website: https://reporter.nih.gov

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Key References

• Befus D, Coeytaux RR, Goldstein KM, et al.  Management of menopause symptoms with acupuncture: an umbrella systematic review and meta-analysis . Journal of Alternative and Complementary Medicine. 2018;24(4):314-323.
• Bleck   R, Marquez E, Gold MA, et al.  A scoping review of acupuncture insurance coverage in the United States . Acupuncture in Medicine. 2020;964528420964214.
• Briggs JP, Shurtleff D.  Acupuncture and the complex connections between the mind and the body. JAMA. 2017;317(24):2489-2490.
• Brinkhaus B, Roll S, Jena S, et al.  Acupuncture in patients with allergic asthma: a randomized pragmatic trial. Journal of Alternative and Complementary Medicine. 2017;23(4):268-277.
• Chan MWC, Wu XY, Wu JCY, et al.  Safety of acupuncture: overview of systematic reviews. Scientific Reports. 2017;7(1):3369.
• Coyle ME, Stupans I, Abdel-Nour K, et al.  Acupuncture versus placebo acupuncture for in vitro fertilisation: a systematic review and meta-analysis. Acupuncture in Medicine. 2021;39(1):20-29.
• Hershman DL, Unger JM, Greenlee H, et al.  Effect of acupuncture vs sham acupuncture or waitlist control on joint pain related to aromatase inhibitors among women with early-stage breast cancer: a randomized clinical trial. JAMA. 2018;320(2):167-176.
• Linde K, Allais G, Brinkhaus B, et al.  Acupuncture for the prevention of episodic migraine. Cochrane Database of Systematic Reviews. 2016;(6):CD001218. Accessed at  cochranelibrary.com on February 12, 2021.
• Linde K, Allais G, Brinkhaus B, et al.  Acupuncture for the prevention of tension-type headache. Cochrane Database of Systematic Reviews. 2016;(4):CD007587. Accessed at  cochranelibrary.com on February 12, 2021.
• MacPherson H, Vertosick EA, Foster NE, et al. The persistence of the effects of acupuncture after a course of treatment: a meta-analysis of patients with chronic pain . Pain. 2017;158(5):784-793.
• Qaseem A, Wilt TJ, McLean RM, et al.  Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American College of Physicians. Annals of Internal Medicine. 2017;166(7):514-530.
• Seidman MD, Gurgel RK, Lin SY, et al.  Clinical practice guideline: allergic rhinitis. Otolaryngology—Head and Neck Surgery. 2015;152(suppl 1):S1-S43.
• Vickers AJ, Vertosick EA, Lewith G, et al. Acupuncture for chronic pain: update of an individual patient data meta-analysis . The Journal of Pain. 2018;19(5):455-474.
• White AR, Rampes H, Liu JP, et al.  Acupuncture and related interventions for smoking cessation. Cochrane Database of Systematic Reviews. 2014;(1):CD000009. Accessed at  cochranelibrary.com on February 17, 2021.
• Zia FZ, Olaku O, Bao T, et al.  The National Cancer Institute’s conference on acupuncture for symptom management in oncology: state of the science, evidence, and research gaps. Journal of the National Cancer Institute. Monographs. 2017;2017(52):lgx005.

.header_greentext{color:green!important;font-size:24px!important;font-weight:500!important;}.header_bluetext{color:blue!important;font-size:18px!important;font-weight:500!important;}.header_redtext{color:red!important;font-size:28px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;font-size:28px!important;font-weight:500!important;}.header_purpletext{color:purple!important;font-size:31px!important;font-weight:500!important;}.header_yellowtext{color:yellow!important;font-size:20px!important;font-weight:500!important;}.header_blacktext{color:black!important;font-size:22px!important;font-weight:500!important;}.header_whitetext{color:white!important;font-size:22px!important;font-weight:500!important;}.header_darkred{color:#803d2f!important;}.Green_Header{color:green!important;font-size:24px!important;font-weight:500!important;}.Blue_Header{color:blue!important;font-size:18px!important;font-weight:500!important;}.Red_Header{color:red!important;font-size:28px!important;font-weight:500!important;}.Purple_Header{color:purple!important;font-size:31px!important;font-weight:500!important;}.Yellow_Header{color:yellow!important;font-size:20px!important;font-weight:500!important;}.Black_Header{color:black!important;font-size:22px!important;font-weight:500!important;}.White_Header{color:white!important;font-size:22px!important;font-weight:500!important;} Other References

• Adams D, Cheng F, Jou H, et al. The safety of pediatric acupuncture: a systematic review. Pediatrics. 2011;128(6):e1575-1587.
• Candon M, Nielsen A, Dusek JA. Trends in insurance coverage for acupuncture, 2010-2019. JAMA Network Open. 2022;5(1):e2142509.
• Cao J, Tu Y, Orr SP, et al. Analgesic effects evoked by real and imagined acupuncture: a neuroimaging study. Cerebral Cortex. 2019;29(8):3220-3231.
• Centers for Medicare & Medicaid Services. Decision Memo for Acupuncture for Chronic Low Back Pain (CAG-00452N). Accessed at https://www.cms.gov/medicare-coverage-database/details/nca-decision-memo.aspx?NCAId=295 on June 25, 2021.
• Chen L, Lin C-C, Huang T-W, et al. Effect of acupuncture on aromatase inhibitor-induced arthralgia in patients with breast cancer: a meta-analysis of randomized controlled trials . The Breast. 2017;33:132-138. 
• Choi G-H, Wieland LS, Lee H, et al. Acupuncture and related interventions for the treatment of symptoms associated with carpal tunnel syndrome. Cochrane Database of Systematic Reviews. 2018;(12):CD011215. Accessed at cochranelibrary.com on January 28, 2021.
• Cui J, Wang S, Ren J, et al. Use of acupuncture in the USA: changes over a decade (2002–2012). Acupuncture in Medicine. 2017;35(3):200-207.
• Federman DG, Zeliadt SB, Thomas ER, et al. Battlefield acupuncture in the Veterans Health Administration: effectiveness in individual and group settings for pain and pain comorbidities. Medical Acupuncture. 2018;30(5):273-278.
• Feng S, Han M, Fan Y, et al. Acupuncture for the treatment of allergic rhinitis: a systematic review and meta-analysis. American Journal of Rhinology & Allergy. 2015;29(1):57-62.
• Franco JV, Turk T, Jung JH, et al. Non-pharmacological interventions for treating chronic prostatitis/chronic pelvic pain syndrome. Cochrane Database of Systematic Reviews. 2018;(5):CD012551. Accessed at cochranelibrary.com on January 28, 2021.
• Freeman MP, Fava M, Lake J, et al. Complementary and alternative medicine in major depressive disorder: the American Psychiatric Association task force report. The Journal of Clinical Psychiatry . 2010;71(6):669-681.
• Giovanardi CM, Cinquini M, Aguggia M, et al. Acupuncture vs. pharmacological prophylaxis of migraine: a systematic review of randomized controlled trials. Frontiers in Neurology. 2020;11:576272.
• Hu C, Zhang H, Wu W, et al. Acupuncture for pain management in cancer: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine. 2016;2016;1720239.
• Jiang C, Jiang L, Qin Q. Conventional treatments plus acupuncture for asthma in adults and adolescent: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine . 2019;2019:9580670.
• Ji M, Wang X, Chen M, et al. The efficacy of acupuncture for the treatment of sciatica: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine.  2015;2015:192808.
• Kaptchuk TJ. Acupuncture: theory, efficacy, and practice. Annals of Internal Medicine . 2002;136(5):374-383.
• Kolasinski SL, Neogi T, Hochberg MC, et al. 2019 American College of Rheumatology/Arthritis Foundation guideline for the management of osteoarthritis of the hand, hip, and knee. Arthritis Care & Research. 2020;72(2):149-162. 
• Langevin H. Fascia mobility, proprioception, and myofascial pain. Life. 2021;11(7):668. 
• Liu Z, Liu Y, Xu H, et al. Effect of electroacupuncture on urinary leakage among women with stress urinary incontinence: a randomized clinical trial. JAMA. 2017;317(24):2493-2501.
• MacPherson H, Hammerschlag R, Coeytaux RR, et al. Unanticipated insights into biomedicine from the study of acupuncture. Journal of Alternative and Complementary Medicine. 2016;22(2):101-107.
• Maeda Y, Kim H, Kettner N, et al. Rewiring the primary somatosensory cortex in carpal tunnel syndrome with acupuncture. Brain. 2017;140(4):914-927.
• Manheimer E, Cheng K, Wieland LS, et al. Acupuncture for hip osteoarthritis. Cochrane Database of Systematic Reviews. 2018;(5):CD013010. Accessed at cochranelibrary.com on February 17, 2021. 
• Moura CC, Chaves ECL, Cardoso ACLR, et al. Auricular acupuncture for chronic back pain in adults: a systematic review and metanalysis. Revista da Escola de Enfermagem da U S P. 2019;53:e03461.
• Nahin RL, Rhee A, Stussman B. Use of complementary health approaches overall and for pain management by US adults. JAMA. 2024;331(7):613-615.
• Napadow V. Neuroimaging somatosensory and therapeutic alliance mechanisms supporting acupuncture. Medical Acupuncture. 2020;32(6):400-402.
• Patnode CD, Henderson JT, Coppola EL, et al. Interventions for tobacco cessation in adults, including pregnant persons: updated evidence report and systematic review for the US Preventive Services Task Force. JAMA. 2021;325(3):280-298.
• Qin Z, Liu X, Wu J, et al. Effectiveness of acupuncture for treating sciatica: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine. 2015;2015;425108.
• Smith CA, Armour M, Lee MS, et al. Acupuncture for depression. Cochrane Database of Systematic Reviews. 2018;(3):CD004046. Accessed at cochranelibrary.com on January 20, 2021.
• US Preventive Services Task Force. Interventions for tobacco smoking cessation in adults, including pregnant persons. US Preventive Services Task Force recommendation statement. JAMA. 2021;325(3):265-279.
• Vase L, Baram S, Takakura N, et al. Specifying the nonspecific components of acupuncture analgesia. Pain. 2013;154(9):1659-1667.
• Wang R, Li X, Zhou S, et al. Manual acupuncture for myofascial pain syndrome: a systematic review and meta-analysis. Acupuncture in Medicine. 2017;35(4):241-250.
• World Health Organization. WHO Traditional Medicine Strategy: 2014–2023. Geneva, Switzerland: World Health Organization, 2013. Accessed at https://www.who.int/publications/i/item/9789241506096 on February 2, 2021.
• Wu M-S, Chen K-H, Chen I-F, et al. The efficacy of acupuncture in post-operative pain management: a systematic review and meta-analysis. PLoS One. 2016;11(3):e0150367.
• Xu S, Wang L, Cooper E, et al. Adverse events of acupuncture: a systematic review of case reports. Evidence-Based Complementary and Alternative Medicine. 2013;2013:581203.
• Yang J, Ganesh R, Wu Q, et al. Battlefield acupuncture for adult pain: a systematic review and meta-analysis of randomized controlled trials. The American Journal of Chinese Medicine. 2021;49(1):25-40.
• Yang Y, Wen J, Hong J. The effects of auricular therapy for cancer pain: a systematic review and meta-analysis. Evidence-Based Complementary and Alternative Medicine. 2020;2020:1618767.  
• Yeh CH, Morone NE, Chien L-C, et al. Auricular point acupressure to manage chronic low back pain in older adults: a randomized controlled pilot study. Evidence-Based Complementary and Alternative Medicine. 2014;2014;375173.
• You F, Ruan L, Zeng L, et al. Efficacy and safety of acupuncture for the treatment of oligoasthenozoospermia: a systematic review. Andrologia. 2020;52(1):e13415.
• Zhang X-C, Chen H, Xu W-T, et al. Acupuncture therapy for fibromyalgia: a systematic review and meta-analysis of randomized controlled trials. Journal of Pain Research. 2019;12:527-542.
• Zheng H, Chen R, Zhao X, et al. Comparison between the effects of acupuncture relative to other controls on irritable bowel syndrome: a meta-analysis. Pain Research and Management. 2019;2019:2871505.

Acknowledgments

NCCIH thanks Pete Murray, Ph.D., David Shurtleff, Ph.D., and Helene M. Langevin, M.D., NCCIH for their review of the 2022 update of this fact sheet. 

This publication is not copyrighted and is in the public domain. Duplication is encouraged.

NCCIH has provided this material for your information. It is not intended to substitute for the medical expertise and advice of your health care provider(s). We encourage you to discuss any decisions about treatment or care with your health care provider. The mention of any product, service, or therapy is not an endorsement by NCCIH.

Related Topics

Pain: Considering Complementary Approaches (eBook)

For Consumers

6 Things To Know When Selecting a Complementary Health Practitioner

For Health Care Providers

Mind and Body Approaches for Chronic Pain

Complementary Psychological and/or Physical Approaches for Cancer Symptoms and Treatment Side Effects

Research Results

New Findings Suggest Acupuncture Stimulation Reduces Systemic Inflammation

How the Body and Brain Achieve Carpal Tunnel Pain Relief via Acupuncture

Related Fact Sheets

Low-Back Pain and Complementary Health Approaches: What You Need To Know

Osteoarthritis: In Depth

Cancer and Complementary Health Approaches: What You Need To Know

Chronic Pain and Complementary Health Approaches

Traditional Chinese Medicine: What You Need To Know

Credentialing, Licensing, and Education

Paying for Complementary and Integrative Health Approaches

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Minim Access Surg
• v.15(3); Jul-Sep 2019

How to write an article: An introduction to basic scientific medical writing

Anil sharma.

Department of Minimal Access, Metabolic and Bariatric Surgery, Institute of Minimal Access, Metabolic and Bariatric Surgery, Max Healthcare Institute Ltd., Saket, New Delhi, India

An original scientific article published in a peer-reviewed professional journal of repute provides great personal satisfaction, adds stature and endows professional respectability to contributing authors. Various types of surgical publications that exist nowadays are case report, cohort study, case–control study, randomised controlled trial narrative review, systematic review, Cochrane review, meta-analysis, editorials and leading articles. A study/research protocol is a standardised document, common to all research projects that typically comprise study objectives, study design, selection of participants, study intervention, study evaluations, safety assessments, statistics and participant rights committees. Once the study protocol is completed and reviewed, it is submitted to the local Institutional Review Board/Institutional Ethics Committee for approval. An outline of the levels of evidence and grades of recommendation is available from the Centre for evidence-based medicine at the University of Oxford. A standardised, structured template exists for scientific presentations in the field of medicine which is also followed in medical writing and publications Introduction Methods Results And Discussion (IMRAD). Instructions to authors would normally include reference to International Committee of Medical Journal Editors and Committee on Publication Ethics guidelines for good and ethical publication practice. It is strongly advised to follow the recommended guidelines appropriate for the published study.

INTRODUCTION

The impact of the published article in a scientific journal of repute is powerful and protracted for as Kenneth Rothman states, ‘The written word reaches the widest audience and constitutes the archival message’. Authorship in a scientific journal implies that the authors have critically analysed and presented a scientific work of merit. ‘Reading maketh a full man, conference a ready man and writing an exact man’, (Francis Bacon). With scientific publishing, surgeons make their contributions to the profession for wide dissemination within their community and in the process create intellectual property that will be preserved down the ages. ‘The universal object of men of letters is reputation’, said John Adams.

A majority of practicing surgeons would not write and would remain engaged in busy surgical practices, bread winning and increasing administrative responsibilities. However, an increasing segment of surgeons in training and academic surgeons now feel the need to write and publish. The reasons for writing and publishing are both egoistic and altruistic.[ 1 ] Egoistic motives are the desire to progress academically and professionally, improve status and develop professional contacts. Altruistic motives are dissemination of knowledge and a moral obligation to publish a significant novel observation in the larger interest of better patient care. In several institutions, for academic appointments and promotions, the pressures to publish are sometimes inordinate. In many teaching institutions, to progress academically to whatever academic title one aspires, one's published output must constantly grow in number and quality. However, good-quality writing and publishing are not just in the domain of academic institutions. Several astute clinicians with clarity of vision from non-academic institutions have made significant contributions to surgical literature. It is imperative that contributions to surgical literature are derived from surgeons (academic and non-academic) at various locations (different continents, regions and nationalities) and workplaces (urban, semi-urban and rural). Such literature would be more relevant to the real world as opposed to surgical practice in highly sophisticated ivory towers. In the final analysis, an original scientific article published in a peer-reviewed professional journal of repute provides great personal satisfaction adds stature and endows professional respectability to contributing authors.

MANUSCRIPT TYPES

‘You don’t write because you want to say something; you write because you have something to say’, (Scot Fitzgerald). The essence of fine surgical writing is to write what you as a surgeon would want to read. Enumerated below is a list of various types of surgical publications that exist nowadays arranged in the order of increasing complexity.

• Letter/communication to the Editor

Case report

Cohort study (non-randomised, observational study), case–control study (non-randomised, observational study).

• Randomised controlled trial (RCT)

Narrative review

Systematic review and cochrane review, meta-analysis.

• Editorials and leading articles.

Letter/communication to the editor

This would be with reference to an article that has previously been published. The letter should be polite, constructive and should provide comments that offer a novel perspective of the published article. The comments should add, detract or critically review the contents of the published article in a fair and reasonable manner. The objective is to closely focus on and examine critical issues that may not have been appropriately addressed.

Many esteemed surgical writers, even journal editors, began a literary career with a time-honoured case report.[ 2 ] The humble case report would probably be the first step that an aspiring surgeon takes in surgical writing. Unfortunately, pressure of space and editorial policies directed at enhancing the impact factor of individual journals have reduced the opportunities for publication of case reports.[ 3 ]

The cohort study, case–control study and RCT constitute ‘original articles’ in surgical publications. The narrative review, systematic review and meta-analysis are ‘review articles’.

A cohort study is when patients are followed forward and assessed from time of exposure until time of consequences of exposure (target outcome). An example is ‘initial experience with single incision laparoscopic cholecystectomy.’

A case–control study is when patients are selected once they have the target outcome or not and researchers look backward to try to determine the factors of exposure. An example is ‘bile duct injury with single incision laparoscopic cholecystectomy.’

Randomised controlled trial

An RCT is performed when investigators want to assess treatment effects, usually considered to be beneficial. An example is ‘an RCT comparing recurrence rates between laparoscopic hernioplasty and Shouldice repair for groin hernias’.

A cohort study is feasible when randomisation of exposure is not possible. A case–control study overcomes temporal delays and may only require small sample size. However, both these studies are susceptible to bias and therefore have limited validity. The advantage of an RCT is that it provides the highest level of evidence. It is therefore useful to disprove efficacy which is important in the present era of technology-driven surgery. There is immense pressure from the manufacturers to use devices and procedures, many of which may not measure up to the scientific scrutiny of a well-conducted RCT. The design and execution of an RCT in surgery, however, is fraught with several difficulties and challenges. The nature of treatment by surgical intervention may lead to ethical issues that make design of the study difficult. Moreover, surgical skills and competence may vary from one hospital and surgeon to another making comparison odious. In most surgical studies, blinding of procedure from assessor is very difficult, and therefore, bias is inevitable.

A narrative review is usually written by invitation to an expert. The expert objectively reviews the subject in a concise and impartial manner. He/she addresses new developments and summarises recent literature. A narrative review leaves an imprint of the approach and thought process of the expert on the subject.

A systematic review involves more rigorous compilation of evidence. A systematic review is designed to present complete and unbiased evidence on the subject that presently exists in the literature. Strict adherence to follow and complete all components of a clearly defined protocol is mandatory.

A meta-analysis is a type of systematic review that uses statistical methods to combine and summarise the results of clinical trials. A meta-analysis must always include a formal examination of heterogeneity as an indicator of similar or divergent results.

Editorials and leading articles

These are usually written by invitation on a specific research area. The opinion and judgement of the editor do not only be based on review of literature but also carry the imprimatur of his/her personal beliefs and experience.

EVIDENCE-BASED MEDICINE

We live in an era of evidence-based medicine where increasingly an evidence-based approach to surgical practice would dictate the refining of systems and processes of patient care. Evidence-based practice is the, explicit and judicious use of the current best evidence in making decisions about the care of individual patients’.[ 4 ] An outline of the levels of evidence and grades of recommendation is available from the Centre for evidence-based medicine at the University of Oxford[ 5 , 6 ] Table 1 describes the levels of evidence for therapeutic studies.[ 7 ]

Levels of evidence for therapeutic studies

CONSTRUCTING THE MANUSCRIPT

‘If you can’t explain it simply, you don’t understand it well enough’, (Albert Einstein).

At the outset, formulation of the study/research protocol is required. The study/research protocol is a standardised document, common to all research projects that should be available in teaching institutions. The protocol template typically comprises the following.

• Study objectives
• Study design
• Selection of participants
• Study intervention
• Study evaluations
• Safety assessments
• Participant rights
• Committees.

Once the study protocol is completed and reviewed, it is submitted to the local Institutional Review Board (IRB)/Institutional Ethics Committee (IEC) for approval. Written consent is obtained and the study is registered at the Clinical Trial Registry of India at www.ctri.in .

‘If you don’t know where you are going, you will end up someplace else’, (Yogi Berra).

A standardised, structured template exists for scientific presentations in the field of medicine, and this is also followed in Medical writing and publications Introduction Methods Results And Discussion (IMRAD).

• Introduction: Why did we start?
• Methods: What did we do?
• Results: What did we find?
• Discussion: Hence, what does it mean?

Enumerated below are the constituent segments and contents therein in an original article of a scientific medical manuscript.

Introduction (two paragraphs)

The Introduction commences with a brief lesson on the subject as described in literature. Current knowledge, insights and recent developments on the subject are briefly stated. A lacuna or gap in knowledge or incomplete information on some aspect of the subject forms the basis and reason to perform the present research/study. The last line in the Introduction section normally reads ‘The aim of this study was…’, ‘We report… or ‘We reviewed…’.

Methods (seven paragraphs)

The Methods section narrates the story of what the authors did. The narration is arranged in a logical framework of time. A logical sequence for presentation is ethical approval, patient selection, surgical intervention, outcome assessments and statistical methods employed.

Results (six paragraphs)

The Results Section is an overall description of the major findings of the study. The Results section presents measurements and data on all stated end-points (primary and secondary) of the study. Data presentation should be clear and concise.

Discussion (seven paragraphs)

The Discussion section summarises the article and presents a perspective of the message in the article. The first paragraph provides a summary of the main aim, methods and results of the study. The last paragraph provides a tentative answer to the research question posed in the study and also a suggestion for future research in a related area of the study. The limitations of the present study are discussed (e.g. nature of study, numbers of patients and limited follow-up). The strengths of the present study, if any, may be enumerated. Similar studies in the literature are discussed and how the present study fits in is analysed. The implications of the present study are discussed in terms of future research, change in patient management policies and suggested amendments to clinical practice.

The title should be descriptive yet concise while conveying the essential features of the contents of the article. The title should contain words that will make the article accessible to workers in the field. Clarity, brevity and above all human interest are the hallmarks of a good title.

Titles and abstracts are freely available to browse across a wide array of databases on the Internet. An attractive title and a concise abstract serve to attract the attention of readers. The abstract serves as a stand-alone summary that describes the major contents and message of the article. The abstract is structured (IMRAD) with a strict word limit. It serves as a quick reference and shortcut for busy researchers.

Keywords are short phrases that capture the main topics of the article. These follow the abstract in the article. Keywords assist in cross-indexing and literature search.

Most journal editors subscribe to guidance from the International Committee of Medical Journal Editors (ICMJE)[ 8 ] also known as the Vancouver group. Contributors who meet all four of the below-mentioned criteria qualify for authorship.

• Substantial contributions to the conception or design of the work or the acquisition, analysis or interpretation of data for the work
• Drafting the work or revising it critically for important intellectual content
• Final approval of the version to be published
• Agreement to be accountable for all aspects of the work.

Acknowledgements

Those whose contributions do not justify authorship may be acknowledged and their contributions should be specified (e.g., ‘served as scientific advisors’, ‘critically reviewed the study proposal’, ‘collected data’, ‘provided and cared for study patients’ and ‘participated in writing or technical editing of the manuscript’).[ 8 ]

Conflict of interest

The ICMJE states that ‘a conflict of interest exists when professional judgement concerning a primary interest (such as patient's welfare or the validity of research) may be influenced by a secondary interest (such as financial gain)’. Public trust in the scientific process and the credibility of published articles depend in part on how transparently conflicts of interest are handled during the planning, implementation, writing, peer review, editing and publication of scientific work. Financial relationships (such as employment, consultancies, stock ownership or options, honoraria, patents and paid expert testimony) are the most easily identifiable conflicts of interest and the most likely to undermine the credibility of the journal, the authors, and science itself.[ 8 ]

A reference to articles serves to guide readers to a connected body of literature. Conference abstracts should not be used as references. They can be cited in the text, in parentheses, but not as page footnotes. References to papers accepted but not yet published should be designated as ‘in press’ or ‘forthcoming’. Information from manuscripts submitted but not accepted should be cited in the text as ‘unpublished observations’ with written permission from the source. Avoid citing a ‘personal communication’ unless it provides essential information not available from a public source, in which case the name of the person and date of communication should be cited in parentheses in the text.[ 8 ]

INSTRUCTIONS TO AUTHORS

It is mandatory to read and follow ‘Instructions to Authors’ provided by the journal where the manuscript is being sent for evaluation. Journals require electronic submission of manuscripts through specially designed editorial software (e.g. edition manager, manuscript central). The instructions provide detailed submission guidelines to Authors for submission of manuscripts. Instructions would normally include reference to ICMJE what an editor expects…pg 1124[ 9 ] and Committee on Publication Ethics (COPE) Guidelines[ 10 ] for good and ethical publication practice.

REPORTING GUIDELINES

It is strongly advised to follow recommended guidelines appropriate for the published study. These guidelines set international standards for reporting different types of research studies. A good checklist is provided for preparing the publication. The guidelines standardise trial design, facilitate accurate reporting and correct interpretation of results [ Table 2 ].[ 11 ]

Reporting guidelines for main study types

ROLE OF BIOSTATISTICIAN

The biostatistician provides invaluable input, advice and suggestions in construction of the manuscript. He/she should be consulted right from the concept and planning stage. He/she assists in protocol development with study design and study evaluations. He/she plans data management by confirming assessment of data on primary and secondary end-points of the study. He/she supervises data collection, archival and analysis. He/she implements and monitors the study on a periodic basis to its conclusion. Finally, the biostatistician assists with reporting results during writing of the manuscript.

DATA MANAGEMENT

Data management is the strategy used for collecting, organising and analysing data. The ultimate aim of conducting a study is to generate data to provide answers to the research question. The quality of data generated plays an important role in the outcome of the study. It follows that if primary data collection and entry are not considerate and meticulous, subsequent data analysis for outcome measures would not be satisfactory. Data need to be ultimately stored in electronic data capturing systems for ease of data management and analysis.

Several data analysis software systems are available that provide statistical results when data are fed into then in a predetermined format (Analyse-it, SPSS, WINKS SDA, Stata, Vitalnet).

WRITING STYLE

An effective writing style is easy to read and simple to understand. The connoisseur writer filters out unnecessary details and distills the essence of his/her communication in the manuscript. A short manuscript presented clear and lucidly is the most effective. Simple sentences in straightforward language convey the most information. A short sentence is easier to read and comprehend than a long rambling one, short, simple and familiar words are more reader-friendly than longer complicated phrases (replace ‘illustrate’ with ‘show’, ‘fundamental’ with ‘basic’ and ‘remainder’ with ‘rest’). A spell check and grammar check are mandatory after completing the manuscript.

New information is provided in a new paragraph. The main point appears at the start and should be clear, succinct and easy to find. The author consciously needs to avoid elitism/triumphalism in the article (the first report, the only study, the largest cohort). Exclamation and quotation marks are avoided in a formal medical manuscript. Proper punctuation marks such as full stops and commas are mandatory.

Text verbatim (copy and paste) from a previously published article or book must be marked as reference source. The author needs to follow the reference style required for submission to the journal. The Vancouver system[ 12 ] is the most commonly used. Abbreviations (INR – international normalised ratio, PT – prothrombin time) and acronyms (IMV – inferior mesenteric vein) should always be defined the first time they are used in the text. Abbreviations are useful to avoid unnecessary and frequent use of long phrases in the text. However, their use should be restricted in the text and never used in the title and abstract. In figures, abbreviations need to be explained in the legend and for tables in the footnote.

Tables and figures must be sufficiently clear, well labelled and interpretable without having to refer to the text. These should be placed in the text as near as possible to the place where they are referred to. Tables should not be used when data can be summarised in text (e.g. population sizes, sex ratios) or where data are better represented in graphs and figures. The legend carries descriptive information on the tables and figures to make them understandable as stand-alone segments. Table legends are placed above the body of the table, and figure legends are placed below the figures. Footnotes in a table explain abbreviations and P values.

PUBLICATION ETHICS

The COPE was founded in 1997 as a voluntary body to attempt to define best practice in the ethics of scientific publishing. The COPE guidelines on good publication practice are useful for authors, editors, editorial board members, readers, owners of journals and publishers. They address study design and ethical approval, data analysis, authorship, conflicts of interest, peer-review process, redundant publication, plagiarism, duties of editors, media relations, advertising and how to deal with misconduct.

• Study design and ethical approval: Good research should be well justified, well planned appropriately designed and ethically approved. To conduct research to a lower standard may constitute misconduct
• Data analysis: Data should be appropriately analysed, but inappropriate analysis does not necessarily amount to misconduct. Fabrication and falsification of data do constitute misconduct
• Authorship: There is no universally agreed definition of authorship although attempts have been made. As a minimum, authors should take responsibility for a particular section of the study

They may be personal, commercial, political, academic or financial. ‘Financial’ interests may include employment, research funding, stock or share ownership, payment for lectures or travel, consultancies and company support for staff

• Peer review: Peer reviewers are external experts chosen by editors to provide written opinions, with the aim of improving the study. Working methods vary from journal to journal, but some use open procedure in which the name of the reviewer is disclosed, together with the full or ‘edited’ report
• Redundant publication: Redundant publication occurs when two or more papers, without full cross-references, share the same hypothesis, data, discussion points, or conclusions
• Plagiarism: Plagiarism ranges from the unreferenced use of others published and unpublished ideas, including research grant applications to submission under ‘new’ authorship of a complete paper, something in a different language. It may occur at any stage of planning, research writing or publication: It applies to print and electronic versions
• Duties of editors: Editors are stewards of journals. They usually take over their journal from the previous editor(s) and always want to hand over the journal in good shape. Most editors provide direction for the journal and build a strong management team. They must consider and balance the interests of many constituents, including readers, authors, staff, owners, editorial board members, advertisers and the media
• Media relations: Medical research findings are of increasing interest to the print and broadcast media. Journalists may attend scientific meetings at which preliminary research findings are presented, leading to their premature publication in the mass media
• Advertising: Many scientific journals and meetings derive significant income from advertising. Reprints may also be lucrative.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.

• Election 2024
• Entertainment
• Newsletters
• Photography
• AP Buyline Personal Finance
• AP Buyline Shopping
• Press Releases
• Israel-Hamas War
• Russia-Ukraine War
• Global elections
• Asia Pacific
• Latin America
• Middle East
• Election results
• Google trends
• AP & Elections
• U.S. Open Tennis
• Paralympic Games
• College football
• Auto Racing
• Movie reviews
• Book reviews
• Financial Markets
• Business Highlights
• Financial wellness
• Artificial Intelligence
• Social Media

Book Review: ‘A Day in September’ examines the lessons from a key Civil War battle

This book cover image released by Norton shows “A Day in September: The Battle of Antietam and the World It Left Behind” by Stephen Budiansky. (Norton via AP)

This cover image released by W.W. Norton shows “A Day in September: The Battle of Antietam and the World It Left Behind,” by Stephen Budiansky. (W.W. Norton via AP)

• Copy Link copied

About 57,000 books have been published on the American Civil War so what possibly could be left to explore ?

Quite a bit, it turns out, particularly regarding the bloodiest battle of the war and in American history, Antietam. In one day of savage fighting, Sept. 17, 1862, an estimated 6,500 soldiers were killed and at least 15,000 wounded.

In 291 brisk, fact-stuffed but engaging, thought-provoking pages, “A Day in September” by Stephen Budiansky examines how ill-prepared we as a nation were for war, but more significantly, what we learned and how those advances led to better military training, rapid improvements in battlefield medical care and the beginnings of a reconciling of the differences in North and South society, values and beliefs.

Some key American institutions at the outbreak of the Civil War were astonishingly primitive and Antietam revealed just how bad. Pre-Civil War, for example, most graduates of the U.S. Military Academy were well-schooled in math and engineering, much less so in military tactics.

Many soldiers lacked even rudimentary training such as target shooting. Militias often behaved like fraternal organizations or a mob, Budiansky writes.

Medical care was primitive. For example, most doctors of the Civil War era did not understand how disease was transmitted. Treatment of the wounded at Antietam typically was chaotic; drivers charged with taking wounded to field hospitals often were drunk, the book observes.

What might have made the book even more engaging would be to carry the lessons learned from these failings to the present day.

For example, can we resolve our current differences peacefully ?

This is an absorbing, illuminating, compelling book that calls on us to consider the advances in military strategy, medical care and diplomacy that Antietam gave us at horrific cost.

It also asks us to consider a rift between science and religion that emerged after the war. The book notes that our religious leaders also fell short, telling the populace on both sides during the Civil War that God was on their side, but as the author quotes Lincoln as observing, one side must be wrong.

Then and now, reasoned discussions and diplomacy largely failed and some Americans are openly talking about a potential Civil War II.

They would not if we absorbed some of the lessons from this book.

AP book reviews: https://apnews.com/hub/book-reviews