synthesis qualitative literature

Log in using your username and password

Search More Search for this keyword Advanced search
Latest content
Current issue
Write for Us
BMJ Journals

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

If you want to inform your practice with answers to questions such as ‘What is it like for people to suffer chronic pain?’ or ‘What are patients’ understanding and experience of hypertension and taking medication’ 1 or ‘What are older people's views about falling and taking part in a falls programme’ 2 then what might you do? You could read individual research papers, but often there are many papers and they can be difficult to track down and access. A qualitative synthesis (sometimes called a qualitative systematic review) systematically searches for research on a topic, and draws the findings from individual studies together. Although systematic reviews of quantitative data are well established, for qualitative research, this is a fairly new field, and methods are still developing.

One type of interpretive qualitative synthesis is meta-ethnography. This was described by Noblit and Hare 7 and has recently been identified as the most widely cited method used in qualitative synthesis 6 . Meta-ethnography can be used with different qualitative methodologies, not just ethnography. The actual themes or concepts or metaphors as described by the authors of the original research become the data for the meta-ethnography. Studies are compared by looking at these concepts and how they relate to each other. You ask questions such as ‘How does this concept in this study relate to concepts in another study’. The next step is ‘translation’. This has been described as ‘reinterpretation and transformation of the analytical and theoretical concepts provided by individual studies into one another’ 3 p. 79. A line of argument is built up; having compared studies, what can now be said—are there new concepts or a new understanding leading to a new theory, only apparent when you have looked at key concepts from the studies and how they might be connected. For example, a qualitative synthesis on medicine taking identified ‘resistance’ as the concept that best helped understand lay responses to prescribed medicines. 9

Another type of interpretative qualitative systematic review is Critical Interpretative Synthesis. 8 , 10 Key concepts come from qualitative studies and from quantitative studies that ‘explicitly allows the integration of qualitative and quantitative evidence through an interpretative process’. 8 p. 39.

So a qualitative synthesis is a developing area. It can be useful in drawing together qualitative research, and often enables new understandings of the data to emerge. This ensures findings from primary research can contribute, through a qualitative synthesis, to a greater understanding of an area, which may be relevant to your practice.

Marshall IJ ,
Wolfe CDA ,
McInnes E ,
Sandelowski M ,
Barnett-Page E ,
Campbell R ,
Dixon-Woods M ,
Britten N ,
Agarwal S ,

Competing interests None.

Read the full text or download the PDF:

Literature Syntheis 101

How To Synthesise The Existing Research (With Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Eunice Rautenbach (DTech) | August 2023

One of the most common mistakes that students make when writing a literature review is that they err on the side of describing the existing literature rather than providing a critical synthesis of it. In this post, we’ll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples.

This post is based on our popular online course, Literature Review Bootcamp . In the course, we walk you through the full process of developing a literature review, step by step. If it’s your first time writing a literature review, you definitely want to use this link to get 50% off the course (limited-time offer).

Overview: Literature Synthesis

What exactly does “synthesis” mean?
Aspect 1: Agreement
Aspect 2: Disagreement
Aspect 3: Key theories
Aspect 4: Contexts
Aspect 5: Methodologies
Bringing it all together

What does “synthesis” actually mean?

As a starting point, let’s quickly define what exactly we mean when we use the term “synthesis” within the context of a literature review.

Simply put, literature synthesis means going beyond just describing what everyone has said and found. Instead, synthesis is about bringing together all the information from various sources to present a cohesive assessment of the current state of knowledge in relation to your study’s research aims and questions .

Put another way, a good synthesis tells the reader exactly where the current research is “at” in terms of the topic you’re interested in – specifically, what’s known , what’s not , and where there’s a need for more research .

So, how do you go about doing this?

Well, there’s no “one right way” when it comes to literature synthesis, but we’ve found that it’s particularly useful to ask yourself five key questions when you’re working on your literature review. Having done so, you can then address them more articulately within your actual write up. So, let’s take a look at each of these questions.

1. Points Of Agreement

The first question that you need to ask yourself is: “Overall, what things seem to be agreed upon by the vast majority of the literature?”

For example, if your research aim is to identify which factors contribute toward job satisfaction, you’ll need to identify which factors are broadly agreed upon and “settled” within the literature. Naturally, there may at times be some lone contrarian that has a radical viewpoint , but, provided that the vast majority of researchers are in agreement, you can put these random outliers to the side. That is, of course, unless your research aims to explore a contrarian viewpoint and there’s a clear justification for doing so.

Identifying what’s broadly agreed upon is an essential starting point for synthesising the literature, because you generally don’t want (or need) to reinvent the wheel or run down a road investigating something that is already well established . So, addressing this question first lays a foundation of “settled” knowledge.

Need a helping hand?

2. Points Of Disagreement

Related to the previous point, but on the other end of the spectrum, is the equally important question: “Where do the disagreements lie?” .

In other words, which things are not well agreed upon by current researchers? It’s important to clarify here that by disagreement, we don’t mean that researchers are (necessarily) fighting over it – just that there are relatively mixed findings within the empirical research , with no firm consensus amongst researchers.

This is a really important question to address as these “disagreements” will often set the stage for the research gap(s). In other words, they provide clues regarding potential opportunities for further research, which your study can then (hopefully) contribute toward filling. If you’re not familiar with the concept of a research gap, be sure to check out our explainer video covering exactly that .

3. Key Theories

The next question you need to ask yourself is: “Which key theories seem to be coming up repeatedly?” .

Within most research spaces, you’ll find that you keep running into a handful of key theories that are referred to over and over again. Apart from identifying these theories, you’ll also need to think about how they’re connected to each other. Specifically, you need to ask yourself:

Are they all covering the same ground or do they have different focal points or underlying assumptions ?
Do some of them feed into each other and if so, is there an opportunity to integrate them into a more cohesive theory?
Do some of them pull in different directions ? If so, why might this be?
Do all of the theories define the key concepts and variables in the same way, or is there some disconnect? If so, what’s the impact of this ?

Simply put, you’ll need to pay careful attention to the key theories in your research area, as they will need to feature within your theoretical framework , which will form a critical component within your final literature review. This will set the foundation for your entire study, so it’s essential that you be critical in this area of your literature synthesis.

If this sounds a bit fluffy, don’t worry. We deep dive into the theoretical framework (as well as the conceptual framework) and look at practical examples in Literature Review Bootcamp . If you’d like to learn more, take advantage of our limited-time offer to get 60% off the standard price.

4. Contexts

The next question that you need to address in your literature synthesis is an important one, and that is: “Which contexts have (and have not) been covered by the existing research?” .

For example, sticking with our earlier hypothetical topic (factors that impact job satisfaction), you may find that most of the research has focused on white-collar , management-level staff within a primarily Western context, but little has been done on blue-collar workers in an Eastern context. Given the significant socio-cultural differences between these two groups, this is an important observation, as it could present a contextual research gap .

In practical terms, this means that you’ll need to carefully assess the context of each piece of literature that you’re engaging with, especially the empirical research (i.e., studies that have collected and analysed real-world data). Ideally, you should keep notes regarding the context of each study in some sort of catalogue or sheet, so that you can easily make sense of this before you start the writing phase. If you’d like, our free literature catalogue worksheet is a great tool for this task.

5. Methodological Approaches

Last but certainly not least, you need to ask yourself the question: “What types of research methodologies have (and haven’t) been used?”

For example, you might find that most studies have approached the topic using qualitative methods such as interviews and thematic analysis. Alternatively, you might find that most studies have used quantitative methods such as online surveys and statistical analysis.

But why does this matter?

Well, it can run in one of two potential directions . If you find that the vast majority of studies use a specific methodological approach, this could provide you with a firm foundation on which to base your own study’s methodology . In other words, you can use the methodologies of similar studies to inform (and justify) your own study’s research design .

On the other hand, you might argue that the lack of diverse methodological approaches presents a research gap , and therefore your study could contribute toward filling that gap by taking a different approach. For example, taking a qualitative approach to a research area that is typically approached quantitatively. Of course, if you’re going to go against the methodological grain, you’ll need to provide a strong justification for why your proposed approach makes sense. Nevertheless, it is something worth at least considering.

Regardless of which route you opt for, you need to pay careful attention to the methodologies used in the relevant studies and provide at least some discussion about this in your write-up. Again, it’s useful to keep track of this on some sort of spreadsheet or catalogue as you digest each article, so consider grabbing a copy of our free literature catalogue if you don’t have anything in place.

Looking at the methodologies of existing, similar studies will help you develop a strong research methodology for your own study.

Bringing It All Together

Alright, so we’ve looked at five important questions that you need to ask (and answer) to help you develop a strong synthesis within your literature review. To recap, these are:

Which things are broadly agreed upon within the current research?
Which things are the subject of disagreement (or at least, present mixed findings)?
Which theories seem to be central to your research topic and how do they relate or compare to each other?
Which contexts have (and haven’t) been covered?
Which methodological approaches are most common?

Importantly, you’re not just asking yourself these questions for the sake of asking them – they’re not just a reflection exercise. You need to weave your answers to them into your actual literature review when you write it up. How exactly you do this will vary from project to project depending on the structure you opt for, but you’ll still need to address them within your literature review, whichever route you go.

The best approach is to spend some time actually writing out your answers to these questions, as opposed to just thinking about them in your head. Putting your thoughts onto paper really helps you flesh out your thinking . As you do this, don’t just write down the answers – instead, think about what they mean in terms of the research gap you’ll present , as well as the methodological approach you’ll take . Your literature synthesis needs to lay the groundwork for these two things, so it’s essential that you link all of it together in your mind, and of course, on paper.

Psst… there’s more!

This post is an extract from our bestselling short course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

excellent , thank you

Thank you for this significant piece of information.

This piece of information is very helpful. Thank you so much and look forward to hearing more literature review from you in near the future.

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Print Friendly

Methods for Qualitative Analysis and Synthesis

First Online: 11 August 2022

Cite this chapter

Rob Dekkers 4 ,
Lindsey Carey 5 &
Peter Langhorne 6

2035 Accesses

The previous chapter has outlined the approaches to qualitative analysis and synthesis, and indicated what criteria and options are for selecting the most appropriate method. The specific methods have only been described in terms of purpose and generic outcomes. To use a specific method, there are specific points to be considered.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime
Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF
Compact, lightweight edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info
Durable hardcover edition

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Almutairi S, Masters K, Donyai P (2018) The health professional experience of using antipsychotic medication for dementia in care homes: a study using grounded theory and focussing on inappropriate prescribing. J Psychiatr Ment Health Nurs 25(5–6):307–318. https://doi.org/10.1111/jpm.12464

Article Google Scholar

AlOmeir O, Patel N, Donyai P (2020) Adherence to adjuvant endocrine therapy among breast cancer survivors: a systematic review and meta-synthesis of the qualitative literature using grounded theory. Support Care Cancer 28(11):5075–5084. https://doi.org/10.1007/s00520-020-05585-9

Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J (2008) Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol 8(1):21. https://doi.org/10.1186/1471-2288-8-21

Bearman M (2016) Quality and literature reviews: beyond reporting standards. Med Educ 50(4):382–384. https://doi.org/10.1111/medu.12984

Bergdahl, E. (2019). Is meta-synthesis turning rich descriptions into thin reductions? A criticism of meta-aggregation as a form of qualitative synthesis. Nursing Inquiry, 26(1), e12273. https://doi.org/10.1111/nin.12273

Booth A, Carroll C, Ilott I, Low LL, Cooper K (2013) Desperately seeking dissonance: identifying the disconfirming case in qualitative evidence synthesis. Qual Health Res 23(1):126–141. https://doi.org/10.1177/1049732312466295

Braun V, Clarke V (2012) Thematic analysis. In: Cooper H, Camic PM, Long DL, Panter AT, Rindskopf D, Sher KJ (eds) APA handbook of research methods in psychology, vol 2. American Psychological Association, Washington, DC, pp 57–71

Google Scholar

Britten N, Garside R, Pope C, Frost J, Cooper C (2017) Asking more of qualitative synthesis: a response to sally thorne. Qual Health Res 27(9):1370–1376. https://doi.org/10.1177/1049732317709010

Cahill M, Robinson K, Pettigrew J, Galvin R, Stanley M (2018) Qualitative synthesis: a guide to conducting a meta-ethnography. Br J Occup Ther 81(3):129–137. https://doi.org/10.1177/0308022617745016

Charmaz K (2014) Constructing grounded theory, 2nd ed. Sage, London

Clarke V, Braun V, Hayfield N (2015) Thematic analysis. In Smith JA (ed) Qualitative psychology: a practical guide to research methods, 3rd ed. Sage, London, pp 222–248

Corbin J, Strauss A (2008) Basics of qualitative research: techniques and procedures for developing grounded theory, 3rd edn. Sage, Thousand Oaks, CA

Book Google Scholar

Corbin J, Strauss A (2015) Basics of qualitative research: techniques and procedures for developing grounded theory, 4th edn. Sage, Thousand Oaks, CA

Donyai P, Alomeir O (2020) Determining how to complete a grounded theory meta-synthesis of research examining patient views on adjuvant hormonal therapy for breast cancer. Accessed from https://methods.sagepub.com/case/grounded-theory-meta-synthesis-patient-views-adjuvant-hormonal-therapy-bc , https://doi.org/10.4135/9781529743418

Eaves YD (2001) A synthesis technique for grounded theory data analysis. J Adv Nurs 35(5):654–663. https://doi.org/10.1046/j.1365-2648.2001.01897.x

EQUATOR Network (2020) Enhancing the QUAlity and transparency of health research. Oxford: UK EQUATOR Centre. Accessed from https://www.equator-network.org

Florczak K (2018) Qualitative synthesis: ontological care please. Nurs Sci Q 31(3):220–223. https://doi.org/10.1177/0894318418774869

France EF, Cunningham M, Ring N, Uny I, Duncan EA, Jepson RG, Maxwell M, Roberts RJ, Turley RL, Booth A, Britten N,Flemming K, Gallagher I, Garside R, Hannes K, Lewin S, Noblit GW, Pope C, Thomas J, Vanstone M, Higginbottom GMA, Noyes J (2019a) Improving reporting of meta-ethnography: the eMERGe reporting guidance. BMC Med Res Methodol 19(1):25. https://doi.org/10.1186/s12874-018-0600-0

France EF, Uny I, Ring N, Turley RL, Maxwell M, Duncan EAS, Jepson RG, Roberts RJ, Noyes, J. (2019b) A methodological systematic review of meta-ethnography conduct to articulate the complex analytical phases. BMC Med Res Methodol 19(1):35. https://doi.org/10.1186/s12874-019-0670-7

Hannes K, Macaitis K (2012) A move to more systematic and transparent approaches in qualitative evidence synthesis: update on a review of published papers. Qual Res 12(4):402–442. https://doi.org/10.1177/1468794111432992

Herber OR, Kastaun S, Wilm S, Barroso J (2019) From qualitative meta-summary to qualitative meta-synthesis: introducing a new situation-specific theory of barriers and facilitators for self-care in patients with heart failure. Qual Health Res 29(1):96–106. https://doi.org/10.1177/1049732318800290

Hutchinson M, East L (2018) Application of meta-summary to derive a measurement instrument from the literature: a method exemplar. Nurse Res 26(3):8–13. https://doi.org/10.7748/nr.2018.e1599

Ibrahim K, Vogt C, Donyai P (2016) Caught in the eye of the storm: a qualitative study of views and experiences of planned drug holidays from methylphenidate in child and adolescent ADHD treatment. Child Adolesc Mental Health 21(4):192–200. https://doi.org/10.1111/camh.12156

Jørgensen MW, Phillips LJ (2002) Discourse analysis as theory and method. Sage, London

Kearney MH (2001) Enduring love: a grounded formal theory of women’s experience of domestic violence. Res Nurs Health 24(4):270–282. https://doi.org/10.1002/nur.1029

Lazar MM (2007) Feminist critical discourse analysis: articulating a feminist discourse praxis. Crit Discourse Stud 4(2):141–164. https://doi.org/10.1080/17405900701464816

Lee RP, Hart RI, Watson RM, Rapley T (2015) Qualitative synthesis in practice: some pragmatics of meta-ethnography. Qual Res 15(3):334–350. https://doi.org/10.1177/1468794114524221

Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, Bohren MA, Tunçalp Ö, Colvin CJ, Garside R, Carlsen B, Langlois EV, Noyes J (2018) Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci 13(1):2. https://doi.org/10.1186/s13012-017-0688-3

Majid U, Vanstone M (2018) Appraising qualitative research for evidence syntheses: a compendium of quality appraisal tools. Qual Health Res 28(13):2115–2131. https://doi.org/10.1177/1049732318785358

Malpass A, Shaw A, Sharp D, Walter F, Feder G, Ridd M, Kessler D (2009) “Medication career” or “Moral career”? The two sides of managing antidepressants: a meta-ethnography of patients’ experience of antidepressants. Soc Sci Med 68(1):154–168. https://doi.org/10.1016/j.socscimed.2008.09.068

Miles MB, Huberman AM (1994) Qualitative data analysis: an expanded sourcebook. Sage, Thousand Oaks, CA

Noblit GW (2018) Meta-ethnography adaptation and return. In: Luis UJ (ed) Cultural constructions of identity: meta-ethnography and theory. Oxford University Press, New York, pp 34–50

Noblit GW, Hare RD (1988) Meta-ethnography: synthesizing qualitative studies. Sage, Newbury Park, CA

Pearson A, Robertson-Malt S, Rittenmayer L (2011) Synthesising qualitative evidence. Lippincott, Williams & Wilkins, Philadelphia, PA

Popper K (2005) The logic of scientific discovery. Routledge, London

Sandelowski M (2000) Whatever happened to qualitative description? Res Nurs Health 23(4):334–340. https://doi.org/10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-G

Sandelowski M, Barroso J (2007) Handbook for synthesizing qualitative research. Springer, New York, NY

The Joanna Briggs Institute (2014) The Joanna Briggs Institute reviewers’ Manual 2014. Adelaide

Thomas J, Harden A (2008) Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 8(1):45. https://doi.org/10.1186/1471-2288-8-45

Thorne S (2017) Metasynthetic madness: what kind of monster have we created? Qual Health Res 27(1):3–12. https://doi.org/10.1177/1049732316679370

Thorne S (2020) Beyond theming: making qualitative studies matter. Nurs Inq 27(1):e12343. https://doi.org/10.1111/nin.12343

Tong A, Flemming K, McInnes E, Oliver S, Craig J (2012) Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol 12(1):181. https://doi.org/10.1186/1471-2288-12-181

Toye F, Pearl J, Vincent K, Barker K (2020) A qualitative evidence synthesis using meta-ethnography to understand the experience of living with pelvic organ prolapse. Int Urogynecol J 31(12):2631–2644. https://doi.org/10.1007/s00192-020-04494-z

Toye F, Seers K, Allcock N, Briggs M, Carr E, Barker K (2014) Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. BMC Med Res Methodol 14(1):80. https://doi.org/10.1186/1471-2288-14-80

Willig C (2003) Discourse analysis. In: Smith JA (ed) Qualitative psychology: a practical guide to research methods. Sage, Thousand Oaks, CA, pp 160–186

Wodak R, Meyer M (2009) Methods of critical discourse studies, 2nd edn. Sage, London

Download references

Author information

Authors and affiliations.

University of Glasgow, Glasgow, UK

Rob Dekkers

Glasgow Caledonian University, Glasgow, UK

Lindsey Carey

Prof. Peter Langhorne

You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

About this chapter

Dekkers, R., Carey, L., Langhorne, P. (2022). Methods for Qualitative Analysis and Synthesis. In: Making Literature Reviews Work: A Multidisciplinary Guide to Systematic Approaches. Springer, Cham. https://doi.org/10.1007/978-3-030-90025-0_11

Download citation

DOI : https://doi.org/10.1007/978-3-030-90025-0_11

Published : 11 August 2022

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-90024-3

Online ISBN : 978-3-030-90025-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

University of Texas Libraries
UT Libraries

Systematic Reviews & Evidence Synthesis Methods

Types of Reviews
Formulate Question
Find Existing Reviews & Protocols
Register a Protocol
Searching Systematically
Supplementary Searching
Managing Results
Deduplication
Critical Appraisal
Glossary of terms
Librarian Support
Video tutorials This link opens in a new window
Systematic Review & Evidence Synthesis Boot Camp

Once you have completed your analysis, you will want to both summarize and synthesize those results. You may have a qualitative synthesis, a quantitative synthesis, or both.

Qualitative Synthesis

In a qualitative synthesis, you describe for readers how the pieces of your work fit together. You will summarize, compare, and contrast the characteristics and findings, exploring the relationships between them. Further, you will discuss the relevance and applicability of the evidence to your research question. You will also analyze the strengths and weaknesses of the body of evidence. Focus on where the gaps are in the evidence and provide recommendations for further research.

Quantitative Synthesis

Whether or not your Systematic Review includes a full meta-analysis, there is typically some element of data analysis. The quantitative synthesis combines and analyzes the evidence using statistical techniques. This includes comparing methodological similarities and differences and potentially the quality of the studies conducted.

Summarizing vs. Synthesizing

In a systematic review, researchers do more than summarize findings from identified articles. You will synthesize the information you want to include.

While a summary is a way of concisely relating important themes and elements from a larger work or works in a condensed form, a synthesis takes the information from a variety of works and combines them together to create something new.

Synthesis :

"The goal of a systematic synthesis of qualitative research is to integrate or compare the results across studies in order to increase understanding of a particular phenomenon, not to add studies together. Typically the aim is to identify broader themes or new theories – qualitative syntheses usually result in a narrative summary of cross-cutting or emerging themes or constructs, and/or conceptual models."

Denner, J., Marsh, E. & Campe, S. (2017). Approaches to reviewing research in education. In D. Wyse, N. Selwyn, & E. Smith (Eds.), The BERA/SAGE Handbook of educational research (Vol. 2, pp. 143-164). doi: 10.4135/9781473983953.n7

Approaches to Reviewing Research in Education from Sage Knowledge

Data synthesis (Collaboration for Environmental Evidence Guidebook)

Interpreting findings and and reporting conduct (Collaboration for Environmental Evidence Guidebook)

Interpreting results and drawing conclusions (Cochrane Handbook, Chapter 15)

Guidance on the conduct of narrative synthesis in systematic reviews (ESRC Methods Programme)

Last Updated: Aug 12, 2024 8:26 AM
URL: https://guides.lib.utexas.edu/systematicreviews

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Meta-synthesis method for qualitative research: a literature review

Affiliation.

1 Department of Midwifery, Faculty of Health, University of Central Lancashire, Preston PR1 2HE, UK. [email protected]
PMID: 15788085
DOI: 10.1111/j.1365-2648.2005.03380.x

Aim: This paper discusses the purpose and stages of meta-synthesis and the epistemological status of knowledge generated from the technique. Particular attention is paid to exploring the contested areas of the method that remain.

Background: There is a growing interest in meta-synthesis as a technique for generating new insights and understanding from qualitative health care research. An increasing number of meta-synthesis papers are appearing in the nursing and midwifery literature.

Methods: Literature on the technique of meta-synthesis and examples of meta-synthesis papers were searched and reviewed. A meta-synthesis exercise was undertaken, and this informed reflection and critique of the method.

Findings: Meta-synthesis attempts to integrate results from a number of different but inter-related qualitative studies. The technique has an interpretive, rather than aggregating, intent, in contrast to meta-analysis of quantitative studies. Examples from the literature indicate that some aspects of the technique are not yet fully established.

Conclusion: Despite the contingent nature of evidence gleaned from meta-synthesis and current lack of consensus about some of its aspects, meta-synthesis is an important technique for qualitative researchers and can deepen understanding of the contextual dimensions of health care.

PubMed Disclaimer

Publication types

Search in MeSH

Related information

Cited in Books

LinkOut - more resources

Full text sources.

Ovid Technologies, Inc.

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

METHODS article

Metasynthesis: an original method to synthesize qualitative literature in psychiatry.

$\r\nJonathan Lachal,,*$

1 AP-HP, Cochin Hospital, Maison de Solenn, Paris, France
2 Université Paris Descartes, Sorbonne Paris Cité, Paris, France
3 CESP, Faculté de médecine, Université Paris-Sud, Faculté de médecine, Université de Versailles Saint-Quentin-en-Yvelines (UVSQ), INSERM, Université Paris-Saclay, Villejuif, France
4 Service Universitaire de Psychiatrie de l’Adolescent, Centre Hospitalier Argenteuil, Argenteuil, France
5 ECSTRA Team, UMR-1153, INSERM, Paris Diderot University, Sorbonne Paris Cité, Paris, France
6 Université Paris-Sud, Paris, France

Background: Metasynthesis—the systematic review and integration of findings from qualitative studies—is an emerging technique in medical research that can use many different methods. Nevertheless, the method must be appropriate to the specific scientific field in which it is used. The objective is to describe the steps of a metasynthesis method adapted from Thematic Synthesis and phenomenology to fit the particularities of psychiatric research.

Method: We detail each step of the method used in a metasynthesis published in 2015 on adolescent and young adults suicidal behaviors. We provide clarifications in several methodological points using the latest literature on metasyntheses. The method is described in six steps: define the research question and the inclusion criteria, select the studies, assess their quality, extract and present the formal data, analyze the data, and express the synthesis.

Conclusion: Metasyntheses offer an appropriate balance between an objective framework, a rigorously scientific approach to data analysis and the necessary contribution of the researcher’s subjectivity in the construction of the final work. They propose a third level of comprehension and interpretation that brings original insights, improve the global understanding in psychiatry, and propose immediate therapeutic implications. They should be included in the psychiatric common research toolkit to become better recognized by clinicians and mental health professionals.

The use of qualitative research is proliferating in medical research ( 1 ). Over the past two decades, numerous studies in the field of psychiatry have used a qualitative protocol ( 2 , 3 ), and it has been recognized as a valuable way to “ obtain knowledge that might not be accessible by other methods and to provide extensive data on how people interpret and act upon their illness symptoms ” ( 4 ). What matters most is the respondent’s perspective and the joint construction by the respondent and the researcher of a context-dependent, multiple, and complex reality ( 5 ). In this respect, the qualitative approach is close to that of the psychiatrist: what is important is what the patient feels and experiences and what emerges during the interaction between the patient and the psychiatrist. Indeed, the subjective coconstruction inherent to most of qualitative methods seems especially close to the psychiatric clinical meeting. Both are useful for building up local theory that helps to increase two important aspects of theory: individually relevant theory for clinical work and field-specific general theory for research ( 6 ). Qualitative research offers a thick description (one that encompass all the complexity of the phenomenon, behavior, or context) of a phenomenon and attempts to document the complexity and multiplicity of its experience ( 6 ). Similarly, in their day-to-day clinical work, psychiatrists attribute great importance to complexity and try to place symptoms within the patient’s history, in all of its intricate context—which again plays a crucial role in therapeutic choices.

Some have expressed concern, however, that because qualitative studies are isolated and rarely used to contribute to practical knowledge, they do not play a significant role in the movement toward evidence-based medicine ( 5 ). To alleviate this concern and enable qualitative work to contribute to this movement, an increasing number of teams have worked to develop and apply synthesis methods to these data. Qualitative syntheses refer to a collection of different methods for systematically reviewing and integrating findings from qualitative studies ( 7 ). The aims of such methods are to capture the increasing volume of qualitative research, to facilitate the transfer of knowledge to improve healthcare, and to bring together a broad range of participants and descriptions ( 8 , 9 ). Qualitative syntheses require not only a systematic approach to collecting, analyzing, and interpreting results across multiple studies, but also to develop overarching interpretation emerging from the joint interpretation of the primary studies included in the synthesis ( 10 , 11 ). Therefore, it involves going beyond the findings of any individual study to make the “whole into something more than the parts alone imply” ( 12 ).

Qualitative syntheses are now recognized as valuable tools for examining participants’ meanings, experiences, and perspectives, both deeply (because of the qualitative approach) and broadly (because of the integration of studies from different healthcare contexts and participants). They have been shown to be particularly useful to identify research gaps, to inform the development of primary studies, and to provide evidence for the development, implementation, and evaluation of health interventions ( 13 ). Because of this growing importance, an important work has been done in the last ten years, in order to ensure the quality of qualitative syntheses, such as: describing the methods to ensure reproducibility, develop tools for assessing the quality of the primary articles, and establish reporting guidelines [see, for example, the ENTREQ statement ( 13 ), the GRADE-Cerqual protocol ( 14 ), and the Cochrane or EVIDENT works ( 15 , 16 )].

However, despite some qualitative syntheses have been successfully conducted in the field of mental health ( 2 , 3 , 17 – 20 ), no study considers the methodological specificities inherent to psychiatric epistemological stance ( 7 ). Filling this gap has been one of the aims of our team since 2011. In this methodological article, we aimed to discuss the challenge of implementing metasynthesis to improve the understanding of youths suicide. In this study, we adapted the Thematic Synthesis developed by Thomas and Harden and incorporate a phenomenological approach in order to deal with new rigor with general as well as psychiatric issues ( 21 ). We will present each step of the method (Figure 1 ) and will propose methodological discussions. The detailed description of the findings can be found elsewhere ( 22 ).

Figure 1 . Distribution in time for articles included in the metasynthesis.

Conducting a Metasynthesis

Before start—constitution of a research group.

The constitution of the research group and the definition of the study method are an important step before engaging in any synthesis work. The researcher must work in collaboration with researchers of diverse backgrounds ( 9 ). A collaborative approach improves quality and rigor and subjects the analytical process to group reflexivity ( 11 ). The research team should include members trained in qualitative synthesis as well as those expert in the topic being studied ( 23 ). As there are many ways to do qualitative syntheses, the research team will have to choose one of them adapted to the research question and to the expertise of the group ( 15 ).

Our team is composed of adolescent and child psychiatrists and psychologists from France and elsewhere (Italy, Chile, and Brazil) and focuses on developing qualitative research ( 24 – 26 ) and metasynthesis in adolescent psychiatry and related fields ( 22 , 27 , 28 ). Our method is adapted from thematic synthesis ( 21 ), which combines and adapts approaches from both metaethnography and grounded theory ( 10 ). Metaethnography, as well as Thematic Synthesis, takes place in six or seven steps from data collection to text coding and finally writing the synthesis. Original authors of metaethnography were trained in grounded theory, a qualitative method developed in the social sciences, laying on conceptual coding combine to construct a new theory. Thematic synthesis allows the researcher to include much more studies in the synthesis and to use tools coming from quantitative reviews, as systematic literature searches. This method perfectly suits to psychiatric research: user-friendliness for both researchers and readers; standardized in its most subjective steps but flexible, to make it adaptable to various patients or situations, such as children, patients with psychological disabilities or psychotic disorders, and to different researchers’ backgrounds (e.g., phenomenology, psychology, or psychoanalysis). We add a phenomenological perspective with a coding close to Smith’s interpretative phenomenological analysis (IPA) ( 29 ). IPA is also a qualitative method of coding a text, laying on phenomenology and hermeneutics. The level of coding is what makes sense to the reader (for example, a letter, a word, a sentence, the absence of a word, or a sentence). Phenomenology allows avoiding never-ending debates about theories of the psyche and focuses on the patient experience which is at the heart of psychiatric care. We understand that published manuscripts provide only thin data sets that are not eligible for a complete phenomenological analysis. Rather we tried to let ourselves guided by the impressions the text generated in us. It was like one article was assimilated as one participant, as it is mainly the voice of the main writer. We applied Smith’s tips on how reading and coding the data.

Define the Research Question and the Selection Criteria

Defining the research question is a crucial substep ( 9 ). This question must be broad enough to be of interest but small enough to be manageable ( 5 , 23 ) and has already been explored by enough studies ( 30 ). Inclusion and exclusion criteria may be fixed on methodological aspects, on participants selected, on thematic focuses or language specificities ( 9 , 31 ).

Youths suicide is a focus that were suitable for qualitative methods. We chose this subject because youth suicide is a major public health issue worldwide as well as a complex disorder that encompasses medical, sociological, anthropological, cultural, psychological, and philosophical issues. It has been widely explored by qualitative research. The lack of effectiveness of current care let us think that new insights could be expected by qualitative exploration. A first selection of articles, as well as an existing literature review on the topic, served to specify some starting information and enable initial decisions, including the definition of the research question, specification of the scope and the inclusion criteria. Then, the questions were constructed through reading and confronting these articles with our first qualitative study in the theme and our clinical knowledge of the theme.

As we wanted to study the therapeutic relationship and barriers to effective care, we decided to include research concerning not only the population being treated (the adolescents and young adults, and their parents), but also the healthcare professionals who care for these patients. A first screening of the literature showed us that optimal scope required a large range of ages, from 15 to 30 years old. The common thread linking all these youths was the importance of their parents in their everyday life. We chose to include only qualitative research, because it remains unclear how to deal with mixed method (combining qualitative and quantitative datasets) ( 23 ). Although databases contain articles in different languages, we chose to include only articles published in English (as most studies are now published in English) and French (as it is our first language) ( 22 , 27 ).

Study Selection

There is a debate on the choice of sampling method, some authors using an exhaustive sampling, some others, an expansive one ( 30 ). We privileged exhaustive systematic searches ( 32 ) since our method allowed large samples and because our target audience was the mental health community, which is accustomed to quantitative systematic reviews ( 9 ). Only journal articles were included, as most scientific data are published in this form ( 33 ). The first selection of articles served to specify the choice of keywords and databases for the electronic search. To ensure both sensitivity and specificity, we decided to use a combined approach of thesaurus terms and free-text terms. This technique maximizes the number of potentially relevant articles retrieved and ensures the highest level of rigor ( 34 ). Keywords were established during research team meetings, and were reported in the article or as supplemental material for more clarity ( 35 ). As each database has its own thesaurus terms, and as keywords encompasses different meanings in each discipline ( 36 ), the keywords were specific for each one.

We used four clusters of keywords: (i) those that concern the topic of interest (such as suicide, obesity, or anorexia nervosa), (ii) those that concern the participants (gender, age, profession, etc.), (iii) those that concern qualitative research (such as qualitative research, interviews, focus groups , or content analysis ), and (iv) those that concern perceptions and understanding, often called “views” ( 33 ) (such as knowledge, perception, self-concept, feeling , or attitude ). The last cluster takes all its importance in the phenomenological perspective of the analysis. An example of the final algorithm used (in the PubMed Web search) is provided in Table 1 .

Table 1 . Algorithm used in the PubMed Web search from Ref. ( 22 ).

Similar work was conducted to select the databases. After consulting reference articles ( 33 , 37 , 38 ), we decided to conduct the search in five electronic databases covering medical, psychological, social, and nursing sciences: MEDLINE, EMBASE, CINAHL, PsycINFO, and Social Science Citation Index (SSCI). Not long ago, CINAHL was the most important database for finding qualitative research, but as qualitative research proliferates in medical research, more and more qualitative articles are referenced in MEDLINE ( 33 ) and EMBASE. PsycINFO was a good database for finding qualitative articles with a psychological approach. We decided to add SSCI to broaden and complexity the outlook with a sociological point of view. We followed recommendations published on MEDLINE ( 39 ), CINAHL ( 40 ), EMBASE ( 41 ), and PsycINFO ( 42 ) for choosing search terms. Finally, we decided not to use the methodological databases’ filters for qualitative research, as these have undergone little replication and validation ( 43 ).

We decided to include articles published only in or after 1990. Two points impelled this decision: first, there was very little qualitative research on suicide before the year 2000 and even less before the 1990s (Figure 2 ). Second, we chose to consider as outdated research findings and results published more than 20 years ago were outdated, given the evolution of medical practices ( 44 ). However, this choice must be adapted to the topic of metasynthesis.

Figure 2 . Flowchart of the metasynthesis steps.

The results of database searches were entered into a bibliographic software program (Zotero©) for automatic removal of duplicates. Then, two authors independently screened all titles and abstracts and selected the studies according to our inclusion criteria (defined earlier). If the abstract was not sufficient, we read the full text. Disagreements were resolved during working group meetings. Full texts of potentially relevant articles were then examined, and a second selection was performed. At this phase, we also checked each article’s reference list looking for new articles we might have overlooked. The final selection represented from 2 to 3% of the total initially obtained. This rate is consistent with the findings of other metasyntheses ( 23 ). For clarity, the selection process was also presented in a flowchart (Figure 3 ). We referred to STARLITE principles to report our literature search ( 45 ) (Table 2 ).

Figure 3 . Flowchart for selecting studies from Ref. ( 22 ).

Table 2 . STARLITE principles applied to the literature search report of Ref. ( 22 ).

Quality Assessment of Included Studies

There is no consensus about whether quality criteria should be applied to qualitative research, or, for those who think they should be, about which criteria to use and how to apply them. Nevertheless a growing number of researchers are choosing to appraise studies for metasyntheses ( 46 ) and some authors state that a good metasynthesis can no longer avoid this methodological step ( 7 ). The reasons and methods for quality assessment fit into three general approaches: assessment of study conduct, appraisal of study reporting, and an implicit judgment of the content and utility of the findings for theory development ( 13 ). There is certainly not one best appraisal tool, but rather a wide choice of good ones ( 8 ).

We chose the Critical Appraisal Skills Program (CASP) ( 47 ), which is the most frequently used instrument ( 46 ), addresses all the principles and assumptions underpinning qualitative research ( 13 ). It is one of the instruments recommended by the Cochrane Collaboration ( 48 ) and has been used in several important thematic analyses of medical topics. As proposed by Boeije et al., we weighted our assessment by applying a three-point scale to each criterion (0 = criterion not met; 1/P = criterion partially met; 2/T = criterion totally met) ( 49 ) (Table 3 ).

Table 3 . Evaluation of the quality of the studies according to the Critical Appraisal Skill Programme (CASP) from Ref. ( 22 ).

We have not excluded any study on quality criteria. We think that the goal of the quality assessment is not to help selecting the more rigorous article. Either, this step is important to improve the overall rigor of the metasynthesis: by easily evaluating the quality of each article, the readers will have the possibility to make their own evaluation of the quality of the results of the metasynthesis ( 9 ). To enhance the rigor of the synthesis, we published the full results of this assessment ( 50 ).

Extracting and Presenting the Formal Data

To understand the context of each study, readers need the formal data about each study: the number and type of participants in each study, its location, and the method of data collection and of analysis. These data must be extracted and presented in a way that enables readers to form their own opinions about the studies included. We presented these data systematically, in a table with the following headings:

– Identification of the study.

– Summary of the study’s aim.

– Country where the study took place.

– Details about the participants: age, gender, type, and number.

– Method of data collection (e.g., semistructured interviews or focus groups).

– Analysis method (grounded theory, phenomenology, thematic, etc.).

Data Analysis

This step is probably the most subjective: its performance is highly influenced by the authors’ backgrounds ( 13 ). There are many ways to analyze, as many as there are authors. All researchers build on their personal knowledge and background for the analysis, sometimes described as bricolage , following Claude Levi-Strauss: “ the bricoleur combines techniques, methods, and materials to work on any number of projects and creations. Whereas a typical construction process might be limited by the history or original use of individual pieces, the bricoleur works outside of such limitations, reorganizing pieces to construct new meaning. In other words, unlike linear, step-by-step processes, the bricoleur steps back and works without exhaustive preliminary specifications ” ( 51 , 52 ). The synthesis will inevitably be only one possible interpretation of the data ( 9 ), as it depends on the authors’ judgment and insights ( 21 ). The qualitative synthesis does not result simply from a coding process, but rather from the researchers’ configuration of segments of coded data “ assembled into a novel whole ” ( 53 ).

In this process, the multidisciplinary team is essential to assess rigor and develop richer and more complex understandings. Triangulation of the analyses is enhanced when researchers with diverse background consider the same data set ( 11 ). “ Collaborative working not only improve quality and rigour, but subjects the analytical process to group reflexivity ” ( 54 ).

The first step of this process involved carefully reading and rereading each study ( 21 ). It is an active reading, with the intention of appraising, familiarizing, identifying, extracting, recording, organizing, comparing, relating, mapping, stimulating and verifying. In other words, it is reading with “ the intention of collating a synthesizable set of accounts ” ( 11 ).

The second step was coding: at least two different researchers coded each part of the data (whole manuscripts), performing a line-by-line coding, close to the phenomenological analysis described by Smith et al. ( 29 ).

In the third step, the codes were grouped and categorized into a hierarchical tree structure. This step is very close to the translation work described by Noblit and Hare ( 12 ). It involves comparing themes across articles to match themes from one article with those from another while ensuring that each key theme captured similar themes from different articles. We obtained a list of descriptive themes very close to the data.

In our example, we highlighted the sentence “You’re going to school, you’re getting an education, but you’re depressed” and coded it depressed . The code is then combined with others in a theme named “The experience of distress.”

Finally, in the last and most subjective step of the analysis, we generated analytical themes, which depended largely on the “judgment and insights of the reviewers” ( 21 ). This step is very similar the development of third-order interpretations—“ the synthesis of both first and second order constructs into a new model or theory about a phenomenon ” ( 23 )—and requires going beyond the content of original studies to achieve a higher level of interpretation and going beyond the descriptive synthesis to propose a more conceptual line-of-argument ( 21 , 23 ). This work has two types of underlying aims. The first type may be theoretical, by enabling a higher level of comprehension of a phenomenon; in medical science, this may be to better describe and understand a pathology. The second type may be to answer clinical questions about pathology and care directly.

In our example, we clearly fulfilled the second aim. The results leaded us to discuss new insights about suicidal youths’ care. The experience of incomprehension shared by all the protagonists of the care interferes with the capacity for empathy of both family members and professionals. We could use the concepts of intersubjectivity to witness the violence driven by the suicidal act.

Writing the Synthesis

Throughout the analysis process, the authors build themes that take place in the story they are telling about the participant’s experience ( 21 ). Then, the expression of the synthesis is our story of the studied phenomenon.

The results of the metasynthesis consist of the themes that we developed in the analysis. They are built by first and second order constructs. We did not define actual third-order themes; rather, third-order constructs helped us to build the synthesis into a story. We organized the themes into superordinate themes, which are interpretations of the themes and can be considered third-order interpretations.

For example, in one of the developed theme called the experience of distress we described that the young people experienced depressive symptoms. The participants described feelings of sadness, sorrow, mental pain, despair, detachment, anger, and irritability . The authors interpreted that as despair . We organized all these closed related feeling into the individual experience of distress . We decide not to speak about depression , first because some healthcare professionals repeated that they may diagnose depression “ but certainly not on a routine basis ” ( 22 ), then because we adopted a phenomenological approach and we felt that distress encompasses a broader and more complex experience.

Metasynthesis results prepare the framework for the discussion, the most interpretative part of the review, where hypothesis and proposals are presented. We offer our understanding of the participants’ experience. Both our presentation and our discourse are influenced by our aim: to answer clinical questions by suggesting specific actions or considerations for care; the discussion and the answers are intended to be useful for the readers of our article, as well as for us ( 23 ).

Our conclusion is that “ the violence of the message of a suicidal act and the fears associated with death lead to incomprehension and interfere with the capacity for empathy of both family members and professionals. The issue in treatment is to be able to witness this violence so that the patient feels understood and heard, and thus to limit recurrences ” ( 22 ). This issue is clear and simple and it leads to an immediate application to clinical practice which is described in the implication for practice chapter.

Finally, we discuss the limitation of the findings. The principal limitations were methodological (with our method, the access to participants’ data is partial), and in the sampling (we didn’t take in consideration the influence of gender on the experience of suicidal behavior). That exercise enhances the credibility of the publication, enabling readers to measure the importance and generalizability of the findings.

The written synthesis has to fulfill the standard for reporting synthesis of qualitative research. We chose to refer to Tong and al. ENTREQ statement ( 13 ) attached to the publication.

Our method is based on Thomas and Harden Thematic Synthesis ( 21 ). After a broad-scale review of literature on the topic of metasynthesis, we have decided to clarify the definition of some aspects of the method and modify or expand others, because we wanted both a medical and a psychological approach. For example, we opted to use a systematic search method and a weighted version of the CASP to assess quality.

Most metasynthesis authors argue that these reviews achieve a third-order level of interpretation, that is, that they are more than the sum of their results. If, as we think, qualitative research can achieve a moderate level of generalization with clinical implications, metasyntheses may transform these findings into more highly abstracted and generalizable theoretical frameworks. We “ push their findings toward the nomothetic end of the idiographic-nomothetic continuum ” ( 44 ). Qualitative specialists certainly do not shy away from stressing the importance of context in their studies, or from arguing that the context of one study may not be applicable to others. It is true that, in a way, metasyntheses decontextualize concepts to attain greater generalizability ( 44 ). But we can relate this act to the response of clinicians reading a qualitative article: they will try to apply the concepts to their own situations ( 21 ). Authors of metasyntheses are proposing their own interpretation of the concept and its generalizability. The scientific value of metasynthesis lies in its role as a summary of several studies and as the interpretation of varying context, as well as in its ability to weight each result and to propose greater generalizability.

Qualitative research is an invaluable method for gaining new insights into mental disorders ( 6 ). Its development in recent years requires that we improve methods for synthesizing their results. We think this way of doing metasynthesis is appropriate to psychiatric research in its intermediate position that stresses both progress in the general comprehension of disorders and direct clinical implications. It offers an appropriate balance between three components: an objective framework, which includes the selection, inclusion, and appraisal of studies; a rigorously scientific approach to data analysis; and the necessary contribution of the researcher’s subjectivity in the construction of the final work. The balance for a qualitative metasynthesis is, we think, very similar to the clinical approach to each patient. It necessitates a robust scientific background, a rigorous step-by-step—symptom by symptom—progression, and finally a part of art that depends on each clinician: the subjective part of therapy.

Finally, we think that metasyntheses enable insights that no other method can provide. Qualitative research sheds new light on scientific questions by emphasizing the participants’ subjective understanding and experience ( 6 ). Metasynthesis proposes a third level of comprehension and interpretation that brings original insights. In our study ( 22 ), we emphasized an original point in the relationship that was no found in any result from each primary study: the difficulty of professionals and parents to understand and cope with suicide as an obstacle to the care of the suicidal adolescent. Therefore, our study’s analysis went deeper and proposed original results.

Author Contributions

Conceived and designed the experiments and final approval: JL, AR-L, MO, and MM. Conducted the literature review: JL and MO. Performed the experiments: JL, MO, and AR-L. Wrote the article: JL (all the article), AR-L (analysis), MO (introduction and analysis), and MM (discussion).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Jo Ann Cahn for revising our English.

1. Shuval K, Harker K, Roudsari B, Groce NE, Mills B, Siddiqi Z, et al. Is qualitative research second class science? A quantitative longitudinal examination of qualitative research in medical journals. PLoS One (2011) 6:e16937. doi:10.1371/journal.pone.0016937

CrossRef Full Text | Google Scholar

2. Malpass A, Shaw A, Sharp D, Walter F, Feder G, Ridd M, et al. “Medication career” or “Moral career”? The two sides of managing antidepressants: a meta-ethnography of patients’ experience of antidepressants. Soc Sci Med (2009) 68:154–68. doi:10.1016/j.socscimed.2008.09.068

3. Cairns VA, Reid GS, Murray C. Family members’ experience of seeking help for first-episode psychosis on behalf of a loved one: a meta-synthesis of qualitative research. Early Interv Psychiatry (2015) 9(3):185–99. doi:10.1111/eip.12157

4. Espíndola CR, Blay SL. Anorexia nervosa’s meaning to patients: a qualitative synthesis. Psychopathology (2009) 42:69–80. doi:10.1159/000203339

5. Evans D, Pearson A. Systematic reviews of qualitative research. Clin Eff Nurs (2001) 5:111–9. doi:10.1054/cein.2001.0219

6. Whitley R, Crawford MJ. Qualitative research in psychiatry. Can J Psychiatry Rev Can Psychiatr (2005) 50:108–14. doi:10.1177/070674370505000206

7. Ring N. A Guide to Synthesising Qualitative Research for Researchers Undertaking Health Technology Assessments and Systematic Reviews . Edinburgh: NHS Quality Improvement Scotland (2011).

Google Scholar

8. Hannes K, Booth A, Harris J, Noyes J. Celebrating methodological challenges and changes: reflecting on the emergence and importance of the role of qualitative evidence in Cochrane reviews. Syst Rev (2013) 2:84. doi:10.1186/2046-4053-2-84

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Toye F, Seers K, Allcock N, Briggs M, Carr E, Barker K. Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. BMC Med Res Methodol (2014) 14:80. doi:10.1186/1471-2288-14-80

10. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol (2009) 9:59. doi:10.1186/1471-2288-9-59

11. Lee RP, Hart RI, Watson RM, Rapley T. Qualitative synthesis in practice: some pragmatics of meta-ethnography. Qual Res (2014) 15:334–50. doi:10.1177/1468794114524221

12. Noblit GW, Hare RD. Meta-Ethnography: Synthesizing Qualitative Studies . Newbury Park: SAGE (1988).

13. Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol (2012) 12:181. doi:10.1186/1471-2288-12-181

14. Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, et al. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med (2015) 12:e1001895. doi:10.1371/journal.pmed.1001895

15. Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, et al. Supplemental Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions (2016). Available from: http://methods.cochrane.org/qi/supplemental-handbook-guidance

16. Booth A. EVIDENT Guidance for Reviewing the Evidence: A Compendium of Methodological Literature and Websites (2016). Available from: https://www.academia.edu/21598179/EVIDENT_Guidance_for_Reviewing_the_Evidence_a_compendium_of_methodological_literature_and_websites

17. Khan N, Bower P, Rogers A. Guided self-help in primary care mental health: meta-synthesis of qualitative studies of patient experience. Br J Psychiatry (2007) 191:206–11. doi:10.1192/bjp.bp.106.032011

18. Knowles SE, Toms G, Sanders C, Bee P, Lovell K, Rennick-Egglestone S, et al. Qualitative meta-synthesis of user experience of computerised therapy for depression and anxiety. PLoS One (2014) 9:e84323. doi:10.1371/journal.pone.0084323

19. Daker-White G, Rogers A. What is the potential for social networks and support to enhance future telehealth interventions for people with a diagnosis of schizophrenia: a critical interpretive synthesis. BMC Psychiatry (2013) 13:279. doi:10.1186/1471-244X-13-279

20. Price O, Baker J, Bee P, Lovell K. Learning and performance outcomes of mental health staff training in de-escalation techniques for the management of violence and aggression. Br J Psychiatry (2015) 206:447–55. doi:10.1192/bjp.bp.114.144576

21. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol (2008) 8:45. doi:10.1186/1471-2288-8-45

22. Lachal J, Orri M, Sibeoni J, Moro MR, Revah-Levy A. Metasynthesis of youth suicidal behaviours: perspectives of youth, parents, and health care professionals. PLoS One (2015) 10:e0127359. doi:10.1371/journal.pone.0127359

23. Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol (2008) 8:21. doi:10.1186/1471-2288-8-21

24. Lachal J, Speranza M, Schmitt A, Spodenkiewicz M, Falissard B, Moro MR, et al. Depression in adolescence: from qualitative research to measurement. Adolesc Psychiatry (2012) 2:296–308. doi:10.2174/2210676611202040296

25. Lachal J, Speranza M, Taïeb O, Falissard B, Lefèvre H, Moro MR, et al. Qualitative research using photo-elicitation to explore the role of food in family relationships among obese adolescents. Appetite (2012) 58:1099–105. doi:10.1016/j.appet.2012.02.045

26. Orri M, Paduanello M, Lachal J, Falissard B, Sibeoni J, Revah-Levy A. Qualitative approach to attempted suicide by adolescents and young adults: the (neglected) role of revenge. PLoS One (2014) 9:e96716. doi:10.1371/journal.pone.0096716

27. Lachal J, Orri M, Speranza M, Falissard B, Lefevre H, Moro M-R, et al. Qualitative studies among obese children and adolescents: a systematic review of the literature. Obes Rev (2013) 14:351–68. doi:10.1111/obr.12010

28. Orri M, Farges O, Clavien P-A, Barkun J, Revah-Lévy A. Being a surgeon – the myth and the reality: a meta-synthesis of surgeons' perspectives about factors affecting their practice and well-being. Ann Surg (2014) 260:721–9. doi:10.1097/SLA.0000000000000962

29. Smith JA. Interpretative phenomenological analysis. Qualitative Psychology: A Practical Guide to Research Methods . Trowbridge: SAGE (2008).

30. Finfgeld-Connett D, Johnson ED. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. J Adv Nurs (2013) 69:194–204. doi:10.1111/j.1365-2648.2012.06037.x

31. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions . Cochrane: John Wiley & Sons (2011).

32. France EF, Ring N, Thomas R, Noyes J, Maxwell M, Jepson R. A methodological systematic review of what’s wrong with meta-ethnography reporting. BMC Med Res Methodol (2014) 14:119. doi:10.1186/1471-2288-14-119

33. Stansfield C, Brunton G, Rees R. Search wide, dig deep: literature searching for qualitative research. An analysis of the publication formats and information sources used for four systematic reviews in public health. Res Synth Methods (2014) 5(2):142–51. doi:10.1002/jrsm.1100

34. Shaw RL, Booth A, Sutton AJ, Miller T, Smith JA, Young B, et al. Finding qualitative research: an evaluation of search strategies. BMC Med Res Methodol (2004) 4:5. doi:10.1186/1471-2288-4-5

35. Dixon-Woods M, Booth A, Sutton AJ. Synthesizing qualitative research: a review of published reports. Qual Res (2007) 7:375–422. doi:10.1177/1468794107078517

36. Bassett R, McGibbon E. A critical participatory and collaborative method for scoping the literature. Qual Quant (2013) 47:3249–59. doi:10.1007/s11135-012-9715-2

37. Booth A. Chapter 3: searching for studies. In: Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, Lockwood C, editors. Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions . Cochrane Collaboration Qualitative Methods Group (2011). Available from: http://cqrmg.cochrane.org/supplemental-handbook-guidance

38. Tong A, Palmer S, Craig JC, Strippoli GFM. A guide to reading and using systematic reviews of qualitative research. Nephrol Dial Transplant (2016) 31(6):897–903. doi:10.1093/ndt/gfu354

39. Wong SS-L, Wilczynski NL, Haynes RB; Hedges Team. Developing optimal search strategies for detecting clinically relevant qualitative studies in MEDLINE. Stud Health Technol Inform (2004) 107:311–6. doi:10.3233/978-1-60750-949-3-311

40. Wilczynski NL, Marks S, Haynes RB. Search strategies for identifying qualitative studies in CINAHL. Qual Health Res (2007) 17:705–10. doi:10.1177/1049732306294515

41. Walters LA, Wilczynski NL, Haynes RB; Hedges Team. Developing optimal search strategies for retrieving clinically relevant qualitative studies in EMBASE. Qual Health Res (2006) 16:162–8. doi:10.1177/1049732305284027

42. McKibbon KA, Wilczynski NL, Haynes RB. Developing optimal search strategies for retrieving qualitative studies in PsycINFO. Eval Health Prof (2006) 29:440–54. doi:10.1177/0163278706293400

43. Flemming K, Briggs M. Electronic searching to locate qualitative research: evaluation of three strategies. J Adv Nurs (2007) 57:95–100. doi:10.1111/j.1365-2648.2006.04083.x

44. Finfgeld-Connett D. Generalizability and transferability of meta-synthesis research findings. J Adv Nurs (2010) 66:246–54. doi:10.1111/j.1365-2648.2009.05250.x

45. Booth A. “Brimful of STARLITE”: toward standards for reporting literature searches. J Med Libr Assoc (2006) 94:421–e205.

46. Hannes K, Macaitis K. A move to more systematic and transparent approaches in qualitative evidence synthesis: update on a review of published papers. Qual Res (2012) 12:402–42. doi:10.1177/1468794111432992

47. CASP. Critical Appraisal Skills Programme (CASP) Qualitative Research Checklist (2013). Available from: http://docs.wixstatic.com/ugd/dded87_25658615020e427da194a325e7773d42.pdf

48. Noyes J, Popay J, Pearson A, Hannes K, Booth A. Chapter 20: qualitative research and Cochrane reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions . Cochrane: John Wiley & Sons (2015), 571–592.

49. Boeije HR, van Wesel F, Alisic E. Making a difference: towards a method for weighing the evidence in a qualitative synthesis: weighing evidence in qualitative synthesis. J Eval Clin Pract (2011) 17:657–63. doi:10.1111/j.1365-2753.2011.01674.x

50. Walsh D, Downe S. Meta-synthesis method for qualitative research: a literature review. J Adv Nurs (2005) 50:204–11. doi:10.1111/j.1365-2648.2005.03380.x

51. Kinn LG, Holgersen H, Ekeland T-J, Davidson L. Metasynthesis and bricolage: an artistic exercise of creating a collage of meaning. Qual Health Res (2013) 23:1285–92. doi:10.1177/1049732313502127

52. Lévi-Strauss C. The Savage Mind . Chicago: University of Chicago Press (1966).

53. Sandelowski M, Leeman J. Writing usable qualitative health research findings. Qual Health Res (2012) 22:1404–13. doi:10.1177/1049732312450368

54. Barry CA, Britten N, Barber N, Bradley C, Stevenson F. Using reflexivity to optimize teamwork in qualitative research. Qual Health Res (1999) 9:26–44. doi:10.1177/104973299129121677

Keywords: qualitative research, metasynthesis, metaethnography, qualitative evidence synthesis, psychiatry, suicide

Citation: Lachal J, Revah-Levy A, Orri M and Moro MR (2017) Metasynthesis: An Original Method to Synthesize Qualitative Literature in Psychiatry. Front. Psychiatry 8:269. doi: 10.3389/fpsyt.2017.00269

Received: 05 September 2017; Accepted: 17 November 2017; Published: 01 December 2017

Reviewed by:

Copyright: © 2017 Lachal, Revah-Levy, Orri and Moro. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Jonathan Lachal, jonathan.lachal@gmail.com

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base
Working with sources
Synthesizing Sources | Examples & Synthesis Matrix

Synthesizing Sources | Examples & Synthesis Matrix

Published on July 4, 2022 by Eoghan Ryan . Revised on May 31, 2023.

Synthesizing sources involves combining the work of other scholars to provide new insights. It’s a way of integrating sources that helps situate your work in relation to existing research.

Synthesizing sources involves more than just summarizing . You must emphasize how each source contributes to current debates, highlighting points of (dis)agreement and putting the sources in conversation with each other.

You might synthesize sources in your literature review to give an overview of the field or throughout your research paper when you want to position your work in relation to existing research.

Example of synthesizing sources, how to synthesize sources, synthesis matrix, other interesting articles, frequently asked questions about synthesizing sources.

Let’s take a look at an example where sources are not properly synthesized, and then see what can be done to improve it.

This paragraph provides no context for the information and does not explain the relationships between the sources described. It also doesn’t analyze the sources or consider gaps in existing research.

Research on the barriers to second language acquisition has primarily focused on age-related difficulties. Building on Lenneberg’s (1967) theory of a critical period of language acquisition, Johnson and Newport (1988) tested Lenneberg’s idea in the context of second language acquisition. Their research seemed to confirm that young learners acquire a second language more easily than older learners. Recent research has considered other potential barriers to language acquisition. Schepens, van Hout, and van der Slik (2022) have revealed that the difficulties of learning a second language at an older age are compounded by dissimilarity between a learner’s first language and the language they aim to acquire. Further research needs to be carried out to determine whether the difficulty faced by adult monoglot speakers is also faced by adults who acquired a second language during the “critical period.”

Scribbr Citation Checker New

The AI-powered Citation Checker helps you avoid common mistakes such as:

Missing commas and periods
Incorrect usage of “et al.”
Ampersands (&) in narrative citations
Missing reference entries

To synthesize sources, group them around a specific theme or point of contention.

As you read sources, ask:

What questions or ideas recur? Do the sources focus on the same points, or do they look at the issue from different angles?
How does each source relate to others? Does it confirm or challenge the findings of past research?
Where do the sources agree or disagree?

Once you have a clear idea of how each source positions itself, put them in conversation with each other. Analyze and interpret their points of agreement and disagreement. This displays the relationships among sources and creates a sense of coherence.

Consider both implicit and explicit (dis)agreements. Whether one source specifically refutes another or just happens to come to different conclusions without specifically engaging with it, you can mention it in your synthesis either way.

Synthesize your sources using:

Topic sentences to introduce the relationship between the sources
Signal phrases to attribute ideas to their authors
Transition words and phrases to link together different ideas

To more easily determine the similarities and dissimilarities among your sources, you can create a visual representation of their main ideas with a synthesis matrix . This is a tool that you can use when researching and writing your paper, not a part of the final text.

In a synthesis matrix, each column represents one source, and each row represents a common theme or idea among the sources. In the relevant rows, fill in a short summary of how the source treats each theme or topic.

This helps you to clearly see the commonalities or points of divergence among your sources. You can then synthesize these sources in your work by explaining their relationship.

Example: Synthesis matrix
	Lenneberg (1967)	Johnson and Newport (1988)	Schepens, van Hout, and van der Slik (2022)
Approach	Primarily theoretical, due to the ethical implications of delaying the age at which humans are exposed to language	Testing the English grammar proficiency of 46 native Korean or Chinese speakers who moved to the US between the ages of 3 and 39 (all participants had lived in the US for at least 3 years at the time of testing)	Analyzing the results of 56,024 adult immigrants to the Netherlands from 50 different language backgrounds
Enabling factors in language acquisition	A critical period between early infancy and puberty after which language acquisition capabilities decline	A critical period (following Lenneberg)	General age effects (outside of a contested critical period), as well as the similarity between a learner’s first language and target language
Barriers to language acquisition	Aging	Aging (following Lenneberg)	Aging as well as the dissimilarity between a learner’s first language and target language

If you want to know more about ChatGPT, AI tools , citation , and plagiarism , make sure to check out some of our other articles with explanations and examples.

ChatGPT vs human editor
ChatGPT citations
Is ChatGPT trustworthy?
Using ChatGPT for your studies
What is ChatGPT?
Chicago style
Paraphrasing

Plagiarism

Types of plagiarism
Self-plagiarism
Avoiding plagiarism
Academic integrity
Consequences of plagiarism
Common knowledge

Synthesizing sources means comparing and contrasting the work of other scholars to provide new insights.

It involves analyzing and interpreting the points of agreement and disagreement among sources.

You might synthesize sources in your literature review to give an overview of the field of research or throughout your paper when you want to contribute something new to existing research.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

Topic sentences help keep your writing focused and guide the reader through your argument.

In an essay or paper , each paragraph should focus on a single idea. By stating the main idea in the topic sentence, you clarify what the paragraph is about for both yourself and your reader.

At college level, you must properly cite your sources in all essays , research papers , and other academic texts (except exams and in-class exercises).

Add a citation whenever you quote , paraphrase , or summarize information or ideas from a source. You should also give full source details in a bibliography or reference list at the end of your text.

The exact format of your citations depends on which citation style you are instructed to use. The most common styles are APA , MLA , and Chicago .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Ryan, E. (2023, May 31). Synthesizing Sources | Examples & Synthesis Matrix. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/working-with-sources/synthesizing-sources/

Is this article helpful?

Eoghan Ryan

Other students also liked, signal phrases | definition, explanation & examples, how to write a literature review | guide, examples, & templates, how to find sources | scholarly articles, books, etc., "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Current Issues
Previous Issues
Editorial Team
Managerial Board
Become a Reviewer
Submission Guidelines
Harvard Citation Style
Vancouver Citation Style
APA Citation Style
Download RIS
Download BibTeX

Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review

Madoda Sitshange (University of Stellenbosch)

Significant poverty levels raise critical questions about the impact of poverty eradication programs. Literature reviews play a critical role in highlighting impactful and ineffective socio-economic approaches. This article presents a review of nine qualitative studies that were reported between 2006 and 2013 in poor urban, semi-rural, and rural communities in South Africa. The main goal of this paper is to describe how low-income communities were impacted by programs to reduce poverty. Themes on strength-based interventions, participation, and holistic-multidimensional approaches, emerge from the content analysis as the best account of the impact of community-based poverty eradication programs. Due to the qualitative nature of the small sample of studies under review, a thematic synthesis of the qualitative data provides baseline evidence for further research to assess progress in the provision of development programs, and to generate more insight to strengthen evidence-informed approaches to address persistently high rates of poverty.

Keywords: poverty eradication, community-based programs, thematic synthesis, systematic review, qualitative evidence, social development

Sitshange, M., (2024) “Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review”, Social Development Issues 46(3): 7. doi: https://doi.org/10.3998/sdi.6771

Downloads: Download XML Download PDF

13 Downloads

Published on 04 sep 2024, peer reviewed, creative commons attribution 4.0, introduction.

A higher rate of poverty indicates that South Africa might not reach the 2030 National Development Plan (NDP) target of reducing poverty-induced hunger to 0 percent ( National Development Plan, 2012 ). The Human Development Report (HDR) presented data showing that 18.9 percent of the population, about 11 million South Africans, live on less than R28 ($1.90) for a day. In terms of the Human Development Index, a measurement of equality developed by the United Nations (UN) that ranks countries by analyzing their quality of life against their rate of industrial development, ranked South Africa 114 out of 189 countries due to its declining standard of living and deepening income inequality ( Human Development Report, 2022 ). Since the COVID-19 pandemic, inequality worsened as the country has dropped two levels lower on the index as a result of more than 2 million job losses ( Human Development Report, 2022 ; World Bank, 2022 ).

Consistent with the statistics that are reported by global development agencies, Statistics South Africa reported that 55.5 percent of the South African population could not afford to meet their basic needs ( Stats SA, 2017 ). In terms of the Gini coefficient index, which measures inequality on a scale from 0 to 1 (where the higher values indicate higher inequality), inequality rose from 0.66 in 1993 to 0.72 in 2006: despite a decrease from 2006 to 0.68 in 2015, South Africa is the most unequal country in the world ( Stats SA, 2017 ). The World Inequality Report presented data showing that in 2014 the richest 10 percent of the population received two-thirds of the national income, while the top 1 percent received 20 percent of the national income ( Alvaredo, Chancel, Piketty, Saez, & Zucman, 2018 ).

Women make up a large percentage of the poor. Compared to male-headed households, poverty is consistently higher in female-headed households. In 2015, poverty was found to be 51.2 percent among female-headed households compared to 31.4 percent in male-headed households ( Stats SA, 2019 ). Gender “…disparities are still predominant in South Africa’s labor market with unemployment at 29.5 for women and 26.1 for men” (National Development Agency, 2019:10). In addition to gender inequality, racial disparities continue to define post-apartheid South Africa socio-economic fault lines. While the post-1994 government social and economic transformation policies improved the living conditions of many black South Africans, many still live in poverty. Unemployment rates are 30.5 percent for black Africans and 8 percent for whites. Race still affects the ability to find a job, as well as the wages received once employed ( World Bank, 2018 ).

Literature Review

The reviewer’s content analysis of the qualitative studies under review was guided by theoretical perspectives in poverty studies. According to Bradshaw (2007) , five main theoretical perspectives provide a comprehensive view of the causes, effects, and solutions to poverty: (1) Individual deficiencies, (2) Cultural belief systems that support subcultures in poverty, (3) Political-economic distortions, (4) Geographical disparities, or (5) Cumulative and circumstantial origins.

Theoretical perspectives on poverty that proffer individualistic, systematic, and cyclical explanations are summarized in Table 1 advanced into a variety of multidimensional perspectives that inform contemporary analysis and research. According to Adetoro, Ngidi, and Danso-Abbeam (2023), “a multidimensional approach has been developed to analyze a wide range of multiple poverty interrelated levels involving severe deprivation of basic human needs, such as health, education, income-generation and living standards”. The Human Development Report (2022) links the concept of multidimensional poverty with a lack of clean water, inadequate access to healthcare services, malnutrition, poor health, and poor housing conditions.

Five theories of poverty

Theory	Causes	Effects	Solutions/Interventions/Programs
Individual	Individual laziness, incompetence, inherent disabilities.	Rewards winners and punish those that don’t work hard or are lucky.	Use training and counselling to help poor individuals to overcome poverty. Safety nets to be accessible to the less fortunate.
Cultural	Adoption of values of a sub-culture that is non-productive and contrary to success.	Re-socialization through the formation of new peer groups.	Asset-based community development. Head-start program for after-school leadership development within subculture.
Political-economic structure	Systematic and structural barriers prevent the poor from accessing jobs, health, education, savings, and assets.	Selection criteria directly or indirectly exclude some groups based on a set of political conditions.	Policies to enforce inclusion and empowerment.
Geographic	Socio-economic advantage is heavily determined by geographic location.	Resource distribution and economics of scale as poverty determining factors.	Area redevelopment programs, rural development policies, and urban revitalization.
Cumulative and cyclical	Spirals of poverty are interdependent and strongly related to community dynamics.	Poverty is systematic and related to community cycles and levels of stability.	Periodic community development programs to build assets targeted at addressing individual deficiencies.

Source: Adopted from Bradshaw (2007) .

There seem to be definite areas of alignment between the South African social security system and the multidimensional poverty formulation. The Department of Social Development provides comprehensive social assistance programs for indigent individuals and families to access a range of benefits such as cash transfers, food aid, and a range of welfare services. Free basic education, free health care, and free social housing are part of social benefits that are provided by housing, health, and education government departments. A multi-departmental approach that collectively provides a range of poverty reduction programs is consistent with a multidimensional theoretical approach to poverty that is applied on a national scale to address inequality, vulnerability, and urban-to-rural poverty ( Mert & Kadioglu, 2016 ). Uni-dimensional assessments of poverty that look at monetary value and consumption, in conjunction with broader multi-dimensional approaches that focus on child poverty, early childhood development and literacy, are indicative of a dynamic theory of poverty that incorporates most of the elements in Bradshaw’s (2007) theory of poverty which considers individualistic to economic-political factors.

Zizzamia, Schotte, and Leibbrandt (2019) posit the concept of poverty dynamics, as referring to a fluid state where individuals, families, and communities experience cyclical periods of chronic poverty, transient poverty, and vulnerable poverty. In expanding on the poverty dynamics perspective, Schotte, Zizzamia, and Leibbrandt (2018) define fluid and cyclical poverty situations as conditions where the chronic poor are trapped in poverty, the transient poor are classified as below the poverty line but with above-average chances of escaping poverty, and the vulnerable are classified as above the poverty line but with above-average chances of falling into poverty. In terms of the poverty dynamics theory, a considerable share of the South African population can be classified as the transient poor and the vulnerable group, estimated as 27 percent of the population ( Zizzamia et al., 2019 ). The poverty dynamics theory draws attention to the rural and urban working poor as most vulnerable due to economic instability and volatile labor markets, irregular forms of employment, and government incompetence, which makes poverty a constant threat in their daily lives, hence are the largest population group that are beneficiaries of poverty alleviation programs.

Contextualization

Meta-synthesizing several qualitative studies sharing similar themes and methods is a well-tested scientific method for assessing and presenting broader experiences ( Graham & Masters-Awatere, 2020 ), which only some research studies can provide. The overriding aim of this article is to present a review of a small sample of primary research studies, that apply qualitative procedures to report data on the impact of poverty eradication programs, by addressing the following questions: In what ways did beneficiaries of community-based poverty eradication programs describe, in their own words, their impact? How did the beneficiaries of poverty eradication programs perceive the extent to which their living conditions improved because of poverty eradication programs?

The two research questions that guide the content analysis of qualitative studies are aimed at highlighting practices and approaches that are associated with positive and negative program outcomes. A review that focuses on program impact might contribute to the knowledge that the implementers of social development interventions need to strengthen the provision of poverty eradication programs. Global, regional, national, and/or local poverty eradication program providers require analysis of program impact to reduce high rates of poverty. Drawing key lessons from systematic reviews, using a review of literature in the paper is in line with the consensus in the research literature that (reviews) play an important role in documenting and disseminating scientific evidence on the impact of programs ( Hlongwa & Hlongwana, 2020 ; Petticrew & Roberts, 2006 ).

A review of poverty eradication programs implemented between 2006 and 2013 is intended to contribute to Lombard’s (2008) 10-year review of the implementation of the White Paper for Social Welfare (1997). The reviewer’s intention in presenting a thematic synthesis of studies focusing on poverty eradication is motivated by a need to provide another perspective on the progress of the social development approach by addressing both the root causes and effects of poverty.

Methodology

A thematic synthesis of qualitative studies on poverty eradication is guided by the question: How were the intended beneficiaries impacted by community-based poverty eradication programs? In addressing the research question, the reviewer sought to highlight themes and trends that may assist in reporting progress in the implementation of social development approaches.

The following keywords in the research report titles, abstracts, keywords, and text, guided the search for qualitative studies: “poverty/poverty-eradication/poverty-alleviation/poverty-reduction/anti-poverty”, “social exclusion”, “community-based program(s)/project(s)”, and “program/project impact”. The reviewer sourced relevant literature through an electronic search using the following databases: Social Science Citation Index on the Web of Science, Google Scholar, and the Social Science Database. A desktop internet search formed a critical part of the search, and some unpublished studies were accessed through the expert opinion of social development researchers. An online search of research reports in development studies, economics, social work, social development, politics, public health, sociology, psychology, and related social sciences and/or humanities disciplines, formed an essential part of the desktop internet search. Peer-reviewed online scientific publications were also examined. University websites were also searched for unpublished dissertations and technical research reports that focus on poverty eradication programs.

The entire search yielded studies that were reported during 2006–2013. This 10-year period is crucial for tracking and assessing progress since the social development approach became official policy, through the White Paper for Social Welfare (1997). The search for qualitative studies produced 76 research reports. The reviewer screened the 76 research studies using the inclusion and exclusion criteria, as presented in Table 2 .

Inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria

After a comprehensive search of the literature, the reviewer applied the inclusion and exclusion criteria on the titles, abstracts, keywords, and text of the 76 reports to remove excluded and duplicated reports. While this is a review of literature, the Preferred Re-porting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) ( Shamseer et al., 2015 ) was applied by the reviewer, to explain the process used to arrive at studies selected for thematic synthesis, as outlined in Figure 1 .

Systematic review process-map.

Of the 76 studies, 16 studies were sourced through a desktop internet search, while an electronic database search yielded 60 studies. Of the 76 studies that represent data on issues related to poverty eradication, social exclusion, and community-based programs, 59 were excluded: 47 mainly because they are quantitative studies, eight use poverty definitions that are inconsistent with the literature review, another set of eight studies did not report the impact of community-based programs, while six studies neither indicate whether ethics clearance was obtained nor peer-review was conducted. Of the 17 studies that complied with the inclusion criteria, further screening narrowed down the number to 15, because two studies were duplicates. After further full-text eligibility assessment, the reviewer narrowed down the number to nine short-listed studies. After re-applying the inclusion and exclusion criteria on the short-listed studies, the reviewer arrived at nine final qualitative studies to be reviewed.

A blind procedure, where a second reviewer, without knowledge of the nine short-listed studies, repeated the process-map that is outlined above, by applying the inclusion and exclusion criteria to the 76 selected studies. The second reviewer confirmed the suitability of the nine studies selected for inclusion in the systematic review. All the included studies were debated to address any inconsistencies and areas of ambiguity as far as the inclusion and exclusion criteria required, and whether the processing of studies through the process-map was standardized and replicable. All nine studies listed in Table 2 are included mainly because of three factors: they all address “‘poverty/poverty-eradication/poverty-alleviation/poverty-reduction/anti-poverty”, “social exclusion”, “community-based program(s)/project(s)”, present qualitative data, and underwent ethics clearance or were peer-reviewed. Adato, Carter, and May (2006) explore household poverty traps and social exclusion. Mashau (2006) and Nkosi (2010) investigate the poverty situation and the impact of a strategy for poverty alleviation in rural and urban areas, while Van der Merwe (2006) provides a description and analysis of the very personal, subjective experience of poverty by Afrikaans-speaking people.

Stephen (2008) , explores factors that might have an impact on the communities’ anti-poverty projects. The study focuses on four projects: two agricultural projects and two non-agricultural projects. Blaauw, Viljoen, and Schenck’s (2011) study sought to determine the prevalence of child-headed households in Gauteng in order to establish a database and to ensure access to aid programs by needy child-headed households. Strydom, Wessels, and Strydom’s (2010) study assesses the effects of health issues and poverty on families in rural areas. Kaeana and Ross (2012) investigated beneficiaries’ perceptions of income-generating projects as alleviators or perpetrators of poverty, and lastly, Sikrweqe (2013) assessed whether a local program contributed towards achieving the goals of poverty reduction.

Table 3 further outlines the sample number and type, the age range of the participants, data collection procedures, and the geographical locations where each study was conducted. All the studies present qualitative data. A study with the smallest sample number reported four participants and a study with the largest sample size reported 700 participants. The age range of the respondents in all nine studies fell within the 18–60 bracket. All the studies (n = 9) interviewed people living and working in underprivileged communities. An overview of geographical areas where the qualitative data were collected shows that the studies were collected in five provinces in South Africa: Gauteng (n = 3), KwaZulu-Natal (n = 2), Limpopo (n = 2), Northern Cape (n = 1), and the Eastern Cape (n = 1).

Studies included in the systematic review.

Aim of study	Sample size	Sample type	Age	Data	Context
Explores household poverty traps and social exclusion	50 households	Poorest households in rural, semi-urban, and urban areas	Not specified	In-depth, semi-structured interviews	KwaZulu-Natal
To investigate the poverty situation and outline a strategy for poverty alleviation in the rural area of Mashau	118 households	Key role-players	18–21	In-depth, semi-structured interviews	Limpopo Province, Mashau Village
To provide a description and analysis of the very personal, subjective experience of poverty by this group of Afrikaans-speaking people	4 households	Residents of a shelter for destitute individuals and families	Not specified	Individual unstructured interviews	Vaal Triangle, Southern Gauteng
To explore factors that might have an impact on the communities’ anti-poverty projects. The study focuses on four projects: two agricultural projects and two non-agricultural projects	49 households	Members of a community-based program	18–35	Self-administered questionnaires and structured interviews	Limpopo Province, Ga-Molepo
To assess the effects of health issues and poverty on families in rural areas.	700 households	Low-income residents	Not specified	Semi-structured interviews	Northern Cape, in Heuningvlei
To assess the impact of the Mashunka Flagship project as an approach to poverty alleviation	20 households	Members of the Mashunka flagship program	18–50	In-depth interviews	KwaZulu-Natal, Msinga Municipality
To determine the prevalence of child-headed households in Gauteng in order to establish a database and to ensure access to aid programs by needy child-headed households	61 households	Residents of Gauteng	Not specified	Semi-structured interviews and questionnaires	Gauteng Province
To investigate beneficiaries’ perceptions of income-generating projects as alleviators or perpetrators of poverty	20 households	Beneficiaries of an income-generating project	20–60	Semi-structured interviews	Sedibeng, Gauteng Province
To establish whether KSD municipality’s IDP contributes towards achieving the goals of poverty reduction	20 households	Residents and municipal directors	30–49	Questionnaire and semi-structured interviews	Eastern Cape, King Sabata Dalindyebo

Guided by relevant theoretical perspectives in poverty, social development, and community-based programs, the reviewer content analyzed the studies by coding of text “line-by-line”; followed by the development of “descriptive themes”; and lastly, the generation of “analytical themes” ( Thomas & Harden, 2008 ; Tong, Palmer, Craig, & Strippoli, 2016 ). “Thematic analysis” completed the identification and confirmation of emerging themes ( Thomas & Harden, 2008 ), and allowed the reviewer to present the qualitative evidence directly from the studies under review. The procedure that was followed by the reviewer enabled explicit translation of the qualitative data by “…synthesizing them in a transparent way, and facilitating the explicit production of new concepts and hypotheses” ( Thomas & Harden, 2008 , p. 1).

Results and Discussion

A thematic synthesis of 2006 to 2013 qualitative studies on the impact of community-based women empowerment programs.

The discussion of findings is presented under three themes: strength-based interventions, participation, and holistic, multi-dimensional approaches. Guided by the literature on poverty eradication, social exclusion, and community-based programs, a comprehensive content analysis of the qualitative data from the studies selected for review enabled the extraction and presentation of the following discussion of the research results.

Strength-based interventions

A content analysis of community-based poverty eradication programs shows that developing the strength of indigent individuals and families is crucial. Strydom et al.’s (2010) study highlights the importance of linkages between the well-being and happiness of beneficiaries and social services providers’ acceptance and enhancement of their (i.e., beneficiaries of poverty-eradication programs) strengths, and material and human resources. The impact of programs, in other words, will be greatly enhanced by leveraging underutilized personal and group coping and survival capabilities. Leveraging underutilized coping and survival capabilities might sustain the structure and functioning of indigent individuals, families, and communities ( Strydom et al., 2010 ). In an urban setting, Van der Merwe (2006 , p.141) posits that psychosocial programs “…need to capitalise on existing strengths and cultivate new personal strengths such as self-confidence, creativity, and capacity for hard work, self-determination, optimism and faith”.

Authors specifically identify social connections, as opposed to social isolation and social exclusion, as crucial for community-based programs to empower indigent individuals and families, to access income-generating opportunities or to cope better in times of periodical cycles of vulnerability to poverty ( Adato et al., 2006 ; Blaauw et al., 2011 ; Sikrweqe, 2013 ; Strydom et al., 2010 ).

Community-based poverty eradication programs that focus on building the strengths of women and children report a crucial area that social service providers need to focus on. Nkosi’s (2010) study found that child-headed and female-headed households went beyond being passive beneficiaries of cash transfers, to using limited savings to access crucial life-skills that translated into increased school attendance, fewer risks of malnutrition, and exposure to abuse. According to Blaauw et al. (2011) , school-based poverty eradication programs play a crucial role in improving the socioeconomic circumstances of child-headed households, primarily by directly linking the development of strengths to child-headed households to directly accessing social services and cash-transfers, rather than relying on adults who might abuse the resources. Even though poverty and the scarcity of resources can cause conflict in households, the respondents in Strydom et al.’s (2010) study felt strongly that the family was their important strength. As a strength that poverty eradication programs must build on, authors present data confirming the family as a form of social capital that is best placed to stabilize basic livelihood levels, owing to the observation that families have the capacity to adapt, change, and become closer in times of social and economic shocks ( Mashau, 2006 ; Stephen, 2008 ).

Gaps in eradicating poverty through the development of individual and family strengths receive great attention in the research literature. According to Adato et al. (2006 , p. 226), for individuals and families that are considered to be living below the poverty line, “…social capital at best helps stabilize livelihoods at low levels and does little to promote upward mobility”. Access to programs that provide a combination of assets with financial value, income-generating capabilities, and access to markets to build on assets over time, could sustainably address both the root causes and effects of poverty, and upward social mobility ( Adato et al., 2006 ; Stephen, 2008 ; Van der Merwe, 2006 ). Focusing poverty eradication programs on individuals and families has its critical limitations. According to Blaauw et al.’s (2011) post-intervention analysis, 26.2 percent of households cannot support even one person with their total monthly income, while less than 40 percent would be able to support a household of two to three members, with only 11.5 percent able to support a household size of four people, and none of the respondents’ households able to support a household size of six members. Holistic and multi-dimensional poverty eradication programs require evidence-informed approaches to supplement and complement strength-based interventions that support individuals, families, and larger communities.

Participation

Participation allowed the beneficiaries of community-based poverty eradication programs to highlight progress and identify gaps in service delivery ( Kaeana & Ross, 2012 ). Sikrweqe’s (2013) study echoes the theme of opening program monitoring and evaluating the voices of the beneficiaries, by presenting data showing that ward committees went beyond improving the participation of beneficiaries, and ensuring that the beneficiaries directly influence decisions about future developments in the neighborhood. The ability of ordinary members of society to influence decisions about development issues deepens democratic practices and governance ( Sikrweqe, 2013 ). Mashau’s (2006) assessment of a flagship local job creation project, highlights a human-centered approach to a collaborative approach to poverty eradication, that brought together the combined strengths of all key stakeholders, ordinary community members, government officials, business people, and non-governmental and faith-based organizations.

At a more practical level, Van der Merwe’s (2006) study emphasizes that, where possible, the beneficiaries must participate in all important areas and phases of program implementation to promote the type of community ownership that will invest in long-term sustainability. In recognition that participation in community-based poverty eradication programs is easier said than done, authors recommend further in-depth research analysis of the impact of participation on the outcomes of poverty eradication programs ( Blaauw et al., 2011 ; Kaeana & Ross, 2012 ; Mashau, 2006 ; Nkosi, 2010 ; Stephen, 2008 ; Van der Merwe, 2006 ). According to Stephen (2008) , least participatory programs tended to have pensioners as the majority of beneficiaries, thereby sensitizing social service providers to be more realistic and strategic in customizing models of participation to be more consistent with the capabilities of the intended beneficiaries.

The main conclusion in Kaeana and Ross’s (2012) study is that income-generating projects achieved their aims to some extent, but there were areas of improvement in terms of the participation of beneficiaries in decision-making. In reiterating the theme of the importance and limitations of participation, Adato et al. (2006) assert that while the impact of the beneficiaries’ participation in poverty eradication programs cannot be denied, there is no compelling evidence that community participation in poverty eradication translated directly into economic advancement and the accumulation of assets with long-term financial value. The link between the level of beneficiary participation in program processes to the reduction of poverty appears to be complex and still to be sufficiently examined, especially when participation occurs within holistic and multi-dimensional approaches.

Holistic and multi-dimensional approaches

A comprehensive and integrated research-informed approach to establish a local and contextually grounded database, according to Blaauw et al. (2011) and Strydom et al. (2010) , sets a standard for poverty to be addressed as the main target of health, development, education, employment creation, and environmental programs. A holistic, multi-disciplinary, and multi-dimensional approach to poverty eradication could be more impactful, in respect of the data that shows that poverty mainly manifests itself in the deprivation of income-generating opportunities, housing, lack of clean water, sanitation, health services, electricity, literacy, public infra-structure, and so on ( Mashau, 2006 ; Sikrweqe, 2012; Stephen, 2008 ; Strydom et al., 2010 ).

According to Kaeana and Ross’s (2012) study, a holistic and multi-pronged approach to poverty eradication needs to integrate income generation and employment creation, the provision of social and physical infrastructure including clinics and schools, measures to address social exclusion and institutionalized racism, xenophobia and sexism, the promotion of sustainable livelihoods, and the dissemination of the type of knowledge and skills that fosters human development at the community level. Community-based programs to raise awareness, through education and skills development are central themes that are frequently reported by the studies under review. For instance, more impactful community-based poverty eradication programs had more beneficiaries with secondary school education, while the worst performing were fewer ( Blaauw et al., 2011 ; Mashau, 2006 ; Nkosi, 2010 ; Van Der Merwe, 2006 ).

Deeper structural changes require innovative and novel approaches in light of growing levels of poverty, unemployment, and socio-economic inequality. According to some authors, the broader problem of “…poverty alleviation seems unlikely to be resolved until deeper structural changes make time and markets work more effectively for the broader community of all South Africans” ( Adato et al., 2006 , p. 245). A theme that cuts across most studies is that current social security programs play a significant role in alleviating poverty, but because of the growing inequality, the social security systems need to be improved to address gaps and shortcomings ( Adato et al., 2006 ; Kaeana & Ross, 2012 ; Sikrweqe, 2012). The research participants in Nkosi’s (2010) study, correctly recommend that gaps and weaknesses in social security programs can be best addressed through intersectoral collaborations between governmental and non-governmental service providers, in conjunction with the training of beneficiaries as a key element towards the sustenance of program impact and comprehensive service delivery. The findings are consistent will the assertion that policymakers recognize the integrated approach as more effective in low-middle-income countries ( Kumar & Cheng, 2024 ).

Recommendations

Similar to systematic reviews, literature reviews analyze an ever-growing scope of research on “best practices” for policy-making and policy evaluation ( Sundberg & Taylor-Gooby, 2013 ; Van Rooyen, Steward, & De Wet, 2012 ). This paper reviews the qualitative evidence to highlight approaches in poverty eradication that can be inferred as impactful and ineffective, subject to more advanced analysis through large-scale reviews that apply qualitative and quantitative methods. This article recommends further systematic reviews that will analyze studies conducted between 2016 and 2023, to provide a more recent and comprehensive picture of the progress and challenges related to social development programs. Reviews place greater emphasis on transparency and accountability ( Thomas & Harden, 2008 ), by providing an overview of impactful and ineffective approaches that no single study can provide. Themes on beneficiaries’ strengths and direct involvement in crucial phases of holistic and multi-multidimensional community-development processes, emerge in the paper as significant to track in forthcoming systematic reviews. Training and research in the three themes outlined above are key areas of focus in assessing progress in the implementation of the social development approach. As noted in a related paper, training “emerges as an important option in expanding the prospects of the intended beneficiaries of community-based programs” ( Sitshange, 2022 ).

Conclusions

Reviews are critical in evaluating the impact of poverty eradication programs. According to the authors, poverty alleviation programs have been ineffective and unsustainable ( Dipela & Mohapi, 2021 ; Raniga, 2018 ), hence high rates of poverty are consistently reported by statisticians. Reviewers of research studies have a responsibility to beyond painting the impact of poverty through numbers, to highlighting the impact of programs using the voices of community members. The thematic synthesis of qualitative research studies in poverty eradication notes a gap between theory and practice. While laws, policies, and institutions are in place to eradicate poverty, reviews need to empower relevant laws, policies, and institutions to prove impact and sustainability using evidence-based frames of reference. While the review that is presented in the paper is qualitative and limited, it lays a basis for more advanced studies on the impact of poverty eradication programs on individuals and groups.

Adato, M., Carter, M. R., & May, J. (2006). Exploring poverty traps and social exclusion in South Africa using qualitative and quantitative data. The Journal of Development Studies , 42(2), 226–247. https://doi.org/10.1080/00220380500405345 https://doi.org/10.1080/00220380500405345

Adetoro, A. A., Ngidi, M. S. C., & Danso-Abbeam, G. (2023). Towards the global zero poverty agenda: Examining the multidimensional poverty situation in South Africa. SN Social Sciences , 3(148), 1–22. https://doi.org/10.1007/s43545-023-00735-2 https://doi.org/10.1007/s43545-023-00735-2

Alvaredo, F., Chancel, L., Piketty, T., Saez, E., & Zucman, G. (2018). World Inequality Report 2018, World Inequality Lab.

Blaauw, D., Viljoen, K., & Schenck, R. (2011). “Life is not pap and vleis”: Poverty in child-headed households in Gauteng. Social Work/Maatskaplike Werk , 47(2), 138–154. https://doi.org/10.15270/47-2-132 https://doi.org/10.15270/47-2-132

Bradshaw, T. K. (2007). Theories of poverty and anti-poverty programs in community development. Journal of the Community Development Society , 38(1), 7–25. https://doi.org/10.1080/15575330709490182 https://doi.org/10.1080/15575330709490182

Dipela, M. P., & Mohapi, B. (2021). Barriers affecting effective monitoring and evaluation of poverty alleviation projects within Waterberg District. Social Work/Maatskaplike Werk , 57(3), 278–301. https://doi.org/10.15270/57-3-948 https://doi.org/10.15270/57-3-948

Graham, R., & Masters-Awatere, B. (2020). Experiences of Māori of Aotearoa New Zealand’s public health system: A systematic review of two decades of published qualitative research. Aust NZ J Public Health , 44(3), 193–200. https://doi.org/10.1111/1753-6405.12971 https://doi.org/10.1111/1753-6405.12971

Hlongwa, M., & Hlongwana, K. (2020). Men’s perspectives on HIV self-testing in sub-Saharan Africa: A qualitative systematic review protocol. JBI Evid Synth , 18(3), 571–575. https://doi.org/10.11124/JBISRIR-D-19-00097 https://doi.org/10.11124/JBISRIR-D-19-00097

Human Development Report. (2022). Report to explore uncertainty in the Anthropocene . United Nations: Geneva.

Kaeana, R., & Ross, E. (2012). Income-generating projects: Alleviating or perpetuating poverty. Social Work/Maatskaplike Werk , 48(1), 17–34. https://doi.org/10.15270/48-1-102 https://doi.org/10.15270/48-1-102

Kearney, M. H. (2001). Enduring love: Grounded formal theory of women’s experience of domestic violence. Research in Nursing and Health , 24(4), 270–282. https://doi.org/10.1002/nur.1029 https://doi.org/10.1002/nur.1029

Kumar, V., & Cheng, S. C. (2024). A comparative literature review of integrated approach in health care in high and low-middle-income countries. Social Development Issues , 46(1), 25–44. https://doi.org/10.3998/sdi.5294 https://doi.org/10.3998/sdi.5294

Lombard, A. (2008). The implementation of the white paper for social welfare. The Social Work Practitioner-Researcher , 20(2), 154–173. http://hdl.handle.net/2263/9739 http://hdl.handle.net/2263/9739

Mashau, T. D. (2006). Towards a strategy for poverty alleviation in Mashau . Master’s diss., North-West University.

Mert, K., & Kadioglu, H. (2016). Nursing interventions to help prevent children from working on the streets. International Nursing Review , 63(3), 429–436. https://doi.org/10.1111/inr.12301. Epub 2016 Jul 19. PMID: 27430362. https://doi.org/10.1111/inr.12301. Epub 2016 Jul 19. PMID: 27430362.

National Development Plan. (2012). National Development Plan 2030: Our future make it work . Pretoria: National Planning Commission.

Nkosi, B. W. G. (2010). An impact of flagship program: An approach to poverty alleviation . Master’s diss., University of Zululand.

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences . Oxford: Blackwell.

Porritt, K., Gomersall, J. C., & Lockwood, C. S. (2014). Study selection and critical appraisal. AJN American Journal of Nurs-ing , 114(6), 47–52. https://doi.org/10.1097/01.NAJ.0000450430.97383.64 https://doi.org/10.1097/01.NAJ.0000450430.97383.64

Raniga, T. (2018). Poverty alleviation, social protection policy and sustainability of economic development cooperatives: Voices of women residing in Bhambhayi, KwaZulu-Natal, South Africa. Social Work/Maatskaplike Werk , 54(4), 394–406. https://doi.org/10.15270/54-4-668 https://doi.org/10.15270/54-4-668

Republic of South Africa (RSA). (1997). Ministry of Welfare and Population. White Paper for Social Welfare. Notice 1008 of 1997. Government Gazette , 368, (18166). Pretoria: Government Printer.

Schotte, S., Zizzamia, R., & Leibbrandt, M. (2018). A poverty dynamics approach to social stratification: The South Africa case. World Development , 110 (2018), 88–103. https://doi.org/10.1016/j.worlddev.2018.05.024 https://doi.org/10.1016/j.worlddev.2018.05.024

Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., et al. (2015). Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P): Elaboration and explanation. BMJ , 586, 1–25. https://doi.org/10.1136/bmj.g7647 https://doi.org/10.1136/bmj.g7647

Sikrweqe, N. P. (2013). Integrated development planning as a poverty reduction strategy in the King Sabata Dalindyebo Municipality, Eastern Cape Province . Master’s diss., University of South Africa.

Sitshange, M. (2022). A thematic synthesis of income-generating studies conducted from 2007 to 2012 in Gauteng, Limpopo and KwaZulu-Natal: A systematic review of the qualitative evidence. Social Work/Maatskaplike Werk , 58(4), 382–393. https://doi.org/10.15270/58-4-1072 https://doi.org/10.15270/58-4-1072

Statistics South Africa (Stats SA). (2017). Poverty Trends in South Africa. An examination of absolute poverty between 2006 and 2015 . Pretoria: Statistics South Africa.

Statistics South Africa (Stats SA). (2019). Marginalised Groups Indicator Report. Report No. 0319-05. Retrieved from https://www.statssa.gov.za/publications/03-19-05/03-19-052019.pdf https://www.statssa.gov.za/publications/03-19-05/03-19-052019.pdf

Stephen, K. (2008). The impact of poverty alleviation projects in Ga-Molepo area in Polokwane Municipality, Limpopo Province . Master’s diss., University of Limpopo.

Strydom, C., Wessels, C., & Strydom, H. (2010). The role of the social worker to empower families in a deep rural community. Social Work/Maatskaplike Werk , 46(2), 175–191. https://doi.org/10.15270/46-2-172 https://doi.org/10.15270/46-2-172

Sundberg, T., & Taylor-Gooby, P. (2013). A systematic review of comparative studies of attitudes to social policy. Social Policy & Administration , 47(4), 416–433. ISSN 0144-5596. https://doi.org/10.1111/spol.12027 https://doi.org/10.1111/spol.12027

Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology , 8(45), 1471–2288. https://doi.org/10.1186/1471-2288-8-45 https://doi.org/10.1186/1471-2288-8-45

Tong, A., Palmer, S., Craig, J. C., & Strippoli, G. F. M. (2016). A guide to reading and using systematic reviews of qualitative research. Nephrology Dialysis Transplantation , 31(6), 897–903. https://doi.org/10.1093/ndt/gfu354 https://doi.org/10.1093/ndt/gfu354

Van der Merwe, K. (2006). The phenomenology of experiencing poverty – An exploration. The Journal for Transdisciplinary Research in Southern Africa , 2(1), 131–144. https://doi.org/10.4102/td.v2i1.311 https://doi.org/10.4102/td.v2i1.311

Van Rooyen, C., Steward, R., & De Wet, T. (2012). The impact of microfinance in Sub-Saharan Africa: A systematic review of the evidence. World Development , 40(11), 2249–2262. https://doi.org/10.1016/j.worlddev.2012.03.012 https://doi.org/10.1016/j.worlddev.2012.03.012

World Bank. (2018). Overcoming poverty and inequality in South Africa: An assessment of drivers, constraints and opportunities. Retrieved from https://documents1.worldbank.org/curated/en/530481521735906534/pdf/124521-rev-ouo-south-africa-povertyand-inequality-assessment-report-2018-fnal-web.pdf https://documents1.worldbank.org/curated/en/530481521735906534/pdf/124521-rev-ouo-south-africa-povertyand-inequality-assessment-report-2018-fnal-web.pdf

World Bank. (2022). Report on inequality in Southern Africa: An assessment of the Southern African Customs Union. Retrieved from https://www.worldbank.org/en/events/2022/03/09/inequality-in-southern-africa-an-assessment-of-the-southern-african-customs-union https://www.worldbank.org/en/events/2022/03/09/inequality-in-southern-africa-an-assessment-of-the-southern-african-customs-union

Zizzamia, R., Schotte, S., & Leibbrandt, M. (2019). Snakes and ladders and loaded dice: Poverty dynamics and inequality in South Africa, 2008-2017 , WIDER Working Paper Series wp-2019-25, World Institute for Development Economic Research (UNU-WIDER).

Madoda Sitshange, Department of Social Work, University of Stellenbosch, South Africa and Department of Politics and International Relations, University of Johannesburg, South Africa. He can be contacted at [email protected]

Harvard-Style Citation

Sitshange, M. (2024) 'Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review', Social Development Issues . 46(3) doi: 10.3998/sdi.6771

Show: Vancouver Citation Style | APA Citation Style

Vancouver-Style Citation

Sitshange, M. Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review. Social Development Issues. 2024 9; 46(3) doi: 10.3998/sdi.6771

Show: Harvard Citation Style | APA Citation Style

APA-Style Citation

Sitshange, M. (2024, 9 4). Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review. Social Development Issues 46(3) doi: 10.3998/sdi.6771

Show: Harvard Citation Style | {% trans 'Vancouver Citation Style' %}

Non Specialist Summary

This article has no summary

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
J Epidemiol Community Health
v.61(3); 2007 Mar

Synthesising quantitative and qualitative research in evidence‐based patient information

Systematic reviews have, in the past, focused on quantitative studies and clinical effectiveness, while excluding qualitative evidence. Qualitative research can inform evidence‐based practice independently of other research methodologies but methods for the synthesis of such data are currently evolving. Synthesising quantitative and qualitative research in a single review is an important methodological challenge.

This paper describes the review methods developed and the difficulties encountered during the process of updating a systematic review of evidence to inform guidelines for the content of patient information related to cervical screening.

Systematic searches of 12 electronic databases (January 1996 to July 2004) were conducted. Studies that evaluated the content of information provided to women about cervical screening or that addressed women's information needs were assessed for inclusion. A data extraction form and quality assessment criteria were developed from published resources. A non‐quantitative synthesis was conducted and a tabular evidence profile for each important outcome (eg “explain what the test involves”) was prepared. The overall quality of evidence for each outcome was then assessed using an approach published by the GRADE working group, which was adapted to suit the review questions and modified to include qualitative research evidence. Quantitative and qualitative studies were considered separately for every outcome.

32 papers were included in the systematic review following data extraction and assessment of methodological quality. The review questions were best answered by evidence from a range of data sources. The inclusion of qualitative research, which was often highly relevant and specific to many components of the screening information materials, enabled the production of a set of recommendations that will directly affect policy within the NHS Cervical Screening Programme.

Conclusions

A practical example is provided of how quantitative and qualitative data sources might successfully be brought together and considered in one review.

The National Health Service Cervical Screening Programme (NHSCSP) first published evidence‐based guidelines addressing the content of screening letters and leaflets in 1997. 1 The NHS Cancer Plan (September 2000) called for honest, comprehensive and understandable screening materials 2 and a White Paper (November 2004) emphasised the need for up‐to‐date and factual health information. 3 An important NHSCSP priority is the continual improvement of the quality of written information sent to women about cervical screening. As such, a review was commissioned in early 2004 to develop updated NHSCSP patient‐information guidelines. The focus of the review was directed and informed by two main questions:

What is the existing research evidence base regarding the content of the written information sent to women at all stages of the cervical screening process?
What are the information needs of women during cervical screening?

The main purpose of the review was to identify important information for inclusion in the screening programme letters and leaflets (introductory, abnormal result and colposcopy). Important information included facts (length of the screening interval), concepts (purpose of the screening test) and general points (suitable attire). For each key information point, a tabular summary was created of the relevant evidence. The full updated guidelines are described elsewhere. 4 A selection of the review recommendations is presented in fig 1 1. . The reviewers were experienced in conducting quantitative and qualitative research related to cancer screening and patient and health professional information.

An external file that holds a picture, illustration, etc.
Object name is ch46110.f1.jpg

Figure 1 The review process (including a selection of the review recommendations).

Systematic reviews

Conventional systematic reviews aim to bring together and summarise the available research addressing a question of interest so that inferences about findings may be guided by the most appropriate forms of evidence. 5 , 6 Evidence synthesis systems have, in the past, given preference to quantitative studies (mainly randomised trials) and clinical effectiveness. Accordingly, explicit methods have been established for the systematic review of trial‐based evidence including comprehensive literature searching, detailed quality appraisal procedures and standard synthesis techniques. 7 , 8 , 9 , 10 Interest is growing in the development of diverse systematic review methods to incorporate different types of evidence including other quantitative study designs as well as qualitative research. 11 Methods for the synthesis of qualitative research are still in the early stages of proposal and evaluation, 12 , 13 and the suitability of fitting various forms of qualitative research within the framework of conventional review methodology is an active research topic. 11 There is, in fact, still debate about whether the synthesis of qualitative evidence is appropriate and whether it is acceptable to combine qualitative studies derived from different traditions. 13 , 14 , 15

Limitations of traditional hierarchies of evidence

Traditional evidence hierarchies were developed specifically to address questions of efficacy and effectiveness 6 and involved assessing research according to study design with the randomised trial as the premier form of evidence. 7 , 16 This becomes problematic in areas of research dominated by non‐trial quantitative evidence owing to the lower quality scores subsequently assigned to these studies. 17 Also, evidence syntheses concentrating on wider issues such as “How does it work?” or “What are patients' experiences?” require a different approach because these types of research questions are best answered by evidence from a variety of sources. 15 , 18 , 19 , 20

Qualitative research

Qualitative evidence has a role to play in answering questions not easily evaluated by experimental studies. 18 , 20 , 21 Although qualitative studies do not establish probability estimates or effect sizes, they can provide important support for quantitative outcomes and identify patient priorities and concerns. 12 However, the usefulness of qualitative research is not limited to the explanation of process measures. In fact, qualitative research may address many healthcare questions directly and inform evidence‐based practice independently of other research methodologies. 12 , 21 , 22

Importance of a range of evidence

Policy‐makers and practitioners assessing complicated healthcare questions draw on diverse sources of evidence during the decision‐making process. 13 , 14 , 23 The development of methods for the synthesis of different types of research in a single review is an important methodological challenge. A number of approaches (narrative summary, thematic analysis, meta‐ethnography and bayesian methods) could be applied to the synthesis of both qualitative and quantitative research. 12 , 13 However, most have evolved from techniques used for primary data analysis and were initially developed for either qualitative or quantitative data—not both. 13 Several groups are currently working on methods for the incorporation of qualitative evidence into systematic reviews including the international Cochrane Qualitative Methods Group, the international Campbell Collaboration, the Evidence for Policy and Practice Information and Coordinating Centre in the UK and the Joanna Briggs Institute in Australia. 23 The UK Economic and Social Research Council has also funded a programme of research in this area. 23 , 24

Box 1 First example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

Colposcopy information leaflets: what women want to know and when they want to receive this information 28

- Qualitative component (Part A) and
- Non‐comparative descriptive component (Part B)
- To determine what information women want to receive about colposcopy and when they want to receive it
- To evaluate how this relates to the NHSCSP guidelines
- Part A: A total of 42 women with abnormal Pap smear results attending a pre‐colposcopy counselling session at a UK cancer centre colposcopy clinic
- Part B: 100 consecutive women with abnormal Pap smear results newly referred to a UK cancer centre colposcopy clinic
- Part A: Observation and documentation
- Part B: Self‐report via questionnaire
- Part A: A list of 38 questions asked by 50% or more of the women at the pre‐colposcopy counselling session (eg, what is an abnormal smear? or what is colposcopy?)
- Part B: Preferred timing of information delivery and information needs identified by women
- The third component of this study involved an assessment of information leaflets from 128 colposcopy clinics and is not described in this text box

Although methods of synthesising quantitative and qualitative research are beginning to be developed, few studies have been published that address the problem from a practical point of view. 9 , 15 , 25 , 26 The current lack of specific guidance and accumulated experience in the synthesis of a range of research evidence leaves would‐be reviewers in a difficult position. The aim of this paper is to contribute to the debate by describing the review methods developed and difficulties encountered during the process of updating evidence‐based guidelines for the content of NHSCSP letters and leaflets.

Box 2 Second example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

An observational study of precolposcopy education sessions: what do women want to know? 29

- Qualitative study
- To observe group counselling educational sessions for women about colposcopy held before the procedure
- To identify specific concerns about cervical cancer, the procedure of colposcopy and any longer term effects of the procedure
- A total of 47 women with abnormal Pap smear results attending 1 of 5 precolposcopy group counselling educational sessions run by two specialist hospital colposcopy clinic nurses
- Observation—participants' questions, comments and non‐verbal communication (eg, laughter or anxiety) were recorded verbatim
- Information needs identified by women

What is the evidence base?

The literature describing screening information content and women's information needs was dispersed among a variety of disciplines, and numerous research designs potentially addressed the review questions. Consequently, the review evidence base was complex and ill defined. This is not a unique problem, many healthcare questions are not easily evaluated solely by experimental methods (ie randomised trials) or other forms of quantitative investigation. 12 , 15 , 17 , 27 Boxes 1–4 provide examples of the range of data sources included in the review (the four studies described were focused on colposcopy information).

Literature searching

Systematic searches of 12 electronic databases (January 1996 to July 2004 inclusive) were conducted. During the search development, a “test” subset of relevant studies was used to assess whether thesaurus or free‐text terms could be included in isolation. A combination of both was required to identify all the “test” studies. As discussed by Shaw et al , 32 the cost of designing a comprehensive search strategy is the lack of precision. Study design filters may help improve precision, however none were incorporated in this work because of concerns about poor indexing in electronic databases—particularly for qualitative studies. 9 , 18 , 20 , 32 , 33 , 34 Instead, a core set of subject specific terms (cervical smears, colposcopy, screening and neoplasms) was used in combination with thesaurus and general free‐text terms such as “leaflet*”, “knowledge” or “understand*” to identify relevant studies.

Box 3 Third example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

Patient‐based evaluation of a colposcopy information leaflet. 30

- Non‐comparative descriptive study
- To evaluate an information leaflet, “Prevention of cervical cancer,” routinely sent to women referred for colposcopy
- To use the findings of the evaluation to propose changes to the leaflet
- A total of 137 women with abnormal Pap smear results newly referred to the Ninewells Hospital colposcopy clinic in Dundee, Scotland
- Self‐report via questionnaire
- Information leaflet text (terms and language used)
- Satisfaction with information provided
- Text of information leaflet included with the study report

Additional references were taken from the table of contents of selected journals, reference sections of relevant studies, and the NHSCSP research literature online database ( http://www.nhs.thescienceregistry.com/main.asp ). Grey literature was sought from internet resources and contact with subject area specialists. In total, 1063 citations were identified (both published and unpublished, with no language restrictions).

Initial citation and full study report assessments

Titles and abstracts of citations were independently prescreened by two reviewers according to review study selection criteria (box 5). Others have noted that when present, the abstracts of qualitative studies vary in content (ie, study design details may not be reported). 32 , 34 The inclusion or exclusion criteria were liberally applied and if doubt existed, individual studies were provisionally included for consideration on the basis of full text reports. Only about 3% of the screened citations lacked an abstract.

A total of 233 studies were retrieved for further evaluation. Two reviewers independently assessed the full study reports; any uncertainty was resolved by discussion. Most of the the exclusions fell into three categories: studies reporting interventions aimed at increasing screening uptake (35), non‐UK based studies investigating non‐generalisable cervical screening issues (41), and studies focused on aspects of cervical screening other than written information materials (51). The remaining 27 excluded studies belonged to several different categories listed in box 5. An important consideration for prospective reviewers is the time required to complete the search strategy development as well as the initial citation and full study report assessments. In this review, around 4 months of full‐time work was dedicated to these preliminary steps.

Box 4 Fourth example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

Is the provision of information leaflets before colposcopy beneficial? A prospective randomised study 31

- Randomised controlled trial
- To assess the usefulness of a leaflet sent to women before colposcopy in reducing anxiety
- A total of 210 women diagnosed with moderate dyskaryosis or less newly referred to the North Staffordshire colposcopy clinic
- Intervention group: information leaflet sent with clinic appointment letter
- Control group: clinic appointment letter only
- Information leaflet assessment (intervention group only)

Data extraction

Data extraction was conducted for 79 studies (52 quantitative and 27 qualitative). A data extraction form was used to record full study details and guide decisions about the relevance of individual studies to the review questions. The form was developed using guidelines produced by the UK National Health Service Centre for Reviews and Dissemination (CRD) 3 and other publications. 35 , 36 , 37 Similar information was extracted for all studies and included: aims, setting, population, research design, methods, interventions (if appropriate), results and conclusions. Data were extracted from relevant studies by one reviewer and checked by a second reviewer.

Identification of key information points

A core set of key information points for the screening programme letters and leaflets was developed from the 1997 guidelines 1 and the extracted data. All the information points from the original guidelines as well as new points identified from the findings of the data extracted studies were included in the core set. In total, 32 studies reported relevant results (the range of study designs is described in box 6). Although a large number of studies underwent data extraction and ultimately many did not report relevant findings, it was essential to consider a variety of potentially applicable studies to obtain a comprehensive set of key information points. The excluded studies fell into the following categories: studies reporting interventions aimed at increasing screening uptake (n = 3), non‐UK based studies investigating non‐generalisable cervical screening issues (n = 3), studies focused on aspects of cervical screening other than written information materials (n = 14) and studies containing information‐related findings not directly relevant to the agreed key information points (n = 20). A further seven studies (five quantitative and two qualitative) contained insufficient information about study design, methods of analysis and findings for a full assessment.

Box 5 Study selection criteria

Inclusion criteria

- Women, 20–64 years
- Organised, systematic cervical screening programme (screening delivered in GP surgeries, community or hospital clinics)
- Assessment of the content of written information materials provided to women about cervical screening at all stages of the cervical screening process
- Investigation of the information needs of women at all stages of the cervical screening process
- All (except for opinion pieces)

Exclusion criteria

Interventions focused on medical professional education, general practice performance and systems, cervical screening technology, protocols and technical aspects of treatment for cervical intra‐epithelial neoplasia and cervical cancer
Interventions that aimed to increase cervical screening uptake (except where the content of participant information materials was evaluated and/or included with the study report)
Studies reporting non‐information based predictors of screening uptake and risk factors for cervical cancer
Studies reporting knowledge, attitudes, health beliefs or barriers towards cervical screening without reference to written information materials or information needs
Non‐UK based studies investigating cervical screening issues not generalisable to the UK screening population and/or setting
Studies reporting insufficient information about study design, methods of analysis and findings for a full assessment
Studies reporting information‐related findings not directly relevant to the core set of key information points

Study methodology—quality scoring

Study quality appraisal is undertaken in quantitative systematic reviews to assess bias—for example, appropriate randomisation for trials—and to identify other study design specific defects. 11 , 38 Many quality appraisal checklists are available for different types of quantitative evidence however, for study designs other than randomised trials, the key elements of quality are not as well agreed. 38 The issue of how or whether to assess qualitative studies is a matter of some debate. 13 , 14 , 33 , 39 , 40 The UK NHS CRD guidance favours structured appraisal of qualitative research but recognises that consensus is lacking on appropriate criteria. 7

Although the exact function of quality appraisal in reviews of quantitative and qualitative evidence is controversial, it is recognised that reviewers should highlight evidence quality issues—in conduct, reporting, or both for review users. 13 As such, the quality of each individual study included in the review was assessed by two reviewers using established checklists. Separate checklists were used for different quantitative study designs whereas a single checklist was developed for all qualitative studies (box 6). 7 , 41 , 42 , 43 , 44 All checklists incorporated a coded comments system that allowed reviewers to record an assessment of each checklist component. After consideration of all items on a given checklist, the methodological quality of each study was rated as: ++ (all or most of the criteria have been fulfilled), + (some of the criteria have been fulfilled) or – (few or no criteria have been fulfilled).

The quality appraisal provided an indication of the strengths and weaknesses of each study. Although study‐design flaws (eg, no intention‐to‐treat analysis conducted for a randomised trial) could prompt the exclusion of quantitative evidence, in practice, none was excluded at this stage. No exclusion rules were applied to the qualitative studies as guidance is lacking on how to exclude “weak” qualitative findings. The quality scores assigned to individual studies were taken into account during the research synthesis.

Research synthesis

The first step in the synthesis process was to construct a tabular summary of all studies related to each key piece of information identified as important for inclusion in the NHSCSP letters and leaflets. An example, the evidence profile created for the colposcopy leaflet information point “Indicate that treatment can occur at the first colposcopy clinic visit”, is presented in table 1 1 (a summary of the study findings is also included).

Study design	Appraisal checklists
Randomised controlled trial (n = 3)	SIGN and CASP
Cluster randomised controlled trial (n = 1)	SIGN and CASP
Quasi‐randomised controlled trial (n = 3)	SIGN and CASP
Retrospective case control study (n = 1)	SIGN and CASP
Cross‐sectional study (n = 3)	SIGN and NZGG
Non‐comparative descriptive study (n = 7)	Reviewer checklist
Non‐comparative time series study (n = 1)	Reviewer checklist
Qualitative study (n = 13)	CASP and UKGCSRO

CASP, Critical appraisal skills programme 41 ; SIGN, Scottish intercollegiate guidelines network 42 ; NZGG, New Zealand guidelines group 43 ; UKGCSRO, UK Government chief social researcher's office. 44

Reviewer checklist included: purpose, population, response rate, outcome definition and assessment.

Many different systems exist for grading quantitative evidence and recommendation strength, 45 however, there is no agreed hierarchy of evidence in qualitative research 18 or across all research methods. 13 Recently, a new system of grading quantitative evidence has been proposed by an international group of experts in the field of systematic reviews (the GRADE working group). 46 For the purposes of this review, a modified GRADE approach was adopted for the quantitative evidence synthesis and a similar but separate system was developed for the qualitative research.

Synthesis of quantitative evidence

The basic principles of the GRADE approach were applied to the synthesis of the quantitative evidence. The quantitative studies were tabulated for each important information point and analysed together in an evidence profile (table 1 1). ). An initial level of evidence (high, low or very low) was assigned to the group of quantitative studies reporting findings related to each information point. The initial level of evidence was then increased or decreased after consideration of the following factors: study design, quality, consistency and directness (box 7 7 ). 46 , 49 Data on the size or magnitude of effects or associations were taken into account whenever possible—for example, if a good quality randomised trial of written information provision showed a beneficial effect on outcomes such as knowledge, understanding, acceptability or anxiety, then the kinds of information included in the trial materials were recommended. Data related to population baseline risks and resource utilisation were not available and were omitted from the review evidence profiles.

Studies	Assessment					Summary of findings
Studies	Design	Quality	Consistency * across studies	Directness †	Other factors‡	Overall assessment	Overall recommendation
Gath, 1995	Non‐comparative descriptive	++	No important inconsistency	DirectDirect	None	Very low
Olamijulo, 1997	Non‐comparative descriptive	+	No important inconsistency	DirectDirect
Howells, 1999	Randomised trial	++		Uncertain

Kuehner, 2001	Qualitative	++	No important inconsistency	Direct	Close conformity based on direct evidence	High
Byrom, 2003	Qualitative	++	No important inconsistency	Direct
Neale, 2003	Qualitative	++		Direct

• The quantitative studies indicated that the provision of a colposcopy leaflet or sheet containing information about the possibility of treatment at the first colposcopy clinic visit was acceptable to women. A need for clearer information about the possibility of receiving treatment at the initial visit was identified.

• The qualitative studies reported that women have unanswered questions about whether treatment will be received on the day of the colposcopy appointment.

*Consistency among quantitative studies refers to the similarity of estimates of effect or observations across studies. 46 Consistency among qualitative studies refers to similarities in developed themes and participant experiences across studies.

†Directness refers to the extent to which people, interventions and findings are similar to the NHSCSP population.

‡Other factors include imprecise or sparse data, strong or very strong association, high risk of reporting bias, evidence of a dose‐response gradient, effect of plausible residual confounding and close conformity of findings based on direct evidence (see box 7 7 ).



High = randomised trial	High = checklist quality score “++”
Low = observational study	Low = checklist quality score “+”
Very low = any other evidence	Very low = checklist quality score “−“
The initial level of evidence assigned was based on the lowest hierachical type of evidence (ie, study design) in the group of studies.	The initial level of evidence assigned was based on the lowest checklist quality score of any study in the group.*
*The use of the lowest checklist quality score enabled any uncertainty in the quality of the available evidence (in conduct, reporting or both) to be incorporated in the initial level of evidence assigned.


Quantitative studies	Qualitative studies

Serious (−1) or very serious (−2) limitation to study quality
Important inconsistency (−1)	Important inconsistency (−1)
Some (−1) or major (−2) uncertainty about directness	Some (−1) or major (−2) uncertainty about directness
Imprecise or sparse data (−1)
High probability of reporting bias (−1)

Strong evidence of association (+1)	Close conformity of findings based on two or more studies rated as ++, directly applicable to the target population and with no major threats to validity (+1)
Very strong evidence of association (+2)
Evidence of a dose response gradient (+1)
All plausible confounders would have reduced the effect (+1)

Synthesis of qualitative evidence

Qualitative studies addressing one particular information point were collated and assigned an initial level of evidence (high ++, low + or very low −) on the basis of the lowest checklist quality score obtained for any study in the group ( (table table 1, box 7 7). ). The initial level of evidence was then modified according to the consistency and directness of the evidence across the group of studies (these additional considerations acted cumulatively). As in the GRADE system, four overall qualitative levels of evidence (high, moderate, low and very low) were used during this phase of the evaluation process. Consistency referred to similarities in developed themes and participant experiences across studies whereas directness referred to the similarity between people, interventions and findings compared with the NHSCSP population. It is important to note that individual studies were not directly penalised for reporting contradictory or less direct evidence. Judgements about consistency and directness were made on the basis of the evidence provided by all of the studies, which in turn influenced the guideline recommendation for that specific information point. The existence of contradictory findings across studies was not taken to be evidence of poor quality research instead; uncertainty was reflected in the strength of the given recommendation.

For example, the qualitative studies in table 1 1 all indicated that women have unanswered questions about the possibility of treatment at the initial colposcopy appointment and that the studies were conducted in directly applicable populations. Therefore the initial level of evidence “high” remained unchanged. If, however, one of the three qualitative studies had received a checklist quality score of “+” instead of “++” then the initial level of evidence assigned to the group of studies would have been “low” instead of “high”. In this situation, the initial level of evidence would have been promoted from “low” to “moderate” owing to the direct and consistent findings reported by the two remaining “++” rated studies. In theory, one study could have found that women prefer to receive treatment information in‐person at the colposcopy clinic. At this point, a further decision regarding the importance of the inconsistency between studies would have been necessary. This type of grading system provides increased flexibility for reviewers when making decisions about evidence.

Recommendation system and review findings

Both overall levels of evidence (quantitative and qualitative) were used to determine which information points should be included in the screening letters and leaflets. Two categories of recommendation “definite” and “suggestive” were developed (fig 1 1). ). Items for which the quantitative and/or qualitative evidence was graded as “high” and/or “moderate” were given a “definite” recommendation for inclusion in the screening information. Items where quantitative and qualitative evidence were graded as “low” and/or “very low” were designated as “suggestive”. In this way, a final recommendation for each information point was included in the review without obscuring the contribution of each type of evidence or complicating the grading system.

Although different levels of evidence were often assigned to the quantitative and qualitative research, contradictory findings were not reported for any of the information points considered. If the qualitative research had been omitted from the review, many recommendations would not have been incorporated or achieved a “definite” status. This applied to the recommendations presented in fig 1 1, , table 1 1 and the following:

The use of statements intended to reassure such as “not to worry” or “no big deal” should be avoided.
The term “wart virus” should be not be included.
The staff members participating in colposcopy and treatment appointments should be identified.

The development of methods for the synthesis of diverse data sources in a single review is a complex challenge. 14 , 15 , 34 It has been said that, “ultimately a subjective and pragmatic judgement must made about methodological issues”. 23 Despite the current lack of guidance, we decided that it was still a valid exercise to attempt to pull together updated evidence‐based guidelines for the production of NHSCSP information using both quantitative and qualitative studies. The purpose of this article is to share our experiences with other researchers and contribute to the ongoing discussion about the best way forward.

The GRADE methodology has been developed 46 and refined by its application to existing systematic reviews composed mainly of randomised trials. 49 The reliability and sensibility of the approach is currently being assessed. 49 The attraction of the GRADE system was the ability to modify the level of evidence assigned to a group of studies through consideration of factors other than study design alone. However, even with the improved flexibility of the GRADE system, study design considerations affected the level of evidence assigned to the quantitative research (it was often rated as “low” or “very low”). On the other hand, the qualitative research often obtained evidence level ratings of “high” or “moderate”. This may partly be explained by the checklist quality scores assigned to the qualitative studies and the increase in evidence level allowed by the close conformity of findings between two or more studies. A sensitivity analysis investigating the effect of the grading system on the results of the review was not attempted–further work is required on this subject.

Study quality appraisal is an important unresolved issue in the synthesis of quantitative and qualitative research. 13 Many different quality criteria for the assessment of qualitative studies have been proposed but no common standards have been agreed. 13 , 50 Controversy also exists about the defining characteristics of good quality qualitative research and the prescription of structured quality criteria. 9 , 13 , 33 , 50 In contrast, quality checklists and formal hierarchies of evidence are included in procedural manuals for quantitative reviews of effectiveness studies. 7 , 8

A pragmatic decision was taken to use a quality checklist modified from published sources 41 , 44 to assess the various qualitative studies included in the review. The checklist quality score was used primarily as a means for highlighting the strengths and weaknesses of each study. 9 Although checklists are convenient tools, it was often difficult to determine whether the quality of reporting or the design and execution of the study was being assessed. For some checklist criteria, the information required to make a decision about whether an item was “well covered”, “adequately addressed” or “poorly addressed” was not always available. Also, separate reviewers applied and interpreted checklist criteria in different ways (even with a coded comments system), although agreement was always reached by consensus. The quality of each study was often better captured by the detailed study assessment notes compiled during the appraisal process.

The review questions were best answered by evidence from a range of data sources. The inclusion of the qualitative research, which was often highly relevant and specific to many issues covered by the screening information, enabled the production of a set of guidelines that will directly affect policy in the NHSCSP. The updated guidelines for the content of the NHSCSP leaflets and letters appear to be sensible standards that confirm and strengthen the previous recommendations. 1 , 28

The systematic review and synthesis of diverse research evidence, though rewarding, is challenging and time‐consuming. Several aspects of the review process that involve considerable time investment on the part of researchers include searching, study appraisal and analysis. As current methods are further developed the process should become increasingly streamlined, transparent and explicit. 13 , 26 , 46 In presenting this research we hope to provide a practical example of how numerous data sources might be brought together to inform guidelines related to the contents of patient information materials.

Acknowledgements

This project was supported by the National Health Service Cervical Screening Programme and Cancer Research, UK. The views expressed are those of the authors and not necessarily of the funders.

Abbreviations

NHSCSP - National Health Service Cervical Screening Programme

Competing interests: None.

Open access
Published: 11 August 2009

Methods for the synthesis of qualitative research: a critical review

Elaine Barnett-Page 1 &
James Thomas 1

BMC Medical Research Methodology volume 9 , Article number: 59 ( 2009 ) Cite this article

201k Accesses

1177 Citations

41 Altmetric

Metrics details

In recent years, a growing number of methods for synthesising qualitative research have emerged, particularly in relation to health-related research. There is a need for both researchers and commissioners to be able to distinguish between these methods and to select which method is the most appropriate to their situation.

A number of methodological and conceptual links between these methods were identified and explored, while contrasting epistemological positions explained differences in approaches to issues such as quality assessment and extent of iteration. Methods broadly fall into 'realist' or 'idealist' epistemologies, which partly accounts for these differences.

Methods for qualitative synthesis vary across a range of dimensions. Commissioners of qualitative syntheses might wish to consider the kind of product they want and select their method – or type of method – accordingly.

Peer Review reports

The range of different methods for synthesising qualitative research has been growing over recent years [ 1 , 2 ], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [ 3 ]. While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe how quantitative research is synthesised, far more terms are used to describe the synthesis of qualitative research. This profusion of terms can mask some of the basic similarities in approach that the different methods share, and also lead to some confusion regarding which method is most appropriate in a given situation. This paper does not argue that the various nomenclatures are unnecessary, but rather seeks to draw together and review the full range of methods of synthesis available to assist future reviewers in selecting a method that is fit for their purpose. It also represents an attempt to guide the reader through some of the varied terminology to spring up around qualitative synthesis. Other helpful reviews of synthesis methods have been undertaken in recent years with slightly different foci to this paper. Two recent studies have focused on describing and critiquing methods for the integration of qualitative research with quantitative [ 4 , 5 ] rather than exclusively examining the detail and rationale of methods for the synthesis of qualitative research. Two other significant pieces of work give practical advice for conducting the synthesis of qualitative research, but do not discuss the full range of methods available [ 6 , 7 ]. We begin our Discussion by outlining each method of synthesis in turn, before comparing and contrasting characteristics of these different methods across a range of dimensions. Readers who are more familiar with the synthesis methods described here may prefer to turn straight to the 'dimensions of difference' analysis in the second part of the Discussion.

Overview of synthesis methods

Meta-ethnography.

In their seminal work of 1988, Noblit and Hare proposed meta-ethnography as an alternative to meta-analysis [ 8 ]. They cited Strike and Posner's [ 9 ] definition of synthesis as an activity in which separate parts are brought together to form a 'whole'; this construction of the whole is essentially characterised by some degree of innovation, so that the result is greater than the sum of its parts. They also borrowed from Turner's theory of social explanation [ 10 ], a key tenet of which was building 'comparative understanding' [[ 8 ], p22] rather than aggregating data.

To Noblit and Hare, synthesis provided an answer to the question of 'how to "put together" written interpretive accounts' [[ 8 ], p7], where mere integration would not be appropriate. Noblit and Hare's early work synthesised research from the field of education.

Three different methods of synthesis are used in meta-ethnography. One involves the 'translation' of concepts from individual studies into one another, thereby evolving overarching concepts or metaphors. Noblit and Hare called this process reciprocal translational analysis (RTA). Refutational synthesis involves exploring and explaining contradictions between individual studies. Lines-of-argument (LOA) synthesis involves building up a picture of the whole (i.e. culture, organisation etc) from studies of its parts. The authors conceptualised this latter approach as a type of grounded theorising.

Britten et al [ 11 ] and Campbell et al [ 12 ] have both conducted evaluations of meta-ethnography and claim to have succeeded, by using this method, in producing theories with greater explanatory power than could be achieved in a narrative literature review. While both these evaluations used small numbers of studies, more recently Pound et al [ 13 ] conducted both an RTA and an LOA synthesis using a much larger number of studies (37) on resisting medicines. These studies demonstrate that meta-ethnography has evolved since Noblit and Hare first introduced it. Campbell et al claim to have applied the method successfully to non-ethnographical studies. Based on their reading of Schutz [ 14 ], Britten et al have developed both second and third order constructs in their synthesis (Noblit and Hare briefly allude to the possibility of a 'second level of synthesis' [[ 8 ], p28] but do not demonstrate or further develop the idea).

In a more recent development, Sandelowski & Barroso [ 15 ] write of adapting RTA by using it to ' integrate findings interpretively, as opposed to comparing them interpretively' (p204). The former would involve looking to see whether the same concept, theory etc exists in different studies; the latter would involve the construction of a bigger picture or theory (i.e. LOA synthesis). They also talk about comparing or integrating imported concepts (e.g. from other disciplines) as well as those evolved 'in vivo'.

Grounded theory

Kearney [ 16 ], Eaves [ 17 ] and Finfgeld [ 18 ] have all adapted grounded theory to formulate a method of synthesis. Key methods and assumptions of grounded theory, as originally formulated and subsequently refined by Glaser and Strauss [ 19 ] and Strauss and Corbin [ 20 , 21 ], include: simultaneous phases of data collection and analysis; an inductive approach to analysis, allowing the theory to emerge from the data; the use of the constant comparison method; the use of theoretical sampling to reach theoretical saturation; and the generation of new theory. Eaves cited grounded theorists Charmaz [ 22 ] and Chesler [ 23 ], as well as Strauss and Corbin [ 20 ], as informing her approach to synthesis.

Glaser and Strauss [ 19 ] foresaw a time when a substantive body of grounded research should be pushed towards a higher, more abstract level. As a piece of methodological work, Eaves undertook her own synthesis of the synthesis methods used by these authors to produce her own clear and explicit guide to synthesis in grounded formal theory. Kearney stated that 'grounded formal theory', as she termed this method of synthesis, 'is suited to study of phenomena involving processes of contextualized understanding and action' [[ 24 ], p180] and, as such, is particularly applicable to nurses' research interests.

As Kearney suggested, the examples examined here were largely dominated by research in nursing. Eaves synthesised studies on care-giving in rural African-American families for elderly stroke survivors; Finfgeld on courage among individuals with long-term health problems; Kearney on women's experiences of domestic violence.

Kearney explicitly chose 'grounded formal theory' because it matches 'like' with 'like': that is, it applies the same methods that have been used to generate the original grounded theories included in the synthesis – produced by constant comparison and theoretical sampling – to generate a higher-level grounded theory. The wish to match 'like' with 'like' is also implicit in Eaves' paper. This distinguishes grounded formal theory from more recent applications of meta-ethnography, which have sought to include qualitative research using diverse methodological approaches [ 12 ].

Thematic Synthesis

Thomas and Harden [ 25 ] have developed an approach to synthesis which they term 'thematic synthesis'. This combines and adapts approaches from both meta-ethnography and grounded theory. The method was developed out of a need to conduct reviews that addressed questions relating to intervention need, appropriateness and acceptability – as well as those relating to effectiveness – without compromising on key principles developed in systematic reviews. They applied thematic synthesis in a review of the barriers to, and facilitators of, healthy eating amongst children.

Free codes of findings are organised into 'descriptive' themes, which are then further interpreted to yield 'analytical' themes. This approach shares characteristics with later adaptations of meta-ethnography, in that the analytical themes are comparable to 'third order interpretations' and that the development of descriptive and analytical themes using coding invoke reciprocal 'translation'. It also shares much with grounded theory, in that the approach is inductive and themes are developed using a 'constant comparison' method. A novel aspect of their approach is the use of computer software to code the results of included studies line-by-line, thus borrowing another technique from methods usually used to analyse primary research.

Textual Narrative Synthesis

Textual narrative synthesis is an approach which arranges studies into more homogenous groups. Lucas et al [ 26 ] comment that it has proved useful in synthesising evidence of different types (qualitative, quantitative, economic etc). Typically, study characteristics, context, quality and findings are reported on according to a standard format and similarities and differences are compared across studies. Structured summaries may also be developed, elaborating on and putting into context the extracted data [ 27 ].

Lucas et al [ 26 ] compared thematic synthesis with textual narrative synthesis. They found that 'thematic synthesis holds most potential for hypothesis generation' whereas textual narrative synthesis is more likely to make transparent heterogeneity between studies (as does meta-ethnography, with refutational synthesis) and issues of quality appraisal. This is possibly because textual narrative synthesis makes clearer the context and characteristics of each study, while the thematic approach organises data according to themes. However, Lucas et al found that textual narrative synthesis is 'less good at identifying commonality' (p2); the authors do not make explicit why this should be, although it may be that organising according to themes, as the thematic approach does, is comparatively more successful in revealing commonality.

Paterson et al [ 28 ] have evolved a multi-faceted approach to synthesis, which they call 'meta-study'. The sociologist Zhao [ 29 ], drawing on Ritzer's work [ 30 ], outlined three components of analysis, which they proposed should be undertaken prior to synthesis. These are meta-data-analysis (the analysis of findings), meta-method (the analysis of methods) and meta-theory (the analysis of theory). Collectively, these three elements of analysis, culminating in synthesis, make up the practice of 'meta-study'. Paterson et al pointed out that the different components of analysis may be conducted concurrently.

Paterson et al argued that primary research is a construction; secondary research is therefore a construction of a construction. There is need for an approach that recognises this, and that also recognises research to be a product of its social, historical and ideological context. Such an approach would be useful in accounting for differences in research findings. For Paterson et al, there is no such thing as 'absolute truth'.

Meta-study was developed to study the experiences of adults living with a chronic illness. Meta-data-analysis was conceived of by Paterson et al in similar terms to Noblit and Hare's meta-ethnography (see above), in that it is essentially interpretive and seeks to reveal similarities and discrepancies among accounts of a particular phenomenon. Meta-method involves the examination of the methodologies of the individual studies under review. Part of the process of meta-method is to consider different aspects of methodology such as sampling, data collection, research design etc, similar to procedures others have called 'critical appraisal' (CASP [ 31 ]). However, Paterson et al take their critique to a deeper level by establishing the underlying assumptions of the methodologies used and the relationship between research outcomes and methods used. Meta-theory involves scrutiny of the philosophical and theoretical assumptions of the included research papers; this includes looking at the wider context in which new theory is generated. Paterson et al described meta-synthesis as a process which creates a new interpretation which accounts for the results of all three elements of analysis. The process of synthesis is iterative and reflexive and the authors were unwilling to oversimplify the process by 'codifying' procedures for bringing all three components of analysis together.

Meta-narrative

Greenhalgh et al [ 32 ]'s meta-narrative approach to synthesis arose out of the need to synthesise evidence to inform complex policy-making questions and was assisted by the formation of a multi-disciplinary team. Their approach to review was informed by Thomas Kuhn's The Structure of Scientific Revolutions [ 33 ], in which he proposed that knowledge is produced within particular paradigms which have their own assumptions about theory, about what is a legitimate object of study, about what are legitimate research questions and about what constitutes a finding. Paradigms also tend to develop through time according to a particular set of stages, central to which is the stage of 'normal science', in which the particular standards of the paradigm are largely unchallenged and seen to be self-evident. As Greenhalgh et al pointed out, Kuhn saw paradigms as largely incommensurable: 'that is, an empirical discovery made using one set of concepts, theories, methods and instruments cannot be satisfactorily explained through a different paradigmatic lens' [[ 32 ], p419].

Greenhalgh et al synthesised research from a wide range of disciplines; their research question related to the diffusion of innovations in health service delivery and organisation. They thus identified a need to synthesise findings from research which contains many different theories arising from many different disciplines and study designs.

Based on Kuhn's work, Greenhalgh et al proposed that, across different paradigms, there were multiple – and potentially mutually contradictory – ways of understanding the concept at the heart of their review, namely the diffusion of innovation. Bearing this in mind, the reviewers deliberately chose to select key papers from a number of different research 'paradigms' or 'traditions', both within and beyond healthcare, guided by their multidisciplinary research team. They took as their unit of analysis the 'unfolding "storyline" of a research tradition over time' [[ 32 ], p417) and sought to understand diffusion of innovation as it was conceptualised in each of these traditions. Key features of each tradition were mapped: historical roots, scope, theoretical basis; research questions asked and methods/instruments used; main empirical findings; historical development of the body of knowledge (how have earlier findings led to later findings); and strengths and limitations of the tradition. The results of this exercise led to maps of 13 'meta-narratives' in total, from which seven key dimensions, or themes, were identified and distilled for the synthesis phase of the review.

Critical Interpretive Synthesis

Dixon-Woods et al [ 34 ] developed their own approach to synthesising multi-disciplinary and multi-method evidence, termed 'critical interpretive synthesis', while researching access to healthcare by vulnerable groups. Critical interpretive synthesis is an adaptation of meta-ethnography, as well as borrowing techniques from grounded theory. The authors stated that they needed to adapt traditional meta-ethnographic methods for synthesis, since these had never been applied to quantitative as well as qualitative data, nor had they been applied to a substantial body of data (in this case, 119 papers).

Dixon-Woods et al presented critical interpretive synthesis as an approach to the whole process of review, rather than to just the synthesis component. It involves an iterative approach to refining the research question and searching and selecting from the literature (using theoretical sampling) and defining and applying codes and categories. It also has a particular approach to appraising quality, using relevance – i.e. likely contribution to theory development – rather than methodological characteristics as a means of determining the 'quality' of individual papers [ 35 ]. The authors also stress, as a defining characteristic, critical interpretive synthesis's critical approach to the literature in terms of deconstructing research traditions or theoretical assumptions as a means of contextualising findings.

Dixon-Woods et al rejected reciprocal translational analysis (RTA) as this produced 'only a summary in terms that have already been used in the literature' [[ 34 ], p5], which was seen as less helpful when dealing with a large and diverse body of literature. Instead, Dixon-Woods et al adopted a lines-of-argument (LOA) synthesis, in which – rejecting the difference between first, second and third order constructs – they instead developed 'synthetic constructs' which were then linked with constructs arising directly from the literature.

The influence of grounded theory can be seen in particular in critical interpretive synthesis's inductive approach to formulating the review question and to developing categories and concepts, rejecting a 'stage' approach to systematic reviewing, and in selecting papers using theoretical sampling. Dixon-Woods et al also claim that critical interpretive synthesis is distinct in its 'explicit orientation towards theory generation' [[ 34 ], p9].

Ecological Triangulation

Jim Banning is the author of 'ecological triangulation' or 'ecological sentence synthesis', applying this method to the evidence for what works for youth with disabilities. He borrows from Webb et al [ 36 ] and Denzin [ 37 ] the concept of triangulation, in which phenomena are studied from a variety of vantage points. His rationale is that building an 'evidence base' of effectiveness requires the synthesis of cumulative, multi-faceted evidence in order to find out 'what intervention works for what kind of outcomes for what kind of persons under what kind of conditions' [[ 38 ], p1].

Ecological triangulation unpicks the mutually interdependent relationships between behaviour, persons and environments. The method requires that, for data extraction and synthesis, 'ecological sentences' are formulated following the pattern: 'With this intervention, these outcomes occur with these population foci and within these grades (ages), with these genders ... and these ethnicities in these settings' [[ 39 ], p1].

Framework Synthesis

Brunton et al [ 40 ] and Oliver et al [ 41 ] have applied a 'framework synthesis' approach in their reviews. Framework synthesis is based on framework analysis, which was outlined by Pope, Ziebland and Mays [ 42 ], and draws upon the work of Ritchie and Spencer [ 43 ] and Miles and Huberman [ 44 ]. Its rationale is that qualitative research produces large amounts of textual data in the form of transcripts, observational fieldnotes etc. The sheer wealth of information poses a challenge for rigorous analysis. Framework synthesis offers a highly structured approach to organising and analysing data (e.g. indexing using numerical codes, rearranging data into charts etc).

Brunton et al applied the approach to a review of children's, young people's and parents' views of walking and cycling; Oliver et al to an analysis of public involvement in health services research. Framework synthesis is distinct from the other methods outlined here in that it utilises an a priori 'framework' – informed by background material and team discussions – to extract and synthesise findings. As such, it is largely a deductive approach although, in addition to topics identified by the framework, new topics may be developed and incorporated as they emerge from the data. The synthetic product can be expressed in the form of a chart for each key dimension identified, which may be used to map the nature and range of the concept under study and find associations between themes and exceptions to these [ 40 ].

'Fledgling' approaches

There are three other approaches to synthesis which have not yet been widely used. One is an approach using content analysis [ 45 , 46 ] in which text is condensed into fewer content-related categories. Another is 'meta-interpretation' [ 47 ], featuring the following: an ideographic rather than pre-determined approach to the development of exclusion criteria; a focus on meaning in context; interpretations as raw data for synthesis (although this feature doesn't distinguish it from other synthesis methods); an iterative approach to the theoretical sampling of studies for synthesis; and a transparent audit trail demonstrating the trustworthiness of the synthesis.

In addition to the synthesis methods discussed above, Sandelowski and Barroso propose a method they call 'qualitative metasummary' [ 15 ]. It is mentioned here as a new and original approach to handling a collection of qualitative studies but is qualitatively different to the other methods described here since it is aggregative; that is, findings are accumulated and summarised rather than 'transformed'. Metasummary is a way of producing a 'map' of the contents of qualitative studies and – according to Sandelowski and Barroso – 'reflect [s] a quantitative logic' [[ 15 ], p151]. The frequency of each finding is determined and the higher the frequency of a particular finding, the greater its validity. The authors even discuss the calculation of 'effect sizes' for qualitative findings. Qualitative metasummaries can be undertaken as an end in themselves or may serve as a basis for a further synthesis.

Dimensions of difference

Having outlined the range of methods identified, we now turn to an examination of how they compare with one another. It is clear that they have come from many different contexts and have different approaches to understanding knowledge, but what do these differences mean in practice? Our framework for this analysis is shown in Additional file 1 : dimensions of difference [ 48 ]. We have examined the epistemology of each of the methods and found that, to some extent, this explains the need for different methods and their various approaches to synthesis.

Epistemology

The first dimension that we will consider is that of the researchers' epistemological assumptions. Spencer et al [ 49 ] outline a range of epistemological positions, which might be organised into a spectrum as follows:

Subjective idealism : there is no shared reality independent of multiple alternative human constructions

Objective idealism : there is a world of collectively shared understandings

Critical realism : knowledge of reality is mediated by our perceptions and beliefs

Scientific realism : it is possible for knowledge to approximate closely an external reality

Naïve realism : reality exists independently of human constructions and can be known directly [ 49 , 45 , 46 ].

Thus, at one end of the spectrum we have a highly constructivist view of knowledge and, at the other, an unproblematized 'direct window onto the world' view.

Nearly all of positions along this spectrum are represented in the range of methodological approaches to synthesis covered in this paper. The originators of meta-narrative synthesis, critical interpretive synthesis and meta-study all articulate what might be termed a 'subjective idealist' approach to knowledge. Paterson et al [ 28 ] state that meta-study shies away from creating 'grand theories' within the health or social sciences and assume that no single objective reality will be found. Primary studies, they argue, are themselves constructions; meta-synthesis, then, 'deals with constructions of constructions' (p7). Greenhalgh et al [ 32 ] also view knowledge as a product of its disciplinary paradigm and use this to explain conflicting findings: again, the authors neither seek, nor expect to find, one final, non-contestable answer to their research question. Critical interpretive synthesis is similar in seeking to place literature within its context, to question its assumptions and to produce a theoretical model of a phenomenon which – because highly interpretive – may not be reproducible by different research teams at alternative points in time [[ 34 ], p11].

Methods used to synthesise grounded theory studies in order to produce a higher level of grounded theory [ 24 ] appear to be informed by 'objective idealism', as does meta-ethnography. Kearney argues for the near-universal applicability of a 'ready-to-wear' theory across contexts and populations. This approach is clearly distinct from one which recognises multiple realities. The emphasis is on examining commonalities amongst, rather than discrepancies between, accounts. This emphasis is similarly apparent in most meta-ethnographies, which are conducted either according to Noblit and Hare's 'reciprocal translational analysis' technique or to their 'lines-of-argument' technique and which seek to provide a 'whole' which has a greater explanatory power. Although Noblit and Hare also propose 'refutational synthesis', in which contradictory findings might be explored, there are few examples of this having been undertaken in practice, and the aim of the method appears to be to explain and explore differences due to context, rather than multiple realities.

Despite an assumption of a reality which is perhaps less contestable than those of meta-narrative synthesis, critical interpretive synthesis and meta-study, both grounded formal theory and meta-ethnography place a great deal of emphasis on the interpretive nature of their methods. This still supposes a degree of constructivism. Although less explicit about how their methods are informed, it seems that both thematic synthesis and framework synthesis – while also involving some interpretation of data – share an even less problematized view of reality and a greater assumption that their synthetic products are reproducible and correspond to a shared reality. This is also implicit in the fact that such products are designed directly to inform policy and practice, a characteristic shared by ecological triangulation. Notably, ecological triangulation, according to Banning, can be either realist or idealist. Banning argues that the interpretation of triangulation can either be one in which multiple viewpoints converge on a point to produce confirming evidence (i.e. one definitive answer to the research question) or an idealist one, in which the complexity of multiple viewpoints is represented. Thus, although ecological triangulation views reality as complex, the approach assumes that it can be approximately knowable (at least when the realist view of ecological triangulation is adopted) and that interventions can and should be modelled according to the products of its syntheses.

While pigeonholing different methods into specific epistemological positions is a problematic process, we do suggest that the contrasting epistemologies of different researchers is one way of explaining why we have – and need – different methods for synthesis.

Variation in terms of the extent of iteration during the review process is another key dimension. All synthesis methods include some iteration but the degree varies. Meta-ethnography, grounded theory and thematic synthesis all include iteration at the synthesis stage; both framework synthesis and critical interpretive synthesis involve iterative literature searching – in the case of critical interpretive synthesis, it is not clear whether iteration occurs during the rest of the review process. Meta-narrative also involves iteration at every stage. Banning does not mention iteration in outlining ecological triangulation and neither do Lucas or Thomas and Harden for thematic narrative synthesis.

It seems that the more idealist the approach, the greater the extent of iteration. This might be because a large degree of iteration does not sit well with a more 'positivist' ideal of procedural objectivity; in particular, the notion that the robustness of the synthetic product depends in part on the reviewers stating up front in a protocol their searching strategies, inclusion/exclusion criteria etc, and being seen not to alter these at a later stage.

Quality assessment

Another dimension along which we can look at different synthesis methods is that of quality assessment. When the approaches to the assessment of the quality of studies retrieved for review are examined, there is again a wide methodological variation. It might be expected that the further towards the 'realism' end of the epistemological spectrum a method of synthesis falls, the greater the emphasis on quality assessment. In fact, this is only partially the case.

Framework synthesis, thematic narrative synthesis and thematic synthesis – methods which might be classified as sharing a 'critical realist' approach – all have highly specified approaches to quality assessment. The review in which framework synthesis was developed applied ten quality criteria: two on quality and reporting of sampling methods, four to the quality of the description of the sample in the study, two to the reliability and validity of the tools used to collect data and one on whether studies used appropriate methods for helping people to express their views. Studies which did not meet a certain number of quality criteria were excluded from contributing to findings. Similarly, in the example review for thematic synthesis, 12 criteria were applied: five related to reporting aims, context, rationale, methods and findings; four relating to reliability and validity; and three relating to the appropriateness of methods for ensuring that findings were rooted in participants' own perspectives. Studies which were deemed to have significant flaws were excluded and sensitivity analyses were used to assess the possible impact of study quality on the review's findings. Thomas and Harden's use of thematic narrative synthesis similarly applied quality criteria and developed criteria additional to those they found in the literature on quality assessment, relating to the extent to which people's views and perspectives had been privileged by researchers. It is worth noting not only that these methods apply quality criteria but that they are explicit about what they are: assessing quality is a key component in the review process for both of these methods. Likewise, Banning – the originator of ecological triangulation – sees quality assessment as important and adapts the Design and Implementation Assessment Device (DIAD) Version 0.3 (a quality assessment tool for quantitative research) for use when appraising qualitative studies [ 50 ]. Again, Banning writes of excluding studies deemed to be of poor quality.

Greenhalgh et al's meta-narrative review [ 32 ] modified a range of existing quality assessment tools to evaluate studies according to validity and robustness of methods; sample size and power; and validity of conclusions. The authors imply, but are not explicit, that this process formed the basis for the exclusion of some studies. Although not quite so clear about quality assessment methods as framework and thematic synthesis, it might be argued that meta-narrative synthesis shows a greater commitment to the concept that research can and should be assessed for quality than either meta-ethnography or grounded formal theory. The originators of meta-ethnography, Noblit and Hare [ 8 ], originally discussed quality in terms of quality of metaphor, while more recent use of this method has used amended versions of CASP (the Critical Appraisal Skills Programme tool, [ 31 ]), yet has only referred to studies being excluded on the basis of lack of relevance or because they weren't 'qualitative' studies [ 8 ]. In grounded theory, quality assessment is only discussed in terms of a 'personal note' being made on the context, quality and usefulness of each study. However, contrary to expectation, meta-narrative synthesis lies at the extreme end of the idealism/realism spectrum – as a subjective idealist approach – while meta-ethnography and grounded theory are classified as objective idealist approaches.

Finally, meta-study and critical interpretive synthesis – two more subjective idealist approaches – look to the content and utility of findings rather than methodology in order to establish quality. While earlier forms of meta-study included only studies which demonstrated 'epistemological soundness', in its most recent form [ 51 ] this method has sought to include all relevant studies, excluding only those deemed not to be 'qualitative' research. Critical interpretive synthesis also conforms to what we might expect of its approach to quality assessment: quality of research is judged as the extent to which it informs theory. The threshold of inclusion is informed by expertise and instinct rather than being articulated a priori.

In terms of quality assessment, it might be important to consider the academic context in which these various methods of synthesis developed. The reason why thematic synthesis, framework synthesis and ecological triangulation have such highly specified approaches to quality assessment may be that each of these was developed for a particular task, i.e. to conduct a multi-method review in which randomised controlled trials (RCTs) were included. The concept of quality assessment in relation to RCTs is much less contested and there is general agreement on criteria against which quality should be judged.

Problematizing the literature

Critical interpretive synthesis, the meta-narrative approach and the meta-theory element of meta-study all share some common ground in that their review and synthesis processes include examining all aspects of the context in which knowledge is produced. In conducting a review on access to healthcare by vulnerable groups, critical interpretive synthesis sought to question 'the ways in which the literature had constructed the problematics of access, the nature of the assumptions on which it drew, and what has influenced its choice of proposed solutions' [[ 34 ], p6]. Although not claiming to have been directly influenced by Greenhalgh et al's meta-narrative approach, Dixon-Woods et al do cite it as sharing similar characteristics in the sense that it critiques the literature it reviews.

Meta-study uses meta-theory to describe and deconstruct the theories that shape a body of research and to assess its quality. One aspect of this process is to examine the historical evolution of each theory and to put it in its socio-political context, which invites direct comparison with meta-narrative synthesis. Greenhalgh et al put a similar emphasis on placing research findings within their social and historical context, often as a means of seeking to explain heterogeneity of findings. In addition, meta-narrative shares with critical interpretive synthesis an iterative approach to searching and selecting from the literature.

Framework synthesis, thematic synthesis, textual narrative synthesis, meta-ethnography and grounded theory do not share the same approach to problematizing the literature as critical interpretive synthesis, meta-study and meta-narrative. In part, this may be explained by the extent to which studies included in the synthesis represented a broad range of approaches or methodologies. This, in turn, may reflect the broadness of the review question and the extent to which the concepts contained within the question are pre-defined within the literature. In the case of both the critical interpretive synthesis and meta-narrative reviews, terminology was elastic and/or the question formed iteratively. Similarly, both reviews placed great emphasis on employing multi-disciplinary research teams. Approaches which do not critique the literature in the same way tend to have more narrowly-focused questions. They also tend to include a more limited range of studies: grounded theory synthesis includes grounded theory studies, meta-ethnography (in its original form, as applied by Noblit and Hare) ethnographies. The thematic synthesis incorporated studies based on only a narrow range of qualitative methodologies (interviews and focus groups) which were informed by a similarly narrow range of epistemological assumptions. It may be that the authors of such syntheses saw no need for including such a critique in their review process.

Similarities and differences between primary studies

Most methods of synthesis are applicable to heterogeneous data (i.e. studies which use contrasting methodologies) apart from early meta-ethnography and synthesis informed by grounded theory. All methods of synthesis state that, at some level, studies are compared; many are not so explicit about how this is done, though some are. Meta-ethnography is one of the most explicit: it describes the act of 'translation' where terms and concepts which have resonance with one another are subsumed into 'higher order constructs'. Grounded theory, as represented by Eaves [ 17 ], is undertaken according to a long list of steps and sub-steps, includes the production of generalizations about concepts/categories, which comes from classifying these categories. In meta-narrative synthesis, comparable studies are grouped together at the appraisal phase of review.

Perhaps more interesting are the ways in which differences between studies are explored. Those methods with a greater emphasis on critical appraisal may tend (although this is not always made explicit) to use differences in method to explain differences in finding. Meta-ethnography proposes 'refutational synthesis' to explain differences, although there are few examples of this in the literature. Some synthesis methods – for example, thematic synthesis – look at other characteristics of the studies under review, whether types of participants and their context vary, and whether this can explain differences in perspective.

All of these methods, then, look within the studies to explain differences. Other methods look beyond the study itself to the context in which it was produced. Critical interpretive synthesis and meta-study look at differences in theory or in socio-economic context. Critical interpretive synthesis, like meta-narrative, also explores epistemological orientation. Meta-narrative is unique in concerning itself with disciplinary paradigm (i.e. the story of the discipline as it progresses). It is also distinctive in that it treats conflicting findings as 'higher order data' [[ 32 ], p420], so that the main emphasis of the synthesis appears to be on examining and explaining contradictions in the literature.

Going 'beyond' the primary studies

Synthesis is sometimes defined as a process resulting in a product, a 'whole', which is more than the sum of its parts. However, the methods reviewed here vary in the extent to which they attempt to 'go beyond' the primary studies and transform the data. Some methods – textual narrative synthesis, ecological triangulation and framework synthesis – focus on describing and summarising their primary data (often in a highly structured and detailed way) and translating the studies into one another. Others – meta-ethnography, grounded theory, thematic synthesis, meta-study, meta-narrative and critical interpretive synthesis – seek to push beyond the original data to a fresh interpretation of the phenomena under review. A key feature of thematic synthesis is its clear differentiation between these two stages.

Different methods have different mechanisms for going beyond the primary studies, although some are more explicit than others about what these entail. Meta-ethnography proposes a 'Line of Argument' (LOA) synthesis in which an interpretation is constructed to both link and explain a set of parts. Critical interpretive synthesis based its synthesis methods on those of meta-ethnography, developing an LOA using what the authors term 'synthetic constructs' (akin to 'third order constructs' in meta-ethnography) to create a 'synthesising argument'. Dixon-Woods et al claim that this is an advance on Britten et al's methods, in that they reject the difference between first, second and third order constructs.

Meta-narrative, as outlined above, focuses on conflicting findings and constructs theories to explain these in terms of differing paradigms. Meta study derives questions from each of its three components to which it subjects the dataset and inductively generates a number of theoretical claims in relation to it. According to Eaves' model of grounded theory [ 17 ], mini-theories are integrated to produce an explanatory framework. In ecological triangulation, the 'axial' codes – or second level codes evolved from the initial deductive open codes – are used to produce Banning's 'ecological sentence' [ 39 ].

The synthetic product

In overviewing and comparing different qualitative synthesis methods, the ultimate question relates to the utility of the synthetic product: what is it for? It is clear that some methods of synthesis – namely, thematic synthesis, textual narrative synthesis, framework synthesis and ecological triangulation – view themselves as producing an output that is directly applicable to policy makers and designers of interventions. The example of framework synthesis examined here (on children's, young people's and parents' views of walking and cycling) involved policy makers and practitioners in directing the focus of the synthesis and used the themes derived from the synthesis to infer what kind of interventions might be most effective in encouraging walking and cycling. Likewise, the products of the thematic synthesis took the form of practical recommendations for interventions (e.g. 'do not promote fruit and vegetables in the same way in the same intervention'). The extent to which policy makers and practitioners are involved in informing either synthesis or recommendation is less clear from the documents published on ecological triangulation, but the aim certainly is to directly inform practice.

The outputs of synthesis methods which have a more constructivist orientation – meta-study, meta-narrative, meta-ethnography, grounded theory, critical interpretive synthesis – tend to look rather different. They are generally more complex and conceptual, sometimes operating on the symbolic or metaphorical level, and requiring a further process of interpretation by policy makers and practitioners in order for them to inform practice. This is not to say, however, that they are not useful for practice, more that they are doing different work. However, it may be that, in the absence of further interpretation, they are more useful for informing other researchers and theoreticians.

Looking across dimensions

After examining the dimensions of difference of our included methods, what picture ultimately emerges? It seems clear that, while similar in some respects, there are genuine differences in approach to the synthesis of what is essentially textual data. To some extent, these differences can be explained by the epistemological assumptions that underpin each method. Our methods split into two broad camps: the idealist and the realist (see Table 1 for a summary). Idealist approaches generally tend to have a more iterative approach to searching (and the review process), have less a priori quality assessment procedures and are more inclined to problematize the literature. Realist approaches are characterised by a more linear approach to searching and review, have clearer and more well-developed approaches to quality assessment, and do not problematize the literature.

Mapping the relationships between methods

What is interesting is the relationship between these methods of synthesis, the conceptual links between them, and the extent to which the originators cite – or, in some cases, don't cite – one another. Some methods directly build on others – framework synthesis builds on framework analysis, for example, while grounded theory and constant comparative analysis build on grounded theory. Others further develop existing methods – meta-study, critical interpretive synthesis and meta-narrative all adapt aspects of meta-ethnography, while also importing concepts from other theorists (critical interpretive synthesis also adapts grounded theory techniques).

Some methods share a clear conceptual link, without directly citing one another: for example, the analytical themes developed during thematic synthesis are comparable to the third order interpretations of meta-ethnography. The meta-theory aspect of meta-study is echoed in both meta-narrative synthesis and critical interpretive synthesis (see 'Problematizing the literature, above); however, the originators of critical interpretive synthesis only refer to the originators of meta-study in relation to their use of sampling techniques.

While methods for qualitative synthesis have many similarities, there are clear differences in approach between them, many of which can be explained by taking account of a given method's epistemology.

However, within the two broad idealist/realist categories, any differences between methods in terms of outputs appear to be small.

Since many systematic reviews are designed to inform policy and practice, it is important to select a method – or type of method – that will produce the kind of conclusions needed. However, it is acknowledged that this is not always simple or even possible to achieve in practice.

The approaches that result in more easily translatable messages for policy-makers and practitioners may appear to be more attractive than the others; but we do need to take account lessons from the more idealist end of the spectrum, that some perspectives are not universal.

Dixon-Woods M, Agarwhal S, Jones D, Young B, Sutton A: Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Pol. 2005, 10 (1): 45-53b. 10.1258/1355819052801804.

Article Google Scholar

Barbour RS, Barbour M: Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract. 2003, 9 (2): 179-186. 10.1046/j.1365-2753.2003.00371.x.

Article PubMed Google Scholar

Mays N, Pope C, Popay J: Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Pol. 2005, 10 (Suppl 1): 6-20. 10.1258/1355819054308576.

Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Shaw RL, Smith J, Sutton A, Young B: How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006, 6: 27-44. 10.1177/1468794106058867.

Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. 2007, Maidenhead: Open University Press

Google Scholar

Thorne S, Jenson L, Kearney MH, Noblit G, Sandelowski M: Qualitative metasynthesis: reflections on methodological orientation and ideological agenda. Qual Health Res. 2004, 14: 1342-1365. 10.1177/1049732304269888.

Centre for Reviews and Dissemination: Systematic Reviews. CRD's Guidance for Undertaking Reviews in Health Care. 2008, York: CRD

Noblit GW, Hare RD: Meta-Ethnography: Synthesizing Qualitative Studies. 1988, London: Sage

Book Google Scholar

Strike K, Posner G: Types of synthesis and their criteria. Knowledge Structure and Use. Edited by: Ward S, Reed L. 1983, Philadelphia: Temple University Press

Turner S: Sociological Explanation as Translation. 1980, New York: Cambridge University Press

Britten N, Campbell R, Pope C, Donovan J, Morgan M, Pill R: Using meta-ethnography to synthesis qualitative research: a worked example. J Health Serv Res. 2002, 7: 209-15. 10.1258/135581902320432732.

Campbell R, Pound P, Pope C, Britten N, Pill R, Morgan M, Donovan J: Evaluating meta-ethnography: a synthesis of qualitative research on lay experiences of diabetes and diabetes care. Soc Sci Med. 2003, 65: 671-84. 10.1016/S0277-9536(02)00064-3.

Pound P, Britten N, Morgan M, Yardley L, Pope C, Daker-White G, Campbell R: Resisting medicines: a synthesis of qualitative studies of medicine taking. Soc Sci Med. 2005, 61: 133-155. 10.1016/j.socscimed.2004.11.063.

Schutz A: Collected Paper. 1962, The Hague: Martinus Nijhoff, 1:

Sandelowski M, Barroso J: Handbook for Synthesizing Qualitative Research. 2007, New York: Springer Publishing Company

Kearney MH: Enduring love: a grounded formal theory of women's experience of domestic violence. Research Nurs Health. 2001, 24: 270-82. 10.1002/nur.1029.

Article CAS Google Scholar

Eaves YD: A synthesis technique for grounded theory data analysis. J Adv Nurs. 2001, 35: 654-63. 10.1046/j.1365-2648.2001.01897.x.

Article CAS PubMed Google Scholar

Finfgeld D: Courage as a process of pushing beyond the struggle. Qual Health Res. 1999, 9: 803-814. 10.1177/104973299129122298.

Glaser BG, Strauss AL: The Discovery of Grounded Theory: Strategies for Qualitative Research. 1967, New York: Aldine De Gruyter

Strauss AL, Corbin J: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 1990, Newbury Park, CA: Sage

Strauss AL, Corbin J: Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 1998, Thousand Oaks, CA: Sage

Charmaz K: The grounded theory method: an explication and interpretation. Contemporary Field Research: A Collection of Readings. Edited by: Emerson RM. 1983, Waveland Press: Prospect Heights, IL, 109-126.

Chesler MA: Professionals' Views of the Dangers of Self-Help Groups: Explicating a Grounded Theoretical Approach. 1987, [Michigan]: Department of Sociology, University of Michigan, Ann Arbour Centre for Research on Social Organisation, Working Paper Series

Kearney MH: Ready-to-wear: discovering grounded formal theory. Res Nurs Health. 1988, 21: 179-186. 10.1002/(SICI)1098-240X(199804)21:2<179::AID-NUR8>3.0.CO;2-G.

Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth. 2008, 8: 45-10.1186/1471-2288-8-45.

Lucas PJ, Arai L, Baird , Law C, Roberts HM: Worked examples of alternative methods for the synthesis of qualitative and quantitative research in systematic reviews. BMC Med Res Meth. 2007, 7 (4):

Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people's views: an example from public health research. J Epidemiol Community H. 2004, 58: 794-800. 10.1136/jech.2003.014829.

Paterson BL, Thorne SE, Canam C, Jillings C: Meta-Study of Qualitative Health Research. A Practical Guide to Meta-Analysis and Meta-Synthesis. 2001, Thousand Oaks, CA: Sage Publications

Zhao S: Metatheory, metamethod, meta-data-analysis: what, why and how?. Sociol Perspect. 1991, 34: 377-390.

Ritzer G: Metatheorizing in Sociology. 1991, Lexington, MA: Lexington Books

CASP (Critical Appraisal Skills Programme). date unknown, [ http://www.phru.nhs.uk/Pages/PHD/CASP.htm ]

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R: Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005, 61: 417-30. 10.1016/j.socscimed.2004.12.001.

Kuhn TS: The Structure of Scientific Revolutions. 1962, Chicago: University of Chicago Press

Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Meth. 2006, 6 (35):

Gough D: Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Applied and Practice-based Research. Edited by: Furlong J, Oancea A. 2007, Special Edition of Research Papers in Education, 22 (2): 213-228.

Webb EJ, Campbell DT, Schwartz RD, Sechrest L: Unobtrusive Measures. 1966, Chicago: Rand McNally

Denzin NK: The Research Act: a Theoretical Introduction to Sociological Methods. 1978, New York: McGraw-Hill

Banning J: Ecological Triangulation. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Triangualtion.pdf ]

Banning J: Ecological Sentence Synthesis. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Ecological%20Sentence%20Synthesis.pdf ]

Brunton G, Oliver S, Oliver K, Lorenc T: A Synthesis of Research Addressing Children's, Young People's and Parents' Views of Walking and Cycling for Transport. 2006, London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London

Oliver S, Rees R, Clarke-Jones L, Milne R, Oakley A, Gabbay J, Stein K, Buchanan P, Gyte G: A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect. 2008, 11: 72-84. 10.1111/j.1369-7625.2007.00476.x.

Pope C, Ziebland S, Mays N: Qualitative research in health care: analysing qualitative data. BMJ. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.

Article CAS PubMed PubMed Central Google Scholar

Ritchie J, Spencer L: Qualitative data analysis for applied policy research. Analysing Qualitative Data. Edited by: Bryman A, Burgess R. 1993, London: Routledge, 173-194.

Miles M, Huberman A: Qualitative Data Analysis. 1984, London: Sage

Evans D, Fitzgerald M: Reasons for physically restraining patients and residents: a systematic review and content analysis. Int J Nurs Stud. 2002, 39: 739-743. 10.1016/S0020-7489(02)00015-9.

Suikkala A, Leino-Kilpi H: Nursing student-patient relationships: a review of the literature from 1984–1998. J Adv Nurs. 2000, 33: 42-50. 10.1046/j.1365-2648.2001.01636.x.

Weed M: 'Meta-interpretation': a method for the interpretive synthesis of qualitative research. Forum: Qual Soc Res. 2005, 6: Art 37-

Gough D, Thomas J: Dimensions of difference in systematic reviews. [ http://www.ncrm.ac.uk/RMF2008/festival/programme/sys1 ]

Spencer L, Ritchie J, Lewis J, Dillon L: Quality in Qualitative Evaluation: a Framework for Assessing Research Evidence. 2003, London: Government Chief Social Researcher's Office

Banning J: Design and Implementation Assessment Device (DIAD) Version 0.3: A response from a qualitative perspective. [ http://mycahs.colostate.edu/James.H.Banning/PDFs/Design%20and%20Implementation%20Assessment%20Device.pdf ]

Paterson BL: Coming out as ill: understanding self-disclosure in chronic illness from a meta-synthesis of qualitative research. Reviewing Research Evidence for Nursing Practice. Edited by: Webb C, Roe B. 2007, [Oxford]: Blackwell Publishing Ltd, 73-83.

Chapter Google Scholar

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1471-2288/9/59/prepub

Download references

Acknowledgements

The authors would like to acknowledge the helpful contributions of the following in commenting on earlier drafts of this paper: David Gough, Sandy Oliver, Angela Harden, Mary Dixon-Woods, Trisha Greenhalgh and Barbara L. Paterson. We would also like to thank the peer reviewers: Helen J Smith, Rosaline Barbour and Mark Rodgers for their helpful reviews. The methodological development was supported by the Department of Health (England) and the ESRC through the Methods for Research Synthesis Node of the National Centre for Research Methods (NCRM). An earlier draft of this paper currently appears as a working paper on the National Centre for Research Methods' website http://www.ncrm.ac.uk/ .

Author information

Authors and affiliations.

Social Science Research Unit, Evidence for Policy and Practice Information and Co-ordinating (EPPI-) Centre, 18 Woburn Square, London, WC1H 0NS, UK

Elaine Barnett-Page & James Thomas

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elaine Barnett-Page .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

Both authors made substantial contributions, with EBP taking a lead on writing and JT on the analytical framework. Both authors read and approved the final manuscript.

Electronic supplementary material

12874_2009_375_moesm1_esm.doc.

Additional file 1: Dimensions of difference. Ranging from subjective idealism through objective idealism and critical realism to scientific realism to naïve realism (DOC 46 KB)

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Barnett-Page, E., Thomas, J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9 , 59 (2009). https://doi.org/10.1186/1471-2288-9-59

Download citation

Received : 09 March 2009

Accepted : 11 August 2009

Published : 11 August 2009

DOI : https://doi.org/10.1186/1471-2288-9-59

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Narrative Synthesis
Theoretical Sampling
Qualitative Synthesis
Order Construct

BMC Medical Research Methodology

ISSN: 1471-2288

General enquiries: [email protected]

Information

Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

Active Journals
Find a Journal
Proceedings Series
For Authors
For Reviewers
For Editors
For Librarians
For Publishers
For Societies
For Conference Organizers
Open Access Policy
Institutional Open Access Program
Special Issues Guidelines
Editorial Process
Research and Publication Ethics
Article Processing Charges
Testimonials
Preprints.org
SciProfiles
Encyclopedia

Article Menu

Subscribe SciFeed
Google Scholar
on Google Scholar
Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A systematic review of synthetic data generation techniques using generative ai.

1. Introduction

2. related work, 3. research methodology, 3.1. research questions.

RQ1. What are the techniques and methods used for synthetic data generation?
RQ2. What frameworks are available for synthetic data generation for specific use cases?
RQ3. What are the limitations of using these techniques?
RQ4. What are the areas of future research?

3.2. Information Source

3.3. eligibility criteria, 3.3.1. inclusion criteria, 3.3.2. exclusion criteria, 3.4. data collection procedure, 3.5. results.

GANs: This category contains all the studies that reveal new GAN models and their implementations. Various modified versions and frameworks of GANs are included in this category.
VAEs: This category contains all studies that introduce the foundations of VAEs and their modified versions.
LLMs. This category comprises various LLM foundational studies on GPT-1, BERT, and their modified versions.
General studies: This category includes all the studies related to the need for differential privacy, biases in machine learning, problems with synthetic data, and the style and format for writing systematic reviews. This category also includes all the web articles used to support the literature.
Surveys and reviews: This category includes all the literature reviews that attempted to explore synthetic data generation, but they did not propose any new model. These studies discuss the applications, challenges, and future research prospects for synthetic data generation.

Click here to enlarge figure

4. Synthetic Data Generation

4.1. programmatic approach, 4.1.1. rule-based systems, 4.1.2. random sampling, 4.2. statistical approach, 4.3. differential privacy approach, 4.4. machine learning approach, 4.4.1. generative adversarial networks (gans), 4.4.2. variational autoencoders (vaes), 4.4.3. large language models (llms), 4.4.4. more studies and frameworks, 5. findings, 5.1. evaluation of models, 5.1.1. evaluation of the specific gan models.

Conditional GANs: cGANs are highly effective in generating data with specific characteristics by conditioning the output on class labels or other variables. This makes them particularly powerful for tasks, such as generating labeled images, text, or data with specific features. cGANs excel in applications requiring control over the generated content, such as medical imaging, where they enhance the diagnostic quality and image segmentation.
Continuous conditional GANs: CcGANs extend cGANs by allowing continuous scalar labels as inputs, which is crucial for tasks, such as regression-based image generation. They outperform traditional cGANs in generating high-quality, diverse images, especially in settings where precise control over continuous variables is required. They are useful in fields such as autonomous driving for generating images with fine control over conditions such as steering angles and in medical imaging for tasks such as generating synthetic cell count images.
Tabular GAN: TGANs are tailored to generate synthetic tabular data with discrete and continuous variables. They effectively maintains inter-column correlations, making them suitable for data science applications in which data relationships are critical. They are primarily used in fields requiring synthetic data generation for tabular datasets, such as finance, healthcare, and relational database modeling.
Conditional tabular GANs: CTGANs improve TGANs by handling complex column distributions and by addressing the challenges of imbalanced training data. Their mode-specific normalization improves their ability to generating accurate tabular data, particularly in privacy-preserving contexts. CTGANs are ideal for generating synthetic data in privacy-sensitive industries, such as healthcare and financial services, where accurate and secure data are critical.
Deep convolutional GANs: DCGANs enhance the basic GAN architecture by incorporating convolutional layers, improving the generation of high-quality images, and enabling improved feature extraction for unsupervised learning. They are widely used in tasks such as image generation, representation learning, and enhancement of the quality of synthesized images for various applications.
CycleGAN: CycleGAN excels in unpaired image-to-image translation, using cycle consistency loss to ensure high-quality transformations between domains without the need for paired training data. It is particularly effective for tasks such as style transfer, object transfiguration, and photo enhancement, but struggles with tasks that require significant geometric transformations.

5.1.2. Evaluation of Specific VAE Models

Auto-Encoding Variational Bayes: AEVB efficiently handles continuous latent variables using the stochastic gradient variational Bayes (SGVB) estimator. This significantly improves variational inference, making it a powerful tool for analyzing large, complex datasets with continuous data. Widely used in probabilistic modeling, AEVB is applicable to tasks such as unsupervised learning, generative modeling, and anomaly detection.
Conditional variational autoencoders: CVAEs are highly effective in structured output prediction by conditioning the generation process on specific input variables, leading to improved resilience in complex prediction tasks. CVAEs outperform deterministic models in tasks requiring probabilistic inference, such as image segmentation and semantic labeling.
Adversarial autoencoders: AAEs combines the strengths of GANs and VAEs to align the latent space with an arbitrary prior, enhancing the generation of meaningful samples across the entire latent space. AAEs are particularly useful in generative modeling and semi-supervised classification tasks, making them effective in domains such as image synthesis and representation learning.
Ladder Variational Autoencoders: LVAEs improve upon traditional VAEs by using a hierarchical structure that enhances the learning of latent representations across multiple layers, leading to a better generative performance. LVAEs are ideal for deep generative modeling tasks, particularly in hierarchical representation learning.
Vector Quantized-Variational autoencoder: The VQ-VAE addresses the issue of “posterior collapse” in VAEs by using discrete latent variables, making it more effective for generating high-quality data, particularly in unsupervised learning scenarios. VQ-VAE is applied in tasks such as unsupervised phoneme learning, speaker conversion, and other tasks involving long-term sequence generation.

5.1.3. Evaluation of Specific LLM Models

GPT-1: GPT-1 was a breakthrough in generating coherent and contextually relevant text, but its unidirectional nature limit its ability to understand long-range dependencies in text. This requires substantial fine-tuning and labeling of data for specific tasks. Useful for basic text generation, translation, and summarization, GPT-1 sets the foundation for future advancements in language models.
GPT-2: GPT-2 was significantly improved over GPT-1 by increasing the model size and training data. It demonstrated strong performance in zero-shot tasks, such as summarization, translation, and question answering without requiring task-specific fine-tuning. GPT-2 is widely used in content creation, machine translation, and dialogue systems, although concerns remain about its potential misuse in generating misleading content.
GPT-3: GPT-3 represents a major leap in scale with 175 billion parameters, allowing it to perform well in few-shot and zero-shot learning tasks. It excels in natural language understanding and generation, making it highly versatile across various tasks without fine-tuning. GPT-3 is used in advanced chatbots, content generation, and translation, demonstrating human-like text generation capabilities. However, it faces challenges with certain datasets and ethical concerns.
BERT: BERT’s bidirectional architecture enables it to understand the context from both the left and right, making it highly effective in a wide range of NLP tasks. It achieves state-of-the-art performance on many benchmarks, including question answering and sentence classification. BERT has been widely adopted for tasks such as sentiment analysis, text classification, and question answering because of its strong contextual understanding.
LaMDA: LaMDA focuses on conversational AI and is designed to generate more contextually accurate and safe responses. It improves factual grounding by integrating external knowledge sources and addresses safety concerns by filtering biased or harmful content. Primarily used in dialogue systems and conversational agents, LaMDA enhances the quality and reliability of AI interactions in applications such as education and customer service.
XLNet: XLNet improves on BERT by combining autoregressive and autoencoding approaches, allowing for better bidirectional context modeling without the limitations of masked language models. It consistently outperformed BERT on various NLP tasks. XLNet excels in tasks such as question answering and sentiment analysis, where understanding complex linguistic relationships is crucial.

5.2. Real World Applications

5.3. threats to validity, 5.3.1. internal threats to validity.

Selection bias: The review primarily incorporated studies published in academic journals and conferences, potentially overlooking valuable insights from proprietary or unpublished industrial applications. This selection bias may have skewed the representation of the effectiveness and applicability of the techniques discussed.
Implementation fidelity: The discussion of the computational and resource demands for implementing generative AI techniques is based on theoretical analysis rather than empirical testing. This can lead to inaccurate assessments of the feasibility and efficiency of these methods in practical applications.

5.3.2. External Threats to Validity

Technological evolution: As generative AI models continue to evolve, the techniques and methods discussed in this review may become increasingly outdated. Future advancements could introduce new capabilities or address the current limitations, thereby affecting the long-term validity of the findings.
Ethical and privacy considerations: Although ethical and privacy issues, particularly those related to LLMs, have been acknowledged, they have not been deeply explored. The responsible deployment of synthetic data generation technologies requires thorough ethical scrutiny, which is crucial for ensuring long-term societal acceptance and regulatory compliance of these technologies.
Interdisciplinary integration: The integration of synthetic data generation techniques with other emerging technologies, such as differential privacy and federated learning, is briefly mentioned but not extensively analyzed. This omission could overlook important synergies and potential advancements in enhancing the privacy and utility of the data.

6. Challenges and Future Research Directions

Data fidelity and diversity: Ensuring that synthetic data accurately represent the diversity and statistical properties of real-world data is a complex task. Overfitting or generating unrealistic data can limit the utility of synthetic data in training AI models and conducting research.
Bias and fairness: Synthetic data generated from biased real-world datasets may perpetuate or amplify these biases, potentially leading to unfair and discriminatory AI models. Identifying and mitigating these biases is crucial for ensuring equitable outcomes.
Computational and environmental costs: High computational power demands, particularly for models such as GPT-3 and BERT, not only increase operational costs but also raise environmental concerns owing to significant energy consumption.
Privacy and security: Although synthetic data can help protect privacy by anonymizing data, ensuring that these data do not inadvertently leak sensitive information remains a challenge. Combining synthetic data generation with privacy-preserving techniques such as differential privacy and federated learning is necessary to enhance security.
Evaluation and validation: Establishing robust metrics and frameworks for evaluating the quality of synthetic data is essential but challenging. Current evaluation methods often lack comprehensive empirical validation, which makes it difficult to assess the real-world applicability of synthetic data.
Practical implementation: There is often a gap between theoretical advancements in generative AI and its practical implementation. Providing clear guidelines and best practices for deploying these models in real-world scenarios is crucial for effective application.

7. Conclusions

Author contributions, data availability statement, conflicts of interest.

Fan, J.; Han, F.; Liu, H. Challenges of Big Data Analysis. Natl. Sci. Rev. 2014 , 1 , 293–314. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Fhom, H. Big Data: Opportunities and Privacy Challenges. In Proceedings of the International Conference on Information Systems and Management Science, Karlsruhe, Germany, 21–23 July 2015; p. 13. [ Google Scholar ] [ CrossRef ]
Poucin, F.; Kraus, A.; Simon, M. Synthetic data shows promising properties to boost the performance of Deep Neural Networks on real-world instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, Montreal, BC, Canada, 11–17 October 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 945–953. [ Google Scholar ]
Abowd, J.M.; Vilhuber, L. How Protective Are Synthetic Data? In Privacy in Statistical Databases ; Springer: Berlin/Heidelberg, Germany, 2008. [ Google Scholar ]
Jävergård, N.; Lyons, R.; Muntean, A.; Forsman, J. Preserving correlations: A Statistical Method for Generating Synthetic Data. arXiv 2024 , arXiv:2403.01471. [ Google Scholar ] [ CrossRef ]
Kingma, D.P.; Welling, M. Auto-Encoding Variational Bayes. arXiv 2013 , arXiv:1312.6114. [ Google Scholar ]
Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. arXiv 2014 . [ Google Scholar ] [ CrossRef ]
Radford, A.; Narasimhan, K. Improving Language Understanding by Generative Pre-Training. OpenAI 2018 . Available online: https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf (accessed on 28 August 2024).
Long, L.; Wang, R.; Xiao, R.; Zhao, J.; Ding, X.; Chen, G.; Wang, H. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey. arXiv 2024 , arXiv:2406.15126. [ Google Scholar ]
Bauer, A.; Trapp, S.; Stenger, M.; Leppich, R.; Kounev, S.; Leznik, M.; Chard, K.; Foster, I. Comprehensive Exploration of Synthetic Data Generation: A Survey. arXiv 2024 , arXiv:2401.02524. [ Google Scholar ]
Hao, S.; Han, W.; Jiang, T.; Li, Y.; Wu, H.; Zhong, C.; Zhou, Z.; Tang, H. Synthetic Data in AI: Challenges, Applications, and Ethical Implications. arXiv 2024 , arXiv:2401.01629. [ Google Scholar ]
Sengar, S.S.; Hasan, A.B.; Kumar, S.; Carroll, F. Generative Artificial Intelligence: A Systematic Review and Applications. arXiv 2024 , arXiv:2405.11029. [ Google Scholar ] [ CrossRef ]
Sufi, F.K. Generative Pre-Tr78. Sufi, F.K. Generative Pre-Trained Transformer (GPT) in Research: A Systematic Review on Data Augmentation. Information 2024 , 15 , 99. [ Google Scholar ] [ CrossRef ]
Guo, X.; Chen, Y. Generative AI for Synthetic Data Generation: Methods, Challenges and the Future. arXiv 2024 , arXiv:2403.04190. [ Google Scholar ]
Lu, Y.; Wang, H.; Wei, W. Machine Learning for Synthetic Data Generation: A Review. arXiv 2023 , arXiv:2302.04062. [ Google Scholar ]
Bandi, A.; Adapa, P.V.; Kuchi, Y.E. The Power of Generative AI: A Review of Requirements, Models, Input-Output Formats, Evaluation Metrics, and Challenges. Future Internet 2023 , 15 , 260. [ Google Scholar ] [ CrossRef ]
Ippolito, D.; Ahn, J.; Cerqueira, J.F.; Huang, M.; Burgess, D. Bias and Fairness in Large Language Models: A Survey. arXiv 2023 , arXiv:2309.00770. [ Google Scholar ]
Eigenschink, P.; Reutterer, T.; Vamosi, S.; Vamosi, R.; Sun, C.; Kalcher, K. Deep Generative Models for Synthetic Data: A Survey. IEEE Access 2023 , 11 , 47304–47320. [ Google Scholar ] [ CrossRef ]
Fonseca, J.; Bação, F. Tabular and Latent Space Synthetic Data Generation: A Literature Review. J. Big Data 2023 , 10 , 1–37. [ Google Scholar ] [ CrossRef ]
Vargas, A.M.; Berlanga, A. A comprehensive review on synthetic data generation and its applications in medical imaging. Neurocomputing 2022 , 482 , 231–247. [ Google Scholar ]
Lu, Y.; Chen, D.; Olaniyi, E.O.; Huang, Y. Generative Adversarial Networks (GANs) for Image Augmentation in Agriculture: A Systematic Review. Comput. Electron. Agric. 2022 , 200 , 107208. [ Google Scholar ] [ CrossRef ]
Wang, S.; Du, Y.; Guo, X.; Pan, B.; Zhao, L. Controllable Data Generation by Deep Learning: A Review. ACM Comput. Surv. 2022 , 56 , 1–38. [ Google Scholar ] [ CrossRef ]
Figueira, Á.; Vaz, B. Survey on Synthetic Data Generation, Evaluation Methods and GANs. Mathematics 2022 , 10 , 2733. [ Google Scholar ] [ CrossRef ]
Kitchenham, B.; Charters, S. Guidelines for Performing Systematic Literature Reviews in Software Engineering. Engineering 2007 , 2 , 1051. [ Google Scholar ]
Petersen, K.; Feldt, R.; Mujtaba, S.; Mattsson, M. Systematic Mapping Studies in Software Engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, Bari, Italy, 26–27 June 2008; pp. 68–77. [ Google Scholar ]
Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Antes, G.; Atkins, D.; Barbour, V.; Barrowman, N.; Berlin, J.A. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009 , 6 , e1000097. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Syntheticus.ai. Guide: Everything You Need to Know about Synthetic Data. Available online: https://syntheticus.ai/guide-everything-you-need-to-know-about-synthetic-data (accessed on 20 May 2024).
DataRobot. What are Parametric Models? Available online: https://www.datarobot.com/blog/what-are-parametric-models/ (accessed on 20 May 2024).
DeepAI. Non-Parametric Model. Available online: https://deepai.org/machine-learning-glossary-and-terms/non-parametric-model (accessed on 20 May 2024).
Singh, A. Protecting your Data Privacy with Differential Privacy: An Introduction. Available online: https://medium.com/dsaid-govtech/protecting-your-data-privacy-with-differential-privacy-an-introduction-abee1d7fcb63 (accessed on 22 May 2024).
Bossert, J.; Lütjen, M.; Kanoun, O. Context-adaptive and activity-aware physical analysis for people with dementia. In Ambient Assisted Living ; Springer: Berlin/Heidelberg, Germany, 2011; pp. 161–168. [ Google Scholar ]
Islam, S.; Aziz, M.T.; Nabil, H.R.; Jim, J.R.; Mridha, M.F.; Kabir, M.M.; Asai, N.; Shin, J. Generative Adversarial Networks (GANs) in Medical Imaging: Advancements, Applications, and Challenges. IEEE Access 2024 , 12 , 35728–35753. [ Google Scholar ] [ CrossRef ]
Strelcenia, E.; Prakoonwit, S. Improving Cancer Detection Classification Performance Using GANs in Breast Cancer Data. IEEE Access 2023 , 11 , 71594–71615. [ Google Scholar ] [ CrossRef ]
Ali, H.; Grönlund, C.; Shah, Z. Leveraging GANs for Data Scarcity of COVID-19: Beyond the Hype. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada, 17–24 June 2023; pp. 659–667. [ Google Scholar ]
Yadav, P.; Gaur, M.; Fatima, N.; Sarwar, S. Qualitative and Quantitative Evaluation of Multivariate Time-Series Synthetic Data Generated Using MTS-TGAN: A Novel Approach. Appl. Sci. 2023 , 13 , 4136. [ Google Scholar ] [ CrossRef ]
Charitou, C.; Dragicevic, S.; Garcez, A.S. Synthetic Data Generation for Fraud Detection using GANs. arXiv 2021 , arXiv:2109.12546. [ Google Scholar ]
Xu, L.; Skoularidou, M.; Cuesta-Infante, A.; Veeramachaneni, K. CTGAN: Synthesizing Tabular Data Using Conditional GANs. Neural Inf. Process. Syst. 2019 . Available online: https://hdl.handle.net/1721.1/128349 (accessed on 28 August 2024).
Xu, L.; Skoularidou, M.; Cuesta-Infante, A.; Veeramachaneni, K. Modeling Tabular Data Using Conditional GAN. arXiv 2019 , arXiv:1907.00503. [ Google Scholar ] [ CrossRef ]
Miyato, T.; Koyama, M. cGANs with Projection Discriminator. arXiv 2018 , arXiv:1802.05637. [ Google Scholar ]
Xie, L.; Lin, K.; Wang, S.; Wang, F.; Zhou, J. Differentially Private Generative Adversarial Network. arXiv 2018 , arXiv:1802.06739. [ Google Scholar ]
Zhu, J.; Park, T.; Isola, P.; Efros, A.A. Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2242–2251. [ Google Scholar ]
Chen, X.; Duan, Y.; Houthooft, R.; Schulman, J.; Sutskever, I.; Abbeel, P. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. arXiv 2016 , arXiv:1606.03657. [ Google Scholar ] [ CrossRef ]
LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015 , 521 , 436–444. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Radford, A.; Metz, L.; Chintala, S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv 2015 , arXiv:1511.06434. [ Google Scholar ]
Mirza, M.; Osindero, S. Conditional Generative Adversarial Nets. arXiv 2014 , arXiv:1411.1784. [ Google Scholar ]
PyTorch. DCGAN Tutorial: Generate Faces Using Deep Convolutional GAN. Available online: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html (accessed on 30 May 2024).
TensorFlow. CycleGAN Tutorial: Using Cycle-Consistent Adversarial Networks for Unpaired Image-to-Image Translation. Available online: https://www.tensorflow.org/tutorials/generative/cyclegan (accessed on 2 June 2024).
Mostofi, F.; Tokdemir, O.B.; Toğan, V. Generating Synthetic Data with Variational Autoencoder to Address Class Imbalance of Graph Attention Network Prediction Model for Construction Management. Adv. Eng. Inform. 2024 , 62 , 102606. [ Google Scholar ] [ CrossRef ]
Wu, J.; Plataniotis, K.N.; Liu, L.; Amjadian, E.; Lawryshyn, Y.A. Interpretation for Variational Autoencoder Used to Generate Financial Synthetic Tabular Data. Algorithms 2023 , 16 , 121. [ Google Scholar ] [ CrossRef ]
Li, H.; Yu, S.; Príncipe, J.C. Causal Recurrent Variational Autoencoder for Medical Time Series Generation. arXiv 2023 , arXiv:2301.06574. [ Google Scholar ] [ CrossRef ]
Saldanha, J.; Chakraborty, S.; Patil, S.A.; Kotecha, K.V.; Kumar, S.; Nayyar, A. Data Augmentation Using Variational Autoencoders for Improvement of Respiratory Disease Classification. PLoS ONE 2022 , 17 , e0266467. [ Google Scholar ] [ CrossRef ]
Kok, S.; Vardhan, L.V. Generating Privacy-Preserving Synthetic Tabular Data Using Oblivious Variational Autoencoders. In Proceedings of the Workshop on Economics of Privacy and Data Labor at the 37th International Conference on Machine Learning, Cambridge, UK, 18 July 2020. [ Google Scholar ]
Islam, Z.; Abdel-Aty, M.A.; Cai, Q.; Yuan, J. Crash Data Augmentation Using Variational Autoencoder. Accid. Anal. Prev. 2020 , 151 , 105950. [ Google Scholar ] [ CrossRef ]
Goyal, P.; Sapienza, M.; Sun, C. Self-Supervised Video Representation Learning with Contrastive Predictive Coding. arxiv 2019 , arXiv:1905.09272. [ Google Scholar ] [ CrossRef ]
van den Oord, A.; Vinyals, O.; Kavukcuoglu, K. Neural Discrete Representation Learning. arxiv 2017 , arXiv:1711.00937. [ Google Scholar ] [ CrossRef ]
Kipf, T.; Welling, M. Variational Graph Auto-Encoders. arXiv 2016 , arXiv:1611.07308. [ Google Scholar ]
Rasmus, A.; Berglund, M.; Honkala, M.; Valpola, H.; Raiko, T. Semi-Supervised Learning with Ladder Networks. arXiv 2015 , arXiv:1507.02672. [ Google Scholar ] [ CrossRef ]
Sohn, K.; Lee, H.; Yan, X. Learning Structured Output Representation using Deep Conditional Generative Models. arXiv 2015 , arXiv:1506.05517. [ Google Scholar ] [ CrossRef ]
Makhzani, A.; Shlens, J.; Jaitly, N.; Goodfellow, I.J. Adversarial Autoencoders. arXiv 2015 , arXiv:1511.05644. [ Google Scholar ]
Li, Z.; Zhu, H.; Lu, Z.; Yin, M. Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations. arXiv 2023 , arXiv:2310.07849. [ Google Scholar ]
Thoppilan, R.; De Freitas, D.; Hall, J.; Shazeer, N.M.; Kulshreshtha, A.; Cheng, H.; Jin, A.; Bos, T.; Baker, L.; Du, Y.; et al. LaMDA: Language Models for Dialog Applications. arXiv 2022 , arXiv:2201.08239. [ Google Scholar ]
Meng, Y.; Huang, J.; Zhang, Y.; Han, J. Generating Training Data with Language Models: Towards Zero-Shot Language Understanding. arXiv 2022 , arXiv:2202.04538. [ Google Scholar ]
Austin, J.; Odena, A.; Nye, M.; Bosma, M.; Michalewski, H.; Dohan, D.; Jiang, E.; Cai, C.J.; Terry, M.; Le, Q.; et al. Program Synthesis with Large Language Models. arXiv 2021 , arXiv:2108.07732. [ Google Scholar ]
Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language Models Are Few-Shot Learners. arXiv 2020 , arXiv:2005.14165. [ Google Scholar ]
Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I. Language Models are Unsupervised Multitask Learners. OpenAI 2019 . Available online: https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf (accessed on 28 August 2024).
Devlin, J.; Chang, M.; Lee, K.; Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Proceedings of the 2019 North American Chapter of the Association for Computational Linguistics, Minneapolis, MN, USA, 2–7 June 2019. [ Google Scholar ]
Yang, Z.; Dai, Z.; Yang, Y.; Carbonell, J.G.; Salakhutdinov, R.; Le, Q.V. XLNet: Generalized Autoregressive Pretraining for Language Understanding. arxiv 2019 , arXiv:1906.08237. [ Google Scholar ] [ CrossRef ]
Dahmen, J.; Cook, D. SynSys: A Synthetic Data Generation System for Healthcare Applications. Sensors 2019 , 19 , 1181. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Google. TF-GAN: A Generative Adversarial Networks Library for TensorFlow. Available online: https://www.tensorflow.org/tutorials/generative/tf-gan (accessed on 10 June 2024).
Gretel.ai. Gretel Synthesis: Generate Synthetic Data with Enhanced Privacy Features. Available online: https://gretel.ai (accessed on 15 June 2024).
Ping, H.; Stoyanovich, J.; Howe, B. DataSynthesizer: Privacy-preserving synthetic datasets. arXiv 2017 . [ Google Scholar ] [ CrossRef ]
Dankar, F.K.; Ibrahim, M.M. Fake It Till You Make It: Guidelines for Effective Synthetic Data Generation. Appl. Sci. 2021 , 11 , 2158. [ Google Scholar ] [ CrossRef ]
D’Amico, S.; Dall’Olio, D.; Sala, C.; Dall’Olio, L.; Sauta, E.; Zampini, M.; Asti, G.; Lanino, L.; Maggioni, G.; Campagna, A.; et al. Synthetic Data Generation by Artificial Intelligence to Accelerate Research and Precision Medicine in Hematology. JCO Clin. Cancer Inform. 2023 , 7 , e2300021. [ Google Scholar ] [ CrossRef ] [ PubMed ]
Jadon, A.; Kumar, S. Leveraging Generative AI Models for Synthetic Data Generation in Healthcare: Balancing Research and Privacy. In Proceedings of the 2023 International Conference on Smart Applications, Communications and Networking (SmartNets), İstanbul, Türkiye, 25–27 July 2023; pp. 1–4. [ Google Scholar ]
Bird, J.J.; Lotfi, A. CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images. IEEE Access 2023 , 12 , 15642–15650. [ Google Scholar ] [ CrossRef ]
Patki, N.; Wedge, R.; Veeramachaneni, K. The synthetic data vault. In Proceedings of the 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Montreal, QC, Canada, 17–19 October 2008; IEEE: Piscataway, NJ, USA, 2016. [ Google Scholar ]
Beaulieu-Jones, B.K.; Wu, Z.S.; Williams, C.; Greene, C.S. Privacy-Preserving Generative Deep Neural Networks Support Clinical Data Sharing. bioRxiv 2019 . [ Google Scholar ] [ CrossRef ]
Choi, E.; Biswal, S.; Malin, B.A.; Duke, J.D.; Stewart, W.F.; Sun, J. Generating Multi-Label Discrete Patient Records using Generative Adversarial Networks. In Proceedings of the 2nd Machine Learning for Healthcare Conference, Boston, MA, USA, 18–19 August 2017. [ Google Scholar ]
Frid-Adar, M.; Klang, E.; Amitai, M.; Goldberger, J.; Greenspan, H. Synthetic Data Augmentation Using GAN for Improved Liver Lesion Classification. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, 4–7 April 2018. [ Google Scholar ]
Yi, X.; Walia, E.; Babyn, P. Generative adversarial network in medical imaging: A review. Med. Image Anal. 2019 , 58 , 101552. [ Google Scholar ] [ CrossRef ]
Esteban, C.; Hyland, S.L.; Rätsch, G. Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs. arXiv 2017 , arXiv:1706.02633. [ Google Scholar ]
Xu, L.; Veeramachaneni, K. Synthesizing Tabular Data Using Generative Adversarial Networks. arXiv 2018 , arXiv:1811.11264. [ Google Scholar ]
Montenegro, H.; Silva, W.; Cardoso, J. Privacy-Preserving Generative Adversarial Network for Case-Based Explainability in Medical Image Analysis. IEEE Access 2021 , 9 , 148037–148047. [ Google Scholar ] [ CrossRef ]
Recent Developments in Generative AI for Audio. Available online: https://www.assemblyai.com/blog/recent-developments-in-generative-ai-for-audio/ (accessed on 17 August 2024).
Bao, H.; Dong, L.; Wei, F. UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training. arXiv 2019 , arXiv:2002.12804. [ Google Scholar ]
Magda, N.; Maciej, M.; Michal, P.; Tomasz, T.; Michal, W. Federated Learning Methods for Combating Attacks and Improving Privacy in IoT Networks. In Computational Intelligence ; Springer: Cham, Switzerland, 2021; pp. 487–500. [ Google Scholar ]
Auditing Bias in Large Language Models. Available online: https://insights.sei.cmu.edu/blog/auditing-bias-in-large-language-models/ (accessed on 17 August 2024).

Primary Study	Year	Overview	Limitations
Long, L. [ ]	2024	This study surveys the potential of large language models (LLMs) for data generation, curation, and evaluation, highlighting their strengths and limitations across various tasks.	The survey lacks specificity in addressing fine-tuning LLMs for domain-specific applications.
Bauer, A. [ ]	2024	This paper presents a comprehensive survey on synthetic data generation, discussing various techniques and their applications, with a focus on machine learning models and privacy concerns.	The review may overlook recent advancements due to its reliance on the earlier literature.
Hao, S. [ ]	2024	The study discusses the challenges, applications, and ethical implications of synthetic data in AI, particularly in healthcare and finance, with a focus on maintaining privacy and fairness.	The review extensively covers challenges but offers limited practical solutions for ethical concerns.
Sengar, S.S. [ ]	2024	This systematic review explores the use of generative AI in various applications, emphasizing its potential in synthetic data generation and the associated ethical challenges.	The review highlights ethical challenges but lacks in-depth strategies for addressing them in generative AI.
Sufi, F.K. [ ]	2024	This paper reviews the use of generative pre-trained transformers (GPTs) for data augmentation in research, highlighting the benefits and limitations of using GPT models in various academic contexts.	The paper primarily focuses on academic contexts, potentially missing broader practical applications of GPTs for data augmentation.
Guo, X. [ ]	2024	The study provides an overview of the methods and challenges in using generative AI for synthetic data generation, focusing on the potential future directions of this technology.	The survey provides a broad overview but may oversimplify the challenges of synthetic data generation in complex real-world scenarios.
Lu, Y. [ ]	2023	This systematic review discusses the role of generative adversarial networks (GANs) in synthetic data generation for agriculture, highlighting their applications in image augmentation and crop monitoring.	The review is comprehensive but could benefit from a more detailed examination of the scalability of GANs in agricultural applications.
Bandi, A. [ ]	2023	The paper reviews the requirements, models, input–output formats, evaluation metrics, and challenges in leveraging generative AI models for synthetic data generation.	The review provides a broad perspective but lacks detailed examples of successful implementations in various domains.
Ippolito, D. [ ]	2023	This survey focuses on the bias and fairness in large language models, providing insights into how these models can be audited and adjusted to minimize ethical risks.	The survey focuses on bias but offers limited actionable recommendations for mitigating it in LLMs.
Eigenschink, P. [ ]	2023	This paper surveys deep generative models for synthetic data, exploring their effectiveness across various fields, including healthcare, finance, and autonomous systems.	The review is extensive but could better address the practical limitations of deep generative models in specific industries.
Fonseca, J. [ ]	2023	The study provides a comprehensive review of methods for generating synthetic tabular and latent space data, discussing their applications and limitations in various domains.	The review is thorough but does not sufficiently explore the practical challenges of implementing synthetic data generation in industrial applications.
Vargas, A. M. [ ]	2022	This paper reviews the use of synthetic data in medical imaging, focusing on the advancements in GANs and VAEs and their impact on data augmentation for diagnostic tasks.	The review focuses on GANs and VAEs but lacks discussion on the integration of synthetic data with existing medical imaging workflows.
Lu, Y. [ ]	2022	This systematic review discusses the role of GANs in agriculture, particularly in the generation of synthetic data for improving crop monitoring and disease detection.	The review is well rounded but lacks insights into the real-world challenges of deploying GANs in agricultural settings.
Wang, S. [ ]	2022	The paper provides a review of controllable data generation using deep learning, with a focus on methods and challenges in generating synthetic data for various applications.	The paper broadly covers controllable data generation but does not delve deeply into the technical challenges of achieving control in diverse data types.
Figueira, Á. [ ]	2022	This survey reviews methods and evaluation techniques for synthetic data generation, focusing on the use of GANs and their effectiveness in generating high-quality data.	The survey is detailed but could be improved by offering more practical evaluation methods for synthetic data quality in applied settings.

Primary Study	Year	Overview
Islam, S. [ ]	2024	This paper reviews the advancements, applications, and challenges of using GANs in medical imaging, particularly in enhancing the quality of medical images for better diagnostic accuracy.
Strelcenia, E. [ ]	2023	This study discusses how GANs have been used to improve breast cancer detection through the generation of synthetic data, leading to enhanced classification performance.
Ali, H. [ ]	2023	The paper explores the application of GANs in generating synthetic COVID-19 data to address data scarcity and improve the performance of diagnostic models.
Yadav, P. [ ]	2023	This paper introduces a novel approach using MTS-TGAN for generating synthetic multivariate time series data, improving the quality and applicability of time series data in various fields.
Charitou, C. [ ]	2021	The study discusses the use of GANs for fraud detection by generating synthetic data that helps improve detection rates in financial services.
Xu, L. [ ]	2019	This paper presents a conditional tabular GAN (CTGAN) model for generating synthetic tabular data, which is particularly effective in handling imbalanced datasets.
Xu, L. [ ]	2019	The TGAN model described in this paper is designed for generating high-quality synthetic tabular data, preserving inter-column relationships.
Miyato, T. [ ]	2018	This paper introduces the concept of cGANs with a Projection Discriminator, improving the quality of class-conditional image generation by modifying the discriminator’s design.
Xie, L. [ ]	2018	The paper discusses a Differentially Private Generative Adversarial Network (DP-GAN), which ensures privacy while generating synthetic data for sensitive applications.
Zhu, J. [ ]	2017	This paper presents CycleGAN, which enables unpaired image-to-image translation, useful in scenarios where paired data are unavailable.
Chen, X. [ ]	2016	CcGAN is introduced in this paper as an interpretable GAN model, focusing on disentangling latent space to generate meaningful representations.
LeCun, Y. [ ]	2015	This seminal paper on deep learning provides an overview of the field and discusses the development of deep neural networks, highlighting their success in various applications.
Radford, A. [ ]	2015	This paper describes the development of a DCGAN, a variant of GANs using deep convolutional layers to improve the quality of generated images.
Mirza, M. [ ]	2014	This paper introduces conditional GANs (cGANs), which improve GANs by conditioning the generation process on additional information, allowing for more controlled data generation.
Goodfellow, I. [ ]	2014	The foundational paper that introduced GANs, describing the adversarial process where two neural networks compete, leading to the generation of realistic data.

Primary Study	Year	Overview
Mostofi, F. [ ]	2024	This study integrates a variational autoencoder (VAE) into a multi-head graph attention network (GAT) to address the class imbalance in construction datasets, significantly improving predictive performance in construction management tasks.
Wu, J. [ ]	2023	This paper focuses on using variational autoencoders (VAEs) for generating synthetic financial data while preserving privacy, demonstrating effectiveness in financial applications where data sensitivity is a concern.
Li, H. [ ]	2023	The study proposes a causal recurrent variational autoencoder (CR-VAE) for generating time series data with an emphasis on learning Granger causality, showing superior performance in medical time series analysis like EEG and fMRI data.
Saldanha, J. [ ]	2022	This paper discusses the use of VAEs to generate synthetic respiratory sounds for the classification of respiratory diseases, addressing class imbalance and improving classification performance in minority classes.
Kok, S. [ ]	2020	The study introduces oblivious variational autoencoders (OVAEs) for generating privacy-preserving synthetic tabular data, particularly in environments requiring strict data privacy like healthcare and finance.
Islam, Z. [ ]	2020	This paper addresses class imbalance in crash datasets using VAEs to generate synthetic data, leading to improved specificity and sensitivity in crash prediction models.
Goyal, P. [ ]	2019	This study focuses on using Contrastive Predictive Coding (CPC) for self-supervised video representation learning, enabling models to learn more effective video features without requiring large, labeled datasets.
van den Oord, A. [ ]	2017	The paper introduces Neural Discrete Representation Learning, which uses discrete latent representations in VAEs to prevent posterior collapse, making it effective in unsupervised learning tasks like phoneme learning.
Kipf, T. [ ]	2016	This study presents variational graph autoencoders (VGAEs) for unsupervised learning on graph-structured data, showing their effectiveness in tasks like link prediction and graph clustering.
Rasmus, A. [ ]	2015	The paper explores Ladder Networks for semi-supervised learning, demonstrating their ability to improve learning performance by leveraging both labeled and unlabeled data.
Sohn, K. [ ]	2015	This study introduces a deep conditional generative model for learning structured output representations, offering advancements in probabilistic inference in complex output spaces like image segmentation.
Makhzani, A. [ ]	2015	This paper discusses adversarial autoencoders (AAEs), which combine VAEs with GANs to align latent space distributions, offering advancements in generative modeling and semi-supervised learning.
Kingma, D.P. [ ]	2013	The foundational paper on Auto-Encoding Variational Bayes (AEVB), introducing a powerful method for variational inference that has become a cornerstone in generative modeling.

Primary Study	Year	Overview
Li, Z. [ ]	2023	This study investigates the potential and limitations of using large language models (LLMs) for generating synthetic datasets specifically for text classification tasks, highlighting challenges related to task subjectivity and model bias.
Thoppilan, R. [ ]	2022	The paper introduces LaMDA (language models for dialog applications), a family of transformer-based models designed for conversational AI, emphasizing safety, factual grounding, and improving the quality of AI-driven dialogues.
Meng, Y. [ ]	2022	This research presents SuperGen, a novel method for zero-shot learning in natural language understanding tasks using pre-trained language models (PLMs), enhancing performance through prompt-guided text generation and label smoothing.
Austin, J. [ ]	2021	The study explores the use of LLMs for program synthesis, demonstrating the ability of these models to generate Python code from natural language descriptions, with an emphasis on challenges related to semantic understanding.
Brown, T.B. [ ]	2020	This paper introduces GPT-3, a groundbreaking autoregressive language model with 175 billion parameters, highlighting its few-shot learning capabilities and its impact on various natural language processing tasks.
Radford, A. [ ]	2019	This study presents GPT-2, a large-scale unsupervised language model that can perform a variety of NLP tasks, such as summarization and question answering, without task-specific fine-tuning.
Devlin, J. [ ]	2019	The BERT model is introduced in this paper, offering significant improvements in various NLP tasks by pre-training deep bidirectional representations from unlabeled text.
Yang, Z. [ ]	2019	This paper introduces XLNet, a generalized autoregressive model that improves BERT by capturing bidirectional context while avoiding the limitations of masked language modeling.
Radford, A. [ ]	2018	This study describes GPT-1, the first generative pre-trained transformer model, demonstrating the effectiveness of unsupervised pre-training for improving performance on various NLP benchmarks.

The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

Goyal, M.; Mahmoud, Q.H. A Systematic Review of Synthetic Data Generation Techniques Using Generative AI. Electronics 2024 , 13 , 3509. https://doi.org/10.3390/electronics13173509

Goyal M, Mahmoud QH. A Systematic Review of Synthetic Data Generation Techniques Using Generative AI. Electronics . 2024; 13(17):3509. https://doi.org/10.3390/electronics13173509

Goyal, Mandeep, and Qusay H. Mahmoud. 2024. "A Systematic Review of Synthetic Data Generation Techniques Using Generative AI" Electronics 13, no. 17: 3509. https://doi.org/10.3390/electronics13173509

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Loading metrics

Open Access

Peer-reviewed

Research Article

What is a heat wave: A survey and literature synthesis of heat wave definitions across the United States

Roles Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Validation, Writing – original draft

* E-mail: [email protected]

Affiliations Department of Geography, Environment, and Spatial Sciences, Michigan State University, East Lansing, Michigan, United States of America, Remote Sensing and GIS Research and Outreach Services, Michigan State University, East Lansing, Michigan, United States of America

Roles Formal analysis, Visualization, Writing – review & editing

Roles Funding acquisition, Methodology, Writing – review & editing

Affiliation Department of Geography, University of Florida, Gainesville, Florida, United States of America

Erin L. Bunting,
Vasily Tolmanov,
David Keellings

Published: September 5, 2024
https://doi.org/10.1371/journal.pclm.0000468
Peer Review
Reader Comments

Heat waves are the last extreme weather events without a formal, on the books, definition. Instead, across the U.S. those working on extreme heat event management, forecasting, and planning are using differing definitions in their work. With such differing definitions being used there are widespread impacts including some to human and environmental health, natural resource management, and long-term emergency management planning. For instance, when should heat advisories for vulnerable populations be released when an event impacts a region using multiple definitions? There are concrete and justifiable reasons for the lack of a formal heat wave definition including, at its simplest, differences in what temperature is extreme enough, compared to the region’s climatological regimens, to be deemed as an extreme heat event or heat wave. This study looks for patterns and commonalities in emergency managers and climatologists, those most commonly addressing or planning for such events, definition of heat wave events through a review of the literature and widespread survey across the United States. Through a short 11-questions survey and subsequent text mining, we find widespread variability in the common heat wave definitions but a consistent pattern of core key term usage including aspects of heat duration, extreme temperature, and humidity. However, we also see little to no usage of non-climatological variables such as exposure, vulnerability, population, and land cover/land use.

Citation: Bunting EL, Tolmanov V, Keellings D (2024) What is a heat wave: A survey and literature synthesis of heat wave definitions across the United States. PLOS Clim 3(9): e0000468. https://doi.org/10.1371/journal.pclm.0000468

Editor: Teodoro Georgiadis, Institute for BioEconomy CNR, ITALY

Received: December 14, 2023; Accepted: July 13, 2024; Published: September 5, 2024

Copyright: © 2024 Bunting et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All data needed to recreate this article is provided in the supplemental materials associated with the article. Identifying data will not be released including the survey takers names or contact information, but these data are not essential for study recreation. This project was deemed exempt from full IRB review. Any questions related to the data, or the article can be addressed by emailing the authors Erin Bunting ( [email protected] ) or David Keellings ( [email protected] ). If you have any data requests or additional questions, and the authors are not available, please feel free to contact MSU RS&GIS ( [email protected] ). This group has prior knowledge of the project and a copy of project and IRB documents on a secure server. If you have any questions or concerns regarding the survey, contact please contact the Michigan State University Human Research Protection Program (HRPP) office at 517-355-2180 or via email at [email protected] . Please use the following MSU Study ID: STUDY00004016 if you communicate with MSU HRPP.

Funding: This research was funded by a grant from the National Science Foundation GSS program (Award #2203235). Multiple people and organizations made this publication possible and the authors wish to express their gratitude, especially to Dr. Laura Myers and Jacob Reed at the University of Alabama, Dan Wanyama, and the Staff of Remote Sensing and GIS Research and Outreach Services (RS&GIS) at Michigan State University. Additionally, we thank the reviewers for their time and effort put into manuscript review.

Competing interests: The authors have declared that no competing interests exist.

1. Introduction

Extreme weather and climate events have been impacting human and natural landscape since the beginning of time. However, with changing climate patterns we are seeing greater impacts of these events globally. As stated in the most recent IPCC report it is an “established fact” that human induced change has resulted in “an increased frequency and intensity of some weather and climate extremes since pre-industrial times” [ 1 ]. With such changing patterns it is important to look at not just the impacts and patterns of such events but the premise of the event definition itself. Extreme weather and climate events are broadly defined as severe weather or climate conditions that induce devastating impacts to the human and natural landscapes. While such events can be weather-related (short in duration), or climate related (long in duration) there is a basic understanding that the event is defined as atypical and beyond the normal. Almost all these extreme events (e.g., hurricanes, tornadoes, blizzards) have standard definitions related to what triggers the event, how they are measured, and their severity classification. For instance, with hurricanes, the Saffir Simpson class is used to define the pressure and wind speeds associated [ 2 ]. With tornadoes, the Enhanced Fujita Scale (EF) uses derived engineering wind estimates in assessing strength and resulting damage [ 3 ].

While almost all climatological events have standard definitions one of the deadliest does not, heat waves. Over the past decade, heat waves of varying durations and intensities have impacted much of the globe. For instance, the 1995 heat wave in the central United States resulted in more than 1000 deaths [ 4 , 5 ]. Further, across France, the large 2003 European heat wave resulted in excess mortality of approximately 15,000 individuals, up 60% from normal mortality patterns [ 6 , 7 ]. In line with the IPCC remarks on trends of extreme events there has been an increased frequency of heat waves not just across the US and France, but across Europe, China, Australia as well [ 1 ]. Additionally, it has been projected that heat waves will be more frequent, intense, and longer-lasting into the future [ 1 , 8 – 10 ], thus raising concerns on planning for such events.

There is no worldwide consensus on a heat wave definition though its usually thought simply as an extended period of extreme heat. Not grounded in academic or climatological literature, the Merriam dictionary defines heat waves as “a period of unusually hot weather”. Additionally, in many published articles, simple definitions based purely on temperature are utilized, including those as simple as “an extreme heat event is defined when the temperature exceeds a given threshold with an appropriate spatial extent” [ 11 ]. Attempts have been made at standardizing a heat wave definition. For instance, in 1996 Environment Canada provided a more scientifically grounded definition of a heat wave as a period of more than three consecutive days of maximum temperatures at or above 32 degrees Celsius [ 12 ]. Additionally, governmental groups both in the US and around the world have developed de facto definitions of heat waves including: the US National Weather Service, NOAA, and UK Met Office. These definitions are being developed for the issuance of heat watches and warnings and therefore should be pertinent for use by emergency managers and state climatologists. With this in mind, one would expect high overlap between national government definitions of heat wave and those from managers and state climatologists. The aforementioned governmental groups define heat waves as such:

US National Weather Service : A period of abnormally hot weather generally lasting more than 2 days. Heat waves can occur with or without high humidity.
NOAA : A period of abnormally and uncomfortably hot and unusually humid weather. Typically, a heat wave lasts two or more days.
UK Met Office : An extended period of hot weather relative to the expected conditions of the areas at that time of year, which may be accompanied by high humidity.
World Meteorological Organization (WMO) : A period where local excess heat accumulates over a sequence of unusually hot days and nights.

What is lacking from these definitions is consistency regarding temperature thresholds, metrics, durations, or number of days used to define such events [ 13 ]. Further, even using these definitions, adverse heat impacts on human health have been documented at lesser extreme temperatures and durations [ 12 , 14 , 15 ]. It’s important to note that the use of various heat wave definitions results in temporal variability in heat wave classification, inability to compare events and synthesize results across regions, and inconsistent terminology in the literature. Many of these challenges can be overcome by finding some consistency across regions and definitions.

Why it is difficult to develop a consistent heat wave definition is fairly obvious, heat waves differ in their intensity (magnitude), extent, duration, and scope of impact [ 16 ]. Numerous studies have used different thresholds of mean or maximum temperature [ 17 ], percentiles of maximum temperature, heat indices, or even combinations of thresholds [ 18 , 19 ]. The common variables of these studies being the use of intensity and duration factors. [ 20 ] looked at 45 definitions of heat waves, combining 5 temperature thresholds, three temperature indicators (daily mean temperature, minimum temperature, and maximum temperature), 5 percentile metrics (90 th , 92.5 th , 95 th , 97.5 th , and 99 th ), and multiple event duration lengths (2, 3, and 4 days) to assess how different definitions align with mortality patterns. Overall, [ 20 ] found the best model fit, and therefore best heat wave definition, using daily mean temperature in the 99 th percentile in combination with a 3-day event duration. Similarly, two studies, one in the US and another from West Africa, found the best fit model was produced by a heat wave definition using both minimum and maximum temperatures in the 90 th percentiles with an event duration of 3 days [ 21 ]. Drawing from these, and other, studies we can see a heat wave definition needs to include factors of: (1) intensity or magnitude: based on a tested index or temperature threshold, (2) duration: defining the persistent of an event to be a heat wave, (3) extent: geographic areas impacted and measures of exposure, and possibly (4) severity [ 22 ].

There are no doubt other factors contributing to heat wave impacts such as frequency, timing, event size, and population density. Incorporation of these factors into a unifying heat wave definition is fraught with complications. For instance, a higher frequency of hot and humid conditions do not necessarily result in a heat wave and severe heat wave impacts [ 12 ]. Similarly, regions experiencing more hot and humid summer conditions already have physiological, behavioral, and infrastructure adaptations to extreme heat, likely reducing the harmful effects [ 12 ]. However, with such heat wave components (intensity, duration, and extent), we can begin to understand the social, cultural, and physical impacts of extreme heat events. For instance, with such a holistic definition we can begin to develop temporary modification to lifestyles to minimize heat stress exposure and impacts. More clearly such a definition enables evasive action and management practice to be developed.

Heat waves can be defined in several ways, through absolute and relative approaches. With an absolute heat wave definition an exact event duration threshold would be set in tandem with a pre-determined temperature and/or heat stress index level [ 12 ]. Whereas, if a relative heat wave definition were developed it would have to take into consideration acclimatization to weather, exposure, and human dimensions in addition to region specific climate trends. Most published studies on heat wave definitions look to model or develop the core metrics, threshold, and durations to define event occurrence. In this study we ask those on the ground for insight into heat wave event definition. Through a simple survey conducted across the United States in 2020 we look at what definitions each respondent is currently using, what variables they see as critical, impacts of differing definitions, and how relative definition approaches are used in their work. Overall, it was hypothesized that (1) emergency managers would have a different perspective on heat wave definition, especially as it relates to human exposure and acclimatation, (2) differing local to regional climate trends across the country would result in different definitions north to south across the U.S., and (3) only atmospheric variables would be considered in respondents’ heat wave definitions.

2. Data and methods

2.a. survey development.

The survey was designed to see how those involved in heat wave forecasting and management define such events and what factors contribute to their definition. The 11-question survey also included space for respondents to provide additional text and information to clarify their definitions and provide other important details. The survey was developed, tested, and IRB approved at Michigan State University and the University of Alabama. Respondents, none of whom were minors, were informed of survey privacy and provided consent language prior to taken the survey. All respondents provided written consent at the onset of the survey. The survey was developed within Qualtrics and consisted of 11 questions ( Fig 1 ).

PPT PowerPoint slide
PNG larger image
TIFF original image

https://doi.org/10.1371/journal.pclm.0000468.g001

The survey commences with questions of occupation, geographic extent of work, and zip code of residence ( Fig 1 ). Question 4–11 are, using free response, asking how, in their professional capacity they, define heat wave events. Additionally, we ask: what variables are associated with the definition (in case this is not mentioned in the free response), what non-atmospheric variables are included, and looks to understand how event duration, size, and time of year factor into their definition.

The survey was open for approximately five months. Distribution of the survey occurred through emails to: (1) state climatologists listserv, (2) emergency management associations, (3) a contact list for all state-recognized emergency managers, and (4) the American Meteorological Association listserv. Overall, the survey was disseminated through associations and email groups related to the parties of interest.

2.b. Data processing and analysis

Once the survey period was completed the Qualtrics survey was closed, and the data were downloaded locally. To clean the data, we first looked at the completeness of responses. Overall, 137 individuals fully completed the survey and 25 partially completed the survey. It was decided if a respondent answered at least half of questions 4–11 (see Fig 1 ) that these would still be included in the analysis. Other responses were removed from the analysis.

Data analysis occurred in several ways. Text mining and pattern analysis were conducted within R [ 23 ]. Initially, analysis began with simple frequency counts of factors such as geographic extent of work, field of work, etc. Next, for questions four and five, text mining was conducted using several R packages including SnowballC [ 24 ]. First, by question, the responses were merged into a corporal collection of phrases containing natural text. From there the tm_map function was used to remove symbols, number, punctuation, and common words (i.e., cause, the, and, is, have, are, was, be, of) from each survey response entry. The resulting text is not in sentence format but rather the key words within each response.

After the data were cleaned for text mining, analysis began with simple frequency counts. Using this simple statistic, the common terms or words utilized by the respondents were tallied by question. Terms or words used more than twice were preserved in the analysis. In addition to the frequency analysis, word clouds were constructed using the wordcloud package in R [ 14 ].

The word cloud represents the extremes of thought, terminology, and definition of heat waves. The word clouds also highlight the frequency in term utilization by the respondents. For the remaining questions simple summary statistics were completed in R.

3.a. Profile of survey respondents

We see large diversity in the response pool, both spatially and across profession. Overall,162 individuals took part in this project with 137 fully completing the survey. This corresponds to an 84.55% completion rate. The 162 responses came from across the US and across a wide variety of subfields related to climatology, emergency management, and meteorology. Spatially, responses were collected from 43 of the 50 states with Hawaii, South Carolina, Massachusetts, Rhode Islands, Connecticut, Virginia, and West Virginia being the exceptions. While there are no direct responses from those states, they are partially represented by those that work regionally, nationally, or at the global scale. While there are survey responses across the country there is a slight skew to the southern portion of the United States. For those respondents that work county to statewide we see the most responses from Alabama, Mississippi, Florida, Arizona, and Texas ( Fig 2 ). However, there are responses at this scale spread evenly across the U.S.

Dots represents those that work at the county level. Colored polygons represent those that work at the state level and the number of people that responded to the survey that work at the state level. Numbers represent those that responded that worked at another scale of geometry (e.g., city, community, multiple counties, townships, etc.). Not represented are those that worked globally (n = 8), nationwide (n = 4), and those that work sub-county level (city or community). Map created in ArcGIS using survey data and a states shapefile from the US Census Bureau ( https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html ).

https://doi.org/10.1371/journal.pclm.0000468.g002

The professional profile of those that completed the survey is diverse and does not skew to one group of individuals or those with certain specialties. Eighty-seven (53.7%) of the completed surveys came from those that identified forecasting as their occupation. Of those 62 worked across multiple counties in a single state, 11 at the state level, 7 at the regional (multiple states or parts of multiple states), 3 at the county level, 2 at an “other level” (city to local level), 1 globally, and 1 at the community level. After forecasting the next highest group of responses came from emergency managers. Overall, 46 (28.2%) emergency managers from across the US completed the survey. Of those emergency managers 32 worked at the county level, 2 worked across multiple counties, 1 worked at the regional scale (across multiple states), 1 worked at the national level, 1 worked at the community level, and the remaining 9 worked at other scales (e.g., tribal lands, city, ecoregion, etc.). Lastly, 29 individuals identified their career as “other”, including: researchers, retired state climatologists, climatologists, land managers, and those in academia. From this group the majority worked at the state to global scale.

3.b Defining heat waves

Text mining of survey responses showed interesting keyword usage patterns ( Fig 3 ). For the question “What is your definition of the climatological term ‘heat wave’”, heat was not the most used word, instead days was mentioned 71 times by respondents. Answers related to this term included definitions like: “Several days of 95+ degrees”, “A period of multiple days beyond normal temperatures”, and “At least 3 consecutive days of high maximum temperatures”. After days, the terms heat (n = 68), period (n = 65), normal (n = 47), and high (n = 42) were the other common terms ( Fig 3 ). Overall, this word usage pattern highlights a highly important and common thought pattern in defining heat waves, such extreme events have a duration aspect that needs to be considered and defined. Heat waves are multi-day extreme events.

Response trend word clouds and frequency counts for the questions: (A) “What is your definition of the climatological term heat wave?”, and (B) “What atmospheric variables are part of your definition of a heat wave?”, and (C) “What other atmospheric variables are part of your definition of a heat wave”.

https://doi.org/10.1371/journal.pclm.0000468.g003

When asked, “What atmospheric variables are part of your definition of a heat wave”, the most selected term was maximum temperature (n = 129) followed by heat index (n = 85), minimum temperature (n = 75), and humidity (n = 74). These four terms were far more common than the next terms of the list, average temperature (n = 45) and other (n = 25). Respondents were given a list of possible terms for this question and asked to select all that apply to their definition of heat waves. The terms included include maximum temperature, minimum temperature, average temperature, humidity, heat index, and other. This word usage pattern highlights that extreme terms are central to the definition of heat waves.

Lastly, respondents were asked “What other atmospheric variables are part of your definition of a heat wave?” This question was asked so respondents have free response, instead of choosing from a bank of options as with the previous questions, the climatological variables that they use in management, forecasting, and planning. It is important to note most respondents did not list any additional atmospheric variables as part of their heat wave temperature. Of those that did the most common response was wet bulb globe temperature (WBGT), wind, cloud cover, and insolation. A few respondents listed, repetitively, humidity and heat index, in response to this question. Overall, this word usage pattern highlights the strong pattern of traditional climatological variables in the definition of heat waves, rather than characteristics of land, demographics, human health, or exposure.

3.c Beyond climatological terms, other factors to considered in defining heat waves

Beyond traditional climatological terms we asked respondent “Does your definition of a heat wave include non-atmospheric variables?” Only 27.5% of respondents replied in the affirmative that they did include non-atmospheric variables in their definition of a heatwave ( Fig 4 ). The common non-atmospheric variables included in the heat wave definition were grouped into categories of: (1) impact on humans, (2) seasonal variation, (3) physical variables, and (4) other. The highest percentage of responses were within the impact to humans categories with 35.3%. Common non-atmospheric variables listed by respondents included: soil moisture, land type, duration, percent impervious surfaces, and crop stress. Most respondents that included non-atmospheric variables in their definition of heat waves were forecasters (n = 25), with only 5 emergency managers and those identified as “other” in their career including non-atmospheric variables. Geographically, those that included non-atmospheric variables mostly worked at the state to multi county scale (n = 22). Others that included non-atmospheric variables worked at varying geographic extents including county scale (n = 7), regional scale (n = 3), and global scale (n = 2). Spatially, that 27.5%, where not clusters in one portion of the US. Those that included non-atmospheric variables spanned from Arizona to Vermont.

Response trends to the questions: (A) “Does your definition of a heat wave include non-atmospheric variables?” and (B) “If yes, what other variables do you include?”.

https://doi.org/10.1371/journal.pclm.0000468.g004

Next, respondents were asked if their heat wave definition included a threshold that must be surpassed to be considered a heat wave and what threshold measurement they used. More than 56% of respondents said yes, their definition included a threshold that must be crossed. Of these 18 were emergency managers, 33 were forecasters, and 16 listed other as their career. As such, of the respondents that completed the survey approximately 38% of forecasters, 39% of emergency managers, and 55% of others included a threshold in their heat wave definition. Spatially, those that included a threshold worked across all geographic extents from community / local to global, though slightly more worked at the statewide or multicounty scale.

It was thought that the main threshold both emergency managers and forecasters would use in their definition would be related to temperature. Many of the provided definitions stated something like:

“Temperature above 95 degrees Fahrenheit for an extended period of time”

“A persistent anomaly in daily surface temperature usually many days about the 98 th percentile.”

Instead, survey results show that only 45.7% of respondents reported using temperature thresholds in their heat wave definitions, followed by heat index (27.9%), heat duration (23.5%), and heat risk ( Fig 5 ). Spatially, if we look at the common temperature threshold mentioned by survey respondents there is an interesting dynamic playing out ( Table 1 ). Those surveyed from the Northeast, Southwest, and Central regions of the United States, as defined by the NOAA climate regions, all listed a temperature threshold of 90°F with little variability. The lowest temperature threshold mentioned was 80°F and it was from a respondent in the south region. Whereas the highest threshold reported was 105°F, occurring in both the East North Central and South regions. Overall, a north to south temperature threshold gradient was expected, as the more southerly regions are subject to climatically higher mean temperatures, but this was not seen in the survey responses.

Response trends to the questions: (A)” Does your definition of a heat wave include a threshold that must be crossed in order to be considered a heat wave?”, and (B) “what is the threshold measurement that you use?”.

https://doi.org/10.1371/journal.pclm.0000468.g005

https://doi.org/10.1371/journal.pclm.0000468.t001

Fig 5B shows that numerous respondents suggest a threshold in their heat wave definition based on duration. During the survey respondents were specifically asked “Does your definition of a heat wave include a minimum duration”. This question was asked, in tandem with the aforementioned in case a respondent did not feel duration was a threshold measure. Over 74% of respondents said yes that event duration was a part of their heat wave definition. Of these 25 were emergency managers (54.3% of EM total population), 56 where forecasters (64.4% of F total population, and 19 listed their career as other. Importantly, the most noted durations for events were 2 and 3 days ( Fig 6 ). Overall, 22.2% or respondents mentioned 2 days as the minimum duration for a heat wave to be defined and 56.6% listed 3 days as the minimum duration. The response pattern of did not vary by geographic extent of work or the region respondents work in.

Response trends to the questions: (A) “Does your definition of a heat wave include a minimum duration?”, and (B)” If yes, how many days?”.

https://doi.org/10.1371/journal.pclm.0000468.g006

At this point, respondents had solely been asked about common aspects of heat wave definitions such as duration, intensity, etc. The next question of the survey asked, “Is size of area affected part of your definition of a heat wave?” Only 21.8% of respondents considered event size in their heat wave definition, equating to just 29 respondents. Of these there was an even split between emergency managers and forecasters, the geographic extent of their work was mostly single or multiple counties, and almost all included duration in their heat wave definition.

The majority of those that responded in the affirmative said that a heat wave had to have a duration of 2 or 3 days to be defined as such an event. The range of durations suggested by those surveyed was as short as 1 day (3 respondents) and as long as 10 days (1 respondent) ( Fig 7 ).

Response trends to the questions: (A) “Is size of area affected part of your definition of a heat wave?”, and (B) “If yes, how is size incorporated into your definition of a heatwave?”.

https://doi.org/10.1371/journal.pclm.0000468.g007

If a heat wave only impacts a small area, does it matter in emergency management and forecasting as much as large events? That question was the impetus for the next survey question, “is size of area affected part of your definition of a heat wave.” The vast majority (78.2%) of people said no, size does not matter. Those that answered yes to this question were almost all forecasters (except 2 out of 14). Those that answered in the affirmative had a clear theme, larger events impact more people compared to isolated events and that is why event size matters. With this theme several people mentioned that an event needs to be state-wide or span multiple counties (44.8% of the previous questions affirmative answers).

The last two questions of the survey look to see how respondents think about defining heat waves across space and time. Respondents were asked “Is your definition of a heat wave dependent on time of year?” This question was almost a 50/50 split in terms of percentages with 53% saying the definition is not time of year dependent and 47% saying it is ( Fig 8 ). For those that answered in the affirmative they were asked how their definition varies through the year. There were a wide range of responses to this follow up question from those mentioning seasonal threshold values to others saying only summer matters as it’s the warmest season. Overall, the consensus of those surveyed is well represented by one respondent’s remakes, “I generally only call things "heat waves" when it is “hot” outside. I wouldn’t call a winter stretch of warmer than normal weather a "heat wave" unless it was drastically warmer than normal.” There is a consistent trend in these types of answers with others saying outright “heat waves have only occurred climatologically in our late spring to early fall months, when it’s hottest”. However, there were a small group (labeled statistical in Fig 8 ) that again link back to thresholds and say, in their opinion, that heat waves can happen if the temperatures exceed the 95 th percentile for that given time frames normal.

Response trends to the questions: (A)” Is your definition of a heat wave dependent on time of year?”, and (B) “If yes, what time of year?”.

https://doi.org/10.1371/journal.pclm.0000468.g008

Lastly, respondents were asked if their definition of a heat wave varies across space. This question was asked to see if one core definition of heat waves cannot be developed and instead a regional definition would be needed. Overall, only 36.3% responded in the affirmative whereas 63.7% said their heat wave definition did not vary across space ( Fig 9 ). Those answering in the affirmative, that their definition varied across space, were then asked how with the options of: (A) regional differences, (B) type of territory, (C) statistically, (D) varies by the gridded data, and (E) other. Overall, 42.9% of respondents that thought heat wave definitions should vary across space thought there were regional differences. When asked to explain their answer respondents had a variety of answers and mentioned topic such as: “definitions can be different across climate regions because citizens are acclimated to different levels” and “In my work the western high plains should have a higher threshold for defining heat waves than the eastern portion of the Southern Plains”, and “Since humans can become acclimated to "normal" conditions, I would vary the definition based upon a certain amount above climatology”. Next, 22.5% of respondent said heat wave definitions should vary because of type of territory. Those that responded in this manner mentioned: “amount of vegetation vs bare ground is important” and “population density and impervious surfaces need to be considered to account for urban heat island effects”. Third, 22.5% said statistically heat wave definitions should vary spatially. These respondents provided further understanding of their response with comments such as: “Heat wave might be the top 0.1% of high temperatures of all time for each location” and “temperature percentile varies by location”. Lastly, only 6.1% of respondents said their definition varies over space because of gridded data. Specifically, those that answered in this manner added comments like: “heat risk is calculated on a spatial grid by entire forecast area” and “gridded temperature data is 2.5 km resolution—so the values for defining the heat wave vary spatially even if the definition remains the same.” Overall, these responses highlight a need to better understand duration, thresholds, and the underlaying population exposure.

Response trends to the questions: (A) “Does your definition of a heat wave vary across space?”, and (B) “How does it vary spatially?”.

https://doi.org/10.1371/journal.pclm.0000468.g009

4. Discussion and conclusions

Heat waves are regarded by the U.S. National Weather Service as a major, in fact leading, cause of weather-related fatalities in the U.S. in most years [ 18 ]. Closing the gap on a formal heat wave definition is important because of such direct affect that these extreme events can induce. A formal definition is important to multiple subfields, beyond emergency management and forecasting, as in the literature it has been consistently noted that the direct adverse effects of heat waves include (1) increased demands for water and electricity [ 25 – 28 ], (2) reduced productivity and overall labor efficiency [ 25 ], (3) drought and overall crop stress or failure, and (4) human health, including cardiovascular and respiratory damage in addition to death. Further, in the literature heat-related studies have been published related to land management, policy development, vulnerability/exposure regarding human health, urban development, emergency planning, and ecosystem health.

Overall, our population is vulnerable to heat waves and therefore we need a better understanding of these extreme events. For instance, the impacts of heat waves on human health are widely documented regarding mortality and morbidity [ 29 , 30 ]. That said, heat wave vulnerability is unequal and unevenly distributed across both human and natural landscapes. The elderly, those residing in nursing homes, and the chronically ill are readily identifiable as susceptible subgroups, high at risk for extreme heat impacts, and studies have criticized the lack of effective heat management for such populations [ 31 – 35 ]. These populations are not the only ones impacted by heat waves and the way such events are managed. Children have a higher sensitivity, outdoor workers have more extensive exposure, and the homeless are just a few of the other vulnerable populations [ 36 – 40 ].

Heat related deaths occur when a rapid temperature increase outpaces the body’s ability to cool itself, though perspiration and increased blood circulation [ 41 ]. There are compounding factors to such mortality risk including high humidity and overall exposure. If we examine the characteristics of heat waves mentioned throughout this study and our survey (duration, intensity, timing, size) its known that these factors negatively impact public health by increasing the risk of heat-related mortality [ 41 ]. For instance, long duration heat events increase exposures, elevate even the daily minimum temperature, and limit the body’s ability to recover [ 17 , 27 ]. Additionally, large-scale events are more likely to expose broader human populations to such extremes, increasing the population vulnerable to such an event. Lastly, the timing of an event can have multiple implications including (1) early extreme events can result in large populations of people unprepared and (2) events during peak summer months can be assumed to be more intense in nature. Beyond health impacts, the timing of such events can alter soil available moisture, impacting plant phenology and productivity. With such changes in soil moisture there are not only connections to widespread drought but also to impacts of cropping and overall harvest quality and quantity, again linking back to human impacts. Overall, in the US, the Center for Disease Control (CDC) estimates that heat related deaths average over 1000 per year [ 42 ]. However, the impacts of extreme heat stretch far beyond the health ramifications and therefore needs to be studied and defined more holistically.

How can a formal unified heat wave definition impact emergency management? Simply, with the development of a formal definition and associated research emergency managers and forecasters can know what to expect, outcome wise and in knowing the vulnerable populations, and plan for during extreme heat events. Much research has gone into analyzing heat wave definitions to see how they align with differing outcome patterns, particularly human health impacts. For instance, Knowlton et al, analyzed heat wave induced hospitalizations and emergency room visits resulting from the 2006 California extreme event and found that a definition based on a higher maximum temperature threshold was associated with a greater relative risk for hospital admission [ 43 , 44 ]. Other studies have tested heat wave definition to decide the ideal scenario for the opening of cooling centers and to consider moving vulnerable people to safer locations [ 45 , 46 ]. Beyond hospitalization and evacuation with a formal heat wave definition managers can understand, through research of trends, what to expect for certain mortality risks. For example, Dong et al. 2016 found that a heat wave definition using the 93 rd percentile of maximum temperature and 5 day event duration was the best way to understand the trends between extreme heat and cardiovascular mortality [ 47 ]. These examples, explicitly related to health outcomes and exposure due to extreme heat are just a few ways in which emergency management could use such information and data.

We live in a warming climate, and with such changing regimes it’s projected that the duration, intensity, and frequency of heat waves will increase [ 48 ]. From the literature and survey, we can surmise a few key aspects of a formal heat wave definition. Factors that need to be considered in the formal definition include duration, intensity (a climatological threshold), and exposure. Survey results show that the majority respondents report a heat wave can be defined as an event that lasts at least 3 days. The mean duration listed by survey respondents was 3.14, the mode was 3 days, and the median was 3 days as well. Overall, the durations listed by respondents ranged from 1–10 days. Regardless of occupation and geographic extent of work most respondents felt a threshold value needed to be crossed for a heat wave to formally be defined. Both percentiles and numeric thresholds were suggested. Additionally, a large group of respondents felt that maximum temperature was the key metric to develop such a threshold. For those that wanted to include percentiles in their heat wave definition the 95 th percentile was the most noted. In terms of absolute temperatures there was a large range of possible thresholds suggested, 90–105 degrees. The mean temperature threshold suggested by respondents was 94.6 degrees. Interestingly, there were no survey response trends suggesting for spatial variation in heat wave definitions. In terms of time of year, most felt that heat waves would only be defined in the hottest months and are not solely associated with any monthly or seasonal temperatures beyond the normal. One of the most important non-climatological aspects brought up by survey respondents was exposure and associated population density. Such survey results do not solidify a formal heat wave definition, but they do show extremely similar thought processes across different occupations and regions of the US. With such similarities in defining heat waves there is no doubt similar patterns of use for such data in emergency management and forecasting, and perhaps for a future unified definition of heat waves.

Supporting information

S1 data. deidentified survey data collected including the full irb information and question in the header of the data file..

https://doi.org/10.1371/journal.pclm.0000468.s001

Acknowledgments

Multiple people and organizations made this publication possible, and the authors wish to express their gratitude. The authors wish to especially thank Dr. Laura Myers and Jacob Reed at the University of Alabama, Dan Wanyama, and the staff of Remote Sensing and GIS Research and Outreach Services (RS&GIS) at Michigan State University. Additionally, we thank the reviewers for their time and effort put into manuscript review.

1. Lee H, Calvin K, Dasgupta D, Krinner G, Mukherji A, Thorne P, et al. IPCC, 2023: Climate Change 2023: Synthesis Report, Summary for Policymakers. Contribution of Working Groups I, II and III to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change [Core Writing Team, Lee H. and Romero J. (eds.)]. IPCC, Geneva, Switzerland. 2023 URL: https://mural.maynoothuniversity.ie/17886/
View Article
Google Scholar
4. Klinenberg E. Heat wave: A social autopsy of disaster in Chicago. University of Chicago press; 2015.
PubMed/NCBI
23. R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2023. Available: https://www.R-project.org/
31. Brown S, Walker G. Understanding heat wave vulnerability in nursing and residential homes. Comfort in a Lower Carbon Society. Routledge; 2013. pp. 59–68.
48. Stocker TF, Qin D, Plattner G-K, Tignor MM, Allen SK, Boschung J, et al. Climate Change 2013: The physical science basis. contribution of working group I to the fifth assessment report of IPCC the intergovernmental panel on climate change. 2014.

COMMENTS

Methods for the synthesis of qualitative research: a critical review
Background. The range of different methods for synthesising qualitative research has been growing over recent years [1,2], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [].While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe ...
Metasynthesis: An Original Method to Synthesize Qualitative Literature
Metasynthesis—the systematic review and integration of findings from qualitative studies—is an emerging technique in medical research that can use many different methods. Nevertheless, the method must be appropriate to the specific scientific field in which it is used. The objective is to describe the steps of a metasynthesis method adapted ...
Qualitative research synthesis: An appreciative and critical
Qualitative research synthesis is a diverse set of methods for combining the data or the results of multiple studies on a topic to generate new knowledge, theory and applications. Use of qualitative research synthesis is rapidly expanding across disciplines. Aggregative and interpretive models of qualitative research synthesis are defined and ...
Qualitative Evidence Synthesis: Where Are We at?
Through a QES, evidence is synthesized from primary qualitative studies with the aim of developing new cumulative knowledge. This differs to a more traditional literature review of qualitative research which seeks to combine studies in a summary format (Flemming & Jones, 2020).Depending on the QES method selected, the process can enable researchers to "go beyond" the individual findings of ...
Synthesizing Qualitative Research: Reflections and Lessons Learnt by
Synthesizing qualitative research has become a useful and popular tool to inform policy- and evidence-based health care in recent years (Noyes, Popay, Pearson, Hannes, & Booth, 2008; Pope & Mays, 2006a).Systematic reviews can prove invaluable for busy practitioners as they combine results from many studies, provide up-to-date summarized evidence, and disseminate them in an unbiased and ...
Synthesizing Qualitative Research
Writing about synthesis methods included the process of choosing be-tween different approaches, selecting what would be appropriate for this ... qualitative literature. We believe that these approaches will generate interest from the international community of researchers, practitioners and policy- ...
What Synthesis Methodology Should I Use? A Review and Analysis of
In fact, the lack of systematization in conventional literature synthesis was the reason for the development of more systematic quantitative , and qualitative -, approaches. Some authors in the field [18] have clarified processes for integrative reviews making them more systematic and rigorous, but most conventional syntheses remain ...
What is a qualitative synthesis?
A qualitative synthesis (sometimes called a qualitative systematic review) systematically searches for research on a topic, and draws the findings from individual studies together. Although systematic reviews of quantitative data are well established, for qualitative research, this is a fairly new field, and methods are still developing.
Meta-synthesis of Qualitative Research
A meta-synthesis of qualitative health research is a structured approach to analyzing primary data across the findings sections of published peer-reviewed papers reporting qualitative research. The analysis of this evidence involves qualitative and sometime quantitative methods with the aim of improving health outcomes, research, services, or ...
Literature Synthesis
The Literature Synthesis can be carried out in either qualitative or quantitative research (Koricheva and Gurevitch 2013). Therefore, the dissimilarities between them are fundamental for selecting the most suitable synthesis technique (Morandi and Camargo 2015 ).
Literature Synthesis 101: How To Guide
In this post, we'll unpack what exactly synthesis means and show you how to craft a strong literature synthesis using practical examples. This post is based on our popular online course, Literature Review Bootcamp. In the course, we walk you through the full process of developing a literature review, step by step.
Methods for Qualitative Analysis and Synthesis
This chapter provides this detail on five specific methods for qualitative synthesis; note that some of the guidance may also be applicable to other methods and literature reviews in general. Section 11.1 presents the method meta-summary, as an example of aggregative qualitative synthesis (see Section 10.2 ), and offers guidelines for ...
Full article: Narrative approaches to systematic review and synthesis
There is a debate in the literature as to what constitutes the quality in qualitative research (Spencer et al. Citation 2003), and this debate extends to the methodological literature on qualitative evidence synthesis (Hannes Citation 2011; see also Snilstveit, (Citation 2012)).
Methods for the thematic synthesis of qualitative research in
We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. ... Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretative synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006, 6: 35-10.1186/1471 ...
PDF Methods for the synthesis of qualitative research: a critical review
Amongst the many syntheses of qualitative research, nine distinct methods of synthesis were identified. These are: meta-narrative, critical interpretive synthesis, meta-study, meta-ethnography, grounded formal theory, thematic synthesis, textual narrative synthesis, framework synthesis and ecological triangulation.
Systematic Reviews & Evidence Synthesis Methods
Qualitative Synthesis. In a qualitative synthesis, you describe for readers how the pieces of your work fit together. You will summarize, compare, and contrast the characteristics and findings, exploring the relationships between them. Further, you will discuss the relevance and applicability of the evidence to your research question.
Meta-synthesis method for qualitative research: a literature review
Methods: Literature on the technique of meta-synthesis and examples of meta-synthesis papers were searched and reviewed. A meta-synthesis exercise was undertaken, and this informed reflection and critique of the method. Findings: Meta-synthesis attempts to integrate results from a number of different but inter-related qualitative studies.
Metasynthesis: An Original Method to Synthesize Qualitative Literature
Metaethnography, as well as Thematic Synthesis, takes place in six or seven steps from data collection to text coding and finally writing the synthesis. Original authors of metaethnography were trained in grounded theory, a qualitative method developed in the social sciences, laying on conceptual coding combine to construct a new theory.
Synthesizing Sources
Synthesizing Sources | Examples & Synthesis Matrix. Published on July 4, 2022 by Eoghan Ryan. Revised on May 31, 2023. Synthesizing sources involves ... This is a tool that you can use when researching and writing your paper, not a part of the final text. In a synthesis matrix, each column represents one source, and each row represents a common ...
Understanding Qualitative Metasynthesis: Issues and Opportunities in
Qualitative metasynthesis is an intentional and coherent approach to analyzing data across qualitative studies. It is a process that enables researchers to identify a specific research question and then search for, select, appraise, summarize, and combine qualitative evidence to address the research question.
Sitshange
Significant poverty levels raise critical questions about the impact of poverty eradication programs. Literature reviews play a critical role in highlighting impactful and ineffective socio-economic approaches. This article presents a review of nine qualitative studies that were reported between 2006 and 2013 in poor urban, semi-rural, and rural communities in South Africa. The main goal of ...
Synthesising quantitative and qualitative research in evidence‐based
Synthesis of qualitative evidence . Qualitative studies addressing one particular information point were collated and assigned an initial level of evidence (high ++, low + or very low −) on the basis of the lowest checklist quality score obtained for any study in the group ( (table table 1, box 7 7).). The initial level of evidence was then ...
Family perspectives on and experiences with advance care planning in
The meta-synthesis focused on the challenges of ACP (Ryan et al., 2017), family members' roles in ACP (Johnson et al., 2016) and informal caregivers' experiences with ACP (Silies et al., 2022). Therefore, this qualitative meta-synthesis study was designed to synthesize family members' experiences with and perspectives on ACP in nursing homes.
Perceived factors influencing the success of pain neuroscience
Materials and Methods . We conducted a meta-synthesis of qualitative studies. Articles were found on MEDLINE via Pubmed, EMBASE, Cochrane Library, CINHAL, and PsycINFO up to April 2023. Eligible qualitative studies focussed on adults (>16 years old) with a diagnosis of chronic primary or secondary MSK pain who performed PNE.
Metasynthesis: An original method to synthesize qualitative literature
Background: Metasynthesis—the systematic review and integration of findings from qualitative studies—is an emerging technique in medical research that can use many different methods. Nevertheless, the method must be appropriate to the specific scientific field in which it is used. The objective is to describe the steps of a metasynthesis method adapted from Thematic Synthesis and ...
Methods for the synthesis of qualitative research: a critical review
The range of different methods for synthesising qualitative research has been growing over recent years [1, 2], alongside an increasing interest in qualitative synthesis to inform health-related policy and practice [].While the terms 'meta-analysis' (a statistical method to combine the results of primary studies), or sometimes 'narrative synthesis', are frequently used to describe how ...
A Systematic Review of Synthetic Data Generation Techniques ...
The methods proposed in the literature to generate synthetic data vary from large language models (LLMs), which are pre-trained on gigantic datasets, to generative adversarial networks (GANs) and variational autoencoders (VAEs). ... The synthesis methods involved the qualitative integration of results without formal meta-analysis or software ...
What is a heat wave: A survey and literature synthesis of heat wave
Heat waves are the last extreme weather events without a formal, on the books, definition. Instead, across the U.S. those working on extreme heat event management, forecasting, and planning are using differing definitions in their work. With such differing definitions being used there are widespread impacts including some to human and environmental health, natural resource management, and long ...
Gender-Based Violence in the Context of the Future of Work: A
Despite the emergence of a burgeoning literature on platform studies, little is known about whether and to what extent platform work exacerbates the risk of gender-based violence among workers. ... and conference papers published between 2016 and 2023 using qualitative content analysis to provide a preliminary understanding of empirical data on ...

Log in using your username and password

You are here

Statistics from Altmetric.com

Read the full text or download the PDF:

Literature Syntheis 101

Overview: Literature Synthesis

What does “synthesis” actually mean?

1. Points Of Agreement

Need a helping hand?

2. Points Of Disagreement

3. Key Theories

4. Contexts

5. Methodological Approaches

Bringing It All Together

Psst… there’s more!

Submit a Comment Cancel reply

Methods for Qualitative Analysis and Synthesis

Cite this chapter

Access this chapter

Author information

Rights and permissions

Copyright information

About this chapter

Download citation

Share this chapter

Systematic Reviews & Evidence Synthesis Methods

Summarizing vs. Synthesizing

Save citation to file

Add to My Bibliography

Meta-synthesis method for qualitative research: a literature review

Similar articles

Publication types

Related information

LinkOut - more resources

METHODS article

Conducting a Metasynthesis

Define the Research Question and the Selection Criteria

Study Selection

Quality Assessment of Included Studies

Extracting and Presenting the Formal Data

Data Analysis

Writing the Synthesis

Author Contributions

Conflict of Interest Statement

Acknowledgments

Have a language expert improve your writing

Synthesizing Sources | Examples & Synthesis Matrix

Table of contents

Scribbr Citation Checker New

Cite this Scribbr article

Is this article helpful?

Eoghan Ryan

Thematically Synthesizing the Qualitative Evidence Reporting the Impact of Poverty Alleviation Programs in Low-income Communities in South Africa: A Review

13 Downloads

Literature Review

Contextualization

Methodology

Results and Discussion

Strength-based interventions

Participation

Holistic and multi-dimensional approaches

Recommendations

Conclusions

Harvard-Style Citation

Vancouver-Style Citation

APA-Style Citation

Non Specialist Summary

Synthesising quantitative and qualitative research in evidence‐based patient information

Conclusions

Systematic reviews

Limitations of traditional hierarchies of evidence

Qualitative research

Importance of a range of evidence

Box 1 First example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

Box 2 Second example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

What is the evidence base?

Literature searching

Box 3 Third example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)

Initial citation and full study report assessments

Box 4 Fourth example of studies included in an updated review of evidence‐based guidelines for the content of NHSCSP letters and leaflets (the studies described were focused on colposcopy information)