examples of quantitative health research

An official website of the United States government

Here's how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Common Data Types in Public Health Research

Quantitative data.

• Quantitative data is measurable, often used for comparisons, and involves counting of people, behaviors, conditions, or other discrete events (Wang, 2013).
• Quantitative data uses numbers to determine the what, who, when, and where of health-related events (Wang, 2013).
• Examples of quantitative data include: age, weight, temperature, or the number of people suffering from diabetes.

Qualitative Data

• Qualitative data is a broad category of data that can include almost any non-numerical data.
• Qualitative data uses words to describe a particular health-related event (Romano).
• This data can be observed, but not measured.
• Involves observing people in selected places and listening to discover how they feel and why they might feel that way (Wang, 2013).
• Examples of qualitative data include: male/female, smoker/non-smoker, or questionnaire response (agree, disagree, neutral).
• Measuring organizational change.
• Measures of clinical leadership in implementing evidence-based guidelines.
• Patient perceptions of quality of care.

Data Sources

Primary data sources.

• Primary data analysis in which the same individual or team of researchers designs, collects, and analyzes the data, for the purpose of answering a research question (Koziol & Arthur, nd).

Advantages to Using Primary Data

• You collect exactly the data elements that you need to answer your research question (Romano).
• You can test an intervention, such as an experimental drug or an educational program, in the purest way (a double-blind randomized controlled trial (Romano).
• You control the data collection process, so you can ensure data quality, minimize the number of missing values, and assess the reliability of your instruments (Romano).

Secondary Data Sources

• Existing data collected for another purposes, that you use to answer your research question (Romano).

Advantages of Working with Secondary Data

• Large samples
• Can provide population estimates : for example state data can be combined across states to get national estimates (Shaheen, Pan, & Mukherjee).
• Less expensive to collect than primary data (Romano)
• It takes less time to collect secondary data (Romano).
• You may not need to worry about informed consent, human subjects restriction (Romano).

Issues in Using Secondary Data

• Study design and data collection already completed (Koziol & Arthur, nd).
• Data may not facilitate particular research question o Information regarding study design and data collection procedures may be scarce.
• Data may potentially lack depth (the greater the breadth the harder it is to measure any one construct in depth) (Koziol & Arthur, nd).
• Certain fields or departments (e.g., experimental programs) may place less value on secondary data analysis (Koziol & Arthur, nd).
• Often requires special techniques to analyze statistically the data.

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Recent quantitative research on determinants of health in high income countries: A scoping review

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Centre for Health Economics Research and Modelling Infectious Diseases, Vaccine and Infectious Disease Institute, University of Antwerp, Antwerp, Belgium

Roles Conceptualization, Data curation, Funding acquisition, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing

• Vladimira Varbanova, 
• Philippe Beutels

• Published: September 17, 2020
• https://doi.org/10.1371/journal.pone.0239031
• Peer Review
• Reader Comments

Identifying determinants of health and understanding their role in health production constitutes an important research theme. We aimed to document the state of recent multi-country research on this theme in the literature.

We followed the PRISMA-ScR guidelines to systematically identify, triage and review literature (January 2013—July 2019). We searched for studies that performed cross-national statistical analyses aiming to evaluate the impact of one or more aggregate level determinants on one or more general population health outcomes in high-income countries. To assess in which combinations and to what extent individual (or thematically linked) determinants had been studied together, we performed multidimensional scaling and cluster analysis.

Sixty studies were selected, out of an original yield of 3686. Life-expectancy and overall mortality were the most widely used population health indicators, while determinants came from the areas of healthcare, culture, politics, socio-economics, environment, labor, fertility, demographics, life-style, and psychology. The family of regression models was the predominant statistical approach. Results from our multidimensional scaling showed that a relatively tight core of determinants have received much attention, as main covariates of interest or controls, whereas the majority of other determinants were studied in very limited contexts. We consider findings from these studies regarding the importance of any given health determinant inconclusive at present. Across a multitude of model specifications, different country samples, and varying time periods, effects fluctuated between statistically significant and not significant, and between beneficial and detrimental to health.

Conclusions

We conclude that efforts to understand the underlying mechanisms of population health are far from settled, and the present state of research on the topic leaves much to be desired. It is essential that future research considers multiple factors simultaneously and takes advantage of more sophisticated methodology with regards to quantifying health as well as analyzing determinants’ influence.

Citation: Varbanova V, Beutels P (2020) Recent quantitative research on determinants of health in high income countries: A scoping review. PLoS ONE 15(9): e0239031. https://doi.org/10.1371/journal.pone.0239031

Editor: Amir Radfar, University of Central Florida, UNITED STATES

Received: November 14, 2019; Accepted: August 28, 2020; Published: September 17, 2020

Copyright: © 2020 Varbanova, Beutels. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files.

Funding: This study (and VV) is funded by the Research Foundation Flanders ( https://www.fwo.be/en/ ), FWO project number G0D5917N, award obtained by PB. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Identifying the key drivers of population health is a core subject in public health and health economics research. Between-country comparative research on the topic is challenging. In order to be relevant for policy, it requires disentangling different interrelated drivers of “good health”, each having different degrees of importance in different contexts.

“Good health”–physical and psychological, subjective and objective–can be defined and measured using a variety of approaches, depending on which aspect of health is the focus. A major distinction can be made between health measurements at the individual level or some aggregate level, such as a neighborhood, a region or a country. In view of this, a great diversity of specific research topics exists on the drivers of what constitutes individual or aggregate “good health”, including those focusing on health inequalities, the gender gap in longevity, and regional mortality and longevity differences.

The current scoping review focuses on determinants of population health. Stated as such, this topic is quite broad. Indeed, we are interested in the very general question of what methods have been used to make the most of increasingly available region or country-specific databases to understand the drivers of population health through inter-country comparisons. Existing reviews indicate that researchers thus far tend to adopt a narrower focus. Usually, attention is given to only one health outcome at a time, with further geographical and/or population [ 1 , 2 ] restrictions. In some cases, the impact of one or more interventions is at the core of the review [ 3 – 7 ], while in others it is the relationship between health and just one particular predictor, e.g., income inequality, access to healthcare, government mechanisms [ 8 – 13 ]. Some relatively recent reviews on the subject of social determinants of health [ 4 – 6 , 14 – 17 ] have considered a number of indicators potentially influencing health as opposed to a single one. One review defines “social determinants” as “the social, economic, and political conditions that influence the health of individuals and populations” [ 17 ] while another refers even more broadly to “the factors apart from medical care” [ 15 ].

In the present work, we aimed to be more inclusive, setting no limitations on the nature of possible health correlates, as well as making use of a multitude of commonly accepted measures of general population health. The goal of this scoping review was to document the state of the art in the recent published literature on determinants of population health, with a particular focus on the types of determinants selected and the methodology used. In doing so, we also report the main characteristics of the results these studies found. The materials collected in this review are intended to inform our (and potentially other researchers’) future analyses on this topic. Since the production of health is subject to the law of diminishing marginal returns, we focused our review on those studies that included countries where a high standard of wealth has been achieved for some time, i.e., high-income countries belonging to the Organisation for Economic Co-operation and Development (OECD) or Europe. Adding similar reviews for other country income groups is of limited interest to the research we plan to do in this area.

In view of its focus on data and methods, rather than results, a formal protocol was not registered prior to undertaking this review, but the procedure followed the guidelines of the PRISMA statement for scoping reviews [ 18 ].

We focused on multi-country studies investigating the potential associations between any aggregate level (region/city/country) determinant and general measures of population health (e.g., life expectancy, mortality rate).

Within the query itself, we listed well-established population health indicators as well as the six world regions, as defined by the World Health Organization (WHO). We searched only in the publications’ titles in order to keep the number of hits manageable, and the ratio of broadly relevant abstracts over all abstracts in the order of magnitude of 10% (based on a series of time-focused trial runs). The search strategy was developed iteratively between the two authors and is presented in S1 Appendix . The search was performed by VV in PubMed and Web of Science on the 16 th of July, 2019, without any language restrictions, and with a start date set to the 1 st of January, 2013, as we were interested in the latest developments in this area of research.

Eligibility criteria

Records obtained via the search methods described above were screened independently by the two authors. Consistency between inclusion/exclusion decisions was approximately 90% and the 43 instances where uncertainty existed were judged through discussion. Articles were included subject to meeting the following requirements: (a) the paper was a full published report of an original empirical study investigating the impact of at least one aggregate level (city/region/country) factor on at least one health indicator (or self-reported health) of the general population (the only admissible “sub-populations” were those based on gender and/or age); (b) the study employed statistical techniques (calculating correlations, at the very least) and was not purely descriptive or theoretical in nature; (c) the analysis involved at least two countries or at least two regions or cities (or another aggregate level) in at least two different countries; (d) the health outcome was not differentiated according to some socio-economic factor and thus studied in terms of inequality (with the exception of gender and age differentiations); (e) mortality, in case it was one of the health indicators under investigation, was strictly “total” or “all-cause” (no cause-specific or determinant-attributable mortality).

Data extraction

The following pieces of information were extracted in an Excel table from the full text of each eligible study (primarily by VV, consulting with PB in case of doubt): health outcome(s), determinants, statistical methodology, level of analysis, results, type of data, data sources, time period, countries. The evidence is synthesized according to these extracted data (often directly reflected in the section headings), using a narrative form accompanied by a “summary-of-findings” table and a graph.

Search and selection

The initial yield contained 4583 records, reduced to 3686 after removal of duplicates ( Fig 1 ). Based on title and abstract screening, 3271 records were excluded because they focused on specific medical condition(s) or specific populations (based on morbidity or some other factor), dealt with intervention effectiveness, with theoretical or non-health related issues, or with animals or plants. Of the remaining 415 papers, roughly half were disqualified upon full-text consideration, mostly due to using an outcome not of interest to us (e.g., health inequality), measuring and analyzing determinants and outcomes exclusively at the individual level, performing analyses one country at a time, employing indices that are a mixture of both health indicators and health determinants, or not utilizing potential health determinants at all. After this second stage of the screening process, 202 papers were deemed eligible for inclusion. This group was further dichotomized according to level of economic development of the countries or regions under study, using membership of the OECD or Europe as a reference “cut-off” point. Sixty papers were judged to include high-income countries, and the remaining 142 included either low- or middle-income countries or a mix of both these levels of development. The rest of this report outlines findings in relation to high-income countries only, reflecting our own primary research interests. Nonetheless, we chose to report our search yield for the other income groups for two reasons. First, to gauge the relative interest in applied published research for these different income levels; and second, to enable other researchers with a focus on determinants of health in other countries to use the extraction we made here.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0239031.g001

Health outcomes

The most frequent population health indicator, life expectancy (LE), was present in 24 of the 60 studies. Apart from “life expectancy at birth” (representing the average life-span a newborn is expected to have if current mortality rates remain constant), also called “period LE” by some [ 19 , 20 ], we encountered as well LE at 40 years of age [ 21 ], at 60 [ 22 ], and at 65 [ 21 , 23 , 24 ]. In two papers, the age-specificity of life expectancy (be it at birth or another age) was not stated [ 25 , 26 ].

Some studies considered male and female LE separately [ 21 , 24 , 25 , 27 – 33 ]. This consideration was also often observed with the second most commonly used health index [ 28 – 30 , 34 – 38 ]–termed “total”, or “overall”, or “all-cause”, mortality rate (MR)–included in 22 of the 60 studies. In addition to gender, this index was also sometimes broken down according to age group [ 30 , 39 , 40 ], as well as gender-age group [ 38 ].

While the majority of studies under review here focused on a single health indicator, 23 out of the 60 studies made use of multiple outcomes, although these outcomes were always considered one at a time, and sometimes not all of them fell within the scope of our review. An easily discernable group of indices that typically went together [ 25 , 37 , 41 ] was that of neonatal (deaths occurring within 28 days postpartum), perinatal (fetal or early neonatal / first-7-days deaths), and post-neonatal (deaths between the 29 th day and completion of one year of life) mortality. More often than not, these indices were also accompanied by “stand-alone” indicators, such as infant mortality (deaths within the first year of life; our third most common index found in 16 of the 60 studies), maternal mortality (deaths during pregnancy or within 42 days of termination of pregnancy), and child mortality rates. Child mortality has conventionally been defined as mortality within the first 5 years of life, thus often also called “under-5 mortality”. Nonetheless, Pritchard & Wallace used the term “child mortality” to denote deaths of children younger than 14 years [ 42 ].

As previously stated, inclusion criteria did allow for self-reported health status to be used as a general measure of population health. Within our final selection of studies, seven utilized some form of subjective health as an outcome variable [ 25 , 43 – 48 ]. Additionally, the Health Human Development Index [ 49 ], healthy life expectancy [ 50 ], old-age survival [ 51 ], potential years of life lost [ 52 ], and disability-adjusted life expectancy [ 25 ] were also used.

We note that while in most cases the indicators mentioned above (and/or the covariates considered, see below) were taken in their absolute or logarithmic form, as a—typically annual—number, sometimes they were used in the form of differences, change rates, averages over a given time period, or even z-scores of rankings [ 19 , 22 , 40 , 42 , 44 , 53 – 57 ].

Regions, countries, and populations

Despite our decision to confine this review to high-income countries, some variation in the countries and regions studied was still present. Selection seemed to be most often conditioned on the European Union, or the European continent more generally, and the Organisation of Economic Co-operation and Development (OECD), though, typically, not all member nations–based on the instances where these were also explicitly listed—were included in a given study. Some of the stated reasons for omitting certain nations included data unavailability [ 30 , 45 , 54 ] or inconsistency [ 20 , 58 ], Gross Domestic Product (GDP) too low [ 40 ], differences in economic development and political stability with the rest of the sampled countries [ 59 ], and national population too small [ 24 , 40 ]. On the other hand, the rationales for selecting a group of countries included having similar above-average infant mortality [ 60 ], similar healthcare systems [ 23 ], and being randomly drawn from a social spending category [ 61 ]. Some researchers were interested explicitly in a specific geographical region, such as Eastern Europe [ 50 ], Central and Eastern Europe [ 48 , 60 ], the Visegrad (V4) group [ 62 ], or the Asia/Pacific area [ 32 ]. In certain instances, national regions or cities, rather than countries, constituted the units of investigation instead [ 31 , 51 , 56 , 62 – 66 ]. In two particular cases, a mix of countries and cities was used [ 35 , 57 ]. In another two [ 28 , 29 ], due to the long time periods under study, some of the included countries no longer exist. Finally, besides “European” and “OECD”, the terms “developed”, “Western”, and “industrialized” were also used to describe the group of selected nations [ 30 , 42 , 52 , 53 , 67 ].

As stated above, it was the health status of the general population that we were interested in, and during screening we made a concerted effort to exclude research using data based on a more narrowly defined group of individuals. All studies included in this review adhere to this general rule, albeit with two caveats. First, as cities (even neighborhoods) were the unit of analysis in three of the studies that made the selection [ 56 , 64 , 65 ], the populations under investigation there can be more accurately described as general urban , instead of just general. Second, oftentimes health indicators were stratified based on gender and/or age, therefore we also admitted one study that, due to its specific research question, focused on men and women of early retirement age [ 35 ] and another that considered adult males only [ 68 ].

Data types and sources

A great diversity of sources was utilized for data collection purposes. The accessible reference databases of the OECD ( https://www.oecd.org/ ), WHO ( https://www.who.int/ ), World Bank ( https://www.worldbank.org/ ), United Nations ( https://www.un.org/en/ ), and Eurostat ( https://ec.europa.eu/eurostat ) were among the top choices. The other international databases included Human Mortality [ 30 , 39 , 50 ], Transparency International [ 40 , 48 , 50 ], Quality of Government [ 28 , 69 ], World Income Inequality [ 30 ], International Labor Organization [ 41 ], International Monetary Fund [ 70 ]. A number of national databases were referred to as well, for example the US Bureau of Statistics [ 42 , 53 ], Korean Statistical Information Services [ 67 ], Statistics Canada [ 67 ], Australian Bureau of Statistics [ 67 ], and Health New Zealand Tobacco control and Health New Zealand Food and Nutrition [ 19 ]. Well-known surveys, such as the World Values Survey [ 25 , 55 ], the European Social Survey [ 25 , 39 , 44 ], the Eurobarometer [ 46 , 56 ], the European Value Survey [ 25 ], and the European Statistics of Income and Living Condition Survey [ 43 , 47 , 70 ] were used as data sources, too. Finally, in some cases [ 25 , 28 , 29 , 35 , 36 , 41 , 69 ], built-for-purpose datasets from previous studies were re-used.

In most of the studies, the level of the data (and analysis) was national. The exceptions were six papers that dealt with Nomenclature of Territorial Units of Statistics (NUTS2) regions [ 31 , 62 , 63 , 66 ], otherwise defined areas [ 51 ] or cities [ 56 ], and seven others that were multilevel designs and utilized both country- and region-level data [ 57 ], individual- and city- or country-level [ 35 ], individual- and country-level [ 44 , 45 , 48 ], individual- and neighborhood-level [ 64 ], and city-region- (NUTS3) and country-level data [ 65 ]. Parallel to that, the data type was predominantly longitudinal, with only a few studies using purely cross-sectional data [ 25 , 33 , 43 , 45 – 48 , 50 , 62 , 67 , 68 , 71 , 72 ], albeit in four of those [ 43 , 48 , 68 , 72 ] two separate points in time were taken (thus resulting in a kind of “double cross-section”), while in another the averages across survey waves were used [ 56 ].

In studies using longitudinal data, the length of the covered time periods varied greatly. Although this was almost always less than 40 years, in one study it covered the entire 20 th century [ 29 ]. Longitudinal data, typically in the form of annual records, was sometimes transformed before usage. For example, some researchers considered data points at 5- [ 34 , 36 , 49 ] or 10-year [ 27 , 29 , 35 ] intervals instead of the traditional 1, or took averages over 3-year periods [ 42 , 53 , 73 ]. In one study concerned with the effect of the Great Recession all data were in a “recession minus expansion change in trends”-form [ 57 ]. Furthermore, there were a few instances where two different time periods were compared to each other [ 42 , 53 ] or when data was divided into 2 to 4 (possibly overlapping) periods which were then analyzed separately [ 24 , 26 , 28 , 29 , 31 , 65 ]. Lastly, owing to data availability issues, discrepancies between the time points or periods of data on the different variables were occasionally observed [ 22 , 35 , 42 , 53 – 55 , 63 ].

Health determinants

Together with other essential details, Table 1 lists the health correlates considered in the selected studies. Several general categories for these correlates can be discerned, including health care, political stability, socio-economics, demographics, psychology, environment, fertility, life-style, culture, labor. All of these, directly or implicitly, have been recognized as holding importance for population health by existing theoretical models of (social) determinants of health [ 74 – 77 ].

https://doi.org/10.1371/journal.pone.0239031.t001

It is worth noting that in a few studies there was just a single aggregate-level covariate investigated in relation to a health outcome of interest to us. In one instance, this was life satisfaction [ 44 ], in another–welfare system typology [ 45 ], but also gender inequality [ 33 ], austerity level [ 70 , 78 ], and deprivation [ 51 ]. Most often though, attention went exclusively to GDP [ 27 , 29 , 46 , 57 , 65 , 71 ]. It was often the case that research had a more particular focus. Among others, minimum wages [ 79 ], hospital payment schemes [ 23 ], cigarette prices [ 63 ], social expenditure [ 20 ], residents’ dissatisfaction [ 56 ], income inequality [ 30 , 69 ], and work leave [ 41 , 58 ] took center stage. Whenever variables outside of these specific areas were also included, they were usually identified as confounders or controls, moderators or mediators.

We visualized the combinations in which the different determinants have been studied in Fig 2 , which was obtained via multidimensional scaling and a subsequent cluster analysis (details outlined in S2 Appendix ). It depicts the spatial positioning of each determinant relative to all others, based on the number of times the effects of each pair of determinants have been studied simultaneously. When interpreting Fig 2 , one should keep in mind that determinants marked with an asterisk represent, in fact, collectives of variables.

Groups of determinants are marked by asterisks (see S1 Table in S1 Appendix ). Diminishing color intensity reflects a decrease in the total number of “connections” for a given determinant. Noteworthy pairwise “connections” are emphasized via lines (solid-dashed-dotted indicates decreasing frequency). Grey contour lines encircle groups of variables that were identified via cluster analysis. Abbreviations: age = population age distribution, associations = membership in associations, AT-index = atherogenic-thrombogenic index, BR = birth rate, CAPB = Cyclically Adjusted Primary Balance, civilian-labor = civilian labor force, C-section = Cesarean delivery rate, credit-info = depth of credit information, dissatisf = residents’ dissatisfaction, distrib.orient = distributional orientation, EDU = education, eHealth = eHealth index at GP-level, exch.rate = exchange rate, fat = fat consumption, GDP = gross domestic product, GFCF = Gross Fixed Capital Formation/Creation, GH-gas = greenhouse gas, GII = gender inequality index, gov = governance index, gov.revenue = government revenues, HC-coverage = healthcare coverage, HE = health(care) expenditure, HHconsump = household consumption, hosp.beds = hospital beds, hosp.payment = hospital payment scheme, hosp.stay = length of hospital stay, IDI = ICT development index, inc.ineq = income inequality, industry-labor = industrial labor force, infant-sex = infant sex ratio, labor-product = labor production, LBW = low birth weight, leave = work leave, life-satisf = life satisfaction, M-age = maternal age, marginal-tax = marginal tax rate, MDs = physicians, mult.preg = multiple pregnancy, NHS = Nation Health System, NO = nitrous oxide emissions, PM10 = particulate matter (PM10) emissions, pop = population size, pop.density = population density, pre-term = pre-term birth rate, prison = prison population, researchE = research&development expenditure, school.ref = compulsory schooling reform, smoke-free = smoke-free places, SO = sulfur oxide emissions, soc.E = social expenditure, soc.workers = social workers, sugar = sugar consumption, terror = terrorism, union = union density, UR = unemployment rate, urban = urbanization, veg-fr = vegetable-and-fruit consumption, welfare = welfare regime, Wwater = wastewater treatment.

https://doi.org/10.1371/journal.pone.0239031.g002

Distances between determinants in Fig 2 are indicative of determinants’ “connectedness” with each other. While the statistical procedure called for higher dimensionality of the model, for demonstration purposes we show here a two-dimensional solution. This simplification unfortunately comes with a caveat. To use the factor smoking as an example, it would appear it stands at a much greater distance from GDP than it does from alcohol. In reality however, smoking was considered together with alcohol consumption [ 21 , 25 , 26 , 52 , 68 ] in just as many studies as it was with GDP [ 21 , 25 , 26 , 52 , 59 ], five. To aid with respect to this apparent shortcoming, we have emphasized the strongest pairwise links. Solid lines connect GDP with health expenditure (HE), unemployment rate (UR), and education (EDU), indicating that the effect of GDP on health, taking into account the effects of the other three determinants as well, was evaluated in between 12 to 16 studies of the 60 included in this review. Tracing the dashed lines, we can also tell that GDP appeared jointly with income inequality, and HE together with either EDU or UR, in anywhere between 8 to 10 of our selected studies. Finally, some weaker but still worth-mentioning “connections” between variables are displayed as well via the dotted lines.

The fact that all notable pairwise “connections” are concentrated within a relatively small region of the plot may be interpreted as low overall “connectedness” among the health indicators studied. GDP is the most widely investigated determinant in relation to general population health. Its total number of “connections” is disproportionately high (159) compared to its runner-up–HE (with 113 “connections”), and then subsequently EDU (with 90) and UR (with 86). In fact, all of these determinants could be thought of as outliers, given that none of the remaining factors have a total count of pairings above 52. This decrease in individual determinants’ overall “connectedness” can be tracked on the graph via the change of color intensity as we move outwards from the symbolic center of GDP and its closest “co-determinants”, to finally reach the other extreme of the ten indicators (welfare regime, household consumption, compulsory school reform, life satisfaction, government revenues, literacy, research expenditure, multiple pregnancy, Cyclically Adjusted Primary Balance, and residents’ dissatisfaction; in white) the effects on health of which were only studied in isolation.

Lastly, we point to the few small but stable clusters of covariates encircled by the grey bubbles on Fig 2 . These groups of determinants were identified as “close” by both statistical procedures used for the production of the graph (see details in S2 Appendix ).

Statistical methodology

There was great variation in the level of statistical detail reported. Some authors provided too vague a description of their analytical approach, necessitating some inference in this section.

The issue of missing data is a challenging reality in this field of research, but few of the studies under review (12/60) explain how they dealt with it. Among the ones that do, three general approaches to handling missingness can be identified, listed in increasing level of sophistication: case-wise deletion, i.e., removal of countries from the sample [ 20 , 45 , 48 , 58 , 59 ], (linear) interpolation [ 28 , 30 , 34 , 58 , 59 , 63 ], and multiple imputation [ 26 , 41 , 52 ].

Correlations, Pearson, Spearman, or unspecified, were the only technique applied with respect to the health outcomes of interest in eight analyses [ 33 , 42 – 44 , 46 , 53 , 57 , 61 ]. Among the more advanced statistical methods, the family of regression models proved to be, by and large, predominant. Before examining this closer, we note the techniques that were, in a way, “unique” within this selection of studies: meta-analyses were performed (random and fixed effects, respectively) on the reduced form and 2-sample two stage least squares (2SLS) estimations done within countries [ 39 ]; difference-in-difference (DiD) analysis was applied in one case [ 23 ]; dynamic time-series methods, among which co-integration, impulse-response function (IRF), and panel vector autoregressive (VAR) modeling, were utilized in one study [ 80 ]; longitudinal generalized estimating equation (GEE) models were developed on two occasions [ 70 , 78 ]; hierarchical Bayesian spatial models [ 51 ] and special autoregressive regression [ 62 ] were also implemented.

Purely cross-sectional data analyses were performed in eight studies [ 25 , 45 , 47 , 50 , 55 , 56 , 67 , 71 ]. These consisted of linear regression (assumed ordinary least squares (OLS)), generalized least squares (GLS) regression, and multilevel analyses. However, six other studies that used longitudinal data in fact had a cross-sectional design, through which they applied regression at multiple time-points separately [ 27 , 29 , 36 , 48 , 68 , 72 ].

Apart from these “multi-point cross-sectional studies”, some other simplistic approaches to longitudinal data analysis were found, involving calculating and regressing 3-year averages of both the response and the predictor variables [ 54 ], taking the average of a few data-points (i.e., survey waves) [ 56 ] or using difference scores over 10-year [ 19 , 29 ] or unspecified time intervals [ 40 , 55 ].

Moving further in the direction of more sensible longitudinal data usage, we turn to the methods widely known among (health) economists as “panel data analysis” or “panel regression”. Most often seen were models with fixed effects for country/region and sometimes also time-point (occasionally including a country-specific trend as well), with robust standard errors for the parameter estimates to take into account correlations among clustered observations [ 20 , 21 , 24 , 28 , 30 , 32 , 34 , 37 , 38 , 41 , 52 , 59 , 60 , 63 , 66 , 69 , 73 , 79 , 81 , 82 ]. The Hausman test [ 83 ] was sometimes mentioned as the tool used to decide between fixed and random effects [ 26 , 49 , 63 , 66 , 73 , 82 ]. A few studies considered the latter more appropriate for their particular analyses, with some further specifying that (feasible) GLS estimation was employed [ 26 , 34 , 49 , 58 , 60 , 73 ]. Apart from these two types of models, the first differences method was encountered once as well [ 31 ]. Across all, the error terms were sometimes assumed to come from a first-order autoregressive process (AR(1)), i.e., they were allowed to be serially correlated [ 20 , 30 , 38 , 58 – 60 , 73 ], and lags of (typically) predictor variables were included in the model specification, too [ 20 , 21 , 37 , 38 , 48 , 69 , 81 ]. Lastly, a somewhat different approach to longitudinal data analysis was undertaken in four studies [ 22 , 35 , 48 , 65 ] in which multilevel–linear or Poisson–models were developed.

Regardless of the exact techniques used, most studies included in this review presented multiple model applications within their main analysis. None attempted to formally compare models in order to identify the “best”, even if goodness-of-fit statistics were occasionally reported. As indicated above, many studies investigated women’s and men’s health separately [ 19 , 21 , 22 , 27 – 29 , 31 , 33 , 35 , 36 , 38 , 39 , 45 , 50 , 51 , 64 , 65 , 69 , 82 ], and covariates were often tested one at a time, including other covariates only incrementally [ 20 , 25 , 28 , 36 , 40 , 50 , 55 , 67 , 73 ]. Furthermore, there were a few instances where analyses within countries were performed as well [ 32 , 39 , 51 ] or where the full time period of interest was divided into a few sub-periods [ 24 , 26 , 28 , 31 ]. There were also cases where different statistical techniques were applied in parallel [ 29 , 55 , 60 , 66 , 69 , 73 , 82 ], sometimes as a form of sensitivity analysis [ 24 , 26 , 30 , 58 , 73 ]. However, the most common approach to sensitivity analysis was to re-run models with somewhat different samples [ 39 , 50 , 59 , 67 , 69 , 80 , 82 ]. Other strategies included different categorization of variables or adding (more/other) controls [ 21 , 23 , 25 , 28 , 37 , 50 , 63 , 69 ], using an alternative main covariate measure [ 59 , 82 ], including lags for predictors or outcomes [ 28 , 30 , 58 , 63 , 65 , 79 ], using weights [ 24 , 67 ] or alternative data sources [ 37 , 69 ], or using non-imputed data [ 41 ].

As the methods and not the findings are the main focus of the current review, and because generic checklists cannot discern the underlying quality in this application field (see also below), we opted to pool all reported findings together, regardless of individual study characteristics or particular outcome(s) used, and speak generally of positive and negative effects on health. For this summary we have adopted the 0.05-significance level and only considered results from multivariate analyses. Strictly birth-related factors are omitted since these potentially only relate to the group of infant mortality indicators and not to any of the other general population health measures.

Starting with the determinants most often studied, higher GDP levels [ 21 , 26 , 27 , 29 , 30 , 32 , 43 , 48 , 52 , 58 , 60 , 66 , 67 , 73 , 79 , 81 , 82 ], higher health [ 21 , 37 , 47 , 49 , 52 , 58 , 59 , 68 , 72 , 82 ] and social [ 20 , 21 , 26 , 38 , 79 ] expenditures, higher education [ 26 , 39 , 52 , 62 , 72 , 73 ], lower unemployment [ 60 , 61 , 66 ], and lower income inequality [ 30 , 42 , 53 , 55 , 73 ] were found to be significantly associated with better population health on a number of occasions. In addition to that, there was also some evidence that democracy [ 36 ] and freedom [ 50 ], higher work compensation [ 43 , 79 ], distributional orientation [ 54 ], cigarette prices [ 63 ], gross national income [ 22 , 72 ], labor productivity [ 26 ], exchange rates [ 32 ], marginal tax rates [ 79 ], vaccination rates [ 52 ], total fertility [ 59 , 66 ], fruit and vegetable [ 68 ], fat [ 52 ] and sugar consumption [ 52 ], as well as bigger depth of credit information [ 22 ] and percentage of civilian labor force [ 79 ], longer work leaves [ 41 , 58 ], more physicians [ 37 , 52 , 72 ], nurses [ 72 ], and hospital beds [ 79 , 82 ], and also membership in associations, perceived corruption and societal trust [ 48 ] were beneficial to health. Higher nitrous oxide (NO) levels [ 52 ], longer average hospital stay [ 48 ], deprivation [ 51 ], dissatisfaction with healthcare and the social environment [ 56 ], corruption [ 40 , 50 ], smoking [ 19 , 26 , 52 , 68 ], alcohol consumption [ 26 , 52 , 68 ] and illegal drug use [ 68 ], poverty [ 64 ], higher percentage of industrial workers [ 26 ], Gross Fixed Capital creation [ 66 ] and older population [ 38 , 66 , 79 ], gender inequality [ 22 ], and fertility [ 26 , 66 ] were detrimental.

It is important to point out that the above-mentioned effects could not be considered stable either across or within studies. Very often, statistical significance of a given covariate fluctuated between the different model specifications tried out within the same study [ 20 , 49 , 59 , 66 , 68 , 69 , 73 , 80 , 82 ], testifying to the importance of control variables and multivariate research (i.e., analyzing multiple independent variables simultaneously) in general. Furthermore, conflicting results were observed even with regards to the “core” determinants given special attention, so to speak, throughout this text. Thus, some studies reported negative effects of health expenditure [ 32 , 82 ], social expenditure [ 58 ], GDP [ 49 , 66 ], and education [ 82 ], and positive effects of income inequality [ 82 ] and unemployment [ 24 , 31 , 32 , 52 , 66 , 68 ]. Interestingly, one study [ 34 ] differentiated between temporary and long-term effects of GDP and unemployment, alluding to possibly much greater complexity of the association with health. It is also worth noting that some gender differences were found, with determinants being more influential for males than for females, or only having statistically significant effects for male health [ 19 , 21 , 28 , 34 , 36 , 37 , 39 , 64 , 65 , 69 ].

The purpose of this scoping review was to examine recent quantitative work on the topic of multi-country analyses of determinants of population health in high-income countries.

Measuring population health via relatively simple mortality-based indicators still seems to be the state of the art. What is more, these indicators are routinely considered one at a time, instead of, for example, employing existing statistical procedures to devise a more general, composite, index of population health, or using some of the established indices, such as disability-adjusted life expectancy (DALE) or quality-adjusted life expectancy (QALE). Although strong arguments for their wider use were already voiced decades ago [ 84 ], such summary measures surface only rarely in this research field.

On a related note, the greater data availability and accessibility that we enjoy today does not automatically equate to data quality. Nonetheless, this is routinely assumed in aggregate level studies. We almost never encountered a discussion on the topic. The non-mundane issue of data missingness, too, goes largely underappreciated. With all recent methodological advancements in this area [ 85 – 88 ], there is no excuse for ignorance; and still, too few of the reviewed studies tackled the matter in any adequate fashion.

Much optimism can be gained considering the abundance of different determinants that have attracted researchers’ attention in relation to population health. We took on a visual approach with regards to these determinants and presented a graph that links spatial distances between determinants with frequencies of being studies together. To facilitate interpretation, we grouped some variables, which resulted in some loss of finer detail. Nevertheless, the graph is helpful in exemplifying how many effects continue to be studied in a very limited context, if any. Since in reality no factor acts in isolation, this oversimplification practice threatens to render the whole exercise meaningless from the outset. The importance of multivariate analysis cannot be stressed enough. While there is no “best method” to be recommended and appropriate techniques vary according to the specifics of the research question and the characteristics of the data at hand [ 89 – 93 ], in the future, in addition to abandoning simplistic univariate approaches, we hope to see a shift from the currently dominating fixed effects to the more flexible random/mixed effects models [ 94 ], as well as wider application of more sophisticated methods, such as principle component regression, partial least squares, covariance structure models (e.g., structural equations), canonical correlations, time-series, and generalized estimating equations.

Finally, there are some limitations of the current scoping review. We searched the two main databases for published research in medical and non-medical sciences (PubMed and Web of Science) since 2013, thus potentially excluding publications and reports that are not indexed in these databases, as well as older indexed publications. These choices were guided by our interest in the most recent (i.e., the current state-of-the-art) and arguably the highest-quality research (i.e., peer-reviewed articles, primarily in indexed non-predatory journals). Furthermore, despite holding a critical stance with regards to some aspects of how determinants-of-health research is currently conducted, we opted out of formally assessing the quality of the individual studies included. The reason for that is two-fold. On the one hand, we are unaware of the existence of a formal and standard tool for quality assessment of ecological designs. And on the other, we consider trying to score the quality of these diverse studies (in terms of regional setting, specific topic, outcome indices, and methodology) undesirable and misleading, particularly since we would sometimes have been rating the quality of only a (small) part of the original studies—the part that was relevant to our review’s goal.

Our aim was to investigate the current state of research on the very broad and general topic of population health, specifically, the way it has been examined in a multi-country context. We learned that data treatment and analytical approach were, in the majority of these recent studies, ill-equipped or insufficiently transparent to provide clarity regarding the underlying mechanisms of population health in high-income countries. Whether due to methodological shortcomings or the inherent complexity of the topic, research so far fails to provide any definitive answers. It is our sincere belief that with the application of more advanced analytical techniques this continuous quest could come to fruition sooner.

Supporting information

S1 checklist. preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (prisma-scr) checklist..

https://doi.org/10.1371/journal.pone.0239031.s001

S1 Appendix.

https://doi.org/10.1371/journal.pone.0239031.s002

S2 Appendix.

https://doi.org/10.1371/journal.pone.0239031.s003

• View Article
• Google Scholar
• PubMed/NCBI
• 75. Dahlgren G, Whitehead M. Policies and Strategies to Promote Equity in Health. Stockholm, Sweden: Institute for Future Studies; 1991.
• 76. Brunner E, Marmot M. Social Organization, Stress, and Health. In: Marmot M, Wilkinson RG, editors. Social Determinants of Health. Oxford, England: Oxford University Press; 1999.
• 77. Najman JM. A General Model of the Social Origins of Health and Well-being. In: Eckersley R, Dixon J, Douglas B, editors. The Social Origins of Health and Well-being. Cambridge, England: Cambridge University Press; 2001.
• 85. Carpenter JR, Kenward MG. Multiple Imputation and its Application. New York: John Wiley & Sons; 2013.
• 86. Molenberghs G, Fitzmaurice G, Kenward MG, Verbeke G, Tsiatis AA. Handbook of Missing Data Methodology. Boca Raton: Chapman & Hall/CRC; 2014.
• 87. van Buuren S. Flexible Imputation of Missing Data. 2nd ed. Boca Raton: Chapman & Hall/CRC; 2018.
• 88. Enders CK. Applied Missing Data Analysis. New York: Guilford; 2010.
• 89. Shayle R. Searle GC, Charles E. McCulloch. Variance Components: John Wiley & Sons, Inc.; 1992.
• 90. Agresti A. Foundations of Linear and Generalized Linear Models. Hoboken, New Jersey: John Wiley & Sons Inc.; 2015.
• 91. Leyland A. H. (Editor) HGE. Multilevel Modelling of Health Statistics: John Wiley & Sons Inc; 2001.
• 92. Garrett Fitzmaurice MD, Geert Verbeke, Geert Molenberghs. Longitudinal Data Analysis. New York: Chapman and Hall/CRC; 2008.
• 93. Wolfgang Karl Härdle LS. Applied Multivariate Statistical Analysis. Berlin, Heidelberg: Springer; 2015.

• Research article
• Open access
• Published: 03 February 2021

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

• Ellesha A. Smith   ORCID: orcid.org/0000-0002-4241-7205 1 ,
• Nicola J. Cooper 1 ,
• Alex J. Sutton 1 ,
• Keith R. Abrams 1 &
• Stephanie J. Hubbard 1  

BMC Public Health volume  21 , Article number:  278 ( 2021 ) Cite this article

4655 Accesses

5 Citations

3 Altmetric

Metrics details

The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate ‘lumping’ of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.

The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.

The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.

Conclusions

Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

Peer Review reports

To make well-informed decisions and provide the best guidance in health care policy, it is essential to have a clear framework for synthesising good quality evidence on the effectiveness and cost-effectiveness of health interventions. There is a broad range of methods available for evidence synthesis. Narrative reviews provide a qualitative summary of the effectiveness of the interventions. Meta-analysis is a statistical method that pools evidence from multiple independent sources [ 1 ]. Meta-analysis and more complex variations of meta-analysis have been extensively applied in the appraisals of clinical interventions and treatments, such as drugs, as the interventions and populations are clearly defined and tested in randomised, controlled conditions. In comparison, public health studies are often more complex in design, making synthesis more challenging [ 2 ].

Many challenges are faced in the synthesis of public health interventions. There is often increased methodological heterogeneity due to the inclusion of different study designs. Interventions are often poorly described in the literature which may result in variation within the intervention groups. There can be a wide range of outcomes, whose definitions are not consistent across studies. Intermediate, or surrogate, outcomes are often used in studies evaluating public health interventions [ 3 ]. In addition to these challenges, public health interventions are often also complex meaning that they are made up of multiple, interacting components [ 4 ]. Recent guidance documents have focused on the synthesis of complex interventions [ 2 , 5 , 6 ]. The National Institute for Health and Care Excellence (NICE) guidance manual provides recommendations across all topics that are covered by NICE and there is currently no guidance that focuses specifically on the public health context.

Research questions

A methodological review of NICE public health intervention guidelines by Achana et al. (2014) found that meta-analysis methods were not being used [ 3 ]. The first part of this paper aims to update and compare, to the original review, the meta-analysis methods being used in evidence synthesis of public health intervention appraisals.

The second part of this paper aims to illustrate what methods are available to address the challenges of public health intervention evidence synthesis. Synthesis methods that go beyond a pairwise meta-analysis are illustrated through the application to a case study in public health and are discussed to understand how evidence synthesis methods can enable more informed decision making.

The third part of this paper presents software, guidance documents and web tools for methods that aim to make appropriate evidence synthesis of public health interventions more accessible. Recommendations for future research and guidance production that can improve the uptake of these methods in a public health context are discussed.

Update of NICE public health intervention guidelines review

Nice guidelines.

The National Institute for Health and Care Excellence (NICE) was established in 1999 as a health authority to provide guidance on new medical technologies to the NHS in England and Wales [ 7 ]. Using an evidence-based approach, it provides recommendations based on effectiveness and cost-effectiveness to ensure an open and transparent process of allocating NHS resources [ 8 ]. The remit for NICE guideline production was extended to public health in April 2005 and the first recommendations were published in March 2006. NICE published ‘Developing NICE guidelines: the manual’ in 2006, which has been updated since, with the most recent in 2018 [ 9 ]. It was intended to be a guidance document to aid in the production of NICE guidelines across all NICE topics. In terms of synthesising quantitative evidence, the NICE recommendations state: ‘meta-analysis may be appropriate if treatment estimates of the same outcome from more than 1 study are available’ and ‘when multiple competing options are being appraised, a network meta-analysis should be considered’. The implementation of network meta-analysis (NMA), which is described later, as a recommendation from NICE was introduced into the guidance document in 2014, with a further update in 2018.

Background to the previous review

The paper by Achana et al. (2014) explored the use of evidence synthesis methodology in NICE public health intervention guidelines published between 2006 and 2012 [ 3 ]. The authors conducted a systematic review of the methods used to synthesise quantitative effectiveness evidence within NICE public health guidelines. They found that only 23% of NICE public health guidelines used pairwise meta-analysis as part of the effectiveness review and the remainder used a narrative summary or no synthesis of evidence at all. The authors argued that despite significant advances in the methodology of evidence synthesis, the uptake of methods in public health intervention evaluation is lower than other fields, including clinical treatment evaluation. The paper concluded that more sophisticated methods in evidence synthesis should be considered to aid in decision making in the public health context [ 3 ].

The search strategy used in this paper was equivalent to that in the previous paper by Achana et al. (2014)[ 3 ]. The search was conducted through the NICE website ( https://www.nice.org.uk/guidance ) by searching the ‘Guidance and Advice List’ and filtering by ‘Public Health Guidelines’ [ 10 ]. The search criteria included all guidance documents that had been published from inception (March 2006) until the 19th August 2019. Since the original review, many of the guidelines had been updated with new documents or merged. Guidelines that remained unchanged since the previous review in 2012 were excluded and used for comparison.

The guidelines contained multiple documents that were assessed for relevance. A systematic review is a separate synthesis within a guideline that systematically collates all evidence on a specific research question of interest in the literature. Systematic reviews of quantitative effectiveness, cost-effectiveness evidence and decision modelling reports were all included as relevant. Qualitative reviews, field reports, expert opinions, surveillance reports, review decisions and other supporting documents were excluded at the search stage.

Within the reports, data was extracted on the types of review (narrative summary, pairwise meta-analysis, network meta-analysis (NMA), cost-effectiveness review or decision model), design of included primary studies (randomised controlled trials or non-randomised studies, intermediate or final outcomes, description of outcomes, outcome measure statistic), details of the synthesis methods used in the effectiveness evaluation (type of synthesis, fixed or random effects model, study quality assessment, publication bias assessment, presentation of results, software). Further details of the interventions were also recorded, including whether multiple interventions were lumped together for a pairwise comparison, whether interventions were complex (made up of multiple components) and details of the components. The reports were also assessed for potential use of complex intervention evidence synthesis methodology, meaning that the interventions that were evaluated in the review were made up of components that could potentially be synthesised using an NMA or a component NMA [ 11 ]. Where meta-analysis was not used to synthesis effectiveness evidence, the reasons for this was also recorded.

Search results and types of reviews

There were 67 NICE public health guidelines available on the NICE website. A summary flow diagram describing the literature identification process and the list of guidelines and their reference codes are provided in Additional files  1 and 2 . Since the previous review, 22 guidelines had not been updated. The results from the previous review were used for comparison to the 45 guidelines that were either newly published or updated.

The guidelines consisted of 508 documents that were assessed for relevance. Table  1 shows which types of relevant documents were available in each of the 45 guidelines. The median number of relevant articles per guideline was 3 (minimum = 0, maximum = 10). Two (4%) of the NICE public health guidelines did not report any type of systematic review, cost-effectiveness review or decision model (NG68, NG64) that met the inclusion criteria. 167 documents from 43 NICE public health guidelines were systematic reviews of quantitative effectiveness, cost-effectiveness or decision model reports and met the inclusion criteria.

Narrative reviews of effectiveness were implemented in 41 (91%) of the NICE PH guidelines. 14 (31%) contained a review that used meta-analysis to synthesise the evidence. Only one (1%) NICE guideline contained a review that implemented NMA to synthesise the effectiveness of multiple interventions; this was the same guideline that used NMA in the original review and had been updated. 33 (73%) guidelines contained cost-effectiveness reviews and 34 (76%) developed a decision model.

Comparison of review types to original review

Table  2 compares the results of the update to the original review and shows that the types of reviews and evidence synthesis methodologies remain largely unchanged since 2012. The proportion of guidelines that only contain narrative reviews to synthesise effectiveness or cost-effectiveness evidence has reduced from 74% to 60% and the proportion that included a meta-analysis has increased from 23% to 31%. The proportion of guidelines with reviews that only included evidence from randomised controlled trials and assessed the quality of individual studies remained similar to the original review.

Characteristics of guidelines using meta-analytic methods

Table  3 details the characteristics of the meta-analytic methods implemented in 24 reviews of the 14 guidelines that included one. All of the reviews reported an assessment of study quality, 12 (50%) reviews included only data from randomised controlled trials, 4 (17%) reviews used intermediate outcomes (e.g. uptake of chlamydia screening rather than prevention of chlamydia (PH3)), compared to the 20 (83%) reviews that used final outcomes (e.g. smoking cessation rather than uptake of a smoking cessation programme (NG92)). 2 (8%) reviews only used a fixed effect meta-analysis, 19 (79%) reviews used a random effects meta-analysis and 3 (13%) did not report which they had used.

An evaluation of the intervention information reported in the reviews concluded that 12 (50%) reviews had lumped multiple (more than two) different interventions into a control versus intervention pairwise meta-analysis. Eleven (46%) of the reviews evaluated interventions that are made up of multiple components (e.g. interventions for preventing obesity in PH47 were made up of diet, physical activity and behavioural change components).

21 (88%) of the reviews presented the results of the meta-analysis in the form of a forest plot and 22 (92%) presented the results in the text of the report. 20 (83%) of the reviews used two or more forms of presentation for the results. Only three (13%) reviews assessed publication bias. The most common software to perform meta-analysis was RevMan in 14 (58%) of the reviews.

Reasons for not using meta-analytic methods

The 143 reviews of effectiveness and cost effectiveness that did not use meta-analysis methods to synthesise the quantitative effectiveness evidence were searched for reasons behind this decision. 70 reports (49%) did not give a reason for not synthesising the data using a meta-analysis and 164 reasons were reported which are displayed in Fig.  1 . Out of the remaining reviews, multiple reasons for not using a meta-analysis were given. 53 (37%) of the reviews reported at least one reason due to heterogeneity. 30 (21%) decision model reports did not give a reason and these are categorised separately. 5 (3%) reviews reported that meta-analysis was not applicable or feasible, 1 (1%) reported that they were following NICE guidelines and 5 (3%) reported that there were a lack of studies.

Frequency and proportions of reasons reported for not using statistical methods in quantitative evidence synthesis in NICE PH intervention reviews

The frequency of reviews and guidelines that used meta-analytic methods were plotted against year of publication, which is reported in Fig.  2 . This showed that the number of reviews that used meta-analysis were approximately constant but there is some suggestion that the number of meta-analyses used per guideline increased, particularly in 2018.

Number of meta-analyses in NICE PH guidelines by year. Guidelines that were published before 2012 had been updated since the previous review by Achana et al. (2014) [ 3 ]

Comparison of meta-analysis characteristics to original review

Table  4 compares the characteristics of the meta-analyses used in the evidence synthesis of NICE public health intervention guidelines to the original review by Achana et al. (2014) [ 3 ]. Overall, the characteristics in the updated review have not much changed from those in the original. These changes demonstrate that the use of meta-analysis in NICE guidelines has increased but remains low. Lumping of interventions still appears to be common in 50% of reviews. The implications of this are discussed in the next section.

Application of evidence synthesis methodology in a public health intervention: motivating example

Since the original review, evidence synthesis methods have been developed and can address some of the challenges of synthesising quantitative effectiveness evidence of public health interventions. Despite this, the previous section shows that the uptake of these methods is still low in NICE public health guidelines - usually limited to a pairwise meta-analysis.

It has been shown in the results above and elsewhere [ 12 ] that heterogeneity is a common reason for not synthesising the quantitative effectiveness evidence available from systematic reviews in public health. Statistical heterogeneity is the variation in the intervention effects between the individual studies. Heterogeneity is problematic in evidence synthesis as it leads to uncertainty in the pooled effect estimates in a meta-analysis which can make it difficult to interpret the pooled results and draw conclusions. Rather than exploring the source of the heterogeneity, often in public health intervention appraisals a random effects model is fitted which assumes that the study intervention effects are not equivalent but come from a common distribution [ 13 , 14 ]. Alternatively, as demonstrated in the review update, heterogeneity is used as a reason to not undertake any quantitative evidence synthesis at all.

Since the size of the intervention effects and the methodological variation in the studies will affect the impact of the heterogeneity on a meta-analysis, it is inappropriate to base the methodological approach of a review on the degree of heterogeneity, especially within public health intervention appraisal where heterogeneity seems inevitable. Ioannidis et al. (2008) argued that there are ‘almost always’ quantitative synthesis options that may offer some useful insights in the presence of heterogeneity, as long as the reviewers interpret the findings with respect to their limitations [ 12 ].

In this section current evidence synthesis methods are applied to a motivating example in public health. This aims to demonstrate that methods beyond pairwise meta-analysis can provide appropriate and pragmatic information to public health decision makers to enable more informed decision making.

Figure  3 summarises the narrative of this part of the paper and illustrates the methods that are discussed. The red boxes represent the challenges in synthesising quantitative effectiveness evidence and refers to the section within the paper for more detail. The blue boxes represent the methods that can be applied to investigate each challenge.

Summary of challenges that are faces in the evidence synthesis of public health interventions and methods that are discussed to overcome these challenges

Evaluating the effect of interventions for promoting the safe storage of cleaning products to prevent childhood poisoning accidents

To illustrate the methodological developments, a motivating example is used from the five year, NIHR funded, Keeping Children Safe Programme [ 15 ]. The project included a Cochrane systematic review that aimed to increase the use of safety equipment to prevent accidents at home in children under five years old. This application is intended to be illustrative of the benefits of new evidence synthesis methods since the previous review. It is not a complete, comprehensive analysis as it only uses a subset of the original dataset and therefore the results are not intended to be used for policy decision making. This example has been chosen as it demonstrates many of the issues in synthesising effectiveness evidence of public health interventions, including different study designs (randomised controlled trials, observational studies and cluster randomised trials), heterogeneity of populations or settings, incomplete individual participant data and complex interventions that contain multiple components.

This analysis will investigate the most effective promotional interventions for the outcome of ‘safe storage of cleaning products’ to prevent childhood poisoning accidents. There are 12 studies included in the dataset, with IPD available from nine of the studies. The covariate, single parent family, is included in the analysis to demonstrate the effect of being a single parent family on the outcome. In this example, all of the interventions are made up of one or more of the following components: education (Ed), free or low cost equipment (Eq), home safety inspection (HSI), and installation of safety equipment (In). A Bayesian approach using WinBUGS was used and therefore credible intervals (CrI) are presented with estimates of the effect sizes [ 16 ].

The original review paper by Achana et al. (2014) demonstrated pairwise meta-analysis and meta-regression using individual and cluster allocated trials, subgroup analyses, meta-regression using individual participant data (IPD) and summary aggregate data and NMA. This paper firstly applies NMA to the motivating example for context, followed by extensions to NMA.

Multiple interventions: lumping or splitting?

Often in public health there are multiple intervention options. However, interventions are often lumped together in a pairwise meta-analysis. Pairwise meta-analysis is a useful tool for two interventions or, alternatively in the presence of lumping interventions, for answering the research question: ‘are interventions in general better than a control or another group of interventions?’. However, when there are multiple interventions, this type of analysis is not appropriate for informing health care providers which intervention should be recommended to the public. ‘Lumping’ is becoming less frequent in other areas of evidence synthesis, such as for clinical interventions, as the use of sophisticated synthesis techniques, such as NMA, increases (Achana et al. 2014) but lumping is still common in public health.

NMA is an extension of the pairwise meta-analysis framework to more than two interventions. Multiple interventions that are lumped into a pairwise meta-analysis are likely to demonstrate high statistical heterogeneity. This does not mean that quantitative synthesis could not be undertaken but that a more appropriate method, NMA, should be implemented. Instead the statistical approach should be based on the research questions of the systematic review. For example, if the research question is ‘are any interventions effective for preventing obesity?’, it would be appropriate to perform a pairwise meta-analysis comparing every intervention in the literature to a control. However, if the research question is ‘which intervention is the most effective for preventing obesity?’, it would be more appropriate and informative to perform a network meta-analysis, which can compare multiple interventions simultaneously and identify the best one.

NMA is a useful statistical method in the context of public health intervention appraisal, where there are often multiple intervention options, as it estimates the relative effectiveness of three or more interventions simultaneously, even if direct study evidence is not available for all intervention comparisons. Using NMA can help to answer the research question ‘what is the effectiveness of each intervention compared to all other interventions in the network?’.

In the motivating example there are six intervention options. The effect of lumping interventions is shown in Fig.  4 , where different interventions in both the intervention and control arms are compared. There is overlap of intervention and control arms across studies and interpretation of the results of a pairwise meta-analysis comparing the effectiveness of the two groups of interventions would not be useful in deciding which intervention to recommend. In comparison, the network plot in Fig.  5 illustrates the evidence base of the prevention of childhood poisonings review comparing six interventions that promote the use of safety equipment in the home. Most of the studies use ‘usual care’ as a baseline and compare this to another intervention. There are also studies in the evidence base that compare pairs of the interventions, such as ‘Education and equipment’ to ‘Equipment’. The plot also demonstrates the absence of direct study evidence between many pairs of interventions, for which the associated treatment effects can be indirectly estimated using NMA.

Network plot to illustrate how pairwise meta-analysis groups the interventions in the motivating dataset. Notation UC: Usual care, Ed: Education, Ed+Eq: Education and equipment, Ed+Eq+HSI: Education, equipment, and home safety inspection, Ed+Eq+In: Education, equipment and installation, Eq: Equipment

Network plot for the safe storage of cleaning products outcome. Notation UC: Usual care, Ed: Education, Ed+Eq: Education and equipment, Ed+Eq+HSI: Education, equipment, and home safety inspection, Ed+Eq+In: Education, equipment and installation, Eq: Equipment

An NMA was fitted to the motivating example to compare the six interventions in the studies from the review. The results are reported in the ‘triangle table’ in Table  5 [ 17 ]. The top right half of the table shows the direct evidence between pairs of the interventions in the corresponding rows and columns by either pooling the studies as a pairwise meta-analysis or presenting the single study results if evidence is only available from a single study. The bottom left half of the table reports the results of the NMA. The gaps in the top right half of the table arise where no direct study evidence exists to compare the two interventions. For example, there is no direct study evidence comparing ‘Education’ (Ed) to ‘Education, equipment and home safety inspection’ (Ed+Eq+HSI). The NMA, however, can estimate this comparison through the direct study evidence as an odds ratio of 3.80 with a 95% credible interval of (1.16, 12.44). The results suggest that the odds of safely storing cleaning products in the Ed+Eq+HSI intervention group is 3.80 times the odds in the Ed group. The results demonstrate a key benefit of NMA that all intervention effects in a network can be estimated using indirect evidence, even if there is no direct study evidence for some pairwise comparisons. This is based on the consistency assumption (that estimates of intervention effects from direct and indirect evidence are consistent) which should be checked when performing an NMA. This is beyond the scope of this paper and details on this can be found elsewhere [ 18 ].

NMA can also be used to rank the interventions in terms of their effectiveness and estimate the probability that each intervention is likely to be the most effective. This can help to answer the research question ‘which intervention is the best?’ out of all of the interventions that have provided evidence in the network. The rankings and associated probabilities for the motivating example are presented in Table  6 . It can be seen that in this case the ‘education, equipment and home safety inspection’ (Ed+Eq+HSI) intervention is ranked first, with a 0.87 probability of being the best intervention. However, there is overlap of the 95% credible intervals of the median rankings. This overlap reflects the uncertainty in the intervention effect estimates and therefore it is important that the interpretation of these statistics clearly communicates this uncertainty to decision makers.

NMA has the potential to be extremely useful but is underutilised in the evidence synthesis of public health interventions. The ability to compare and rank multiple interventions in an area where there are often multiple intervention options is invaluable in decision making for identifying which intervention to recommend. NMA can also include further literature in the analysis, compared to a pairwise meta-analysis, by expanding the network to improve the uncertainty in the effectiveness estimates.

Statistical heterogeneity

When heterogeneity remains in the results of an NMA, it is useful to explore the reasons for this. Strategies for dealing with heterogeneity involve the inclusion of covariates in a meta-analysis or NMA to adjust for the differences in the covariates across studies [ 19 ]. Meta-regression is a statistical method developed from meta-analysis that includes covariates to potentially explain the between-study heterogeneity ‘with the aim of estimating treatment-covariate interactions’ (Saramago et al. 2012). NMA has been extended to network meta-regression which investigates the effect of trial characteristics on multiple intervention effects. Three ways have been suggested to include covariates in an NMA: single covariate effect, exchangeable covariate effects and independent covariate effects which are discussed in more detail in the NICE Technical Support Document 3 [ 14 ]. This method has the potential to assess the effect of study level covariates on the intervention effects, which is particularly relevant in public health due to the variation across studies.

The most widespread method of meta-regression uses study level data for the inclusion of covariates into meta-regression models. Study level covariate data is when the data from the studies are aggregated, e.g. the proportion of participants in a study that are from single parent families compared to dual parent families. The alternative to study level data is individual participant data (IPD), where the data are available and used as a covariate at the individual level e.g. the parental status of every individual in a study can be used as a covariate. Although IPD is considered to be the gold standard for meta-analysis, aggregated level data is much more commonly used as it is usually available and easily accessible from published research whereas IPD can be hard to obtain from study authors.

There are some limitations to network meta-regression. In our motivating example, using the single parent covariate in a meta-regression would estimate the relative difference in the intervention effects of a population that is made up of 100% single parent families compared to a population that is made up of 100% dual parent families. This interpretation is not as useful as the analysis that uses IPD, which would give the relative difference of the intervention effects in a single parent family compared to a dual parent family. The meta-regression using aggregated data would also be susceptible to ecological bias. Ecological bias is where the effect of the covariate is different at the study level compared to the individual level [ 14 ]. For example, if each study demonstrates a relationship between a covariate and the intervention but the covariate is similar across the studies, a meta-regression of the aggregate data would not demonstrate the effect that is observed within the studies [ 20 ].

Although meta-regression is a useful tool for investigating sources of heterogeneity in the data, caution should be taken when using the results of meta-regression to explain how covariates affect the intervention effects. Meta-regression should only be used to investigate study characteristics, such as the duration of intervention, which will not be susceptible to ecological bias and the interpretation of the results (the effect of intervention duration on intervention effectiveness) would be more meaningful for the development of public health interventions.

Since the covariate of interest in this motivating example is not a study characteristic, meta-regression of aggregated covariate data was not performed. Network meta-regression including IPD and aggregate level data was developed by Samarago et al. (2012) [ 21 ] to overcome the issues with aggregated data network meta-regression, which is discussed in the next section.

Tailored decision making to specific sub-groups

In public health it is important to identify which interventions are best for which people. There has been a recent move towards precision medicine. In the field of public health the ‘concept of precision prevention may [...] be valuable for efficiently targeting preventive strategies to the specific subsets of a population that will derive maximal benefit’ (Khoury and Evans, 2015). Tailoring interventions has the potential to reduce the effect of inequalities in social factors that are influencing the health of the population. Identifying which interventions should be targeted to which subgroups can also lead to better public health outcomes and help to allocate scarce NHS resources. Research interest, therefore, lies in identifying participant level covariate-intervention interactions.

IPD meta-analysis uses data at the individual level to overcome ecological bias. The interpretation of IPD meta-analysis is more relevant in the case of using participant characteristics as covariates since the interpretation of the covariate-intervention interaction is at the individual level rather than the study level. This means that it can answer the research question: ‘which interventions work best in subgroups of the population?’. IPD meta-analyses are considered to be the gold standard for evidence synthesis since it increases the power of the analysis to identify covariate-intervention interactions and it has the ability to reduce the effect of ecological bias compared to aggregated data alone. IPD meta-analysis can also help to overcome scarcity of data issues and has been shown to have higher power and reduce the uncertainty in the estimates compared to analysis including only summary aggregate data [ 22 ].

Despite the advantages of including IPD in a meta-analysis, in reality it is often very time consuming and difficult to collect IPD for all of the studies [ 21 ]. Although data sharing is becoming more common, it remains time consuming and difficult to collect IPD for all studies in a review. This results in IPD being underutilised in meta-analyses. As an intermediate solution, statistical methods have been developed, such as the NMA in Samarago et al. (2012), that incorporates both IPD and aggregate data. Methods that simultaneously include IPD and aggregate level data have been shown to reduce uncertainty in the effect estimates and minimise ecological bias [ 20 , 21 ]. A simulation study by Leahy et al. (2018) found that an increased proportion of IPD resulted in more accurate and precise NMA estimates [ 23 ].

An NMA including IPD, where it is available, was performed, based on the model presented in Samarago et al. (2012) [ 21 ]. The results in Table  7 demonstrates the detail that this type of analysis can provide to base decisions on. More relevant covariate-intervention interaction interpretations can be obtained, for example the regression coefficients for covariate-intervention interactions are the individual level covariate intervention interactions or the ‘within study interactions’ that are interpreted as the effect of being in a single parent family on the effectiveness of each of the interventions. For example, the effect of Ed+Eq compared to UC in a single parent family is 1.66 times the effect of Ed+Eq compared to UC in a dual parent family but this is not an important difference as the credible interval crosses 1. The regression coefficients for the study level covariate-intervention interactions or the ‘between study interactions’ can be interpreted as the relative difference in the intervention effects of a population that is made up of 100% single parent families compared to a population that is made up of 100% dual parent families.

• Complex interventions

In many public health research settings the complex interventions are comprised of a number of components. An NMA can compare all of the interventions in a network as they are implemented in the original trials. However, NMA does not tell us which components of the complex intervention are attributable to this effect. It could be that particular components, or the interacting effect of multiple components, are driving the effectiveness and other components are not as effective. Often, trials have not directly compared every combination of components as there are so many component combination options, it would be inefficient and impractical. Component NMA was developed by Welton et al. (2009) to estimate the effect of each component of the complex interventions and combination of components in a network, in the absence of direct trial evidence and answers the question: ‘are interventions with a particular component or combination of components effective?’ [ 11 ]. For example, for the motivating example, in comparison to Fig.  5 , which demonstrates the interventions that an NMA can estimate effectiveness, Fig.  6 demonstrates all of the possible interventions of which the effectiveness can be estimated in a component NMA, given the components present in the network.

Network plot that illustrates how component network meta-analysis can estimate the effectiveness of intervention components and combinations of components, even when they are not included in the direct evidence. Notation UC: Usual care, Ed: Education, Eq: Equipment, Installation, Ed+Eq: Education and equipment, Ed+HSI: Education and home safety inspection, Ed+In: Education and installation, Eq+HSI: Equipment and home safety inspection, Eq+In: equipment and installation, HSI+In: Home safety inspection and installation, Ed+Eq+HSI: Education, equipment, and home safety inspection, Ed+Eq+In: Education, equipment and installation, Eq+HSI+In: Equipment, home safety inspection and installation, Ed+Eq+HSI+In: Education, equipment, home safety inspection and installation

The results of the analyses of the main effects, two way effects and full effects models are shown in Table  8 . The models, proposed in the original paper by Welton et al. (2009), increase in complexity as the assumptions regarding the component effects relax [ 24 ]. The main effects component NMA assumes that the components in the interventions each have separate, independent effects and intervention effects are the sum of the component effects. The two-way effects models assumes that there are interactions between pairs of the components, so the effects of the interventions are more than the sum of the effects. The full effects model assumes that all of the components and combinations of the components interact. Component NMA did not provide further insight into which components are likely to be the most effective since all of the 95% credible intervals were very wide and overlapped 1. There is a lot of uncertainty in the results, particularly in the 2-way and full effects models. A limitation of component NMA is that there are issues with uncertainty when data is scarce. However, the results demonstrate the potential of component NMA as a useful tool to gain better insights from the available dataset.

In practice, this method has rarely been used since its development [ 24 – 26 ]. It may be challenging to define the components in some areas of public health where many interventions have been studied. However, the use of meta-analysis for planning future studies is rarely discussed and component NMA would provide a useful tool for identifying new component combinations that may be more effective [ 27 ]. This type of analysis has the potential to prioritise future public health research, which is especially useful where there are multiple intervention options, and identify more effective interventions to recommend to the public.

Further methods / other outcomes

The analysis and methods described in this paper only cover a small subset of the methods that have been developed in meta-analysis in recent years. Methods that aim to assess the quality of evidence supporting a NMA and how to quantify how much the evidence could change due to potential biases or sampling variation before the recommendation changes have been developed [ 28 , 29 ]. Models adjusting for baseline risk have been developed to allow for different study populations to have different levels of underlying risk, by using the observed event rate in the control arm [ 30 , 31 ]. Multivariate methods can be used to compare the effect of multiple interventions on two or more outcomes simultaneously [ 32 ]. This area of methodological development is especially appealing within public health where studies assess a broad range of health effects and typically have multiple outcome measures. Multivariate methods offer benefits over univariate models by allowing the borrowing of information across outcomes and modelling the relationships between outcomes which can potentially reduce the uncertainty in the effect estimates [ 33 ]. Methods have also been developed to evaluate interventions with classes or different intervention intensities, known as hierarchical interventions [ 34 ]. These methods were not demonstrated in this paper but can also be useful tools for addressing challenges of appraising public health interventions, such as multiple and surrogate outcomes.

This paper only considered an example with a binary outcome. All of the methods described have also been adapted for other outcome measures. For example, the Technical Support Document 2 proposed a Bayesian generalised linear modelling framework to synthesise other outcome measures. More information and models for continuous and time-to-event data is available elsewhere [ 21 , 35 – 38 ].

Software and guidelines

In the previous section, meta-analytic methods that answer more policy relevant questions were demonstrated. However, as shown by the update to the review, methods such as these are still under-utilised. It is suspected from the NICE public health review that one of the reasons for the lack of uptake of methods in public health could be due to common software choices, such as RevMan, being limited in their flexibility for statistical methods.

Table  9 provides a list of software options and guidance documents that are more flexible than RevMan for implementing the statistical methods illustrated in the previous section to make these methods more accessible to researchers.

In this paper, the network plot in Figs.  5 and 6 were produced using the networkplot command from the mvmeta package [ 39 ] in Stata [ 61 ]. WinBUGS was used to fit the NMA in this paper by adapting the code in the book ‘Evidence Synthesis for Decision Making in Healthcare’ which also provides more detail on Bayesian methods and assessing convergence of Bayesian models [ 45 ]. The model for including IPD and summary aggregate data in an NMA was based on the code in the paper by Saramago et al. (2012). The component NMA in this paper was performed in WinBUGS through R2WinBUGS, [ 47 ] using the code in Welton et al. (2009) [ 11 ].

WinBUGS is a flexible tool for fitting complex models in a Bayesian framework. The NICE Decision Support Unit produced a series of Evidence Synthesis Technical Support Documents [ 46 ] that provide a comprehensive technical guide to methods for evidence synthesis and WinBUGS code is also provided for many of the models. Complex models can also be performed in a frequentist framework. Code and commands for many models are available in R and STATA (see Table  9 ).

The software, R2WinBUGS, was used in the analysis of the motivating example. Increasing numbers of researchers are using R and so packages that can be used to link the two softwares by calling BUGS models in R, packages such as R2WinBUGS, can improve the accessibility of Bayesian methods [ 47 ]. The new R package, BUGSnet, may also help to facilitate the accessibility and improve the reporting of Bayesian NMA [ 48 ]. Webtools have also been developed as a means of enabling researchers to undertake increasingly complex analyses [ 52 , 53 ]. Webtools provide a user-friendly interface to perform statistical analyses and often help in the reporting of the analyses by producing plots, including network plots and forest plots. These tools are very useful for researchers that have a good understanding of the statistical methods they want to implement as part of their review but are inexperienced in statistical software.

This paper has reviewed NICE public health intervention guidelines to identify the methods that are currently being used to synthesise effectiveness evidence to inform public health decision making. A previous review from 2012 was updated to see how method utilisation has changed. Methods have been developed since the previous review and these were applied to an example dataset to show how methods can answer more policy relevant questions. Resources and guidelines for implementing these methods were signposted to encourage uptake.

The review found that the proportion of NICE guidelines containing effectiveness evidence summarised using meta-analysis methods has increased since the original review, but remains low. The majority of the reviews presented only narrative summaries of the evidence - a similar result to the original review. In recent years, there has been an increased awareness of the need to improve decision making by using all of the available evidence. As a result, this has led to the development of new methods, easier application in standard statistical software packages, and guidance documents. Based on this, it would have been expected that their implementation would rise in recent years to reflect this, but the results of the review update showed no such increasing pattern.

A high proportion of NICE guideline reports did not provide a reason for not applying quantitative evidence synthesis methods. Possible explanations for this could be time or resource constraints, lack of statistical expertise, being unaware of the available methods or poor reporting. Reporting guidelines, such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), should be updated to emphasise the importance of documenting reasons for not applying methods, as this can direct future research to improve uptake.

Where it was specified, the most common reported reason for not conducting a meta-analysis was heterogeneity. Often in public health, the data is heterogeneous due to the differences between studies in population, design, interventions or outcomes. A common misconception is that the presence of heterogeneity implies that it is not possible to pool the data. Meta-analytic methods can be used to investigate the sources of heterogeneity, as demonstrated in the NMA of the motivating example, and the use of IPD is recommended where possible to improve the precision of the results and reduce the effect of ecological bias. Although caution should be exercised in the interpretation of the results, quantitative synthesis methods provide a stronger basis for making decisions than narrative accounts because they explicitly quantify the heterogeneity and seek to explain it where possible.

The review also found that the most common software to perform the synthesis was RevMan. RevMan is very limited in its ability to perform advanced statistical analyses, beyond that of pairwise meta-analysis, which might explain the above findings. Standard software code is being developed to help make statistical methodology and application more accessible and guidance documents are becoming increasingly available.

The evaluation of public health interventions can be problematic due to the number and complexity of the interventions. NMA methods were applied to a real Cochrane public health review dataset. The methods that were demonstrated showed ways to address some of these issues, including the use of NMA for multiple interventions, the inclusion of covariates as both aggregated data and IPD to explain heterogeneity, and the extension to component network meta-analysis for guiding future research. These analyses illustrated how the choice of synthesis methods can enable more informed decision making by allowing more distinct interventions, and combinations of intervention components, to be defined and their effectiveness estimated. It also demonstrated the potential to target interventions to population subgroups where they are likely to be most effective. However, the application of component NMA to the motivating example has also demonstrated the issues around uncertainty if there are a limited number of studies observing the interventions and intervention components.

The application of methods to the motivating example demonstrated a key benefit of using statistical methods in a public health context compared to only presenting a narrative review – the methods provide a quantitative estimate of the effectiveness of the interventions. The uncertainty from the credible intervals can be used to demonstrate the lack of available evidence. In the context of decision making, having pooled estimates makes it much easier for decision makers to assess the effectiveness of the interventions or identify when more research is required. The posterior distribution of the pooled results from the evidence synthesis can also be incorporated into a comprehensive decision analytic model to determine cost-effectiveness [ 62 ]. Although narrative reviews are useful for describing the evidence base, the results are very difficult to summarise in a decision context.

Although heterogeneity seems to be inevitable within public health interventions due to their complex nature, this review has shown that it is still the main reported reason for not using statistical methods in evidence synthesis. This may be due to guidelines that were originally developed for clinical treatments that are tested in randomised conditions still being applied in public health settings. Guidelines for the choice of methods used in public health intervention appraisals could be updated to take into account the complexities and wide ranging areas in public health. Sophisticated methods may be more appropriate in some cases than simpler models for modelling multiple, complex interventions and their uncertainty, given the limitations are also fully reported [ 19 ]. Synthesis may not be appropriate if statistical heterogeneity remains after adjustment for possible explanatory covariates but details of exploratory analysis and reasons for not synthesising the data should be reported. Future research should focus on the application and dissemination of the advantages of using more advanced methods in public health, identifying circumstances where these methods are likely to be the most beneficial, and ways to make the methods more accessible, for example, the development of packages and web tools.

There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making. This review has shown that the uptake of statistical methods for evaluating the effectiveness of public health interventions is slow, despite advances in methods that address specific issues in public health intervention appraisal and the publication of guidance documents to complement their application.

Availability of data and materials

The dataset supporting the conclusions of this article is included within the article.

Abbreviations

National institute for health and care excellence

• Network meta-analysis

Individual participant data

Home safety inspection

Installation

Credible interval

Preferred reporting items for systematic reviews and meta-analyses

Dias S, Welton NJ, Sutton AJ, Ades A. NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials: National Institute for Health and Clinical Excellence; 2011, p. 98. (Technical Support Document in Evidence Synthesis; TSD2).

Higgins JPT, López-López JA, Becker BJ, et al.Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Global Health. 2019; 4(Suppl 1):e000858. https://doi.org/10.1136/bmjgh-2018-000858 .

Article   PubMed   PubMed Central   Google Scholar  

Achana F, Hubbard S, Sutton A, Kendrick D, Cooper N. An exploration of synthesis methods in public health evaluations of interventions concludes that the use of modern statistical methods would be beneficial. J Clin Epidemiol. 2014; 67(4):376–90.

Article   PubMed   Google Scholar  

Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new medical research council guidance. Int J Nurs Stud. 2013; 50(5):587–92.

Caldwell DM, Welton NJ. Approaches for synthesising complex mental health interventions in meta-analysis. Evidence-Based Mental Health. 2016; 19(1):16–21.

Melendez-Torres G, Bonell C, Thomas J. Emergent approaches to the meta-analysis of multiple heterogeneous complex interventions. BMC Med Res Methodol. 2015; 15(1):47.

Article   CAS   PubMed   PubMed Central   Google Scholar  

NICE. NICE: Who We Are. https://www.nice.org.uk/about/who-we-are . Accessed 19 Sept 2019.

Kelly M, Morgan A, Ellis S, Younger T, Huntley J, Swann C. Evidence based public health: a review of the experience of the national institute of health and clinical excellence (NICE) of developing public health guidance in England. Soc Sci Med. 2010; 71(6):1056–62.

NICE. Developing NICE Guidelines: The Manual. https://www.nice.org.uk/process/pmg20/chapter/introduction-and-overview . Accessed 19 Sept 2019.

NICE. Public Health Guidance. https://www.nice.org.uk/guidance/published?type=ph . Accessed 19 Sept 2019.

Welton NJ, Caldwell D, Adamopoulos E, Vedhara K. Mixed treatment comparison meta-analysis of complex interventions: psychological interventions in coronary heart disease. Am J Epidemiol. 2009; 169(9):1158–65.

Ioannidis JP, Patsopoulos NA, Rothstein HR. Reasons or excuses for avoiding meta-analysis in forest plots. BMJ. 2008; 336(7658):1413–5.

Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002; 21(11):1539–58.

Article   Google Scholar  

Dias S, Sutton A, Welton N, Ades A. NICE DSU Technical Support Document 3: Heterogeneity: Subgroups, Meta-Regression, Bias and Bias-Adjustment: National Institute for Health and Clinical Excellence; 2011, p. 76.

Kendrick D, Ablewhite J, Achana F, et al.Keeping Children Safe: a multicentre programme of research to increase the evidence base for preventing unintentional injuries in the home in the under-fives. Southampton: NIHR Journals Library; 2017.

Google Scholar  

Lunn DJ, Thomas A, Best N, et al.WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility. Stat Comput. 2000; 10:325–37. https://doi.org/10.1023/A:1008929526011 .

Dias S, Caldwell DM. Network meta-analysis explained. Arch Dis Child Fetal Neonatal Ed. 2019; 104(1):8–12. https://doi.org/10.1136/archdischild-2018-315224. http://arxiv.org/abs/https://fn.bmj.com/content/104/1/F8.full.pdf.

Dias S, Welton NJ, Sutton AJ, Caldwell DM, Lu G, Ades A. NICE DSU Technical Support Document 4: Inconsistency in Networks of Evidence Based on Randomised Controlled Trials: National Institute for Health and Clinical Excellence; 2011. (NICE DSU Technical Support Document in Evidence Synthesis; TSD4).

Cipriani A, Higgins JP, Geddes JR, Salanti G. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013; 159(2):130–7.

Riley RD, Steyerberg EW. Meta-analysis of a binary outcome using individual participant data and aggregate data. Res Synth Methods. 2010; 1(1):2–19.

Saramago P, Sutton AJ, Cooper NJ, Manca A. Mixed treatment comparisons using aggregate and individual participant level data. Stat Med. 2012; 31(28):3516–36.

Lambert PC, Sutton AJ, Abrams KR, Jones DR. A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. J Clin Epidemiol. 2002; 55(1):86–94.

Article   CAS   PubMed   Google Scholar  

Leahy J, O’Leary A, Afdhal N, Gray E, Milligan S, Wehmeyer MH, Walsh C. The impact of individual patient data in a network meta-analysis: an investigation into parameter estimation and model selection. Res Synth Methods. 2018; 9(3):441–69.

Freeman SC, Scott NW, Powell R, Johnston M, Sutton AJ, Cooper NJ. Component network meta-analysis identifies the most effective components of psychological preparation for adults undergoing surgery under general anesthesia. J Clin Epidemiol. 2018; 98:105–16.

Pompoli A, Furukawa TA, Efthimiou O, Imai H, Tajika A, Salanti G. Dismantling cognitive-behaviour therapy for panic disorder: a systematic review and component network meta-analysis. Psychol Med. 2018; 48(12):1945–53.

Rücker G, Schmitz S, Schwarzer G. Component network meta-analysis compared to a matching method in a disconnected network: A case study. Biom J. 2020. https://doi.org/10.1002/bimj.201900339 .

Efthimiou O, Debray TP, van Valkenhoef G, Trelle S, Panayidou K, Moons KG, Reitsma JB, Shang A, Salanti G, Group GMR. GetReal in network meta-analysis: a review of the methodology. Res Synth Methods. 2016; 7(3):236–63.

Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta-analysis. PLoS ONE. 2014; 9(7):99682.

Article   CAS   Google Scholar  

Phillippo DM, Dias S, Welton NJ, Caldwell DM, Taske N, Ades A. Threshold analysis as an alternative to grade for assessing confidence in guideline recommendations based on network meta-analyses. Ann Intern Med. 2019; 170(8):538–46.

Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document 5: Evidence Synthesis in the Baseline Natural History Model: National Institute for Health and Clinical Excellence; 2011, p. 29. (NICE DSU Technical Support Document in Evidence Synthesis; TSD5).

Achana FA, Cooper NJ, Dias S, Lu G, Rice SJ, Kendrick D, Sutton AJ. Extending methods for investigating the relationship between treatment effect and baseline risk from pairwise meta-analysis to network meta-analysis. Stat Med. 2013; 32(5):752–71.

Riley RD, Jackson D, Salanti G, Burke DL, Price M, Kirkham J, White IR. Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples. BMJ (Clinical research ed.) 2017; 358:j3932. https://doi.org/10.1136/bmj.j3932 .

Achana FA, Cooper NJ, Bujkiewicz S, Hubbard SJ, Kendrick D, Jones DR, Sutton AJ. Network meta-analysis of multiple outcome measures accounting for borrowing of information across outcomes. BMC Med Res Methodol. 2014; 14(1):92.

Owen RK, Tincello DG, Keith RA. Network meta-analysis: development of a three-level hierarchical modeling approach incorporating dose-related constraints. Value Health. 2015; 18(1):116–26.

Jansen JP. Network meta-analysis of individual and aggregate level data. Res Synth Methods. 2012; 3(2):177–90.

Donegan S, Williamson P, D’Alessandro U, Garner P, Smith CT. Combining individual patient data and aggregate data in mixed treatment comparison meta-analysis: individual patient data may be beneficial if only for a subset of trials. Stat Med. 2013; 32(6):914–30.

Saramago P, Chuang L-H, Soares MO. Network meta-analysis of (individual patient) time to event data alongside (aggregate) count data. BMC Med Res Methodol. 2014; 14(1):105.

Thom HH, Capkun G, Cerulli A, Nixon RM, Howard LS. Network meta-analysis combining individual patient and aggregate data from a mixture of study designs with an application to pulmonary arterial hypertension. BMC Med Res Methodol. 2015; 15(1):34.

Gasparrini A, Armstrong B, Kenward MG. Multivariate meta-analysis for non-linear and other multi-parameter associations. Stat Med. 2012; 31(29):3821–39.

Chaimani A, Higgins JP, Mavridis D, Spyridonos P, Salanti G. Graphical tools for network meta-analysis in stata. PLoS ONE. 2013; 8(10):76654.

Rücker G, Schwarzer G, Krahn U, König J. netmeta: Network meta-analysis with R. R package version 0.5-0. 2014. R package version 0.5-0. Availiable: http://CRAN.R-project.org/package=netmeta .

van Valkenhoef G, Kuiper J. gemtc: Network Meta-Analysis Using Bayesian Methods. R package version 0.8-2. 2016. Available online at: https://CRAN.R-project.org/package=gemtc .

Lin L, Zhang J, Hodges JS, Chu H. Performing arm-based network meta-analysis in R with the pcnetmeta package. J Stat Softw. 2017; 80(5):1–25. https://doi.org/10.18637/jss.v080.i05 .

Rücker G, Schwarzer G. Automated drawing of network plots in network meta-analysis. Res Synth Methods. 2016; 7(1):94–107.

Welton NJ, Sutton AJ, Cooper N, Abrams KR, Ades A. Evidence Synthesis for Decision Making in Healthcare, vol. 132. UK: Wiley; 2012.

Book   Google Scholar  

Dias S, Welton NJ, Sutton AJ, Ades AE. Evidence synthesis for decision making 1: introduction. Med Decis Making Int J Soc Med Decis Making. 2013; 33(5):597–606. https://doi.org/10.1177/0272989X13487604 .

Sturtz S, Ligges U, Gelman A. R2WinBUGS: a package for running WinBUGS from R. J Stat Softw. 2005; 12(3):1–16.

Béliveau A, Boyne DJ, Slater J, Brenner D, Arora P. Bugsnet: an r package to facilitate the conduct and reporting of bayesian network meta-analyses. BMC Med Res Methodol. 2019; 19(1):196.

Neupane B, Richer D, Bonner AJ, Kibret T, Beyene J. Network meta-analysis using R: a review of currently available automated packages. PLoS ONE. 2014; 9(12):115065.

White IR. Multivariate random-effects meta-analysis. Stata J. 2009; 9(1):40–56.

Chaimani A, Salanti G. Visualizing assumptions and results in network meta-analysis: the network graphs package. Stata J. 2015; 15(4):905–50.

Owen RK, Bradbury N, Xin Y, Cooper N, Sutton A. MetaInsight: An interactive web-based tool for analyzing, interrogating, and visualizing network meta-analyses using R-shiny and netmeta. Res Synth Methods. 2019; 10(4):569–81. https://doi.org/10.1002/jrsm.1373 .

Freeman SC, Kerby CR, Patel A, Cooper NJ, Quinn T, Sutton AJ. Development of an interactive web-based tool to conduct and interrogate meta-analysis of diagnostic test accuracy studies: MetaDTA. BMC Med Res Methodol. 2019; 19(1):81.

Nikolakopoulou A, Higgins JPT, Papakonstantinou T, Chaimani A, Del Giovane C, Egger M, Salanti G. CINeMA: An approach for assessing confidence in the results of a network meta-analysis. PLoS Med. 2020; 17(4):e1003082. https://doi.org/10.1371/journal.pmed.1003082 .

Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010; 36(3):1–48.

Freeman SC, Carpenter JR. Bayesian one-step ipd network meta-analysis of time-to-event data using royston-parmar models. Res Synth Methods. 2017; 8(4):451–64.

Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med. 2008; 27(11):1870–93.

Debray TP, Moons KG, van Valkenhoef G, Efthimiou O, Hummel N, Groenwold RH, Reitsma JB, Group GMR. Get real in individual participant data (ipd) meta-analysis: a review of the methodology. Res Synth Methods. 2015; 6(4):293–309.

Tierney JF, Vale C, Riley R, Smith CT, Stewart L, Clarke M, Rovers M. Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use. PLoS Med. 2015; 12(7):e1001855. https://doi.org/10.1371/journal.pmed.1001855 .

Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the prisma-ipd statement. JAMA. 2015; 313(16):1657–65.

StataCorp. Stata Statistical Software: Release 16. College Station: StataCorp LLC; 2019.

Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A. Comprehensive decision analytical modelling in economic evaluation: a bayesian approach. Health Econ. 2004; 13(3):203–26.

Download references

Acknowledgements

We would like to acknowledge Professor Denise Kendrick as the lead on the NIHR Keeping Children Safe at Home Programme that originally funded the collection of the evidence for the motivating example and some of the analyses illustrated in the paper.

ES is funded by a National Institute for Health Research (NIHR), Doctoral Research Fellow for this research project. This paper presents independent research funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. The funding bodies played no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and affiliations.

Department of Health Sciences, University of Leicester, Lancaster Road, Leicester, UK

Ellesha A. Smith, Nicola J. Cooper, Alex J. Sutton, Keith R. Abrams & Stephanie J. Hubbard

You can also search for this author in PubMed   Google Scholar

Contributions

ES performed the review, analysed the data and wrote the paper. SH supervised the project. SH, KA, NC and AS provided substantial feedback on the manuscript. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Ellesha A. Smith .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

KA is supported by Health Data Research (HDR) UK, the UK National Institute for Health Research (NIHR) Applied Research Collaboration East Midlands (ARC EM), and as a NIHR Senior Investigator Emeritus (NF-SI-0512-10159). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. KA has served as a paid consultant, providing unrelated methodological advice, to; Abbvie, Amaris, Allergan, Astellas, AstraZeneca, Boehringer Ingelheim, Bristol-Meyers Squibb, Creativ-Ceutical, GSK, ICON/Oxford Outcomes, Ipsen, Janssen, Eli Lilly, Merck, NICE, Novartis, NovoNordisk, Pfizer, PRMA, Roche and Takeda, and has received research funding from Association of the British Pharmaceutical Industry (ABPI), European Federation of Pharmaceutical Industries & Associations (EFPIA), Pfizer, Sanofi and Swiss Precision Diagnostics. He is a Partner and Director of Visible Analytics Limited, a healthcare consultancy company.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Key for the Nice public health guideline codes. Available in NICEGuidelinesKey.xlsx .

Additional file 2

NICE public health intervention guideline review flowchart for the inclusion and exclusion of documents. Available in Flowchart.JPG .

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Smith, E.A., Cooper, N.J., Sutton, A.J. et al. A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines. BMC Public Health 21 , 278 (2021). https://doi.org/10.1186/s12889-021-10162-8

Download citation

Received : 22 September 2020

Accepted : 04 January 2021

Published : 03 February 2021

DOI : https://doi.org/10.1186/s12889-021-10162-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Meta-analysis
• Systematic review
• Public health
• Decision making
• Evidence synthesis

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

• Privacy Policy

Home » Quantitative Research – Methods, Types and Analysis

Quantitative Research – Methods, Types and Analysis

Table of Contents

Quantitative Research

Quantitative research is a type of research that collects and analyzes numerical data to test hypotheses and answer research questions . This research typically involves a large sample size and uses statistical analysis to make inferences about a population based on the data collected. It often involves the use of surveys, experiments, or other structured data collection methods to gather quantitative data.

Quantitative Research Methods

Quantitative Research Methods are as follows:

Descriptive Research Design

Descriptive research design is used to describe the characteristics of a population or phenomenon being studied. This research method is used to answer the questions of what, where, when, and how. Descriptive research designs use a variety of methods such as observation, case studies, and surveys to collect data. The data is then analyzed using statistical tools to identify patterns and relationships.

Correlational Research Design

Correlational research design is used to investigate the relationship between two or more variables. Researchers use correlational research to determine whether a relationship exists between variables and to what extent they are related. This research method involves collecting data from a sample and analyzing it using statistical tools such as correlation coefficients.

Quasi-experimental Research Design

Quasi-experimental research design is used to investigate cause-and-effect relationships between variables. This research method is similar to experimental research design, but it lacks full control over the independent variable. Researchers use quasi-experimental research designs when it is not feasible or ethical to manipulate the independent variable.

Experimental Research Design

Experimental research design is used to investigate cause-and-effect relationships between variables. This research method involves manipulating the independent variable and observing the effects on the dependent variable. Researchers use experimental research designs to test hypotheses and establish cause-and-effect relationships.

Survey Research

Survey research involves collecting data from a sample of individuals using a standardized questionnaire. This research method is used to gather information on attitudes, beliefs, and behaviors of individuals. Researchers use survey research to collect data quickly and efficiently from a large sample size. Survey research can be conducted through various methods such as online, phone, mail, or in-person interviews.

Quantitative Research Analysis Methods

Here are some commonly used quantitative research analysis methods:

Statistical Analysis

Statistical analysis is the most common quantitative research analysis method. It involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis can be used to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.

Regression Analysis

Regression analysis is a statistical technique used to analyze the relationship between one dependent variable and one or more independent variables. Researchers use regression analysis to identify and quantify the impact of independent variables on the dependent variable.

Factor Analysis

Factor analysis is a statistical technique used to identify underlying factors that explain the correlations among a set of variables. Researchers use factor analysis to reduce a large number of variables to a smaller set of factors that capture the most important information.

Structural Equation Modeling

Structural equation modeling is a statistical technique used to test complex relationships between variables. It involves specifying a model that includes both observed and unobserved variables, and then using statistical methods to test the fit of the model to the data.

Time Series Analysis

Time series analysis is a statistical technique used to analyze data that is collected over time. It involves identifying patterns and trends in the data, as well as any seasonal or cyclical variations.

Multilevel Modeling

Multilevel modeling is a statistical technique used to analyze data that is nested within multiple levels. For example, researchers might use multilevel modeling to analyze data that is collected from individuals who are nested within groups, such as students nested within schools.

Applications of Quantitative Research

Quantitative research has many applications across a wide range of fields. Here are some common examples:

• Market Research : Quantitative research is used extensively in market research to understand consumer behavior, preferences, and trends. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform marketing strategies, product development, and pricing decisions.
• Health Research: Quantitative research is used in health research to study the effectiveness of medical treatments, identify risk factors for diseases, and track health outcomes over time. Researchers use statistical methods to analyze data from clinical trials, surveys, and other sources to inform medical practice and policy.
• Social Science Research: Quantitative research is used in social science research to study human behavior, attitudes, and social structures. Researchers use surveys, experiments, and other quantitative methods to collect data that can inform social policies, educational programs, and community interventions.
• Education Research: Quantitative research is used in education research to study the effectiveness of teaching methods, assess student learning outcomes, and identify factors that influence student success. Researchers use experimental and quasi-experimental designs, as well as surveys and other quantitative methods, to collect and analyze data.
• Environmental Research: Quantitative research is used in environmental research to study the impact of human activities on the environment, assess the effectiveness of conservation strategies, and identify ways to reduce environmental risks. Researchers use statistical methods to analyze data from field studies, experiments, and other sources.

Characteristics of Quantitative Research

Here are some key characteristics of quantitative research:

• Numerical data : Quantitative research involves collecting numerical data through standardized methods such as surveys, experiments, and observational studies. This data is analyzed using statistical methods to identify patterns and relationships.
• Large sample size: Quantitative research often involves collecting data from a large sample of individuals or groups in order to increase the reliability and generalizability of the findings.
• Objective approach: Quantitative research aims to be objective and impartial in its approach, focusing on the collection and analysis of data rather than personal beliefs, opinions, or experiences.
• Control over variables: Quantitative research often involves manipulating variables to test hypotheses and establish cause-and-effect relationships. Researchers aim to control for extraneous variables that may impact the results.
• Replicable : Quantitative research aims to be replicable, meaning that other researchers should be able to conduct similar studies and obtain similar results using the same methods.
• Statistical analysis: Quantitative research involves using statistical tools and techniques to analyze the numerical data collected during the research process. Statistical analysis allows researchers to identify patterns, trends, and relationships between variables, and to test hypotheses and theories.
• Generalizability: Quantitative research aims to produce findings that can be generalized to larger populations beyond the specific sample studied. This is achieved through the use of random sampling methods and statistical inference.

Examples of Quantitative Research

Here are some examples of quantitative research in different fields:

• Market Research: A company conducts a survey of 1000 consumers to determine their brand awareness and preferences. The data is analyzed using statistical methods to identify trends and patterns that can inform marketing strategies.
• Health Research : A researcher conducts a randomized controlled trial to test the effectiveness of a new drug for treating a particular medical condition. The study involves collecting data from a large sample of patients and analyzing the results using statistical methods.
• Social Science Research : A sociologist conducts a survey of 500 people to study attitudes toward immigration in a particular country. The data is analyzed using statistical methods to identify factors that influence these attitudes.
• Education Research: A researcher conducts an experiment to compare the effectiveness of two different teaching methods for improving student learning outcomes. The study involves randomly assigning students to different groups and collecting data on their performance on standardized tests.
• Environmental Research : A team of researchers conduct a study to investigate the impact of climate change on the distribution and abundance of a particular species of plant or animal. The study involves collecting data on environmental factors and population sizes over time and analyzing the results using statistical methods.
• Psychology : A researcher conducts a survey of 500 college students to investigate the relationship between social media use and mental health. The data is analyzed using statistical methods to identify correlations and potential causal relationships.
• Political Science: A team of researchers conducts a study to investigate voter behavior during an election. They use survey methods to collect data on voting patterns, demographics, and political attitudes, and analyze the results using statistical methods.

How to Conduct Quantitative Research

Here is a general overview of how to conduct quantitative research:

• Develop a research question: The first step in conducting quantitative research is to develop a clear and specific research question. This question should be based on a gap in existing knowledge, and should be answerable using quantitative methods.
• Develop a research design: Once you have a research question, you will need to develop a research design. This involves deciding on the appropriate methods to collect data, such as surveys, experiments, or observational studies. You will also need to determine the appropriate sample size, data collection instruments, and data analysis techniques.
• Collect data: The next step is to collect data. This may involve administering surveys or questionnaires, conducting experiments, or gathering data from existing sources. It is important to use standardized methods to ensure that the data is reliable and valid.
• Analyze data : Once the data has been collected, it is time to analyze it. This involves using statistical methods to identify patterns, trends, and relationships between variables. Common statistical techniques include correlation analysis, regression analysis, and hypothesis testing.
• Interpret results: After analyzing the data, you will need to interpret the results. This involves identifying the key findings, determining their significance, and drawing conclusions based on the data.
• Communicate findings: Finally, you will need to communicate your findings. This may involve writing a research report, presenting at a conference, or publishing in a peer-reviewed journal. It is important to clearly communicate the research question, methods, results, and conclusions to ensure that others can understand and replicate your research.

When to use Quantitative Research

Here are some situations when quantitative research can be appropriate:

• To test a hypothesis: Quantitative research is often used to test a hypothesis or a theory. It involves collecting numerical data and using statistical analysis to determine if the data supports or refutes the hypothesis.
• To generalize findings: If you want to generalize the findings of your study to a larger population, quantitative research can be useful. This is because it allows you to collect numerical data from a representative sample of the population and use statistical analysis to make inferences about the population as a whole.
• To measure relationships between variables: If you want to measure the relationship between two or more variables, such as the relationship between age and income, or between education level and job satisfaction, quantitative research can be useful. It allows you to collect numerical data on both variables and use statistical analysis to determine the strength and direction of the relationship.
• To identify patterns or trends: Quantitative research can be useful for identifying patterns or trends in data. For example, you can use quantitative research to identify trends in consumer behavior or to identify patterns in stock market data.
• To quantify attitudes or opinions : If you want to measure attitudes or opinions on a particular topic, quantitative research can be useful. It allows you to collect numerical data using surveys or questionnaires and analyze the data using statistical methods to determine the prevalence of certain attitudes or opinions.

Purpose of Quantitative Research

The purpose of quantitative research is to systematically investigate and measure the relationships between variables or phenomena using numerical data and statistical analysis. The main objectives of quantitative research include:

• Description : To provide a detailed and accurate description of a particular phenomenon or population.
• Explanation : To explain the reasons for the occurrence of a particular phenomenon, such as identifying the factors that influence a behavior or attitude.
• Prediction : To predict future trends or behaviors based on past patterns and relationships between variables.
• Control : To identify the best strategies for controlling or influencing a particular outcome or behavior.

Quantitative research is used in many different fields, including social sciences, business, engineering, and health sciences. It can be used to investigate a wide range of phenomena, from human behavior and attitudes to physical and biological processes. The purpose of quantitative research is to provide reliable and valid data that can be used to inform decision-making and improve understanding of the world around us.

Advantages of Quantitative Research

There are several advantages of quantitative research, including:

• Objectivity : Quantitative research is based on objective data and statistical analysis, which reduces the potential for bias or subjectivity in the research process.
• Reproducibility : Because quantitative research involves standardized methods and measurements, it is more likely to be reproducible and reliable.
• Generalizability : Quantitative research allows for generalizations to be made about a population based on a representative sample, which can inform decision-making and policy development.
• Precision : Quantitative research allows for precise measurement and analysis of data, which can provide a more accurate understanding of phenomena and relationships between variables.
• Efficiency : Quantitative research can be conducted relatively quickly and efficiently, especially when compared to qualitative research, which may involve lengthy data collection and analysis.
• Large sample sizes : Quantitative research can accommodate large sample sizes, which can increase the representativeness and generalizability of the results.

Limitations of Quantitative Research

There are several limitations of quantitative research, including:

• Limited understanding of context: Quantitative research typically focuses on numerical data and statistical analysis, which may not provide a comprehensive understanding of the context or underlying factors that influence a phenomenon.
• Simplification of complex phenomena: Quantitative research often involves simplifying complex phenomena into measurable variables, which may not capture the full complexity of the phenomenon being studied.
• Potential for researcher bias: Although quantitative research aims to be objective, there is still the potential for researcher bias in areas such as sampling, data collection, and data analysis.
• Limited ability to explore new ideas: Quantitative research is often based on pre-determined research questions and hypotheses, which may limit the ability to explore new ideas or unexpected findings.
• Limited ability to capture subjective experiences : Quantitative research is typically focused on objective data and may not capture the subjective experiences of individuals or groups being studied.
• Ethical concerns : Quantitative research may raise ethical concerns, such as invasion of privacy or the potential for harm to participants.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

One-to-One Interview – Methods and Guide

Quasi-Experimental Research Design – Types...

Explanatory Research – Types, Methods, Guide

Applied Research – Types, Methods and Examples

Phenomenology – Methods, Examples and Guide

Qualitative Research – Methods, Analysis Types...

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Using data for...

Using data for improvement

Read the full collection.

• Related content
• Peer review
• Amar Shah , chief quality officer and consultant forensic psychiatrist, national improvement lead for the Mental Health Safety Improvement Programme
• East London NHS Foundation Trust, London, E1 8DE, UK
• amarshah{at}nhs.net @DrAmarShah

What you need to know

Both qualitative and quantitative data are critical for evaluating and guiding improvement

A family of measures, incorporating outcome, process, and balancing measures, should be used to track improvement work

Time series analysis, using small amounts of data collected and displayed frequently, is the gold standard for using data for improvement

We all need a way to understand the quality of care we are providing, or receiving, and how our service is performing. We use a range of data in order to fulfil this need, both quantitative and qualitative. Data are defined as “information, especially facts and numbers, collected to be examined and considered and used to help decision-making.” 1 Data are used to make judgements, to answer questions, and to monitor and support improvement in healthcare ( box 1 ). The same data can be used in different ways, depending on what we want to know or learn.

Defining quality improvement 2

Quality improvement aims to make a difference to patients by improving safety, effectiveness, and experience of care by:

Using understanding of our complex healthcare environment

Applying a systematic approach

Designing, testing, and implementing changes using real-time measurement for improvement

Within healthcare, we use a range of data at different levels of the system:

Patient level—such as blood sugar, temperature, blood test results, or expressed wishes for care)

Service level—such as waiting times, outcomes, complaint themes, or collated feedback of patient experience

Organisation level—such as staff experience or financial performance

Population level—such as mortality, quality of life, employment, and air quality.

This article outlines the data we need to understand the quality of care we are providing, what we need to capture to see if care is improving, how to interpret the data, and some tips for doing this more effectively.

Sources and selection criteria

This article is based on my experience of using data for improvement at East London NHS Foundation Trust, which is seen as one of the world leaders in healthcare quality improvement. Our use of data, from trust board to clinical team, has transformed over the past six years in line with the learning shared in this article. This article is also based on my experience of teaching with the Institute for Healthcare Improvement, which guides and supports quality improvement efforts across the globe.

What data do we need?

Healthcare is a complex system, with multiple interdependencies and an array of factors influencing outcomes. Complex systems are open, unpredictable, and continually adapting to their environment. 3 No single source of data can help us understand how a complex system behaves, so we need several data sources to see how a complex system in healthcare is performing.

Avedis Donabedian, a doctor born in Lebanon in 1919, studied quality in healthcare and contributed to our understanding of using outcomes. 4 He described the importance of focusing on structures and processes in order to improve outcomes. 5 When trying to understand quality within a complex system, we need to look at a mix of outcomes (what matters to patients), processes (the way we do our work), and structures (resources, equipment, governance, etc).

Therefore, when we are trying to improve something, we need a small number of measures (ideally 5-8) to help us monitor whether we are moving towards our goal. Any improvement effort should include one or two outcome measures linked explicitly to the aim of the work, a small number of process measures that show how we are doing with the things we are actually working on to help us achieve our aim, and one or two balancing measures ( box 2 ). Balancing measures help us spot unintended consequences of the changes we are making. As complex systems are unpredictable, our new changes may result in an unexpected adverse effect. Balancing measures help us stay alert to these, and ought to be things that are already collected, so that we do not waste extra resource on collecting these.

Different types of measures of quality of care

Outcome measures (linked explicitly to the aim of the project).

Aim— To reduce waiting times from referral to appointment in a clinic

Outcome measure— Length of time from referral being made to being seen in clinic

Data collection— Date when each referral was made, and date when each referral was seen in clinic, in order to calculate the time in days from referral to being seen

Process measures (linked to the things you are going to work on to achieve the aim)

Change idea— Use of a new referral form (to reduce numbers of inappropriate referrals and re-work in obtaining necessary information)

Process measure— Percentage of referrals received that are inappropriate or require further information

Data collection— Number of referrals received that are inappropriate or require further information each week divided by total number of referrals received each week

Change idea— Text messaging patients two days before the appointment (to reduce non-attendance and wasted appointment slots)

Process measure— Percentage of patients receiving a text message two days before appointment

Data collection— Number of patients each week receiving a text message two days before their appointment divided by the total number of patients seen each week

Process measure— Percentage of patients attending their appointment

Data collection— Number of patients attending their appointment each week divided by the total number of patients booked in each week

Balancing measures (to spot unintended consequences)

Measure— Percentage of referrers who are satisfied or very satisfied with the referral process (to spot whether all these changes are having a detrimental effect on the experience of those referring to us)

Data collection— A monthly survey to referrers to assess their satisfaction with the referral process

Measure— Percentage of staff who are satisfied or very satisfied at work (to spot whether the changes are increasing burden on staff and reducing their satisfaction at work)

Data collection— A monthly survey for staff to assess their satisfaction at work

How should we look at the data?

This depends on the question we are trying to answer. If we ask whether an intervention was efficacious, as we might in a research study, we would need to be able to compare data before and after the intervention and remove all potential confounders and bias. For example, to understand whether a new treatment is better than the status quo, we might design a research study to compare the effect of the two interventions and ensure that all other characteristics are kept constant across both groups. This study might take several months, or possibly years, to complete, and would compare the average of both groups to identify whether there is a statistically significant difference.

This approach is unlikely to be possible in most contexts where we are trying to improve quality. Most of the time when we are improving a service, we are making multiple changes and assessing impact in real-time, without being able to remove all confounding factors and potential bias. When we ask whether an outcome has improved, as we do when trying to improve something, we need to be able to look at data over time to see how the system changes as we intervene, with multiple tests of change over a period. For example, if we were trying to improve the time from a patient presenting in the emergency department to being admitted to a ward, we would likely be testing several different changes at different places in the pathway. We would want to be able to look at the outcome measure of total time from presentation to admission on the ward, over time, on a daily basis, to be able to see whether the changes made lead to a reduction in the overall outcome. So, when looking at a quality issue from an improvement perspective, we view smaller amounts of data but more frequently to see if we are improving over time. 2

What is best practice in using data to support improvement?

Best practice would be for each team to have a small number of measures that are collectively agreed with patients and service users as being the most important ways of understanding the quality of the service being provided. These measures would be displayed transparently so that all staff, service users, and patients and families or carers can access them and understand how the service is performing. The data would be shown as time series analysis, to provide a visual display of whether the service is improving over time. The data should be available as close to real-time as possible, ideally on a daily or weekly basis. The data should prompt discussion and action, with the team reviewing the data regularly, identifying any signals that suggest something unusual in the data, and taking action as necessary.

The main tools used for this purpose are the run chart and the Shewhart (or control) chart. The run chart ( fig 1 ) is a graphical display of data in time order, with a median value, and uses probability-based rules to help identify whether the variation seen is random or non-random. 2 The Shewhart (control) chart ( fig 2 ) also displays data in time order, but with a mean as the centre line instead of a median, and upper and lower control limits (UCL and LCL) defining the boundaries within which you would predict the data to be. 6 Shewhart charts use the terms “common cause variation” and “special cause variation,” with a different set of rules to identify special causes.

A typical run chart

• Download figure
• Open in new tab
• Download powerpoint

A typical Shewhart (or control) chart

Is it just about numbers?

We need to incorporate both qualitative and quantitative data to help us learn about how the system is performing and to see if we improve over time. Quantitative data express quantity, amount, or range and can be measured numerically—such as waiting times, mortality, haemoglobin level, cash flow. Quantitative data are often visualised over time as time series analyses (run charts or control charts) to see whether we are improving.

However, we should also be capturing, analysing, and learning from qualitative data throughout our improvement work. Qualitative data are virtually any type of information that can be observed and recorded that is not numerical in nature. Qualitative data are particularly useful in helping us to gain deeper insight into an issue, and to understand meaning, opinion, and feelings. This is vital in supporting us to develop theories about what to focus on and what might make a difference. 7 Examples of qualitative data include waiting room observation, feedback about experience of care, free-text responses to a survey.

Using qualitative data for improvement

One key point in an improvement journey when qualitative data are critical is at the start, when trying to identify “What matters most?” and what the team’s biggest opportunity for improvement is. The other key time to use qualitative data is during “Plan, Do, Study, Act” (PDSA) cycles. Most PDSA cycles, when done well, rely on qualitative data as well as quantitative data to help learn about how the test fared compared with our original theory and prediction.

Table 1 shows four different ways to collect qualitative data, with advantages and disadvantages of each, and how we might use them within our improvement work.

Different ways to collect qualitative data for improvement

• View inline

Tips to overcome common challenges in using data for improvement?

One of the key challenges faced by healthcare teams across the globe is being able to access data that is routinely collected, in order to use it for improvement. Large volumes of data are collected in healthcare, but often little is available to staff or service users in a timescale or in a form that allows it to be useful for improvement. One way to work around this is to have a simple form of measurement on the unit, clinic, or ward that the team own and update. This could be in the form of a safety cross 8 or tally chart. A safety cross ( fig 3 ) is a simple visual monthly calendar on the wall which allows teams to identify when a safety event (such as a fall) occurred on the ward. The team simply colours in each day green when no fall occurred, or colours in red the days when a fall occurred. It allows the team to own the data related to a safety event that they care about and easily see how many events are occurring over a month. Being able to see such data transparently on a ward allows teams to update data in real time and be able to respond to it effectively.

Example of a safety cross in use

A common challenge in using qualitative data is being able to analyse large quantities of written word. There are formal approaches to qualitative data analyses, but most healthcare staff are not trained in these methods. Key tips in avoiding this difficulty are ( a ) to be intentional with your search and sampling strategy so that you collect only the minimum amount of data that is likely to be useful for learning and ( b ) to use simple ways to read and theme the data in order to extract useful information to guide your improvement work. 9 If you want to try this, see if you can find someone in your organisation with qualitative data analysis skills, such as clinical psychologists or the patient experience or informatics teams.

Education into practice

What are the key measures for the service that you work in?

Are these measures available, transparently displayed, and viewed over time?

What qualitative data do you use in helping guide your improvement efforts?

How patients were involved in the creation of this article

Service users are deeply involved in all quality improvement work at East London NHS Foundation Trust, including within the training programmes we deliver. Shared learning over many years has contributed to our understanding of how best to use all types of data to support improvement. No patients have had input specifically into this article.

This article is part of a series commissioned by The BMJ based on ideas generated by a joint editorial group with members from the Health Foundation and The BMJ , including a patient/carer. The BMJ retained full editorial control over external peer review, editing, and publication. Open access fees and The BMJ ’s quality improvement editor post are funded by the Health Foundation.

Competing interests: I have read and understood the BMJ Group policy on declaration of interests and have no relevant interests to declare.

Provenance and peer review: Commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

• ↵ Cambridge University Press. Cambridge online dictionary , 2008. https://dictionary.cambridge.org/ .
• Provost LP ,
• Braithwaite J
• Neuhauser D
• Donabedian A
• Mohammed MA
• Davidoff F ,
• Dixon-Woods M ,
• Leviton L ,
• ↵ Flynn M. Quality & Safety—The safety cross system: simple and effective. https://www.inmo.ie/MagazineArticle/PrintArticle/11155 .

• Research article
• Open access
• Published: 01 December 2006

Using quantitative and qualitative data in health services research – what happens when mixed method findings conflict? [ISRCTN61522618]

• Suzanne Moffatt 1 ,
• Martin White 1 ,
• Joan Mackintosh 1 &
• Denise Howel 1  

BMC Health Services Research volume  6 , Article number:  28 ( 2006 ) Cite this article

85k Accesses

128 Citations

16 Altmetric

Metrics details

In this methodological paper we document the interpretation of a mixed methods study and outline an approach to dealing with apparent discrepancies between qualitative and quantitative research data in a pilot study evaluating whether welfare rights advice has an impact on health and social outcomes among a population aged 60 and over.

Quantitative and qualitative data were collected contemporaneously. Quantitative data were collected from 126 men and women aged over 60 within a randomised controlled trial. Participants received a full welfare benefits assessment which successfully identified additional financial and non-financial resources for 60% of them. A range of demographic, health and social outcome measures were assessed at baseline, 6, 12 and 24 month follow up. Qualitative data were collected from a sub-sample of 25 participants purposively selected to take part in individual interviews to examine the perceived impact of welfare rights advice.

Separate analysis of the quantitative and qualitative data revealed discrepant findings. The quantitative data showed little evidence of significant differences of a size that would be of practical or clinical interest, suggesting that the intervention had no impact on these outcome measures. The qualitative data suggested wide-ranging impacts, indicating that the intervention had a positive effect. Six ways of further exploring these data were considered: (i) treating the methods as fundamentally different; (ii) exploring the methodological rigour of each component; (iii) exploring dataset comparability; (iv) collecting further data and making further comparisons; (v) exploring the process of the intervention; and (vi) exploring whether the outcomes of the two components match.

The study demonstrates how using mixed methods can lead to different and sometimes conflicting accounts and, using this six step approach, how such discrepancies can be harnessed to interrogate each dataset more fully. Not only does this enhance the robustness of the study, it may lead to different conclusions from those that would have been drawn through relying on one method alone and demonstrates the value of collecting both types of data within a single study. More widespread use of mixed methods in trials of complex interventions is likely to enhance the overall quality of the evidence base.

Combining quantitative and qualitative methods in a single study is not uncommon in social research, although, 'traditionally a gulf is seen to exist between qualitative and quantitative research with each belonging to distinctively different paradigms'. [ 1 ] Within health research there has, more recently, been an upsurge of interest in the combined use of qualitative and quantitative methods, sometimes termed mixed methods research [ 2 ] although the terminology can vary. [ 3 ] Greater interest in qualitative research has come about for a number of reasons: the numerous contributions made by qualitative research to the study of health and illness [ 4 – 6 ]; increased methodological rigor [ 7 ] within the qualitative paradigm, which has made it more acceptable to researchers or practitioners trained within a predominantly quantitative paradigm [ 8 ]; and, because combining quantitative and qualitative methods may generate deeper insights than either method alone. [ 9 ] It is now widely recognised that public health problems are embedded within a range of social, political and economic contexts. [ 10 ] Consequently, a range of epidemiological and social science methods are employed to research these complex issues. [ 11 ] Further legitimacy for the use of qualitative methods alongside quantitative has resulted from the recognition that qualitative methods can make an important contribution to randomised controlled trials (RCTs) evaluating complex health service interventions. There is published work on the various ways that qualitative methods are being used in RCTs (e.g. [ 12 , 13 ] but little on how they can optimally enhance the usefulness and policy relevance of trial findings. [ 14 , 15 ]

A number of mixed methods publications outline the various ways in which qualitative and quantitative methods can be combined. [ 1 , 2 , 9 , 16 ] For the purposes of this paper with its focus on mixed methods in the context of a pilot RCT, the significant aspects of mixed methods appear to be: purpose, process and, analysis and interpretation. In terms of purpose, qualitative research may be used to help identify the relevant variables for study [ 17 ], develop an instrument for quantitative research [ 18 ], to examine different questions (such as acceptability of the intervention, rather than its outcome) [ 19 ]; and to examine the same question with different methods (using, for example participant observation or in depth interviews [ 1 ]). Process includes the priority accorded to each method and ordering of both methods which may be concurrent, sequential or iterative. [ 20 ] Bryman [ 9 ] points out that, 'most researchers rely primarily on a method associated with either quantitative or qualitative methods and then buttress their findings with a method associated with the other tradition' (p128). Both datasets may be brought together at the 'analysis/interpretation' phase, often known as 'triangulation' [ 21 ]. Brannen [ 1 ] suggests that most researchers have taken this to mean more than one type of data, but she stresses that Denzin's original conceptualisation involved methods, data, investigators or theories. Bringing different methods together almost inevitably raises discrepancies in findings and their interpretation. However, the investigation of such differences may be as illuminating as their points of similarity. [ 1 , 9 ]

Although mixed methods are now widespread in health research, quantitative and qualitative methods and results are often published separately. [ 22 , 23 ] It is relatively rare to see an account of the methodological implications of the strategy and the way in which both methods are combined when interpreting the data within a particular study. [ 1 ] A notable exception is a study showing divergence between qualitative and quantitative findings of cancer patients' quality of life using a detailed case study approach to the data. [ 13 ]

By presenting quantitative and qualitative data collected within a pilot RCT together, this paper has three main aims: firstly, to demonstrate how divergent quantitative and qualitative data led us to interrogate each dataset more fully and assisted in the interpretation process, producing a greater research yield from each dataset; secondly, to demonstrate how combining both types of data at the analysis stage produces 'more than the sum of its parts'; and thirdly, to emphasise the complementary nature of qualitative and quantitative methods in RCTs of complex interventions. In doing so, we demonstrate how the combination of quantitative and qualitative data led us to conclusions different from those that would have been drawn through relying on one or other method alone.

The study that forms the basis of this paper, a pilot RCT to examine the impact of welfare rights advice in primary care, was funded under the UK Department of Health's Policy Research Programme on tackling health inequalities, and focused on older people. To date, little research has been able to demonstrate how health inequalities can be tackled by interventions within and outside the health sector. Although living standards have risen among older people, a common experience of growing old is worsening material circumstances. [ 24 ] In 2000–01 there were 2.3 million UK pensioners living in households with below 60 per cent of median household income, after housing costs. [ 25 ] Older people in the UK may be eligible for a number of income- or disability-related benefits (the latter could be non-financial such as parking permits or adaptations to the home), but it has been estimated that approximately one in four (about one million) UK pensioner households do not claim the support to which they are entitled. [ 26 ] Action to facilitate access to and uptake of welfare benefits has taken place outside the UK health sector for many years and, more recently, has been introduced within parts of the health service, but its potential to benefit health has not been rigorously evaluated. [ 27 – 29 ]

There are a number of models of mixed methods research. [ 2 , 16 , 30 ] We adopted a model which relies of the principle of complementarity, using the strengths of one method to enhance the other. [ 30 ] We explicitly recognised that each method was appropriate for different research questions. We undertook a pragmatic RCT which aimed to evaluate the health effects of welfare rights advice in primary care among people aged over 60. Quantitative data included standardised outcome measures of health and well-being, health related behaviour, psycho-social interaction and socio-economic status ; qualitative data used semi-structured interviews to explore participants' views about the intervention, its outcome, and the acceptability of the research process.

Following an earlier qualitative pilot study to inform the selection of appropriate outcome measures [ 31 ], contemporaneous quantitative and qualitative data were collected. Both datasets were analysed separately and neither compared until both analyses were complete. The sampling strategy mirrored the embedded design; probability sampling for the quantitative study and theoretical sampling for the qualitative study, done on the basis of factors identified in the quantitative study.

Approval for the study was obtained from Newcastle and North Tyneside Joint Local Research Ethics Committee and from Newcastle Primary Care Trust.

The intervention

The intervention was delivered by a welfare rights officer from Newcastle City Council Welfare Rights Service in participants' own homes and comprised a structured assessment of current welfare status and benefits entitlement, together with active assistance in making claims where appropriate over the following six months, together with necessary follow-up for unresolved claims.

Quantitative study

The design presented ethical dilemmas as it was felt problematic to deprive the control group of welfare rights advice, since there is adequate evidence to show that it leads to significant financial gains. [ 32 ] To circumvent this dilemma, we delivered welfare rights advice to the control group six months after the intervention group. A single-blinded RCT with allocation of individuals to intervention (receipt of welfare rights consultation immediately) and control condition (welfare rights consultation six months after entry into the trial) was undertaken.

Four general practices located at five surgeries across Newcastle upon Tyne took part. Three of the practices were located in the top ten per cent of most deprived wards in England using the Index of Multiple Deprivation (two in the top one percent – ranked 30 th and 36 th most deprived); the other practice was ranked 3,774 out of a total of 8,414 in England. [ 33 ]

Using practice databases, a random sample of 100 patients aged 60 years or over from each of four participating practices was invited to take part in the study. Only one individual per household was allowed to participate in the trial, but if a partner or other adult household member was also eligible for benefits, they also received welfare rights advice. Patients were excluded if they were permanently hospitalised or living in residential or nursing care homes.

Written informed consent was obtained at the baseline interview. Structured face to face interviews were carried out at baseline, six, 12 and 24 months using standard scales covering the areas of demographics, mental and physical health (SF36) [ 34 ], Hospital Anxiety and Depression Scale (HADS) [ 35 ], psychosocial descriptors (e.g. Social Support Questionnaire [ 36 ] and the Self-Esteem Inventory, [ 37 ], and socioeconomic indicators (e.g. affordability and financial vulnerability). [ 38 ] Additionally, a short semi-structured interview was undertaken at 24 months to ascertain the perceived impact of additional resources for those who received them.

All health and welfare assessment data were entered onto customised MS Access databases and checked for quality and completeness. Data were transferred to the Statistical Package for the Social Sciences (SPSS) v11.0 [ 39 ] and STATA v8.0 for analysis. [ 40 ]

Qualitative study

The qualitative findings presented in this paper focus on the impact of the intervention. The sampling frame was formed by those (n = 96) who gave their consent to be contacted during their baseline interview for the RCT. The study sample comprised respondents from intervention and control groups purposively selected to include those eligible for the following resources: financial only; non-financial only; both financial and non financial; and, none. Sampling continued until no new themes emerged from the interviews; until data 'saturation' was reached. [ 21 ]

Initial interviews took place between April and December 2003 in participants' homes after their welfare rights assessment; follow-up interviews were undertaken in January and February 2005. The semi-structured interview schedule covered perceptions of: impact of material and/or financial benefits; impact on mental and/or physical health; impact on health related behaviours; social benefits; and views about the link between material resources and health. All participants agreed to the interview being audio-recorded. Immediately afterwards, observational field notes were made. Interviews were transcribed in full.

Data analysis largely followed the framework approach. [ 41 ] Data were coded, indexed and charted systematically; and resulting typologies discussed with other members of the research team, 'a pragmatic version of double coding'. [ 42 ] Constant comparison [ 43 ] and deviant case analysis [ 44 ] were used since both methods are important for internal validation. [ 7 , 42 ] Finally, sets of categories at a higher level of abstraction were developed.

A brief semi-structured interview was undertaken (by JM) with all participants who received additional resources. These interview data explored the impact data of additional resources on all of those who received them, not just the qualitative sub-sample. The data were independently coded by JM and SM using the same coding frame. Discrepant codes were examined by both researchers and a final code agreed.

One hundred and twenty six people were recruited into the study; there were 117 at 12 month follow-up and 109 at 24 months (five deaths, one moved, the remainder declined).

Table 1 shows the distribution of financial and non-financial benefits awarded as a result of the welfare assessments. Sixty percent of participants were awarded some form of welfare benefit, and just over 40% received a financial benefit. Some households received more than one type of benefit.

Table 2 compares the quantitative and qualitative sub-samples on a number of personal, economic, health and lifestyle factors at baseline. Intervention and control groups were comparable.

Table 3 compares outcome measures by award group, i.e. no award, non-financial and financial and shows only small differences between the mean changes across each group, none of which were statistically significant. Other analyses of the quantitative data compared the changes seen between baseline and six months (by which time the intervention group had received the welfare rights advice but the control group had not) and found little evidence of differences between the intervention and control groups of any practical importance. The only statistically significant difference between the groups was a small decrease in financial vulnerability in the intervention group after six months. [ 45 ]

There was little evidence for differences in health and social outcomes measures as a result of the receipt of welfare advice of a size that would be of major practical or clinical interest. However, this was a pilot study, with only the power to detect large differences if they were present. One reason for a lack of difference may be that the scales were less appropriate for older people and did not capture all relevant outcomes. Another reason for the lack of differences may be that insufficient numbers of people had received their benefits for long enough to allow any health outcomes to have changed when comparisons were made. Fourteen per cent of participants found to be eligible for financial benefits had not started receiving their benefits by the time of the first follow-up interview after their benefit assessment (six months for intervention, 12 months for control); and those who had, had only received them for an average of 2 months. This is likely to have diluted any impact of the intervention effect, and might account, to some extent, for the lack of observed effect.

Twenty five interviews were completed, fourteen of whom were from the intervention group. Ten participants were interviewed with partners who made active contributions. Twenty two follow-up interviews were undertaken between twelve and eighteen months later (three individuals were too ill to take part).

Table 1 (fifth column) shows that 14 of the participants in the qualitative study received some financial award. The median income gain was (€84, $101) (range £10 (€15, $18) -£100 (€148, $178)) representing a 4%-55% increase in weekly income. 18 participants were in receipt of benefit, either as a result of the current intervention or because of claims made prior to this study.

By the follow-up (FU) interviews all but one participant had been receiving their benefits for between 17 and 31 months. The intervention was viewed positively by all interviewees irrespective of outcome. However, for the fourteen participants who received additional financial resources the impact was considerable and accounts revealed a wide range of uses for the extra money. Participants' accounts revealed four linked categories, summarised on Table 4 . Firstly, increased affordability of necessities , without which maintaining independence and participating in daily life was difficult. This included accessing transport, maintaining social networks and social activities, buying better quality food, stocking up on food, paying bills, preventing debt and affording paid help for household activities. Secondly, occasional expenses such as clothes, household equipment, furniture and holidays were more affordable. Thirdly, extra income was used to act as a cushion against potential emergencies and to increase savings . Fourthly, all participants described the easing of financial worries as bringing ' peace of mind' .

Without exception, participants were of the view that extra money or resources would not improve existing health problems. The reasons behind these strongly held views about individual health conditions was generally that their poor health was attributed to specific health conditions and a combination of family history or fate, which were immune to the effects of money. Most participants had more than one chronic condition and felt that because of these conditions, plus their age, additional money would have no effect.

However, a number of participants linked the impact of the intervention with improved ways of coping with their conditions because of what the extra resources enabled them to do:

Mrs T: Having money is not going to improve his health, we could win the lottery and he would still have his health problems.

Mr T: No, but we don't need to worry if I wanted .... Well I mean I eat a lot of honey and I think it's very good, very healthful for you ... at one time we couldn't have afforded to buy these things. Now we can go and buy them if I fancy something, just go and get it where we couldn't before .

Mrs T: Although the Attendance Allowance is actually his [partners], it's made me relax a bit more ... I definitely worry less now (N15, female, 62 and partner)

Despite the fact that no-one expected their own health conditions to improve, most people believed that there was a link between resources and health in a more abstract sense, either because they experienced problems affording necessities such as healthy food or maintaining adequate heat in their homes, or because they empathised with those who lacked money. Participants linked adequate resources to maintaining health and contributing to a sense of well-being.

Money does have a lot to do with health if you are poor. It would have a lot to do with your health ... I don't buy loads and loads of luxuries, but I know I can go out and get the food we need and that sort of thing. I think that money is a big part of how a house, or how people in that house are . (N13, female, 72)

Comparing the results from the two datasets

When the separate analyses of the quantitative and qualitative datasets after the 12 month follow-up structured interviews were completed, the discrepancy in the findings became apparent. The quantitative study showed little evidence of a size that would be of practical or clinical interest, suggesting that the intervention had no impact on these outcome measures. The qualitative study found a wide-ranging impact, indicating that the intervention had a positive effect. The presence of such inter-method discrepancy led to a great deal of discussion and debate, as a result of which we devised six ways of further exploring these data.

(i) Treating the methods as fundamentally different

This process of simultaneous qualitative and quantitative dataset interrogation enables a deeper level of analysis and interpretation than would be possible with one or other alone and demonstrates how mixed methods research produces more than the sum of its parts. It is worth emphasising however, that it is not wholly surprising that each method comes up with divergent findings since each asked different, but related questions, and both are based on fundamentally different theoretical paradigms. Brannen [ 1 ] and Bryman [ 9 ] argue that it is essential to take account of these theoretical differences and caution against taking a purely technical approach to the use of mixed methods, a simple 'bolting together' of techniques. [ 17 ] Combining the two methods for crossvalidation (triangulation) purposes is not a viable option because it rests on the premise that both methods are examining the same research problem. [ 1 ] We have approached the divergent findings as indicative of different aspects of the phenomena in question and searched for reasons which might explain these inconsistencies. In the approach that follows, we have treated the datasets as complementary, rather than attempt to integrate them, since each approach reflects a different view on how social reality ought to be studied.

(ii) Exploring the methodological rigour of each component

It is standard practice at the data analysis and interpretation phases of any study to scrutinise methodological rigour. However, in this case, we had another dataset to use as a yardstick for comparison and it became clear that our interrogation of each dataset was informed to some extent by the findings of the other. It was not the case that we expected to obtain the same results, but clearly the divergence of our findings was of great interest and made us more circumspect about each dataset. We began by examining possible reasons why there might be problems with each dataset individually, but found ourselves continually referring to the results of the other study as a benchmark for comparison.

With regard to the quantitative study, it was a pilot, of modest sample size, and thus not powered to detect small differences in the key outcome measures. In addition there were three important sources of dilution effects: firstly, only 63% of intervention group participants received some type of financial award; secondly, we found that 14% of those in the trial eligible for financial benefits did not receive their money until after the follow up assessments had been carried out; and thirdly, many had received their benefits for only a short period, reducing the possibility of detecting any measurable effects at the time of follow-up. All of these factors provide some explanation for the lack of a measurable effect between intervention and control group and between those who did and did not receive additional financial resources.

The number of participants in the qualitative study who received additional financial resources as a result of this intervention was small (n = 14). We would argue that the fieldwork, analysis and interpretation [ 46 ] were sufficiently transparent to warrant the degree of methodological rigour advocated by Barbour [ 7 , 17 ] and that the findings were therefore an accurate reflection of what was being studied. However, there still remained the possibility that a reason for the discrepant findings was due to differences between the qualitative sub-sample and the parent sample, which led us to step three.

(iii) Exploring dataset comparability

We compared the qualitative and quantitative samples on a number of social and economic factors (Table 2 ). In comparison to the parent sample, the qualitative sub-sample was slightly older, had fewer men, a higher proportion with long-term limiting illness, but fewer current smokers. However, there was nothing to indicate that such small differences would account for the discrepancies. There were negligible differences in SF-36 (Physical and Mental) and HAD (Anxiety and Depression) scores between the groups at baseline, which led us to discount the possibility that those in the quantitative sub sample were markedly different to the quantitative sample on these outcome measures.

(iv) Collection of additional data and making further comparisons

The divergent findings led us to seek further funding to undertake collection of additional quantitative and qualitative data at 24 months. The quantitative and qualitative follow-up data verified the initial findings of each study. [ 45 ] We also collected a limited amount of qualitative data on the perceived impact of resources, from all participants who had received additional resources. These data are presented in figure 1 which shows the uses of additional resources at 24 month follow-up for 35 participants (N = 35, 21 previously in quantitative study only, 14 in both). This dataset demonstrates that similar issues emerged for both qualitative and quantitative datasets: transport, savings and 'peace of mind' emerged as key issues, but the data also showed that the additional money was used on a wide range of items. This follow-up confirmed the initial findings of each study and further, indicated that the perceived impact of the additional resources was the same for a larger sample than the original qualitative sub-sample, further confirming our view that the positive findings extended beyond the fourteen participants in the qualitative sub-sample, to all those receiving additional resources.

Use of additional resources at 2 year follow up (N = 35)*.

(v) Exploring whether the intervention under study worked as expected

The qualitative study revealed that many participants had received welfare benefits via other services prior to this study, revealing the lack of a 'clean slate' with regard to the receipt of benefits, which we had not anticipated. We investigated this further in the quantitative dataset and found that 75 people (59.5%) had received benefits prior to the study; if the first benefit was on health grounds, a later one may have been because their health had deteriorated further.

(vi) Exploring whether the outcomes of the quantitative and qualitative components match

'Probing certain issues in greater depth' as advocated by Bryman (p134) [ 1 ] focussed our attention on the outcome measures used in the quantitative part of the study and revealed several challenges. Firstly, the qualitative study revealed a number of dimensions not measured by the quantitative study, such as, 'maintaining independence' which included affording paid help, increasing and improving access to facilities and managing better within the home. Secondly, some of the measures used with the intention of capturing dimensions of mental health did not adequately encapsulate participants' accounts of feeling 'less stressed' and 'less depressed' by financial worries. Probing both datasets also revealed congruence along the dimension of physical health. No differences were found on the SF36 physical scale and participants themselves did not expect an improvement in physical health (for reasons of age and chronic health problems). The real issue would appear to be measuring ways in which older people are better able to cope with existing health problems and maintain their independence and quality of life, despite these conditions.

Qualitative study results also led us to look more carefully at the quantitative measures we used. Some of the standardised measures were not wholly applicable to a population of older people. Mallinson [ 47 ] also found this with the SF36 when she demonstrated some of its limitations with this age group, as well as how easy it is to, 'fall into the trap of using questionnaires like a form of laboratory equipment and forget that ... they are open to interpretation'. The data presented here demonstrate the difficulties of trying to capture complex phenomena quantitatively. However, they also demonstrate the usefulness of having alternative data forms on which to draw whether complementary (where they differ but together generate insights) or contradictory (where the findings conflict). [ 30 ] In this study, the complementary and contradictory findings of the two datasets proved useful in making recommendations for the design of a definitive study.

Many researchers understand the importance, indeed the necessity, of combining methods to investigate complex health and social issues. Although quantitative research remains the dominant paradigm in health services research, qualitative research has greater prominence than before and is no longer, as Barbour [ 42 ] points out regarded as the 'poor relation to quantitative research that it has been in the past' (p1019). Brannen [ 48 ] argues that, despite epistemological differences there are 'more overlaps than differences'. Despite this, there is continued debate about the authority of each individual mode of research which is not surprising since these different styles, 'take institutional forms, in relation to cultures of and markets for knowledge' (p168). [ 49 ] Devers [ 50 ] points out that the dominance of positivism, especially within the RCT method, has had an overriding influence on the criteria used to assess research which has had the inevitable result of viewing qualitative studies unfavourably. We advocate treating qualitative and quantitative datasets as complementary rather than in competition for identifying the true version of events. This, we argue, leads to a position which exploits the strengths of each method and at the same time counters the limitations of each. The process of interpreting the meaning of these divergent findings has led us to conclude that much can be learned from scientific realism [ 51 ]which has 'sought to position itself as a model of scientific explanation which avoids the traditional epistemological poles of positivism and relativism' (p64). This stance enables investigators to take account of the complexity inherent in social interventions and reinforces, at a theoretical level, the problems of attempting to measure the impact of a social intervention via experimental means. However, the current focus on evidence based health care [ 52 ] now includes public health [ 53 , 54 ] and there is increased attention paid to the results of trials of public health interventions, attempting as they do, to capture complex social phenomena using standardised measurement tools. We would argue that at the very least, the inclusion of both qualitative and quantitative elements in such studies, is essential and ultimately more cost-effective, increasing the likelihood of arriving at a more thoroughly researched and better understood set of results.

The findings of this study demonstrate how the use of mixed methods can lead to different and sometimes conflicting accounts. This, we argue, is largely due to the outcome measures in the RCT not matching the outcomes emerging from the qualitative arm of the study. Instead of making assumptions about the correct version, we have reported the results of both datasets together rather than separately, and advocate six steps to interrogate each dataset more fully. The methodological strategy advocated by this approach involves contemporaneous qualitative and quantitative data collection, analysis and reciprocal interrogation to inform interpretation in trials of complex interventions. This approach also indicates the need for a realistic appraisal of quantitative tools. More widespread use of mixed methods in trials of complex interventions is likely to enhance the overall quality of the evidence base.

Brannen J: Mixing Methods: qualitative and quantitative research. 1992, Aldershot, Ashgate

Google Scholar  

Tashakkori A, Teddlie C: Handbook of Mixed Methods in Social and Behavioural Research. 2003, London, Sage

Morgan DL: Triangulation and it's discontents: Developing pragmatism as an alternative justification for combining qualitative and quantitative methods. Cambridge, 11-12 July.. 2005.

Pill R, Stott NCH: Concepts of illness causation and responsibility: some preliminary data from a sample of working class mothers. Social Science and Medicine. 1982, 16: 43-52. 10.1016/0277-9536(82)90422-1.

Article   CAS   PubMed   Google Scholar  

Scambler G, Hopkins A: Generating a model of epileptic stigma: the role of qualitative analysis. Social Science and Medicine. 1990, 30: 1187-1194. 10.1016/0277-9536(90)90258-T.

Townsend A, Hunt K, Wyke S: Managing multiple morbidity in mid-life: a qualitative study of attitudes to drug use. BMJ. 2003, 327: 837-841. 10.1136/bmj.327.7419.837.

Article   PubMed   PubMed Central   Google Scholar  

Barbour RS: Checklists for improving rigour in qualitative research: the case of the tail wagging the dog?. British Medical Journal. 2001, 322: 1115-1117.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Pope C, Mays N: Qualitative Research in Health Care. 2000, London, BMJ Books

Bryman A: Quantity and Quality in Social Research. 1995, London, Routledge

Ashton J: Healthy cities. 1991, Milton Keynes, Open University Press

Baum F: Researching Public Health: Behind the Qualitative-Quantitative Methodological Debate. Social Science and Medicine. 1995, 40: 459-468. 10.1016/0277-9536(94)E0103-Y.

Donovan J, Mills N, Smith M, Brindle L, Jacoby A, Peters T, Frankel S, Neal D, Hamdy F: Improving design and conduct of randomised trials by embedding them in qualitative research: (ProtecT) study. British Medical Journal. 2002, 325: 766-769.

Cox K: Assessing the quality of life of patients in phase I and II anti-cancer drug trials: interviews versus questionnaires. Social Science and Medicine. 2003, 56: 921-934. 10.1016/S0277-9536(02)00100-4.

Article   PubMed   Google Scholar  

Lewin S: Mixing methods in complex health service randomised controlled trials: research 'best practice'. Mixed Methods in Health Services Research Conference 23rd November 2004, Sheffield University.

Cresswell JW: Mixed methods research and applications in intervention studies.: ; Cambridge, July 11-12. 2005.

Cresswell JW: Research Design. Qualitative, Quantitative and Mixed Methods Approaches. 2003, London, Sage

Barbour RS: The case for combining qualitative and quanitative approaches in health services research. Journal of Health Services Research and Policy. 1999, 4: 39-43.

Gabriel Z, Bowling A: Quality of life from the perspectives of older people. Ageing & Society. 2004, 24: 675-691. 10.1017/S0144686X03001582.

Article   Google Scholar  

Koops L, Lindley RL: Thrombolysis for acute ischaemic stroke: consumer involvement in design of new randomised controlled trial. BMJ. 2002, 325: 415-418. 10.1136/bmj.325.7361.415.

O'Cathain A, Nicholl J, Murphy E: Making the most of mixed methods. Mixed Methods in Health Services Research Conference 23rd November 2004, Sheffield University.

Denzin NK: The Research Act. 1978, New York, McGraw-Hill Book Company

Roberts H, Curtis K, Liabo K, Rowland D, DiGuiseppi C, Roberts I: Putting public health evidence into practice: increasing the prevalance of working smoke alarms in disadvantaged inner city housing. Journal of Epidemiology & Community Health. 2004, 58: 280-285. 10.1136/jech.2003.007948.

Article   CAS   Google Scholar  

Rowland D, DiGuiseppi C, Roberts I, Curtis K, Roberts H, Ginnelly L, Sculpher M, Wade A: Prevalence of working smoke alarms in local authority inner city housing: randomised controlled trial. British Medical Journal. 2002, 325: 998-1001.

Vincent J: Old Age. 2003, London, Routledge

Chapter   Google Scholar  

Department for Work and Pensions: Households below average income statistics 2000/01. 2002, London, Department for Work and Pensions

National Audit Office: Tackling Pensioner Pverty: Encouraging take-up of entitlements. 2002, London, National Audit Office

Paris JAG, Player D: Citizens advice in general practice. British Medical Journal. 1993, 306: 1518-1520.

Abbott S: Prescribing welfare benefits advice in primary care: is it a health intervention, and if so, what sort?. Journal of Public Health Medicine. 2002, 24: 307-312. 10.1093/pubmed/24.4.307.

Harding R, Sherr L, Sherr A, Moorhead R, Singh S: Welfare rights advice in primary care: prevalence, processes and specialist provision. Family Practice. 2003, 20: 48-53. 10.1093/fampra/20.1.48.

Morgan DL: Practical strategies for combining qualitative and quantitative methods: applications to health research. Qualitative Health Research. 1998, 8: 362-376.

Moffatt S, White M, Stacy R, Downey D, Hudson E: The impact of welfare advice in primary care: a qualitative study. Critical Public Health. 2004, 14: 295-309. 10.1080/09581590400007959.

Thomson H, Hoskins R, Petticrew M, Ogilvie D, Craig N, Quinn T, Lindsey G: Evaluating the health effects of social interventions. British Medical Journal. 2004, 328: 282-285.

Department for the Environment TR: Measuring Multiple Deprivation at the Small Area Level: The Indices of Deprivation. 2000, London, Department for the Envrionment, Trransport and the Regions

Ware JE, Sherbourne CD: The MOS 36 item short form health survey (SF-36). Conceptual framework and item selection. Medical Care. 1992, 30: 473-481.

Snaith RP, Zigmond AS: The hospital anxiety and depression scale. Acta Psychiatrica Scandinivica. 1983, 67: 361-370.

Sarason I, Carroll C, Maton K: Assessing social support: the social support questionnaire. Journal of Personality and Social Psychology. 1983, 44: 127-139. 10.1037//0022-3514.44.1.127.

Ward R: The impact of subjective age and stigma on older persons. Journal of Gerontology. 1977, 32: 227-232.

Ford G, Ecob R, Hunt K, Macintyre S, West P: Patterns of class inequality in health through the lifespan: class gradients at 15, 35 and 55 years in the west of Scotland. Social Science and Medicine. 1994, 39: 1037-1050. 10.1016/0277-9536(94)90375-1.

SPSS.: v. 11.0 for Windows [program]. 2003, Chicago, Illinois.

STATA.: Statistical Software [program]. 8.0 version. 2003, Texas, College Station

Ritchie J, Lewis J: Qualitative Research Practice. A Guide for Social Scientists. 2003, London, Sage

Barbour RS: The Newfound Credibility of Qualitative Research? Tales of Technical Essentialism and Co-Option. Qualitative Health Research. 2003, 13: 1019-1027. 10.1177/1049732303253331.

Silverman D: Doing qualitative research. 2000, London, Sage

Clayman SE, Maynard DW: Ethnomethodology and conversation analysis. Situated Order: Studies in the Social Organisation of Talk and Embodied Activities. Edited by: Have PT and Psathas G. 1994, Washington, D.C., University Press of America

White M, Moffatt S, Mackintosh J, Howel D, Sandell A, Chadwick T, Deverill M: Randomised controlled trial to evaluate the health effects of welfare rights advice in primary health care: a pilot study. Report to the Department of Health, Policy Research Programme. 2005, Newcastle upon Tyne, University of Newcastle upon Tyne

Moffatt S: "All the difference in the world". A qualitative study of the perceived impact of a welfare rights service provided in primary care. 2004, , University College London

Mallinson S: Listening to reposndents: a qualitative assessment of the Short-Form 36 Health Status Questionnaire. Social Science and Medicine. 2002, 54: 11-21. 10.1016/S0277-9536(01)00003-X.

Brannen J: Mixing Methods: The Entry of Qualitative and Quantitative Approaches into the Research Process. International Journal of Social Research Methodology. 2005, 8: 173-184. 10.1080/13645570500154642.

Green A, Preston J: Editorial: Speaking in Tongues- Diversity in Mixed Methods Research. International Journal of Social Research Methodology. 2005, 8: 167-171. 10.1080/13645570500154626.

Devers KJ: How will we know "good" qualitative research when we see it? Beginning the dialogue in Health Services Research. Health Services Research. 1999, 34: 1153-1188.

CAS   PubMed   PubMed Central   Google Scholar  

Pawson R, Tilley N: Realistic Evaluation. 2004, London, Sage

Miles A, Grey JE, Polychronis A, Price N, Melchiorri C: Current thinking in the evidence-based health care debate. Journal of Evaluation in Clinical Practice. 2003, 9: 95-109. 10.1046/j.1365-2753.2003.00438.x.

Pencheon D, Guest C, Melzer D, Gray JAM: Oxford Handbook of Public Health Practice. 2001, Oxford, Oxford University Press

Wanless D: Securing Good Health for the Whole Population. 2004, London, HMSO

Pre-publication history

The pre-publication history for this paper can be accessed here: http://www.biomedcentral.com/1472-6963/6/28/prepub

Download references

Acknowledgements

We wish to thank: Rosemary Bell, Jenny Dover and Nick Whitton from Newcastle upon Tyne City Council Welfare Rights Service; all the participants and general practice staff who took part; and for their extremely helpful comments on earlier drafts of this paper, Adam Sandell, Graham Scambler, Rachel Baker, Carl May and John Bond. We are grateful to referees Alicia O'Cathain and Sally Wyke for their insightful comments. The views expressed in this paper are those of the authors and not necessarily those of the Department of Health.

Author information

Authors and affiliations.

Public Health Research Group, School of Population & Health Sciences, Faculty of Medical Sciences, William Leech Building, Framlington Place, Newcastle upon Tyne, NE2 4HH, UK

Suzanne Moffatt, Martin White, Joan Mackintosh & Denise Howel

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Suzanne Moffatt .

Additional information

Competing interests.

The author(s) declare that they have no competing interests.

Authors' contributions

SM and MW had the original idea for the study, and with the help of DH, Adam Sandell and Nick Whitton developed the proposal and gained funding. JM collected the data for the quantitative study, SM designed and collected data for the qualitative study. JM, DH and MW analysed the quantitative data, SM analysed the qualitative data. All authors contributed to interpretation of both datasets. SM wrote the first draft of the paper, JM, MW and DH commented on subsequent drafts. All authors have read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions.

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Moffatt, S., White, M., Mackintosh, J. et al. Using quantitative and qualitative data in health services research – what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Serv Res 6 , 28 (2006). https://doi.org/10.1186/1472-6963-6-28

Download citation

Received : 29 September 2005

Accepted : 08 March 2006

Published : 01 December 2006

DOI : https://doi.org/10.1186/1472-6963-6-28

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mixed Method
• Welfare Benefit
• Mixed Method Research
• Divergent Finding
• Financial Vulnerability

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Fact sheets
• Facts in pictures

Publications

• Questions and answers
• Tools and toolkits
• Endometriosis
• Excessive heat
• Mental disorders
• Polycystic ovary syndrome
• All countries
• Eastern Mediterranean
• South-East Asia
• Western Pacific
• Data by country
• Country presence 
• Country strengthening 
• Country cooperation strategies 
• News releases
• Feature stories
• Press conferences
• Commentaries
• Photo library
• Afghanistan
• Cholera 
• Coronavirus disease (COVID-19)
• Greater Horn of Africa
• Israel and occupied Palestinian territory
• Disease Outbreak News
• Situation reports
• Weekly Epidemiological Record
• Surveillance
• Health emergency appeal
• International Health Regulations
• Independent Oversight and Advisory Committee
• Classifications
• Data collections
• Global Health Observatory
• Global Health Estimates
• Mortality Database
• Sustainable Development Goals
• Health Inequality Monitor
• Global Progress
• World Health Statistics
• Partnerships
• Committees and advisory groups
• Collaborating centres
• Technical teams
• Organizational structure
• Initiatives
• General Programme of Work
• WHO Academy
• Investment in WHO
• WHO Foundation
• External audit
• Financial statements
• Internal audit and investigations 
• Programme Budget
• Results reports
• Governing bodies
• World Health Assembly
• Executive Board
• Member States Portal
• Health topics /

Research is indispensable for resolving public health challenges – whether it be tackling diseases of poverty, responding to rise of chronic diseases,  or ensuring that mothers have access to safe delivery practices.

Likewise, shared vulnerability to global threats, such as severe acute respiratory syndrome, Ebola virus disease, Zika virus and avian influenza has mobilized global research efforts in support of enhancing capacity for preparedness and response. Research is strengthening surveillance, rapid diagnostics and development of vaccines and medicines.

Public-private partnerships and other innovative mechanisms for research are concentrating on neglected diseases in order to stimulate the development of vaccines, drugs and diagnostics where market forces alone are insufficient.

Research for health spans 5 generic areas of activity:

• measuring the magnitude and distribution of the health problem;
• understanding the diverse causes or the determinants of the problem, whether they are due to biological, behavioural, social or environmental factors;
• developing solutions or interventions that will help to prevent or mitigate the problem;
• implementing or delivering solutions through policies and programmes; and
• evaluating the impact of these solutions on the level and distribution of the problem.

High-quality research is essential to fulfilling WHO’s mandate for the attainment by all peoples of the highest possible level of health. One of the Organization’s core functions is to set international norms, standards and guidelines, including setting international standards for research.

Under the “WHO strategy on research for health”, the Organization works to identify research priorities, and promote and conduct research with the following 4 goals:

• Capacity - build capacity to strengthen health research systems within Member States.
• Priorities - support the setting of research priorities that meet health needs particularly in low- and middle-income countries.
• Standards - develop an enabling environment for research through the creation of norms and standards for good research practice.
• Translation - ensure quality evidence is turned into affordable health technologies and evidence-informed policy.
• Prequalification of medicines by WHO
• Global Observatory on Health R&D
• Global Observatory on Health Research and Development
• Implementation research toolkit
• Ethics in implementation research: participant's guide
• Ethics in implementation research: facilitator's guide
• Ethics in epidemics, emergencies and disasters: Research, surveillance and patient care: WHO training manual
• WHA58.34 Ministerial Summit on Health Research
• WHA60.15 WHO's role and responsibilities in health research
• WHA63.21 WHO's role and responsibilities in health research
• EB115/30 Ministerial Summit on Health Research: report by the Secretariat
• Science division

WHO consults on action plan for sustainable clinical research infrastructure

WHO advisory group convenes its first meeting on responsible use of the life sciences in Geneva

Challenging harmful masculinities and engaging men and boys in sexual and reproductive health

Stakeholders convene in Uganda on responsible use of the life sciences

Ebola and Marburg disease outbreaks: infection prevention and control research priorities in health care...

Ebola virus (EBOV) and Marburg virus (MARV) are associated with severe, potentially fatal, systemic diseases. During the development of the Infection Prevention...

Target product profiles for animal plasma-derived antivenoms: antivenoms for treatment of snakebite envenoming...

We describe here WHO public-benefit Target Product Profiles (TPPs) for antivenoms intended to be used for treatment of snakebite envenoming in South Asia,...

WHO global research priorities for sexually transmitted infections

Sexually transmitted infections (STIs) are widespread globally and negatively affect sexual and reproductive health. Gaps in evidence and in available...

Report of the sixth meeting of the WHO Diagnostic Technical Advisory Group for Neglected Tropical Diseases:...

The World Health Organization’s Global Neglected Tropical Diseases Programme (WHO/NTD) manages a diverse portfolio of 21 diseases and disease groups,1 ...

Coordinating R&D on antimicrobial resistance

Ensuring responsible use of life sciences research

Prioritizing diseases for research and development in emergency contexts

Promoting research on Buruli ulcer

Research in maternal, perinatal, and adolescent health

Undertaking health law research

Feature story

One year on, Global Observatory on Health R&D identifies striking gaps and inequalities

Video: Open access to health: WHO joins cOAlition S

Video: Multisectional research on sleeping sickness in Tanzania in the context of climate change

Related health topics

Clinical trials

Global health ethics

Health Laws

Intellectual property and trade

Related links

Research and Development Blueprint

WHO Collaborating Centres

R&D Blueprint for Action to Prevent Epidemics

International Clinical Trials Registry Platform

PH717 Module 1B - Descriptive Tools

Descriptive epidemiology & descriptive statistics.

•   Page:
•   1  
• |   2  
• |   3  
• |   4  
• |   5  
• |   6  
• |   7  
• |   8  
• |   9  

Quantitative and Qualitative Methods for Public Health

This course focuses on quantitative methods, which are designed to precisely estimate population parameters and measure the association between biologic, social, environmental, and behavioral factors and health conditions in order to define the determinants of health and disease and, ultimately, to understand causal pathways.

However, it is important to acknowledge the importance of qualitative methods which provide a means of understanding public health problems in greater depth by providing contextual information regarding a population's beliefs, opinions, norms, and behaviors. This type of information is difficult to capture using traditional quantitative methods, yet it can be vitally important for understanding the "why" for many health problems and also the "how" in terms of how to achieve improvements in health outcomes.

These two approaches might be thought of as the positivist and the constructivist approaches. In positivist research data are more easily quantified, but they are disconnected from the context in which they occur. For example, people of lower socioeconomic status are more likely to smoke tobacco, but the data collected does not indicate why. However, with a constructivist approach, the exposures that people are subjected to (or choose) are better understood in the context of their personal circumstances and the significance that people attribute to things in their environment.

Qualitative methods provide a means of understanding health problems and potential barriers and solutions in greater detail, and they provide an opportunity to understand the "how" and "why" and to identify overlooked issues and themes.

The table below, from the introductory course on fundamentals of public health, highlights some of the major differences between quantitative and qualitative research.

return to top | previous page

Content ©2020. All Rights Reserved. Date last modified: September 10, 2020. Wayne W. LaMorte, MD, PhD, MPH

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 21, Issue 4
• How to appraise quantitative research
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

This article has a correction. Please see:

• Correction: How to appraise quantitative research - April 01, 2019

• Xabi Cathala 1 ,
• Calvin Moorley 2
• 1 Institute of Vocational Learning , School of Health and Social Care, London South Bank University , London , UK
• 2 Nursing Research and Diversity in Care , School of Health and Social Care, London South Bank University , London , UK
• Correspondence to Mr Xabi Cathala, Institute of Vocational Learning, School of Health and Social Care, London South Bank University London UK ; cathalax{at}lsbu.ac.uk and Dr Calvin Moorley, Nursing Research and Diversity in Care, School of Health and Social Care, London South Bank University, London SE1 0AA, UK; Moorleyc{at}lsbu.ac.uk

https://doi.org/10.1136/eb-2018-102996

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Some nurses feel that they lack the necessary skills to read a research paper and to then decide if they should implement the findings into their practice. This is particularly the case when considering the results of quantitative research, which often contains the results of statistical testing. However, nurses have a professional responsibility to critique research to improve their practice, care and patient safety. 1  This article provides a step by step guide on how to critically appraise a quantitative paper.

Title, keywords and the authors

The authors’ names may not mean much, but knowing the following will be helpful:

Their position, for example, academic, researcher or healthcare practitioner.

Their qualification, both professional, for example, a nurse or physiotherapist and academic (eg, degree, masters, doctorate).

This can indicate how the research has been conducted and the authors’ competence on the subject. Basically, do you want to read a paper on quantum physics written by a plumber?

The abstract is a resume of the article and should contain:

Introduction.

Research question/hypothesis.

Methods including sample design, tests used and the statistical analysis (of course! Remember we love numbers).

Main findings.

Conclusion.

The subheadings in the abstract will vary depending on the journal. An abstract should not usually be more than 300 words but this varies depending on specific journal requirements. If the above information is contained in the abstract, it can give you an idea about whether the study is relevant to your area of practice. However, before deciding if the results of a research paper are relevant to your practice, it is important to review the overall quality of the article. This can only be done by reading and critically appraising the entire article.

The introduction

Example: the effect of paracetamol on levels of pain.

My hypothesis is that A has an effect on B, for example, paracetamol has an effect on levels of pain.

My null hypothesis is that A has no effect on B, for example, paracetamol has no effect on pain.

My study will test the null hypothesis and if the null hypothesis is validated then the hypothesis is false (A has no effect on B). This means paracetamol has no effect on the level of pain. If the null hypothesis is rejected then the hypothesis is true (A has an effect on B). This means that paracetamol has an effect on the level of pain.

Background/literature review

The literature review should include reference to recent and relevant research in the area. It should summarise what is already known about the topic and why the research study is needed and state what the study will contribute to new knowledge. 5 The literature review should be up to date, usually 5–8 years, but it will depend on the topic and sometimes it is acceptable to include older (seminal) studies.

Methodology

In quantitative studies, the data analysis varies between studies depending on the type of design used. For example, descriptive, correlative or experimental studies all vary. A descriptive study will describe the pattern of a topic related to one or more variable. 6 A correlational study examines the link (correlation) between two variables 7  and focuses on how a variable will react to a change of another variable. In experimental studies, the researchers manipulate variables looking at outcomes 8  and the sample is commonly assigned into different groups (known as randomisation) to determine the effect (causal) of a condition (independent variable) on a certain outcome. This is a common method used in clinical trials.

There should be sufficient detail provided in the methods section for you to replicate the study (should you want to). To enable you to do this, the following sections are normally included:

Overview and rationale for the methodology.

Participants or sample.

Data collection tools.

Methods of data analysis.

Ethical issues.

Data collection should be clearly explained and the article should discuss how this process was undertaken. Data collection should be systematic, objective, precise, repeatable, valid and reliable. Any tool (eg, a questionnaire) used for data collection should have been piloted (or pretested and/or adjusted) to ensure the quality, validity and reliability of the tool. 9 The participants (the sample) and any randomisation technique used should be identified. The sample size is central in quantitative research, as the findings should be able to be generalised for the wider population. 10 The data analysis can be done manually or more complex analyses performed using computer software sometimes with advice of a statistician. From this analysis, results like mode, mean, median, p value, CI and so on are always presented in a numerical format.

The author(s) should present the results clearly. These may be presented in graphs, charts or tables alongside some text. You should perform your own critique of the data analysis process; just because a paper has been published, it does not mean it is perfect. Your findings may be different from the author’s. Through critical analysis the reader may find an error in the study process that authors have not seen or highlighted. These errors can change the study result or change a study you thought was strong to weak. To help you critique a quantitative research paper, some guidance on understanding statistical terminology is provided in  table 1 .

• View inline

Some basic guidance for understanding statistics

Quantitative studies examine the relationship between variables, and the p value illustrates this objectively.  11  If the p value is less than 0.05, the null hypothesis is rejected and the hypothesis is accepted and the study will say there is a significant difference. If the p value is more than 0.05, the null hypothesis is accepted then the hypothesis is rejected. The study will say there is no significant difference. As a general rule, a p value of less than 0.05 means, the hypothesis is accepted and if it is more than 0.05 the hypothesis is rejected.

The CI is a number between 0 and 1 or is written as a per cent, demonstrating the level of confidence the reader can have in the result. 12  The CI is calculated by subtracting the p value to 1 (1–p). If there is a p value of 0.05, the CI will be 1–0.05=0.95=95%. A CI over 95% means, we can be confident the result is statistically significant. A CI below 95% means, the result is not statistically significant. The p values and CI highlight the confidence and robustness of a result.

Discussion, recommendations and conclusion

The final section of the paper is where the authors discuss their results and link them to other literature in the area (some of which may have been included in the literature review at the start of the paper). This reminds the reader of what is already known, what the study has found and what new information it adds. The discussion should demonstrate how the authors interpreted their results and how they contribute to new knowledge in the area. Implications for practice and future research should also be highlighted in this section of the paper.

A few other areas you may find helpful are:

Limitations of the study.

Conflicts of interest.

Table 2 provides a useful tool to help you apply the learning in this paper to the critiquing of quantitative research papers.

Quantitative paper appraisal checklist

• 1. ↵ Nursing and Midwifery Council , 2015 . The code: standard of conduct, performance and ethics for nurses and midwives https://www.nmc.org.uk/globalassets/sitedocuments/nmc-publications/nmc-code.pdf ( accessed 21.8.18 ).
• Gerrish K ,
• Moorley C ,
• Tunariu A , et al
• Shorten A ,

Competing interests None declared.

Patient consent Not required.

Provenance and peer review Commissioned; internally peer reviewed.

Correction notice This article has been updated since its original publication to update p values from 0.5 to 0.05 throughout.

Linked Articles

• Miscellaneous Correction: How to appraise quantitative research BMJ Publishing Group Ltd and RCN Publishing Company Ltd Evidence-Based Nursing 2019; 22 62-62 Published Online First: 31 Jan 2019. doi: 10.1136/eb-2018-102996corr1

Read the full text or download the PDF:

• Prospective Students
• Current Students
• Residents & Fellows
• Patient Care
• Give to SMHS

Health Services Research

Health Services Research addresses important issues about access to health care, quality and cost, and utilization of health services. This multidisciplinary field of research studies how social factors, financing systems, technologies, organizational structures and processes, and personal behaviors contribute to health and well-being. Research studies often include patients, clinicians, policymakers and the community to innovate and inform health care. Many researchers explore why-if guidelines exist, or interventions were implemented—the problem still exists. Health services research fills the key gap between cures to care.

SMHS is launching a new, monthly journal club featuring recent health services research publications from top faculty. You’ll keep up with the latest GW research and find new ways to translate knowledge to enhance your clinical practice. Your in-person attendance is highly encouraged.

All are Welcome! Come join your colleagues for a lively discussion on today’s most compelling topics.

Review the Research Paper for September's Meeting

View Journal Club Calendar

An Introduction to Health Services Research has been designed for clinicians interested in developing the skills necessary to engage in health services research (HSR). Learners will be introduced to HSR conceptual models in a learning health system, designing a research trajectory, quantitative, qualitative, and mixed-method methodologies, community-based research strategies, ethical considerations in HSR, utilization of big data/EMR, funding mechanisms as well as writing and dissemination.  Each of the nine modules incorporates case studies, lectures, readings, and discussions with experts. Upon course completion, learners will have the knowledge and skills to begin their personal health services research line of inquiry.

View HSR Course Overview

• Scholarly Resources
• Other Resources

Methods for Quantitative Research in Psychology

• Conducting Research

Psychological Research

August 2023

This seven-hour course provides a comprehensive exploration of research methodologies, beginning with the foundational steps of the scientific method. Students will learn about hypotheses, experimental design, data collection, and the analysis of results. Emphasis is placed on defining variables accurately, distinguishing between independent, dependent, and controlled variables, and understanding their roles in research.

The course delves into major research designs, including experimental, correlational, and observational studies. Students will compare and contrast these designs, evaluating their strengths and weaknesses in various contexts. This comparison extends to the types of research questions scientists pose, highlighting how different designs are suited to different inquiries.

A critical component of the course is developing the ability to judge the quality of sources for literature reviews. Students will learn criteria for evaluating the credibility, relevance, and reliability of sources, ensuring that their understanding of the research literature is built on a solid foundation.

Reliability and validity are key concepts addressed in the course. Students will explore what it means for an observation to be reliable, focusing on consistency and repeatability. They will also compare and contrast different forms of validity, such as internal, external, construct, and criterion validity, and how these apply to various research designs.

The course concepts are thoroughly couched in examples drawn from the psychological research literature. By the end of the course, students will be equipped with the skills to design robust research studies, critically evaluate sources, and understand the nuances of reliability and validity in scientific research. This knowledge will be essential for conducting high-quality research and contributing to the scientific community.

Learning objectives

• Describe the steps of the scientific method.
• Specify how variables are defined.
• Compare and contrast the major research designs.
• Explain how to judge the quality of a source for a literature review.
• Compare and contrast the kinds of research questions scientists ask.
• Explain what it means for an observation to be reliable.
• Compare and contrast forms of validity as they apply to the major research designs.

This program does not offer CE credit.

More in this series

Introduces applying statistical methods effectively in psychology or related fields for undergraduates, high school students, and professionals.

August 2023 On Demand Training

Introduces the importance of ethical practice in scientific research for undergraduates, high school students, and professionals.

Kia ora - welcome to our new website. We've tried to ensure old links will redirect to the correct pages, but some links may no longer work. Find out more .

About us Mō mātou

About the Ministry of Health and the New Zealand health system. 

Quick links

• Consultations

New Zealand’s health system

• Government Policy Statement
• Health system roles and organisations
• Overview and statutory framework
• Vote Health

Organisation and leadership

• Chief clinical officers
• Executive Governance Team
• Ministry directorates
• Public Health Agency
• Statutory roles

Corporate publications

• Annual reports
• Briefings to incoming Ministers
• Health and Independence reports
• Mental health annual reports
• Strategic intentions
• Current vacancies
• Employee benefits
• Information for job applicants

Regulation & legislation Ngā here me ngā ture

Health providers and products we regulate, and laws we administer.

• Legislation we administer
• Rest home audit reports

Assisted dying

Certification of health care services, drug checking, health practitioners competence assurance act, medicinal cannabis, medicines control, mental health and addiction, notifiable diseases, pharmacy licensing, psychoactive substances, smoked tobacco, vaping, herbal smoking and smokeless tobacco, strategies & initiatives he rautaki, he tūmahi hou.

How we’re working to improve health outcomes for all New Zealanders.

Health strategies

• New Zealand Health Strategy
• Health of Disabled People Strategy
• Pae Tū: Hauora Māori Strategy
• Rural Health Strategy
• Te Mana Ola: The Pacific Health Strategy
• Women’s Health Strategy

Programmes and initiatives

• Gambling harm
• Health workforce
• Mental health, addiction and suicide prevention
• Pacific health
• Smokefree 2025

Māori health Hauora Māori

Increasing access to health services, achieving equity and improving outcomes for Māori.

• Subscribe for Māori health updates

Ao Mai te Rā anti-racism initiative

Māori health models, te tiriti o waitangi framework, wai 2575 health services and outcomes inquiry, whakamaua: māori health action plan, statistics & research he tatauranga, he rangahau.

Data and insights from our health surveys, research and monitoring.

• New Zealand Health Survey
• Health targets

System monitoring

• Mental health and addiction targets
• Planning and performance data
• New Zealand Gambling Survey
• Past surveys

Statistics and data sets

• Māori health data and statistics
• Mental health data and statistics
• Obesity data and statistics
• Suicide data and statistics
• Tobacco data and statistics
• COVID-19 and immunisation research projects
• Gambling harm research and evaluation
• Our role in health research

Measuring public health behaviours and intentions

In June 2023, the Public Health Agency, within the Ministry of Health, commissioned research agency Verian to undertake a series of research projects and population surveys into attitudes and behaviours related to public health measures following the COVID-19 pandemic. The purpose of this research was to examine the impact of COVID-19 on New Zealanders’ attitudes and behaviours towards public health measures and use this knowledge to better prepare the Ministry, and other decision makers, for future pandemics and other major health threats.

This work is split into two parts, each with two reports. The first part is two repeated behavioural surveys to monitor adherence to public health measures over time, covering both intention and actual behaviour. The second part is a qualitative analysis to explore drivers of COVID-19 related public health behaviours, and to quantify the impact of barriers and other factors on adherence to public health measures. This involves qualitative interviews followed by a survey.

This report, Measuring public health behaviours and intentions is one of the two based on findings from the behavioural survey. It is the second of the four reports to be published.

Key findings

• Half of the adults who had flu symptoms and interacted with others at a place of work or study, said they interacted with others while unwell. A third of children interacted with other children at their school while they were unwell.
• Parents/caregivers are more likely to keep their children home from school if they’re unwell than adults are to keep themselves away from work (83% vs 55% ‘at least somewhat likely’).
• Around 50% of New Zealanders said COVID-19 hasn’t impacted their intention to get vaccinated for the flu, other illnesses, and a new pandemic. The remaining 50% are relatively evenly split between being more likely to get vaccinated now than they were before COVID-19 and being less likely. Most parents/caregivers are just as likely to get vaccinations for their children, as they were before COVID-19 (22% ‘more likely’, 61% ‘just as likely’).

The report is informed by 1,642 surveys conducted online using online research panels. The sample is structured to be demographically representative of the population by age, gender, and region. Māori and Pacific peoples over-sampled relative to population to ensure sufficient sample sizes for analysis – 369 of the 1,642 interviews were with Māori and 200 were with Pacific peoples (30 people identified as both Māori and Pacific).​ An additional 197 surveys were conducted by telephone – 109 with Māori and 101 with Pacific peoples (13 people identified as both Māori and Pacific).​ Surveying was conducted 31 October to 29 November 2023.​

The report will help enhance the Ministry’s understanding of people’s recent and intended public health behaviours and how these have been impacted by the COVID-19 pandemic.

• Document download Measuring Public Health Behaviours and Intentions ( pdf , 389.25 KB )

Publishing information

Third-party content. Please check the document or email the Web Manager to find out how to obtain permission to re-use content.

• email Email
• print Print
• chat Feedback

© Ministry of Health – Manatū Hauora

• Open access
• Published: 27 August 2024

Perinatal mental health and active-duty military spouses: a scoping review

• Kelly Pretorius 1 , 2 , 3 ,
• Margaret F. Sposato 1 &
• Wendy Trueblood-Miller 1  

BMC Pregnancy and Childbirth volume  24 , Article number:  557 ( 2024 ) Cite this article

74 Accesses

Metrics details

Introduction

Mental health conditions (i.e. depression or anxiety) are the most common complication of pregnancy and childbirth in the United States (U.S.) and are associated with increased maternal and infant morbidity and mortality. Research has demonstrated a relationship between stress and mental health diagnoses in pregnancy; therefore, it is concerning that military families face unique challenges which contribute to additional stressors among spouses of active-duty (AD) military personnel during the perinatal period. The objective of this scoping review was to understand the current state of research on perinatal stress or perinatal mental health among American spouses of AD military personnel.

The Boolean phrase was created in consultation with 2 health science librarians and the following databases searched in October 2023: PubMed, Embase, Military and Government Collection, CINAHL, and PsychINFO. 2 reviewers identified 481 studies for screening once duplicates were removed. After applying inclusion and exclusion criteria, 21 studies remained for data extraction and analysis.

Most of the studies were quantitative, took place in the southern U.S., and the most represented military branch was Air Force. Most of the studies included both AD military members and AD spouses; 28% focused solely on AD spouses. Samples were not racially diverse, and findings identified racial disparities in perinatal mental health conditions. There was a wide variety in outcome measures, including the following general categories: (1) stress, anxiety, and/or depression, (2) maternal-infant attachment, (3) group prenatal care, and (4) deployment focus. Our review identified the following concepts: spouses most at risk for perinatal mental health conditions, the need for perinatal mental health screening, and the need for social support.

Conclusions

Findings from the identified studies indicate a need for additional research in this area. Additionally, findings highlight circumstances unique to this population that result in an increased risk of stress and/or mental health conditions during the perinatal period. Such challenges demand improved mental health screening and additional resources for this population. Meeting the needs of this unique population also requires significant funding and policy change to allow for increased access to mental health resources and to ensure the health of the birthing person and infant.

Peer Review reports

• Perinatal mental health

The most common complication of pregnancy and childbirth in the United States (U.S.) is mental health conditions [ 1 , 2 ]. Mental health conditions during the perinatal period include depression, bipolar disorder, anxiety disorder, obsessive-compulsive disorder, posttraumatic stress disorder, and substance use disorder [ 3 ]. These diagnoses are associated with an increased risk of maternal and infant morbidity and mortality [ 3 ]. Perinatal distress originating from anxiety, depression, or stress has also been found to negatively influence child health [ 4 ] and development [ 5 , 6 ].

In a study on socioeconomically diverse new mothers, worries about housing, income, family health, and relationships were prevalent [ 7 ]. This is worrisome since recent research has discovered that precursors to perinatal mental health conditions, such as generalized perinatal distress, also have negative health outcomes, including adverse infant socio-cognitive outcomes, such as delayed cognitive development and difficulty with emotional regulation [ 8 ]. Additionally, chronic stressors, such as perceived stress or anxiety, poverty, intimate partner violence and racism have long been associated with an increased incidence of preterm birth [ 9 ].

Thus, it is concerning that stress-related symptoms are now reported as the most common complication of pregnancy, even impacting healthy pregnancies and spanning socioeconomic statuses [ 8 ]. Left untreated, the consequences of mental health conditions are severe. Though underreported, data indicate suicide and overdose deaths are a leading cause of maternal death in the first year following childbirth in the U.S. [ 10 , 11 ].

Military family challenges

Military families face many unique challenges relative to the general U.S. population that may contribute to additional stressors and mental health conditions. As of 2022, there were 2,071,451 million active-duty (AD) and reserve members of the U.S. military and 2,482,499 family members [ 12 ]. In 2021, 81% of AD military spouses surveyed had experienced a change of duty station, meaning the family was required to relocate [ 13 ]. Financial implications incur with a change in duty station. For instance, a change in duty station brings additional obstacles, including finding employment for the AD spouse (resulting in loss of income), moving costs, settling claims for damaged or missing household goods, move coordination, waiting for housing to become available, and re-establishing childcare. It is therefore not surprising that a change in duty station increases the odds of spousal unemployment and unemployment increases the odds of food insecurity [ 13 ]. In 2021, 25% of AD spouses reported food insecurity [ 13 ].

The emotional impact of being a military family is also extensive. 41% of AD military spouses surveyed in 2021 reported a partner’s deployment to a combat zone and 48% reported concerns about the mental health of their spouse returning from deployment [ 13 ]. There has also been an increase in use of counselling services among AD spouses (44% in 2021 compared to 37% in 2012) [ 13 ]. Additionally, a Government Accountability Office (GAO) study determined that 36% of beneficiaries in the Department of Defense (DoD) Tricare program (the healthcare insurance of AD members and beneficiaries) received a mental health diagnosis between the years 2017 and 2019; the most common diagnoses being anxiety disorders, depressive disorders, and trauma and stressor-related disorders [ 14 ].

Furthermore, military families operate with perceived pressure to adhere to military standards, which leads to additional strain and overall dissatisfaction [ 15 ]. Military spouses report a culture “where the soldier and his career come first” and have self-proclaimed difficulties in managing their stress [ 16 ]. Military spouses have also reported higher levels of stress and depression, especially during their AD partner’s deployments [ 17 , 18 , 19 ]. Lastly, according to a GAO report to Congress on domestic abuse in the DoD, there were 8,055 incidents recorded in 2019; thereby implying significant domestic abuse among military households that possibly result from and contribute to extenuating stressors [ 20 ]. It is worth noting that this report states the incidents are likely grossly underreported [ 20 ]. Overall, there has been a general decline in spousal satisfaction in American military life since 2012 [ 13 ].

Pregnant, active-duty spouses

Pregnant spouses of AD military personnel are therefore at increased risk for perinatal mental health conditions, especially during AD spousal deployments [ 14 , 21 ]. In the U.S., childbearing people are not adequately supported by national policies nor funding, leading to long term consequences for the mother and child [ 1 ]. Pregnant spouses of American AD military personnel, despite their unique circumstances, also lack support. For example, a recent news article [ 22 ] highlights deficiencies identified by a Pentagon Inspector General Report in the military healthcare system, which is directed by the Defense Health Agency (DHA). Barriers to obtaining medical care, including mental healthcare, at military treatment facilities (MTFs) both in the U.S. and abroad have been identified [ 23 ]. For example, in June 2023, pregnant Air Force dependents stationed in Okinawa, Japan were informed they would either need to deliver at a Japanese hospital or return stateside for delivery, because of staffing shortages [ 24 ]. Circumstances such as these result in additional stressors among an already marginalized population.

Rationale for scoping review

It is imperative to describe the existing literature on perinatal stress and perinatal mental health among American, AD military spouses due to their unique circumstances and needs. We narrowly defined our population of interest and focused solely on the U.S. military per the recommendation of Levac, Colquhoun, and O’Brien [ 25 ] and because military culture and support programs vary significantly on an international scale. A preliminary search of the literature identified a general lack of studies on this topic and one scoping review published in 2018 to inquire if United Kingdom (UK) military spouses were at risk for poor perinatal health [ 21 ]. The Godier-McBard et al. paper identified studies conducted in the U.S. but was completed with a focus on UK military personnel and families [ 26 ].

Our scoping review differs in our research question, our search terms, our population of interest, and is warranted since it has been 6 years since the last review was published. According to Arksey and O’Malley [ 27 ], scoping reviews are helpful in examining the extent of research activity, to see if a systematic review is feasible, to summarize findings, and identify research gaps. Given the limited amount of research on our topic of interest, we therefore proceeded with a scoping review rather than a systematic review of the literature in hopes of providing an overview of the evidence [ 28 ].

This review aims to answer the following research question: What is the current state of research on perinatal stress or perinatal mental health among American spouses of AD military personnel?

Search strategy

To ensure a quality review and rigorous methodological approach, this scoping review followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist and expansion [ 29 ]. The protocol was registered with OSF registries as the following: https://doi.org/10.17605/OSF.IO/TYSEZ .

To ensure a comprehensive search, we consulted with two professional librarians. The text words contained in the titles and abstracts of relevant articles, as well as the index terms describing the articles were used to develop a full search strategy for this review. The final search terms were selected after many iterations. The main concepts were perinatal mental health and the military family or spouse. We initially included the concept of stress in our search strategy but surprisingly, this did not identify any additional articles. We also initially included the concept of “overseas”, but this severely limited our results; therefore, we adjusted the concepts to exclude the concept of “overseas” in our search strategy. To demonstrate our search strategy, the database search strategy for PubMed is included as Appendix I .

We searched the following databases in October 2023: PubMed, Embase, Military and Government Collection, CINAHL, and PsychINFO. No limits were placed during this initial search. The search strategy, including all identified keywords and index terms, was adapted for each database.

Study selection process

Following the initial search, the citations were uploaded into Covidence [ 30 ], a web-based software platform that streamlines the production of literature reviews. Once duplicates were removed, 481 articles remained to review. Following a pilot test, titles and abstracts were screened by two independent reviewers for assessment against the inclusion and exclusion criteria. An additional 2 articles were identified through backward search. After application of inclusion and exclusion criteria, 21 articles were included in this scoping review for full review. The results of the search and the study inclusion process are presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram (see Fig.  1 ).

PRISMA-ScR flow diagram

Inclusion and exclusion criteria

We included qualitative and quantitative research studies written in English, conducted either in the U.S. or abroad, with a study population that included American, AD spouses, focused on perinatal mental health or stress, and published in a peer-review journal. We included both experimental and quasi-experimental study designs, including randomized controlled trials (RCT), non-randomized controlled trials, before and after studies and interrupted time-series studies. In addition, analytical observational studies including prospective and retrospective cohort studies, case-control studies and analytical cross-sectional studies were considered for inclusion. This review also considered descriptive observational study designs including case series, individual case reports and descriptive cross-sectional studies for inclusion. Qualitative studies were also included.

We excluded commentaries, case studies, literature reviews, letters to the editor, books, dissertations, and monographs. We also excluded systematic reviews so that the extracted studies would not demonstrate redundancy. To remain current, we only included studies published since the year 2000. An ancestry search was done to identify other additional studies. Disagreements that arose between the reviewers at any stage of the selection process were resolved via discussion.

Data extraction

Data was extracted by two independent reviewers using a data extraction tool developed by the reviewers (see Appendix II ). This extraction tool was created to address the objective and research question and therefore includes specific details about: methods, setting, population, perinatal period, details of intervention or program, findings or perinatal outcomes, and implications of the study. This extraction tool was not modified in the extraction process. However, after trends were noted, we opted to present the body of literature in additional tables to demonstrate trends. Any disagreements about the data in the extraction table were resolved via discussion. The charted data was then discussed among the team members to further identify trends and derive concepts. A critical appraisal of the selected studies was not completed for this scoping review.

Study design

The majority (21/22; 95%) of the identified articles were quantitative, with 1 article reporting mixed methods [ 31 ] and 1 article utilizing qualitative methods [ 32 ]. Of the quantitative articles, 24% (5/21) were intervention studies with a RCT design [ 31 , 33 , 34 , 35 , 36 ]. Two of the RCT studies involved implementing group prenatal care [ 31 , 34 ]. The remaining methods were as follows (in order of most frequent): 6 retrospective studies [ 37 , 38 , 39 , 40 , 41 , 42 , 43 ], 4 cross-sectional surveys [ 44 , 45 , 46 ] including a comparative descriptive study [ 47 ], and 3 prospective or longitudinal studies [ 48 , 49 , 50 ].

Study setting

Table  1 and Appendix II describe details of the setting and the region in which the studies occurred. The majority of studies (42%) took place in the southern U.S. [ 36 , 38 , 44 , 45 , 46 , 47 , 48 , 50 ], followed by 23% in the Western U.S. [ 31 , 34 , 41 , 42 , 43 ], 9% in the U.S. - not specified [ 32 , 49 ], and 4% in the Midwestern U.S. [ 33 ]. 19% of the selected studies utilized national health data information that represented the entire military personnel population and was not regionally specific [ 37 , 39 , 40 , 51 ]. Other than the studies that utilized national health data information [ 37 , 39 , 40 , 51 ], studies primarily obtained data from subjects seeking healthcare at MTFs located on base (see Table  1 ).

Study population and perinatal time period

The number of participants among all included studies totals 340,666 (one study did not provide sample size, thus, this data was not included [ 51 ]). The range of samples for studies varied significantly with the largest samples representing studies utilizing national health data information and the smallest utilizing qualitative methods. For example, 161,454 pregnancies were included to assess for odds of postpartum depression (PPD) relative to spousal deployment [ 39 ] versus 10 women interviewed to understand the lived experience of widowhood during pregnancy [ 32 ]. The majority (61%) of the studies included persons that were both AD and spouses of AD [ 31 , 34 , 35 , 36 , 37 , 39 , 40 , 44 , 45 , 46 , 48 , 50 , 51 ]. However, many (28%) studies did focus solely on AD spouses [ 33 , 41 , 42 , 43 , 47 , 49 ]. One study included AD, AD spouses, AD daughters, and Veteran Affairs patients [ 38 ] and one study specified that both civilians and AD spouses were included [ 32 ].

Samples were predominantly White [ 33 , 34 , 35 , 36 , 37 , 44 , 49 , 50 ], representing a younger population (median age of 27 [ 37 ], means of 26 years [ 41 , 50 ], means of 28 years [ 34 , 35 ] range between 18 and 28 years [ 33 ]), and educated [ 32 , 35 , 36 , 49 , 50 ]. While the samples represented all branches of the military, the Air Force was most represented [ 33 , 35 , 36 , 47 , 49 , 50 ], followed by the Marine Corps [ 43 , 44 , 45 , 46 ], the Army [ 39 , 42 , 48 ], and the Navy [ 31 ] (see Table  2 ).

Pregnant and postpartum women were included in the identified studies. The specific perinatal time in which women were studied include the following (in order of prevalence): pregnancy and postpartum [ 31 , 32 , 33 , 34 , 38 , 41 , 42 , 43 , 45 , 48 , 49 ], pregnancy only [ 35 , 36 , 44 , 46 , 50 ] pregnancy and up to 1 year postpartum [ 37 , 39 ], postpartum up to 1 year [ 40 , 51 ], and postpartum only [ 47 ]. Additionally, one study focused on widowed spouses [ 32 ].

Perinatal outcomes

The studies assessed for a variety of outcomes, such as: incidence and prevalence of mental health conditions, predictors or risk factors for mental health conditions, compliance of PPD screening, predictors of maternal-infant attachment, outcomes of various interventions, and effects of deployment on maternal acceptance of pregnancy and PPD. The various outcomes fall into the following general categories: (1) stress, anxiety, and/or depression, (2) maternal-infant attachment, (3) group prenatal care, and (4) deployment-focused. The details will be briefly discussed below, and the specific tools utilized by the studies are depicted in Table  3 .

Stress, anxiety, and depression

Some retrospective cohort studies utilized national health data to identify incidence and prevalence of specific mental health conditions. For example, one study analyzed national health data to identify incidences of psychiatric disorders, including PPD, and predictors of such diagnoses [ 37 ]. Findings identified significant differences in outcomes based on race/ethnicity: incidence of psychiatric disorders during pregnancy was significantly lower among White women compared with Asian or African American women and higher among single women [ 37 ].

Rates of antidepressant use was assessed in one study, where it was determined that the rate of usage was higher among: younger women, women of lower socioeconomic status, and women with a history of military service [ 40 ]. Haas et al. [ 44 , 45 , 46 ] conducted 3 separate cross-sectional surveys to assess for self-reported or self-perceived stress during pregnancy and found the following were associated with higher stress: women with 2 or more children at home before delivery [ 44 , 45 , 46 ], not having a support person, and having a deployed partner [ 44 , 46 ], especially to a combat zone during pregnancy [ 44 ].

A descriptive study examined protective and risk factors of PPD and identified elevated depressive symptoms in more than 50% of the women (consistent with at risk populations) and identified that family change or strain, lower self-reliance, and lower social support were predictors of PPD among military wives [ 47 ]. Another study assessed for influencers of Edinburgh Postnatal Depression Scale (EPDS) scores and found higher EPDS scores were associated with isolation, a personal history of depression, and having a husband/partner deploy during pregnancy [ 41 ].

Regarding screening, a retrospective study assessed for screening of depression in a large MTF OB (obstetrics) department and identified excellent screening rates during the first trimester and room for improvement at the 28-week and postpartum visits [ 38 ].

Maternal-infant attachment

Many studies included maternal-infant attachment in their approach. One longitudinal study aimed to identify predictors of postpartum maternal-infant attachment, including the impact of spousal deployments and found that spousal deployment in the 1st trimester, having a history of depression, and a higher EPDS score impacted satisfaction with motherhood and infant care [ 49 ]. An RCT compared an intervention versus routine care in facilitating maternal role adaptation and assessed for prenatal adaption, postpartum adaption, external resources, social support, and internal resources [ 33 ]. This intervention, “Baby Boot Camp” may have enhanced external and internal resources to facilitate prenatal and postpartum maternal role adaptation, however not all differences were statistically significant [ 33 ].

Group prenatal care

The RCTs primarily involved comparing group prenatal care versus routine care [ 31 , 34 ]. For example, Kennedy et al. (2011) assessed the impact of group prenatal care on mental health, stress, depressive symptoms during pregnancy, and PPD [ 31 ] while Tubay et al. (2019) assessed for the impact of group prenatal care on anxiety, depression, and infant birthweight [ 34 ]. Both RCTs identified no significant differences in outcome measurements; however, group prenatal care participants were less likely to report feelings of guilt or shame in one study [ 31 ] and the group prenatal care was found to be more accessible and convenient in the other study [ 34 ].

Another RCT compared a prenatal support program (“Mentors Offering Maternal Support”) versus routine care and assessed for prenatal maternal adaptation, maternal/fetal attachment, community support, and self-esteem [ 36 ], finding no significant differences between the groups. However, higher levels of contact with deployed husbands did result in higher scores of self-esteem and lower anxiety [ 36 ]. This same program was later tested again, and the assessed outcomes were prenatal adaption, self-esteem, resilience, and depression [ 35 ]. When repeated, the intervention did reduce prenatal anxiety and identified single participants and those with deployed husbands as being at greater risk of anxiety [ 35 ].

Deployment-focused

Most of the articles (57%) conducted studies that included information relating to deployments [ 35 , 39 , 41 , 42 , 43 , 44 , 45 , 46 , 48 , 49 , 50 ]. Retrospective studies assessed for the effects of deployment on depression screening scores during (1) pregnancy [ 42 ], and (2) pregnancy and postpartum [ 43 ]. Findings included that deployment status significantly affected the prevalence of depression in pregnant women [ 42 , 43 ] and postpartum [ 43 ].

Another study assessed the effects of deployment on pregnancy outcomes, finding that deployment of a spouse to a combat zone during the pregnancy resulted in a 3-fold increased risk of PPD [ 48 ]. Another longitudinal study evaluated the effect of deployment on maternal acceptance of pregnancy and included measures on social support [ 50 ]. This study found that women with deployed husbands had a significant increase in conflict associated with acceptance of pregnancy, that conflict with acceptance of pregnancy decreased with social support, and that social support located on base (versus off base) resulted in greater acceptance of pregnancy [ 50 ].

Lastly, another study analyzed national health data to determine the odds of PPD relative to spousal deployment [ 39 ] and found the odds of PPD were higher among: those of younger maternal age, experiencing spousal deployment during pregnancy, with a diagnosis of depression or anxiety prior to pregnancy, alcohol/drug/tobacco use while pregnant, being married to a White (non-Hispanic) spouse, being affiliated with the Army, and complications with their infant upon delivery. Additionally, the highest levels of PPD were noted among women with husbands deployed during delivery [ 39 ].

Concepts identified

Spouses most at risk.

In reviewing the articles, the profile of the American, AD military spouse most at risk for perinatal stress and/or depression is the following: identifying as Asian or African American [ 37 ], being of younger maternal age [ 39 , 40 ], being married to a White (non-Hispanic) spouse [ 39 ], being affiliated with the Army [ 39 ], being of lower socio-economic status [ 40 ], having children at home before delivery [ 44 , 45 , 46 ], having a deployed partner [ 35 , 39 , 41 , 42 , 43 , 44 , 46 , 49 , 50 ], not having a social support or reporting isolation [ 41 , 44 , 47 , 50 ], having an antepartum diagnosis of depression or anxiety [ 39 , 41 , 49 ], alcohol/drug/tobacco use in pregnancy [ 39 ], having pregnancy complications [ 37 ], and experiencing neonatal complications [ 39 ].

Need for perinatal mental health screening

Many of the identified articles addressed screenings for PPD and/or anxiety either in their study design or when discussing implications. Screening military personnel and dependents during the postpartum period is recommended [ 51 ], especially during deployments [ 39 ]. Additionally, it is suggested that there be focused screening for suicidal behavior among persons already diagnosed with PPD [ 51 ]. One study found compliance of depression screening in military settings was excellent at initial visit but required an improved commitment for maintenance throughout the entire pregnancy [ 38 ].

Need for social support

A prevalent concept that emerged among the articles was the need for social support among pregnant or postpartum military spouses. The need for social support has been explicitly requested by military spouses [ 36 ] and is especially needed when dealing with unanticipated external stressors that can occur in military life [ 32 ]. Social support was found to positively influence maternal acceptance of pregnancy, which facilitates maternal-fetal and infant attachment [ 50 ]. Not having a support person during pregnancy correlated with higher stress [ 44 ] and feeling isolated significantly influenced the risk for a diagnosis of PPD [ 41 ]. Lower social support also predicted PPD [ 47 ]. This is concerning given the geographic challenges of military life; one study found that 1/5 pregnant persons did not have a support person beyond their partner [ 45 ].

Key findings and associated limitations of the evidence base

Overall, the identified research demonstrates that research is lacking regarding perinatal mental health among AD military spouses. This lack of research may be related to barriers that limit research among military personnel and their dependents [ 52 ]. For example, obtaining access to military personnel and data is typically restricted to those affiliated with the DoD or to those affiliated with select universities. Additionally, military leaders often prioritize medical readiness over other medical, indirect outcomes that may be the focus of research studies [ 52 ]. This restriction on conducting research is reflected in the limited amount of literature to review, the limited variety of journals, and the predominance of publication of studies in Military Medicine , by government employees (see Table  4 ). It is worth noting that many of the authors of the identified articles are nurses, demonstrating nursing’s interest in this area of research.

Study design, setting, population, and outcomes

Since many of the articles were quantitative, further investigation into the perspective of the AD pregnant, military spouse is warranted and qualitative research might be helpful to ensure specialized needs are met. Additionally, while 5 RCTs were identified, continued intervention studies are needed given that many of the tested interventions were questionable in their efficacy.

Additionally, the Southern U.S. was over-represented in the sample and therefore other regions, and associated military bases, might have been overlooked. Furthermore, despite the large number of military dependents stationed overseas (approximately 170,000 dependents in 2021 [ 12 ]), none of the identified articles focused on AD spouses stationed overseas. In reviewing pertinent research, we identified two articles related to being stationed overseas. One was a systematic review of the impact of foreign postings among military spouses (both American and British). The review identified studies published primarily in the 1980s and early 1990s and findings included spousal satisfaction being linked to spousal employment status [ 53 ]. The other study, completed in 2001, compared prenatal care needs in the U.S. and abroad, yet this comparison did not include any mental health factors [ 54 ]. However, this study did find that those stationed overseas reported challenges to obtaining care within a different culture and barriers to prenatal care were identified [ 54 ]. Thus, future research should include all regions of the U.S. as well as American spouses stationed abroad. Conducting research with the overseas population is critical, since this population may face additional stressors that impact perinatal mental health outcomes.

Furthermore, while 28% of the identified studies focused on AD spouses only, the majority included both AD and spouses of AD. While AD personnel do require support and warrant research, the AD spousal population has unique needs and should be studied separately. Since many of the samples were predominantly White, there is also concern that underrepresented populations were not adequately sampled. However, the identified articles did represent both the pregnant and postpartum population and it was reassuring to see some extend to 1 year postpartum given the national emphasis to care for the mother during this extended period [ 55 ]. Future research should aim to include more racially diverse samples of AD spouses and extend outcome measurements to at least 1 year postpartum.

Despite the differences among military branches, there was not equal representation in the studies since the Air Force was predominantly represented (see Table  2 ). This warrants further investigation given the varying stressors that impact the branches differently. For example, military personnel in the Army, Navy, and Marines are more likely to be deployed than the Air Force, and experience higher levels of PTSD and anxiety [ 56 ].

The outcome measures of the identified articles fell into the following general categories: (1) perinatal stress, anxiety, and/or depression, (2) maternal-infant attachment, (3) group prenatal care interventions, and (4) deployment-focused. Our identified outcome categories were similar to themes identified in the 2018 scoping review on perinatal mental health: deployment increases prenatal stress and depression, protective or risk factors from perinatal mental health problems, and support interventions for this population [ 26 ]. Since some of these outcomes are now well understood, such as the negative impact of deployment and the insignificant difference of many of the interventions, future research should aim to evaluate the effect of new and innovative support programs on perinatal mental health outcomes. Additionally, since precursors to perinatal mental health conditions are known to have negative health outcomes [ 8 , 9 ], studying acute or chronic stress within this population is warranted.

Identified concepts

Our review identified the following concepts: spouses most at risk for perinatal mental health conditions, a need for perinatal mental health screening, and the need for social support among perinatal American AD-spouses. From the articles reviewed, the profile of the pregnant, AD-spouse most at risk has also been identified: Asian or African American, younger maternal age, married to a White (non-Hispanic) spouse, an affiliation with the Army, lower socio-economic status, having children at home before delivery, having a deployed partner, not having social support or reporting isolation, having an antepartum diagnosis of depression or anxiety, using alcohol/drug/tobacco in pregnancy, and experiencing pregnancy or neonatal complications. Additionally, our review identified continued racial disparities in perinatal mental health outcomes. These findings are supported by an integrative review that also found racial disparities among perinatal outcomes among beneficiaries of the military health system [ 57 ].

Our review also found evidence highlighting the need for continued perinatal mental health screening among perinatal American AD military spouses. This in line with the current recommendation that mental health screening occur among all Tricare beneficiaries in the prenatal and postpartum time periods [ 14 ]. Furthermore, programs need to be in place to ensure continued adherence to mental health screening among perinatal persons. Accessibility to appropriate support programs is also critical for this population.

The need for adequate social support and mental health resources for perinatal American AD military spouses was also highlighted in our review. The need for social support among this specific population is previously known and there are existing programs within the military network [ 58 ], such as the “New Parent Support Program”, which was established to help military families “navigate through pregnancy, transition successfully into parenthood and provide a nurturing environment” [ 59 ]. However, our review suggests that such programs need to be bolstered and potentially adjusted to meet the specific needs of the AD perinatal military spouse. Additionally, future research and screening or support programs should target the most at risk persons to address disparities.

Limitations

This study is not without limitations. As per the guidelines for conducting a scoping review, we did not complete a risk of bias or a thorough assessment regarding the quality of the included studies. Thus, our results and implications are limited in their impact. Despite our best effort to construct a thorough search and maintain methodological rigor throughout the entire review process, it is possible we missed relevant studies. Additionally, application of our inclusion and exclusion criteria, such as only including studies published since 2000, may have excluded sentinel papers on this topic. Despite these limitations, this scoping review makes an important contribution to the current body of knowledge by highlighting a critical gap in the literature about perinatal mental health among American spouses of AD military personnel. This review only identified 21 studies; thus, there is a need for additional research in this area. Furthermore, this review emphasizes the unique circumstances facing this specific population and identifies areas of directed improvement.

Perinatal mental health is of vital importance; ensuring appropriate screening and support for the pregnant, AD spouse is necessary to prevent the increasing burden of maternal and infant morbidity and mortality in the U.S. Findings from this scoping review have identified key areas relative to practice and future research. For example, persons most at risk for perinatal mental health conditions within this population have clearly been identified and additional resources should be made available to this specific population. This need for tailored support is supported by prior research, given that their needs differ from the typical perinatal person [ 26 ]. Additionally, many (57%) of the articles included in this review addressed deployment; however, the need for support of AD military spouses despite a spousal’s deployment status is warranted. Given the additional challenges that come with being stationed abroad, the minimal research on the impact of living abroad among military families is concerning. Thus, more research on the pregnant AD military spouse stationed abroad is warranted.

There is also room for improvement in both screening and providing much needed services for those most at risk. Meeting the needs of this unique population requires significant funding and policy change to allow for increased access to mental health resources and to ensure the health of the mother and infant. Barriers to obtaining medical care, including mental healthcare services, have been documented at MTFs both in the U.S. and abroad [ 23 ]. Efforts should be made by the DHA to address these barriers and provide crucial resources, which is in line with the GAO’s suggestions [ 14 ].

In conclusion, military families face unique challenges, which contribute to additional stressors among spouses of U.S. military AD personnel during the perinatal period. Meeting the needs of this unique population requires significant funding and policy change from the highest level, additional and specific research, improvements in screening, and additional support interventions. Only then can we adequately identify those who need assistance, increase access to mental health resources, and ensure the health of the mother and infant.

Data availability

All data generated or analyzed during this study are included in this published article (and its supplementary information files).

Abbreviations

Active-duty

Defense Health Agency

Department of Defense

Edinburgh Postnatal Depression Scale

Government Accountability Office

Military treatment facility

• Postpartum depression

Randomized controlled trial

United States

Griffen A, et al. Perinatal Mental Health Care in the United States: an overview of policies and programs: study examines perinatal mental health care policies and programs in the United States. Health Aff. 2021;40(10):1543–50.

Article   Google Scholar  

Fawcett EJ et al. The prevalence of anxiety disorders during pregnancy and the Postpartum period: a multivariate bayesian Meta-analysis. J Clin Psychiatry, 2019. 80(4).

Kendig S, et al. Consensus Bundle on Maternal Mental Health: perinatal depression and anxiety. Obstet Gynecol. 2017;129(3):422–30.

Article   PubMed   PubMed Central   Google Scholar  

Yang F, et al. Prenatal exposure to severe stress and the risk of heart failure up to Middle-Age. JACC Heart Fail; 2024.

Kingston D, Tough S, Whitfield H. Prenatal and postpartum maternal psychological distress and Infant Development: a systematic review. Child Psychiatry Hum Dev. 2012;43(5):683–714.

Article   PubMed   Google Scholar  

Mughal MK, et al. Trajectories of maternal stress and anxiety from pregnancy to three years and child development at 3 years of age: findings from the all our families (AOF) pregnancy cohort. J Affect Disord. 2018;234:318–26.

Guyol GG, et al. The impact of maternal worry on Infant Care practices. Acad Pediatr. 2022;22(6):935–41.

Wu Y, et al. Association of Elevated Maternal Psychological Distress, altered fetal brain, and offspring cognitive and social-emotional outcomes at 18 months. JAMA Netw open. 2022;5(4):e229244–229244.

Latendresse G. The Interaction between chronic stress and pregnancy: Preterm Birth from a Biobehavioral Perspective. J Midwifery Women’s Health. 2009;54(1):8–17.

Trost SL, et al. Preventing pregnancy-related Mental Health deaths: insights from 14 US Maternal Mortality Review Committees, 2008–17: study examines maternal mortality and mental health. Health Aff. 2021;40(10):1551–9.

Mangla K, et al. Maternal self-harm deaths: an unrecognized and preventable outcome. Am J Obstet Gynecol. 2019;221(4):295–303.

Policy O. o.t.D.A.S.o.D.f.M.C.a.F. 2021 Demographics: Profile of the Military Community. 2021 [cited 2023 October 18]; https://download.militaryonesource.mil/12038/MOS/Reports/2021-demographics-report.pdf

2021 Active Duty Spouse Survey (ADSS). Defense Personnel Analytics Center 2023 [cited 2023 September 6]; https://download.militaryonesource.mil/12038/MOS/Presentations/2021-active-duty-spouse-overview-briefing.pdf

Hundrup AM, DEFENSE HEALTH CARE. Prevalence of and efforts to screen and Treat Mental Health Conditions in prenatal and Postpartum TRICARE beneficiaries. GAO Rep, 2022: p. i–48.

Drummet AR, Coleman M, Cable S. Military families under stress: implications for Family Life Education. Fam Relat. 2003;52(3):279–87.

Mailey EL, et al. Everything else comes first: a mixed-methods analysis of barriers to health behaviors among military spouses. BMC Public Health. 2018;18(1):1013–1013.

Dimiceli EE, Steinhardt MA, Smith SE. Stressful experiences, coping strategies, and predictors of Health-related outcomes among wives of Deployed Military Servicemen. Armed Forces Soc. 2010;36(2):351–73.

Esposito-Smythers C, et al. Military Youth and the Deployment cycle: Emotional Health consequences and recommendations for intervention. J Fam Psychol. 2011;25(4):497–507.

Verdeli H, et al. The case for treating Depression in Military Spouses. J Fam Psychol. 2011;25(4):488–96.

Office USGA, Domestic Abuse. Actions Needed to Enhance DoD’s Prevention, Response, and Oversight. GAO-21-289 Report to Congressional Committees 2021 [cited 2023 December 15]; https://www.gao.gov/assets/gao-21-289.pdf

Lauren Rose G-M, et al. Military spouses with deployed partners are at greater risk of poor perinatal mental health: a scoping review. J R Army Med Corps. 2019;165(5):363.

Ziezulewicz G. DOD watchdog report warns of issues across military health care system, in MilitaryTimes. MilitaryTimes; 2023.

Management Advisory. Concerns with Access to care and staffing shortages in the Military Health System (DODIG-2024-033). Department of Defense Office of Inspector General; 2023.

Burke M. Pregnant military mothers dodge delivery-care drama on Okinawa, in Stars and Stripes. 2023.

Levac D, Colquhoun H. O’Brien, scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69.

Godier-McBard LR, et al. Military spouses with deployed partners are at greater risk of poor perinatal mental health: a scoping review. J R Army Med Corps. 2019;165(5):363–70.

Arksey HOM. Scoping studies: towards a Methodological Framework. Int J Social Res Methodology: Theory Pract. 2005;8(1):19–32.

Munn Z, et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Tricco AC, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

Ltd VHIcovidence. 2023; https://www.covidence.org

Kennedy HP, et al. A randomized clinical trial of group prenatal care in two military settings. Mil Med. 2011;176(10):1169–77.

Doherty ME, Scannell-Desch E. The lived experience of widowhood during pregnancy. J Midwifery Womens Health. 2008;53(2):103–9.

Schachman KA, Lee RK, Lederma RP. Baby boot camp: facilitating maternal role adaptation among military wives. Nurs Res. 2004;53(2):107–15.

Tubay AT, et al. The effects of Group prenatal care on infant birthweight and maternal Well-Being: a Randomized Controlled Trial. Mil Med. 2019;184(5–6):e440–6.

Weis KL, et al. Mentors Offering maternal support reduces prenatal, pregnancy-specific anxiety in a sample of Military Women. JOGNN: J Obstetric Gynecologic Neonatal Nurs. 2017;46(5):669–85.

Weis KL, Ryan TW. Mentors offering maternal support: a support intervention for military mothers. J Obstet Gynecol Neonatal Nurs. 2012;41(2):303–14.

Andriotti T, et al. Psychiatric conditions during pregnancy and Postpartum in a universally insured American Population. Mil Med. 2022;187(7–8):e795–801.

Gisseman J, et al. Depression Screening during pregnancy: compliance and effectiveness in a Military Population. Mil Med. 2021;186(9/10):e951–5.

Levine JA, et al. Postpartum depression and timing of spousal military deployment relative to pregnancy and delivery. Arch Gynecol Obstet. 2015;292(3):549–58.

Roberts TA, Hansen S. Association of Hormonal Contraception with depression in the postpartum period. Contraception, 2017. 96(3): p. N.PAG-N.PAG.

Robrecht DT, et al. Spousal military deployment as a risk factor for postpartum depression. J Reproductive Medicine ® . 2008;53(11):860–4.

PubMed   Google Scholar  

Smith DC, et al. Effects of deployment on depression screening scores in pregnancy at an army military treatment facility. Obstet Gynecol. 2010;116(3):679–84.

Spooner S, Rastle M, Elmore K. Maternal depression screening during prenatal and postpartum care at a Navy and Marine Corps military treatment facility. Mil Med. 2012;177(10):1208–11.

Haas DM, Pazdernik LA. Partner deployment and stress in pregnant women. J Reprod Med. 2007;52(10):901–6.

Haas DM, Pazdernik LA. A cross-sectional survey of stressors for postpartum women during wartime in a military medical facility. Mil Med. 2006;171(10):1020–3.

Haas DM, Pazdernik LA, Olsen CH. A cross-sectional survey of the relationship between partner deployment and stress in pregnancy during wartime. Womens Health Issues. 2005;15(2):48–54.

Schachman K, Lindsey L. A resilience perspective of Postpartum Depressive Symptomatology in Military wives. JOGNN: J Obstetric Gynecologic Neonatal Nurs. 2013;42(2):157–67.

Tarney CM, et al. Association of Spouse Deployment on Pregnancy Outcomes in a U.S. Military Population. Obstet Gynecol (New York 1953). 2015;126(3):569–74.

Weis K, Lederman RP. Prenatal spousal military deployment and maternal prenatal adaptation as predictors of postpartum maternal-infant attachment. South Online J Nurs Res. 2010;10(3):p7–7.

Google Scholar  

Weis KL, et al. The relationship of military imposed marital separations on maternal acceptance of pregnancy. Res Nurs Health. 2008;31(3):196–207.

Do T, et al. Depression and suicidality during the postpartum period after first time deliveries, active component service women and dependent spouses, U.S. Armed forces, 2007–2012. Msmr. 2013;20(9):2–7.

Rhon DI, Oh RC, Teyhen DS. Challenges with engaging military stakeholders for Clinical Research at the point of care in the U.S. Military Health System. Mil Med. 2022;187(7–8):209–14.

Blakely G, et al. A systematic review of the impact of foreign postings on accompanying spouses of military personnel. Nurs Health Sci. 2012;14(1):121–32.

Sylvia BM, et al. Prenatal care needs, availability, accessibility, use, and satisfaction: a comparison of military women within and outside the continental United States. Mil Med. 2001;166(5):443–8.

Article   CAS   PubMed   Google Scholar  

Ranji UG, I.l Salganicoff A. Expanding Postpartum Medicaid Coverage. Women’s Health Policy 2021 [cited 2024 February 22]; https://www.kff.org/womens-health-policy/issue-brief/expanding-postpartum-medicaid-coverage/

Dedeaux JA, Lewis RK. Does the physical and mental health of veterans differ by military branch? J Prev Interv Community. 2021;49(3):207–20.

Iobst SE, et al. Integrative Review of Racial Disparities in perinatal outcomes among beneficiaries of the Military Health System. JOGNN: J Obstetric Gynecologic Neonatal Nurs. 2022;51(1):16–28.

Recame MA. Childbirth Education and Parental Support Programs within the U.S. Military Population. Int J Childbirth Educ. 2013;28(1):67–70.

New Parent Support Program. 2024; https://www.militaryonesource.mil/benefits/new-parent-support-program/

Download references

Acknowledgements

Authors would like to acknowledge the librarians who assisted in development of the search terms: Anna Liss Jacobsen at Indiana University and Lynn Bostwick at The University of Texas at Austin.

This research was supported by Grant Number T32 NR018407 from the National Institute of Nursing Research.

Author information

Authors and affiliations.

Indiana University School of Nursing, 600 Barnhill Dr, Indianapolis, IN, 46202, USA

Kelly Pretorius, Margaret F. Sposato & Wendy Trueblood-Miller

St. Edward’s University School of Nursing, 3001 S Congress Ave, Austin, TX, 78704, USA

Kelly Pretorius

Texas Children’s Hospital Pediatric Hospital Medicine, 9835 N. Lake Creek Parkway, Austin, TX, 78717, USA

You can also search for this author in PubMed   Google Scholar

Contributions

KP conceived the idea and designed the study under the guidance of WM. KP and MS completed the review process and data extractions. KP wrote the manuscript. MS and WM edited and proofread the manuscript.

Corresponding author

Correspondence to Kelly Pretorius .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Pretorius, K., Sposato, M.F. & Trueblood-Miller, W. Perinatal mental health and active-duty military spouses: a scoping review. BMC Pregnancy Childbirth 24 , 557 (2024). https://doi.org/10.1186/s12884-024-06727-1

Download citation

Received : 19 March 2024

Accepted : 29 July 2024

Published : 27 August 2024

DOI : https://doi.org/10.1186/s12884-024-06727-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Active-duty spouse
• Military family
• Postpartum anxiety
• American military
• Scoping review

BMC Pregnancy and Childbirth

ISSN: 1471-2393

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Religion, Spirituality and Health Research: Warning of Contaminated Scales

• IMPRESSIONISTIC REPORTING
• Published: 28 August 2024

Cite this article

• Harold G. Koenig   ORCID: orcid.org/0000-0003-2573-6121 1 , 2 , 3 , 5 &
• Lindsay B. Carey 4 , 6 , 7  

The relationship between religiosity, spirituality and health has received increasing attention in the academic literature. Studies involving quantitative measurement of religiosity and/or spirituality (R/S) and health have reported many positive associations between these constructs. The quality of various measures, however, is very important in this field, given concerns that some measures of R/S have been contaminated with indicators of mental health. When this occurs, that is when R/S is defined and measured a priori, this subsequently guarantees a positive association between R/S and health (especially mental health). Such associations are called tautological, which involves correlating a construct with itself, thus producing associations that are uninterpretable and misleading. In this article, concerns about the measurement of R/S are discussed, examples of contaminated and potentially probelmatic measures of R/S are noted, and recommendations are made regarding uncontaminated measures of R/S that should be used in future studies of R/S and health.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Defining “spirituality": Please note that it is not the intent nor within the scope of this review to consider the differences in meaning between "spirituality" and the actual practice of spiritual care or pastoral care. Please refer to Carey et al. ( 2024 , p. 2, and associated Footnote ) for a brief comparative discussion.

Ai, A. L., Wink, P., Paloutzian, R. F., & Harris, K. A. (Eds.). (2021). Assessing spirituality in a diverse world . Springer International Publishing.

Google Scholar  

Al Zaben, F., Khalifa, D. A., Sehlo, M. G., Al Shohaib, S., Binzaqr, S. A., Badreg, A. M., & Koenig, H. G. (2015). Religious involvement and health in dialysis patients in Saudi Arabia. Journal of Religion and Health, 54 , 713–730. https://doi.org/10.1007/s10943-014-9962-8

Article   PubMed   Google Scholar  

Al Zaben, F., Sehlo, M. G., Khalifa, D. A., & Koenig, H. G. (2015b). Test–retest reliability of the Muslim religiosity scale: Follow-up to “religious involvement and health among dialysis patients in Saudi Arabia.” Journal of Religion and Health, 54 , 1144–1147. https://doi.org/10.1007/s10943-015-0025-6

Artress, L. (1996). Walking a sacred path: Rediscovering the labyrinth as a spiritual practice . Penguin.

Carey, L. B., Koenig, H. G., Hill, T., Drummond, D., Gabbay, E., Cohen, J., Aiken, C., & Carey, J. R. (2024). Spirituality, mental health, and COVID-19. Journal of Religion and Health, 63 (1), 1–5. https://doi.org/10.1007/s10943-024-02000-z

Daaleman, T. P., & Frey, B. B. (2004). The spirituality index of well-being: A new instrument for health-related quality-of-life research. Annals of Family Medicine, 2 (5), 499–503. https://doi.org/10.1370/afm.89

Article   PubMed   PubMed Central   Google Scholar  

de BritoSena, M. A., Damiano, R. F., Lucchetti, G., & Peres, M. F. P. (2021). Defining spirituality in healthcare: A systematic review and conceptual framework. Frontiers in Psychology, 12 , 756080. https://doi.org/10.3389/fpsyg.2021.756080

Article   Google Scholar  

Drummond, D. A., & Carey, L. B. (2019). Assessing spiritual well-being in residential aged care: An exploratory review. Journal of Religion and Health, 58 , 372–390. https://doi.org/10.1007/s10943-018-0717-9

Exline, J. J., Pargament, K. I., Grubbs, J. B., & Yali, A. M. (2014). The religious and spiritual struggles scale: Development and initial validation. Psychology of Religion and Spirituality, 6 (3), 208. https://doi.org/10.1037/a0036465

Francis, L. J., Lewis, J. M., Philipchalk, R., Brown, L. B., & Lester, D. (1995). The internal consistency reliability and construct validity of the Francis Scale of Attitude toward Christianity (adult) among undergraduate students in the UK, USA, Australia and Canada. Personality and Individual Differences, 19 (6), 949–953. https://doi.org/10.1016/S0191-8869(95)00131-X

Francis, L. J., Santosh, Y. R., Robbins, M., & Vij, S. (2008). Assessing attitude toward Hinduism: The Santosh-Francis Scale. Mental Health, Religion and Culture, 11 (6), 609–621. https://doi.org/10.1080/13674670701846469

Garssen, B., & Visser, A. (2016). Spiritual wellbeing predicting depression: Is it relevant? Journal of Behavioral Medicine, 39 (2), 369–369. https://doi.org/10.1007/s10865-016-9719-9

Garssen, B., Visser, A., & de JagerMeezenbroek, E. (2016). Examining whether spirituality predicts subjective well-being: How to avoid tautology. Psychology of Religion and Spirituality, 8 (2), 141–148. https://doi.org/10.1037/rel0000025

Gomez, R., & Fisher, J. W. (2003). Domains of spiritual well-being and development and validation of the Spiritual Well-Being Questionnaire. Personality and Individual Differences, 35 (8), 1975–1991. https://doi.org/10.1016/S0191-8869(03)00045-X

Gomez, R., & Watson, S. (2023). A reevaluation of the factor structure, reliability, and validity of the Spiritual Well-Being Questionnaire (SWBQ). Journal of Religion and Health, 62 (3), 2112–2130. https://doi.org/10.1007/s10943-022-01619-0

Granqvist, P., & Hagekull, B. (2001). Seeking security in the new age: On attachment and emotional compensation. Journal for the Scientific Study of Religion, 40 (3), 527–545. https://doi.org/10.1111/0021-8294.0007

Hill, P. C., & Maltby, L. E. (2009). Measuring religiousness and spirituality: issues, existing measures, and the implications for education and wellbeing. In M. de Souza, L. J. Francis, J. O’Higgins-Norman, D. Scott (Eds.), International handbook of education for spirituality, care and wellbeing. International Handbooks of Religion and Education (vol 3). Springer.

Hill, P. C., & Hood, R. W., Jr. (1999). Measures of religiosity . Religious Education Press.

Hill, P. C., Hood, R. W., Jong, J., & Harris, K. A. (Eds.). (2024). Measures of religiosity and spirituality . Springer.

Hoge, D. R. (1972). A validated intrinsic religious motivation scale. Journal for the Scientific Study of Religion, 11 , 369–376. https://doi.org/10.2307/1384677

Idler, E. L., Musick, M. A., Ellison, C. G., George, L. K., Krause, N., Ory, M. G., et al. (2003). Measuring multiple dimensions of religion and spirituality for health research conceptual background and findings from the 1998 General Social Survey. Research on Aging, 25 (4), 327–365. https://doi.org/10.2307/1384677

JORH. (2021). Remembering 9/11, moral injury, COVID-19 and measuring religion, spirituality and health. Journal of Religion and Health , 60 (5). https://link.springer.com/journal/10943/volumes-and-issues/60-5 .

JORH. (2023). Care of the aged, women’s health, and measuring religion, spirituality and health. Journal of Religion and Health, 62 (5). https://link.springer.com/journal/10943/volumes-and-issues/62-5

JORH. (2024). Tribal healing, suicide, ethical issues, cancer and measuring religiosity and spirituality. Journal of Religion and Health , 63 (2). https://link.springer.com/journal/10943/volumes-and-issues/63-2

Koenig, H. G., Al-Zaben, F., Khalifa, D. A., & Al Shohaib, S. (2015). Measures of religiosity, Chapter 19. In B. J. Boyle, D. H. Saklofske, G. Mathews (Eds.), Measures of personality and social psychological constructs . Academic Press.

Koenig, H. G. (2008). Concerns about measuring “spirituality” in research. Journal of Nervous and Mental Disease, 196 (5), 349–355. https://doi.org/10.1097/NMD.0b013e31816ff796

Koenig, H. G. (2011). Spirituality and health research: Methods, measurement, statistics, and resources . Templeton Press.

Koenig, H. G., & Büssing, A. (2010). The Duke University Religion Index (DUREL): A five-item measure for use in epidemological studies. Religions, 1 (1), 78–85. https://doi.org/10.3390/rel1010078

Koenig, H. G., Cohen, H. J., Blazer, D. G., Pieper, C., Meador, K. G., Shelp, F., & DiPasquale, B. (1992). Religious coping and depression among elderly, hospitalized medically ill men. American Journal of Psychiatry, 149 (12), 1693–1700. https://doi.org/10.1176/ajp.149.12.1693

Article   CAS   PubMed   Google Scholar  

Koenig, H. G., Meador, K. G., & Parkerson, G. (1997). Religion index for psychiatric research. American Journal of Psychiatry, 154 (6), 885–886. https://doi.org/10.1176/ajp.154.6.885b

Koenig, H. G., Nelson, B., Shaw, S. F., Al Zaben, F., Wang, Z., & Saxena, S. (2014). Belief into Action scale: A brief but comprehensive measure of religious commitment. Open Journal of Psychiatry , 5 (01), 66.

Koenig, H. G., VanderWeele, T. J., & Peteet, J. R. (2024). Measurement (Chapter 2). In H. G. Koenig, T. J. VanderWeele, & J. R. Peteet (Eds.), Handbook of religion and health (3rd ed., pp. 15–29). Oxford University Press.

Chapter   Google Scholar  

Krause, N. (1999). Religious support. In Multidimensional measurement of religiousness/spirituality for use in health research : A report of the Fetzer Institute/National Institute on Aging Workshop Group (pp. 57–63). Fetzer Foundation.

Lesmana, C. B. J., Tiliopoulos, N., & Francis, L. J. (2011). The internal consistency reliability of the Santosh-Francis scale of attitude toward Hinduism among Balinese Hindus. International Journal of Hindu Studies, 15 (3), 293–301. https://doi.org/10.1007/s11407-011-9108-5

Levin, J. (2020). Religion and medicine: A history of the encounter between humanity’s two greatest institutions . Oxford University Press.

Book   Google Scholar  

Moreira-Almeida, A., & Koenig, H. G. (2006). Retaining the meaning of the words religiousness and spirituality: A commentary on the WHOQOL SRPB group’s “A cross-cultural study of spirituality, religion, and personal beliefs as components of quality of life” (62: 6, 2005, 1486–1497). Social Science and Medicine, 63 (4), 843–845. https://doi.org/10.1016/j.socscimed.2006.03.001

Paloutzian, R. F., & Ellison, C. W. (1982). Loneliness, spiritual well-being, and the quality of life. In L. A. Peplau & D. Perlman (Eds.), Loneliness: A sourcebook of current theory, research and therapy (pp. 224–236). John Wiley & Sons.

Pargament, K. I., Smith, B. W., Koenig, H. G., & Perez, L. (1998). Patterns of positive and negative religious coping with major life stressors. Journal for the Scientific Study of Religion . https://doi.org/10.2307/1388152

Peterman, A. H., Fitchett, G., Brady, M. J., Hernandez, L., & Cella, D. (2002). Measuring spiritual well-being in people with cancer: The functional assessment of chronic illness therapy—spiritual well-being scale (FACIT-Sp). Annals of Behavioral Medicine, 24 (1), 49–58. https://doi.org/10.2307/1388152

Phillips, R. E., Michelle Cheng, C., Oemig, C., Hietbrink, L., & Vonnegut, E. (2012). Validation of a Buddhist coping measure among primarily non-Asian Buddhists in the United States. Journal for the Scientific Study of Religion, 51 (1), 156–172. https://doi.org/10.1111/j.1468-5906.2012.01620.x

Piedmont, R. L. (1999). Does spirituality represent the sixth factor of personality? Spiritual transcendence and the five-factor model. Journal of Personality, 67 (6), 985–1013. https://doi.org/10.1111/1467-6494.00080

Piedmont, R. L. (2001). Spiritual transcendence and the scientific study of spirituality. Journal of Rehabilitation, 67 (1), 4–14.

Puchalski, C. M., Vitillo, R., Hull, S. K., & Reller, N. (2014). Improving the spiritual dimension of whole person care: Reaching national and international consensus. Journal of Palliative Medicine, 17 (6), 642–656. https://doi.org/10.1089/jpm.2014.9427

Rosmarin, D. H., Pargament, K. I., Krumrei, E. J., & Flannelly, K. J. (2009). Religious coping among Jews: Development and initial validation of the JCOPE. Journal of Clinical Psychology, 65 (7), 670–683. https://doi.org/10.1002/jclp.20574

Salander, P. (2006). Who needs the concept of ‘spirituality’? Psycho-Oncology, 15 (7), 647–649.  https://onlinelibrary.wiley.com/doi/10.1002/pon.1060

Salander, P. (2012). The emperor’s new clothes: Spirituality. A concept based on questionable ontology and circular findings. Archive for the Psychology of Religion, 34 (1), 17–32. https://doi.org/10.1163/157361212X645241

Sheldrake, P. (2007). A brief history of spirituality . Blackwell Publishing.

Tsuang, M. T., & Simpson, J. C. (2008). Commentary on Koenig (2008): “Concerns about measuring ‘spirituality’ in research.” Journal of Nervous and Mental Disease, 196 (8), 647–649. https://doi.org/10.1097/NMD.0b013e31816ff796

Underwood, L. G., & Teresi, J. A. (2002). The daily spiritual experience scale. Annals of Behavioral Medicine, 24 (1), 22–33. https://doi.org/10.1207/S15324796ABM2401_04

WHOQoL SRPB Group. (2006). A cross-cultural study of spirituality, religion, and personal beliefs as components of quality of life. Social Science and Medicine, 62 (6), 1486–1497. https://doi.org/10.1016/j.socscimed.2005.08.001

Worthington, E. L., Jr., Wade, N. G., Hight, T. L., Ripley, J. S., McCullough, M. E., Berry, J. W., & O’Connor, L. (2003). The religious commitment inventory–10: Development, refinement, and validation of a brief scale for research and counseling. Journal of Counseling Psychology, 50 (1), 84. https://doi.org/10.1037/0022-0167.50.1.84

Download references

Acknowledgements

The authors wish to acknowledge [TBA]

No specific funding was applied or granted for this submission.

Author information

Authors and affiliations.

Department of Psychiatry & Behavioral Sciences, Duke University Medical Center, Durham, NC, USA

Harold G. Koenig

Department of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia

Department of Psychiatry, Shiraz University of Medical Sciences, Shiraz, Iran

School of Psychology & Public Health, College of Science, Health & Engineering La Trobe University, Victoria, 3086, Australia

Lindsay B. Carey

Duke University Medical Center, Box 3400, Durham, NC, 27710, USA

Institute for Ethics and Society, University of Notre Dame, Sydney, New South Wales, 2007, Australia

College of Sport, Health and Engineering, Victoria University, Melbourne, New South Wales, 3011, Australia

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Harold G. Koenig .

Ethics declarations

Conflicts of interest.

The authors are academic editors of research submissions relating to religion and/or spirituality to the Journal of Religion and Health (Author 1 & 2), the International Journal of Psychiatry in Medicine (Author 1) and the Health and Social Care Chaplaincy (Author 2).

Human and Animal Participants

While some research noted within this publication has made use of human participants, this review did not directly involve the use of any human or other animal subjects.

Informed Consent

Not Applicable.

Consent for Publication

Appropriate author and publisher permissions were acquired with regard to the republishing of all figures within this manuscript.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

See Table 3 .

Rights and permissions

Reprints and permissions

About this article

Koenig, H.G., Carey, L.B. Religion, Spirituality and Health Research: Warning of Contaminated Scales. J Relig Health (2024). https://doi.org/10.1007/s10943-024-02112-6

Download citation

Accepted : 19 August 2024

Published : 28 August 2024

DOI : https://doi.org/10.1007/s10943-024-02112-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Spirituality
• Measurement
• Contaminated scales
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• JRSM Short Rep
• v.4(6); 2013 Jun

Using mixed methods in health research

Shema tariq.

1 School of Health Sciences, City University London, EC1A 7QN, London, UK

Jenny Woodman

2 MRC Centre of Epidemiology for Child Health, UCL Institute of Child Health, WC1N 1EH, London, UK

Mixed methods research is the use of quantitative and qualitative methods in a single study or series of studies. It is an emergent methodology which is increasingly used by health researchers, especially within health services research. There is a growing literature on the theory, design and critical appraisal of mixed methods research. However, there are few papers that summarize this methodological approach for health practitioners who wish to conduct or critically engage with mixed methods studies. The objective of this paper is to provide an accessible introduction to mixed methods for clinicians and researchers unfamiliar with this approach. We present a synthesis of key methodological literature on mixed methods research, with examples from our own work and that of others, to illustrate the practical applications of this approach within health research. We summarize definitions of mixed methods research, the value of this approach, key aspects of study design and analysis, and discuss the potential challenges of combining quantitative and qualitative methods and data. One of the key challenges within mixed methods research is the successful integration of quantitative and qualitative data during analysis and interpretation. However, the integration of different types of data can generate insights into a research question, resulting in enriched understanding of complex health research problems.

Introduction

Mixed methods research is the use of quantitative and qualitative methods in one study. Research is often dichotomized as quantitative or qualitative. Quantitative research, such as clinical trials or observational studies, generates numerical data. On the other hand qualitative approaches tend to generate non-numerical data, using methods such as semi-structured interviews, focus group discussions and participant observation. Historically, quantitative methods have dominated health research. However, qualitative methods have been increasingly accepted by the health research community in the past two decades, with a rise in publication of qualitative studies. 1 As the value of qualitative approaches has been recognized, there has been a growing interest in combining qualitative and quantitative methods. A recent review of health services research within England has shown an increase in the proportion of studies classified as mixed methods from 17% in the mid-1990s to 30% in the early 2000s. 2 In this paper, we present a synthesis of key literature on mixed methods research, with examples from our own work and that of others to illustrate the practical applications of this approach. This paper is aimed at health researchers and practitioners who are new to the field of mixed methods research and may only have experience of either quantitative or qualitative approaches and methodologies. We wish to provide these readers with an accessible introduction to the increasingly popular methodology of mixed methods research. We hope this will help readers to consider whether their research questions might best be answered by a mixed methods study design, and to engage critically with health research that uses this approach.

The authors each independently carried out a narrative literature review and met to discuss findings. Literature was identified via searches of PubMed, Google and Google Scholar, and hand-searches of the Journal of Mixed Methods Research, with relevant publications selected after discussion. An important consideration was that papers either had a methodological focus or contained a detailed description of their mixed methods design. For PubMed and Google searches, similar terms were used. For example, the PubMed strategy consisted of title and abstract searches for: ((mixed methods) OR ((mixed OR (qualitative AND quantitative)) AND methods)). We also drew upon recommendations from mixed methods conferences and seminars, and reference lists from key publications.

What is mixed methods research?

The most widely accepted definition of mixed methods research is research that ‘focuses on collecting, analysing, and mixing both quantitative and qualitative data in a single study or a series of studies’. 3 Central to the definition is the use of both quantitative and qualitative methods in one study (or a series of connected studies). Separate quantitative and qualitative studies addressing the same research question independently would not be considered ‘mixed methods’ as there would be no integration of approaches at the design, analysis or presentation stage. A recent innovation in mixed methods research is the mixed methods systematic review, which sets out to systematically appraise both quantitative and qualitative literature on a subject area and then synthesize the findings.

Why are mixed methods approaches used?

The underlying assumption of mixed methods research is that it can address some research questions more comprehensively than by using either quantitative or qualitative methods alone. 3 Questions that profit most from a mixed methods design tend to be broad and complex, with multiple facets that may each be best explored by quantitative or qualitative methods. See Boxes 1 and ​ and2 2 for examples from our own work.

Examples of authors’ mixed methods research – JW.

Examples of authors’ mixed methods research – ST.

Usually, quantitative research is associated with a positivist stance and a belief that reality that can be measured and observed objectively. Most commonly, it sets out to test an a priori hypothesis and is therefore conventionally described as ‘deductive’. Strengths of quantitative research include its procedures to minimize confounding and its potential to generate generalizable findings if based on samples that are both large enough and representative. It remains the dominant paradigm in health research. However, this deductive approach is less suited to generating hypotheses about how or why things are happening, or explaining complex social or cultural phenomena.

Qualitative research most often comes from an interpretive framework and is usually informed by the belief that there are multiple realities shaped by personal viewpoints, context and meaning. In-depth qualitative research aims to provide a rich description of views, beliefs and meaning. It also tends to acknowledge the role of researcher and context in shaping and producing the data. Qualitative approaches are described as ‘inductive’ as questions are often open-ended with the analysis allowing hypotheses to emerge from data. High-quality qualitative research can generate robust theory that is applicable to contexts outside of the study area in question, helping to guide practitioners and policy-makers. 8 However, for research that aims to directly impact on policy and practice, the findings of qualitative research can be limited by the small sample sizes that are necessary for in-depth exploratory work and the consequent lack of generalizabilty.

Mixed methods research therefore has the potential to harness the strengths and counterbalance the weaknesses of both approaches and can be especially powerful when addressing complex, multifaceted issues such as health services interventions 9 and living with chronic illness. 10

There are many reasons why researchers choose to combine quantitative and qualitative methods in a study. 11 , 12 We list some common reasons below, using a hypothetical research question about adolescents’ adherence to anticonvulsant medication to illustrate real world applications.

• Complementarity: Using data obtained by one method to illustrate results from another. An example of this would be a survey of adolescents with epilepsy demonstrating poor levels of adherence. Semi-structured interviews with a sub-group of those surveyed may allow us to explore barriers to adherence.
• Development: Using results from one method to develop or inform the use of the other method. A focus group conducted with a group of adolescents with epilepsy may identify mobile phone technology as a potentially important tool in adherence support. We could then develop a mobile phone ‘app’ that reminds patients to take their medication and conduct an intervention study to assess its impact on adherence levels.
• Initiation: Using results from different methods specifically to look for areas of incongruence in order to generate new insights. An illustration of this would be a study exploring the discrepancy between reported adherence in clinic consultations and actual medication adherence. A review of case notes may find adherence levels of over 90% in a clinic population; however, semi-structured interviews with peer researchers may reveal lower levels of adherence and barriers to open discussion with clinicians.
• Expansion: Setting out to examine different aspects of a research question, where each aspect warrants different methods. We may wish to conduct a study that explores adherence more broadly. A large-scale survey of adolescents with epilepsy would provide information on adherence levels and associations whilst interviews and focus groups may allow us to engage with individual experiences of chronic illness and medication in adolescence.
• Triangulation: Using data obtained by both methods to corroborate findings. For example, we could conduct a clinical study measuring drug levels in individuals and documenting self-reported adherence. Qualitative methods such as video diaries may confirm adherence levels.

To this list we would also add political commitment. That is to say, researchers may recognize, and wish to deploy, the strengths of quantitative research in producing generalizable results but may also be committed to representing the voice of participants in their work.

Whatever the reasons for mixing methods, it is important that authors present these explicitly as it allows us to assess if a mixed methods study design is appropriate for answering the research question. 3 , 13

How is mixed methods research conducted?

When embarking on a mixed methods research project it is important to consider:

• the methods that will be used;
• the priority of the methods;
• the sequence in which the methods are to be used.

A wide variety of methods exists by which to collect both quantitative and qualitative data. Both the research question and the data required will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team’s skills and experience and time constraints.

Priority of methods relates to the emphasis placed on each method in the study. For instance, the study may be predominantly a quantitative study with a small qualitative component, or vice versa. Alternatively, both quantitative and qualitative methods and data may have equal weighting. The emphasis given to each component of the study will be driven mainly by the research question, the skills of the research team and feasibility.

Finally, researchers must decide when each method is to be used in the study. For instance a team may choose to start with a quantitative phase followed by a qualitative phase, or vice versa. Some studies use both quantitative and qualitative methods concurrently. Again the choice of when to use each method is largely dependent on the research question.

The priority and sequence of mixing methods have been elaborated in a typology of mixed methods research models. See Table 1 for typology and specific examples.

Examples of studies using mixed methods.

How is data analysed in a mixed methods project?

The most important, and perhaps most difficult, aspect of mixed methods research is integrating the qualitative and quantitative data. One approach is to analyse the two data types separately and to then undertake a second stage of analysis where the data and findings from both studies are compared, contrasted and combined. 19 The quantitative and qualitative data are kept analytically distinct and are analysed using techniques usually associated with that type of data; for example, statistical techniques could be used to analyse survey data whilst thematic analysis may be used to analyse interview data. In this approach, the integrity of each data is preserved whilst also capitalizing on the potential for enhanced understanding from combining the two data and sets of findings.

Another approach to mixed methods data analysis is the integrative strategy. 20 Rather than keeping the datasets separate, one type of data may be transformed into another type. That is to say that qualitative data may be turned into quantitative data (‘quantitizing’) or quantitative data may be converted into qualitative data (‘qualitizing’). 21 The former is probably the most common method of this type of integrated analysis. Quantitative transformation is achieved by the numerical coding of qualitative data to create variables that may relate to themes or constructs, allowing statements such as ‘six of 10 participants spoke of the financial barriers to accessing health care’. These data can then be combined with the quantitative dataset and analysed together. Transforming quantitative data into qualitative data is less common. An example of this is the development of narrative psychological ‘types’ from numerical data obtained by questionnaires. 22

Potential challenges in conducting mixed methods research

Despite its considerable strengths as an approach, mixed methods research can present researchers with challenges. 23 , 24

Firstly, combining methodologies has sometimes been seen as problematic because of the view that quantitative and qualitative belong to separate and incompatible paradigms. In this context, paradigms are the set of practices and beliefs held by an academic community at a given point in time. 25 Researchers subscribing to this view argue that it is neither possible nor desirable to combine quantitative and qualitative methods in a study as they represent essentially different and conflicting ways of viewing the world and how we collect information about it. 8 Other researchers take a more pragmatic view, believing that concerns about the incommensurability of worldviews can be set aside if the combination of quantitative and qualitative methods addresses the research question effectively. This pragmatic view informs much applied mixed methods research in health services or policy. 8

Secondly, combining two methods in one study can be time consuming and requires experience and skills in both quantitative and qualitative methods. This can mean, in reality, that a mixed methods project requires a team rather than a lone researcher in order to conduct the study rigorously and within the specified time frame. However, it is important that a team comprising members from different disciplines work well together, rather than becoming compartmentalized. 26 We believe that a project leader with experience in both quantitative and qualitative methods can act as an important bridge in a mixed methods team.

Thirdly, achieving true integration of the different types of data can be difficult. We have suggested various analytic strategies above but this can be hard to achieve as it requires innovative thinking to move between different types of data and make meaningful links between them. It is therefore important to reflect on the results of a study and ask if your understanding has been enriched by the combination of different types of data. If this is not the case then integration may not have occurred sufficiently. 23

Finally, many researchers cite the difficulty in presenting the results of mixed methods study as a barrier to conducting this type of research. 23 Researchers may decide to present their quantitative and qualitative data separately for different audiences. This strategy may involve a decision to publish additional work focusing on the interpretations and conclusions which come from comparing and contrasting findings from the different data types. See Box 1 for an example of this type of publication strategy. Many journals in the medical sciences have a distinct methodological base and relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. However, as the number of mixed methods studies increases in the health research literature we would expect researchers to feel more confident in the presentation of this type of work.

Many of the areas we explore in health are complex and multifaceted. Mixed methods research (combining quantitative and qualitative methods in one study) is an innovative and increasingly popular way of addressing these complexities. Although mixed methods research presents some challenges, in much the same way as every methodology does, this approach provides the research team with a wider range of tools at their disposal in order to answer a question. We believe that the production and integration of different types of data and the combination of skill sets in a team can generate insights into a research question, resulting in enriched understanding.

DECLARATIONS

Competing interests.

None declared

This work was funded by the Medical Research Council (MRC) [grant number: G0701648 to ST], and the MRC with the Economic and Social Research Council (ESRC) [grant number: G0800112 to JW]

Ethical approval

No ethical approval was required for this work

Contributorship

This work was conceived by both ST and JW who each carried out an independent literature review and collaborated on the structure and content of this report. ST wrote the manuscript with revisions and editing done by JW

Acknowledgements

We thank Professors Jonathan Elford and Ruth Gilbert for their comments on draft manuscripts

This article was submitted by the authors and peer reviewed by Geoffrey Harding

• Open access
• Published: 30 August 2024

Exploring medical and dental practitioner perspectives and developing a knowledge attitude and practice (KAP) evaluation tool for the common risk factor approach in managing non-communicable and periodontal diseases

• Lakshmi Puzhankara   ORCID: orcid.org/0000-0002-5559-5887 1 ,
• Vineetha Karuveettil   ORCID: orcid.org/0000-0002-5358-4391 2 ,
• Chandrashekar Janakiram   ORCID: orcid.org/0000-0003-1907-8708 2 ,
• Ramprasad Vasthare   ORCID: orcid.org/0000-0002-0181-7069 3 ,
• Sowmya Srinivasan   ORCID: orcid.org/0000-0001-8236-0103 4 &
• Angel Fenol   ORCID: orcid.org/0000-0001-5088-8368 5  

BMC Oral Health volume  24 , Article number:  1017 ( 2024 ) Cite this article

Metrics details

The Common Risk Factor Approach (CRFA) is one of the methods to achieve medical-dental integration. CRFA addresses shared risk factors among major Non-communicable Diseases (NCDs). This study aimed to explore the perspectives of dental and medical practitioners concerning CRFA for managing NCDs and periodontal diseases and to create and validate a tool to evaluate the Knowledge, Attitude, and Practice (KAP) of medical and dental practitioners in relation to utilization of CRFA for management of NCDs and Periodontal diseases.

This research employed a concurrent mixed-method model and was carried out from January 2021 to February 2022, focusing on medical and dental practitioners in South India. In the qualitative phase, online interviews were conducted with dental and medical practitioners, recorded, and transcribed. Thematic analysis was applied after achieving data saturation. In the quantitative phase, a KAP questionnaire was developed. The sample size was determined by using the G power statistical power analysis program. A sample size of 220 in each group (dentists and medical practitioners) was estimated. Systematic random sampling was used to recruit the potential participants. The data obtained through the online dissemination of KAP tool was analysed and scores were standardized to categorize the KAP.

Qualitative thematic analysis identified four major themes: understanding of common risk factors, risk factor reduction and disease burden, integrating CRFA into clinical practice, and barriers to CRFA. In addition, thematic analysis revealed seventeen subthemes. For the quantitative phase, standardization was applied to a 14-item KAP questionnaire for medical practitioners and a 19-item KAP questionnaire for dental practitioners. The total KAP score for medical practitioners in the study was 21.84 ± 2.87, while dental practitioners scored 22.82 ± 3.21, which indicated a high level of KAP regarding CRFA. Meta integration of qualitative and quantitative data identified eight overarching themes: four were concordant, three were discordant, and one theme provided the explanatory component.

The study’s structured, validated questionnaire showed that both medical and dental professionals had a high knowledge of CRFA. However, they were not appreciably aware of the risk factors that are shared between NCDs and periodontal disease. Both groups were interested in the idea of using CRFA in integrated medical and dental care.

Peer Review reports

Introduction

Non-communicable diseases (NCDs) account for more than 41 million deaths globally each year [ 1 ]. These diseases are influenced by both non-modifiable and modifiable risk factors [ 2 ]. Periodontal disease, another multifactorial non-communicable ailment, shares several risk factors with NCDs. Individuals with periodontal diseases, particularly periodontitis, face a heightened risk of losing multiple teeth, leading to compromised masticatory function and altered dietary habits [ 3 ]. This not only affects the quality of life and self-esteem of affected individuals but also imposes significant socio-economic burdens and healthcare costs [ 4 ]. Despite the evident connections between periodontal disease and NCDs [ 5 , 6 ], there persists a historical divide between oral and general healthcare [ 7 ], further reinforced by the establishment of medical insurance [ 8 ]. This separation has contributed to out-of-pocket expenditures (OOPE) on dental care, accounting for approximately 14% of OOPE in Organisation for Economic Co-operation and Development (OECD) countries. [ 9 ] A recent study in South India revealed that 15.4% of sanitary workers experienced Catastrophic Dental Health Expenditure (CDHE) [ 10 ]. Additionally, a global study involving 41 low- and middle-income countries found that 7% of households faced CDHE [ 11 ].

The integration of dental and medical care would bring substantial benefits to the general population. Oral health has a significant impact on general health. Simple, non-invasive periodontal therapy was found to result in a remarkable (40–70%) reduction in medical costs and hospitalizations for individuals with conditions such as diabetes, coronary artery disease, or during pregnancy [ 12 ]. This underscores the potential advantages of addressing oral health within the broader spectrum of healthcare, leading to improved overall health outcomes and reduced healthcare costs.

Several methods of integrating medical and dental care have been explored, [ 13 , 14 , 15 ] and one such strategy is risk reduction for disease prevention. Common risk factors such as smoking, obesity, poor nutrition, low socioeconomic status, stress, and inadequate oral hygiene are shared by both periodontitis and NCDs [ 5 ]. Traditional health promotion tends to focus on specific diseases, potentially contributing to the separation of oral health from general health. An alternative approach, the Common Risk Factor Approach (CRFA), addresses shared risk factors among major NCDs, including oral diseases. CRFA emphasizes managing contributing elements to enhance overall population health.

The approaches within CRFA aim to mitigate the impact of common chronic diseases [ 13 ] and include integrated action against shared risk factors and altering one risk factor that may influence others, leading to a cascade effect. For instance, changing smoking behavior could impact related behaviors like alcohol consumption and diet. Collaborative efforts across sectors, concentrating upstream on basic etiological factors, can lead to progress in oral health improvement and decreased oral health inequalities [ 16 ]. Given the clustering of both modifiable and non-modifiable risk factors in patients with NCDs and periodontal diseases, CRFA emerges as a cost-effective and rational approach [ 13 ]. Of these risk factors, modifiable risk factors can be controlled or changed. The control or modification of a few key risk factors can have a substantial impact on managing numerous chronic conditions.

The World Health Organization (WHO) advocates a global strategy for enhancing oral health alongside overall health, emphasizing shared risk factors [ 17 ]. Implementing CRFA for overall health, including oral health, presents opportunities to integrate oral health promotion into broader health policies, such as those related to food [ 15 ]. However, successful implementation requires appropriate evidence, guidelines, and policies due to perceived challenges in applying CRFA for oral health promotion [ 15 ].

To comprehensively assess the potential initiation of the CRFA for NCDs, including periodontal disease, it is crucial to understand the knowledge, attitudes, and practices of medical and dental practitioners regarding shared risk factors. While previous studies have explored knowledge about periodontitis risk factors among medical practitioners and the general population, [ 18 , 19 ] there is a notable gap in understanding the KAP of both medical and dental practitioners regarding shared risk factors between NCDs and periodontitis and the integration of CRFA into medical and dental practices.

Capacity-building measures are essential for implementing CRFA-based programs [ 15 ], and assessing the baseline KAP of the target population will bridge the evidence gap for integrations. Despite the pivotal role of CRFA in addressing health issues, there is currently no standardized instrument tailored to assess practitioners’ KAP in this context. Questionnaires are commonly used for KAP assessment [ 20 ], and a structured, validated questionnaire is essential for obtaining clear information on practitioners’ understanding and application of CRFA in managing NCDs and periodontal diseases.

The objectives of this mixed-method study are to address these gaps by understanding practitioners’ opinions on CRFA and developing a validated structured instrument to assess the Knowledge, Attitude, and Practice of medical and dental practitioners toward the use of CRFA for managing NCDs and periodontal diseases. The study will employ both quantitative and qualitative methods, utilizing a structured questionnaire to capture practitioners’ perspectives and incorporating open-ended communication to gain insights into the reasons behind their opinions, support, and potential hurdles in implementing CRFA in the Indian context.

The mixed-method study received ethical approval from the institutional ethics committee and institutional review board, and informed consent was obtained from the participants during the conduct of the study.

Research design

The study employed a concurrent mixed-methods model, incorporating both qualitative and quantitative arms, to holistically investigate the research questions. This approach combines the advantages of qualitative and quantitative data, allowing for a comprehensive exploration of the CRFA. The qualitative arm provides in-depth insights into the complex phenomena associated with CRFA, offering a contextual richness that complements the quantitative results. The lists of potential participants were obtained from the list of dentists and medical practitioners of Kerala, Karnataka, Tamil Nadu, Andhra Pradesh, Telangana, and Goa available through the regional Indian Dental Association (IDA), Indian Medical Association (IMA), and directories of medical and dental practitioners. Based on the data obtained from the directories, a state-wise distribution of samples was done. Systematic random sampling was used to select the possible participants for the study from January 2021 to February 2022.

Qualitative arm

Study context and population.

The qualitative segment of the study sought to delve into the viewpoints of experts in medicine and general dental practice, particularly those possessing relevant expertise related to the CRFA. Participants were selected from specialties such as endocrinology, gynaecology, otorhinolaryngology, periodontology, general medicine, and general dentistry, based on their relevance to the shared risk factors between periodontal disease and various medical conditions. Purposive sampling was employed to recruit a diverse group of medical and dental practitioners, and the sampling units were identified from the directories of professional associations like the Indian Dental Association (IDA) and the Indian Medical Association (IMA). Participation in the online interviews using the ‘Zoom Meetings’ online platform was voluntary. After obtaining their consent, the link for the Zoom meeting was shared with the participants. Participants received acknowledgment certificates as an incentive. No explicit exclusion criteria were set, ensuring a broad representation of perspectives across the selected fields.

In-depth interviews

The qualitative phase of the study utilized in-depth interview guides that covered similar topics for both dental and medical practitioners. These guides included components related to the understanding of common risk factors, risk factor reduction, and disease burden, suggested methods for integrating CRFA into clinical practice, and barriers to CRFA. The semi-structured questions were developed a priori, drawing from existing literature. The interviews were conducted with consent, and a note-keeper recorded the proceedings, while in-depth interviews were recorded for transcription. The recordings were transformed into verbatim transcripts at the end of each day.

The number of participants for in-depth interviews was determined based on achieving data saturation, ensuring that the sample size was sufficient to capture a diverse range of perspectives until no new information or themes emerged. Data saturation enhances the credibility and trustworthiness of study findings, signifying theoretical sufficiency. The analysis methodology involved progressive analysis throughout the study, allowing for the incremental identification and incorporation of themes and sub-themes after each interview. This iterative process facilitated the continual refinement of emerging data patterns.

The decision to conclude interviews was guided by the observation of the ceased emergence of new themes, indicating data saturation. Close monitoring of interview data helped identify a point where further sessions yielded no novel insights or themes. After achieving data saturation, a comprehensive final thematic analysis was conducted following guidelines by Braun and Clark [ 21 ] and reiterated by Kiger et al [ 22 ]. This analysis involved data review, coding, categorization, and synthesis to derive conclusive themes and sub-themes. Each transcript underwent review by two researchers, and emerging themes were developed, involving a third author in cases of disagreement. Consensus on codes, categories, and themes was reached through regular discussions. The data was organized and managed using computer-assisted qualitative research software, QDA Miner Lite (Version 2.0.7; Provalis Research).

Quantitative arm

The quantitative segment of the mixed-method study focused on developing and validating a KAP questionnaire on the CRFA for the integration of medical and dental care. Distinct questionnaires were created for medical and dental practitioners. The development of the questionnaire occurred in two stages.

In the first stage, item and domain development took place, involving a deductive approach to form initial questions, followed by content validation and test-retest reliability. The second stage involved the validation of the questionnaire through item response theory, exploratory factor analysis, and internal consistency reliability assessment. This two-stage process ensured the robustness and appropriateness of the questionnaire for assessing the KAP of medical and dental practitioners regarding CRFA in the context of managing NCDs and Periodontal diseases.

Study population

The study included both medical practitioners and dental practitioners, encompassing those with and without a postgraduate degree or specialization. This diverse inclusion aimed to capture perspectives from practitioners with varying levels of education and expertise, providing a comprehensive understanding of the knowledge, attitudes, and practices related to the CRFA among professionals in both fields.

Sample size

The sample size was determined by using the G power statistical power analysis program. Based on the findings from a previous study [ 23 ] a sample size of 220 dentists and medical practitioners was estimated. This was done by taking into account the Chi-square test’s effect size of 0.30, the study’s power of 0.95, and the number of groups of medical and dental practitioners that could be used to compare mean knowledge, attitude, and practice scores.

Data collection

The study utilized a systematic approach for sampling dental and medical practitioners from Kerala, Karnataka, Tamil Nadu, Andhra Pradesh, Telangana, and Goa. The directories of the regional Indian Dental Association (IDA) and Indian Medical Association (IMA) were consulted to compile a list of practitioners (both specialists and general practitioners). To ensure a representative sample, the distribution of participants was organized by state (Table  1 ).

Systematic random sampling was employed to select potential participants, minimizing bias in participant selection. Contact details were then used to send a web-based questionnaire via Google Forms, accompanied by an invitation to participate. Anticipating a 50% non-response rate, the questionnaires were distributed to twice the required number of participants. The final analysis included responses from 225 medical practitioners and 307 dental practitioners across South India.

Questionnaire development

The development and validation of the KAP questionnaire occurred in two distinct stages. In the first stage, item and domain development were undertaken through a three-step process: (i) Deductive approach, (ii) Content validation, (iii) Test-retest reliability. The second stage involved the validation of the questionnaire using: (i) Item response theory, (ii) Exploratory factor analysis, (iii) Internal consistency reliability assessment. Subsequently, scores were standardized to categorize the KAP of the population into low, medium, and high categories. This multi-stage process ensured the reliability and validity of the questionnaire for assessing participants’ knowledge, attitude, and practice regarding the CRFA.

Stage one: item and domain development

The deductive approach was employed to develop items for the questionnaire based on existing literature related to the CRFA in the management of periodontal disease and NCDs. Eight referenced articles contributed to the conceptual definition of knowledge, attitude, and practice regarding CRFA [ 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ]. The definition of CRFA emphasized its role in creating cross-disciplinary health promotion programs that address common risk factors for diseases. Knowledge, attitude, and practice were defined in terms of awareness, thoughts, behaviors, and understanding of shared risk factors and etiology related to periodontal disease and NCDs, as well as CRFA.

The initial questionnaire, developed in English, consisted of 28 items for the dental questionnaire and 24 items for the medical questionnaire, distributed across four domains: (1) Demography of participants; (2) Knowledge towards CRFA for NCDs and oral health; (3) Attitude towards CRFA for NCDs and oral health; and (4) Practice towards implementing CRFA for NCDs and oral health. To ensure content validity, the initial questionnaire underwent review by an expert panel comprising dental and medical practitioners. The test-retest reliability of the questionnaire was assessed by administering it twice to 30 participants within a one-month timeframe.

Stage two: questionnaire validation

The study included responses from 225 medical practitioners and 307 dental practitioners across six states in South India to evaluate the additional psychometric properties of the questionnaire. Data analysis was conducted using JMETRIK software.

Item response theory (IRT)

In the knowledge domain, a two-parameter logistic item response theory (2-PL IRT) analysis was conducted using responses categorized as either correct or incorrect. The analysis was performed in JMETRIK (version 4.0.0, Charlottesville, Virginia, USA) using the RASCH (log odds ratio) limited package. The analysis considered the range of difficulty (-4 to + 4) and discrimination (0.20 to infinity) as the cut-off values for evaluating psychometric properties. Item fit was assessed using chi-square goodness-of-fit per item, and p values were reported. The modified parallel analysis was employed to evaluate one-dimensionality.

Exploratory factor analysis

The adequacy of sampling was assessed using the Kaiser–Meyer–Olkin measure (KMO) and Bartlett’s test of sphericity [ 20 ]. A KMO value above 0.5 and a significant result in Bartlett’s test ( p  < 0.001) were considered indicative of a sufficient sample.

Internal consistency reliability

The internal consistency (IC) of the items was calculated using the coefficient of Cronbach’s alpha [ 31 ] and correlation between items.

Standardization of scores

The responses to the questions in the Knowledge, Attitude, and Practice groups were coded, and scores were calculated for each group. The scores were then split into percentiles for standardization. The total KAP score was also calculated and interpreted as low KAP (0 to 24th percentile), medium KAP (25th to 75th percentile), and high KAP (76th to 100th percentile) based on the percentile scores [ 32 ].

In-depth interviews involved five medical practitioners specializing in endocrinology, gynaecology, otorhinolaryngology, and general medicine, along with five general dental practitioners and five periodontists. The qualitative thematic analysis identified four major themes: understanding of common risk factors, risk factor reduction and disease burden, integrating CRFA into clinical practice, and barriers to CRFA. Subsequently, seventeen subthemes emerged, encompassing topics such as enumerating risk factors, transitioning from disease-specific to risk factor approaches, diagnosing systemic NCDs through identifying risk factors and oral signs, controlling risk factors and NCD burden, the impact of periodontal therapy on NCD burden, the influence of medical practitioners over periodontists, measures for integrating CRFA, barriers to integration, and more.

Theme 1: understanding shared common risk factors

The study revealed that medical and dental practitioners, including periodontists, demonstrated awareness of the association between diabetes and periodontal disease, as well as the shared risk factor of smoking. However, their knowledge regarding risk factors common to other major NCDs and periodontal disease was limited. Many practitioners were unable to identify shared risk factors such as obesity, the presence of oral pathogens, and nutritional deficiency [ 5 ]. This knowledge gap may be attributed to the prevailing practice of treating patients based on specific diseases rather than targeting shared risk factors. Although there is a gradual shift toward a risk factor-based approach in certain specialties, there remains a general scepticism about patient compliance with long-term risk factor reduction strategies. The subthemes that emerged under this major theme are: (i) Enumeration of the risk factors (ii) Transition from disease specific to risk factor approach (iii) Diagnosis of a systemic NCD through identification of presence of risk factors and oral signs .

‘There are many risk factors, ranging from smoking to genetics. Very common ones are smoking, alcohol, lifestyle. Each and every factor has a specific role. Genetics has a significant role. If a parent is diabetic by his or her 50s then the next generation will become diabetic by 30s’. (MP1)

There was a consensus regarding the need for a change from a disease specific approach to a risk factor approach.

MP1 had supported CRFA. ‘This is a very good approach. Common risk factors are present for many diseases. So, if we can create an awareness regarding smoking, alcohol, and sedentary lifestyle, it can significantly reduce the development of many diseases.’

The identification of clustering of risk factors for periodontal disease and NCDs in patients, in addition to the occurrence of oral signs, can sometimes lead to the diagnosis of systemic diseases.

PR5 ‘In diabetes we have noticed. They come with multiple abscesses, then we advise them to check the blood glucose level and they are diagnosed with diabetes. They are not aware of the condition before. So, once we treat the patient and with the consultation with the diabetologist, we have noticed an improvement in the status.’

Theme 2: risk factor reduction and disease burden

All practitioners concurred on the potential positive impact of early identification of risk factors, counselling, and reducing risk factors to mitigate disease burden. Nevertheless, medical practitioners acknowledged that a significant portion of them tend to overlook oral health, possibly due to a lack of awareness regarding its association with systemic conditions. The thematic analysis revealed subthemes such as (i) Control of Risk Factors and Impact on NCD Burden (ii) The Role of Periodontal Therapy in Alleviating the NCD Burden (iii) Reciprocal Impact of Other NCD Therapies on Periodontal Disease Burden (iv) Influence of Medical Practitioners in Shaping Patient Decisions Over Periodontists. These subthemes underscored the interconnectedness of risk factors, diverse therapies, and the collaborative role of medical and dental practitioners in addressing both oral and systemic health.

PR1 stated that ‘Lifestyle modification…I have been following the periodontal patients in my clinic. There are patients whom I have been following for last 6 to 7 years. Patients who have been motivated to maintain the oral hygiene, their rate of progression (of periodontal disease) and diabetic control is much better than patients who are not maintaining their oral hygiene properly.’

The dental practitioners have observed that periodontal therapy can result in improving the NCD status and that a better compliance is observed when the advice is given by a medical practitioner.

‘Yes, after periodontal treatment, sugar level often reduces as noticed in diabetes. Diabetics with uncontrolled sugar levels, fluctuating sugar levels, after periodontal therapy usually have better controlled sugar levels’, GP1 said.

PR3 said, ‘Yes definitely, when a physician refers the patient to us, they are more willing to listen to us and adapt to whatever changes we say.’

Theme 3: methods suggested for integrating CRFA into clinical practice

Various approaches have been proposed to integrate the CRFA into clinical practice. These strategies encompass capacity building initiatives to promote medical-dental integration, such as establishing NCD clinics; raising awareness among the medical community regarding the interconnectedness of medical and dental health; advocating for policies that underscore the significance of CRFA integration in clinical settings; developing effective healthcare referral systems and cross-disciplinary health promotion strategies, including oral health care; and encouraging patient education and motivation. The subthemes within this overarching theme are: (i) Capacity Building (ii) Advocacy and Policy Implications (iii) Healthcare Partnerships Involving Referrals and Cross-Disciplinary Health Promotion Strategies (iv) Patient Education and Motivation.

The interviews highlighted diverse strategies for capacity building, including the implementation of regular check-ups and screening camps as integral components of healthcare services. Furthermore, suggestions encompassed the use of awareness posters and videos, adoption of evidence-based practices, and the establishment of NCD clinicsx [ 33 ]. NCD clinics, as proposed, would serve as essential hubs for screening, diagnosing, and managing NCDs. These clinics would offer comprehensive examinations, including diet counselling, lifestyle management, and home-based care. Patients could be referred to these clinics by other healthcare centres, health workers, or they could directly report to the clinic, enabling the identification and management of complications or advanced stages of NCDs. MP1 stated, ‘In government clinics, there are NCD clinic. Along with the NCD clinic, if a dental clinic can be set up, a lot of cases with oral manifestations will be diagnosed. So integrated clinics with NCD and dental will be very useful.’

Advocacy enables stakeholders and government decision-makers to have discussions and bring out suggestions and recommendations to a prevailing policy that is of interest to them.

MP1 Suggested that ‘Even for a job opportunity, basic examination is physical examination and evaluation for systemic diseases. Oral examination may be included in the basic fitness requirement for the job.’

Interdisciplinary collaboration is also essential for medical dental integration as stated by MP3, ‘There should be a rapport between the medical and dental practitioner so that there is communication regarding the cases and there is a follow-up of the cases.’

Communication through mass media and other visual aids, generating social and cultural awareness for patient education, and motivation for holistic health care have also been suggested to facilitate the implementation of integrated care delivery.

PR2 has mentioned, ‘When this gets published, apart from journals, this should reach the common population also. The common population rarely see the journal articles. So, it should be brought forth in mass media so that it reaches the population.’

Theme 4: barriers for implementation of CRFA

CRFA is considered a relatively novel approach, as the comprehensive exploration of shared risk factors and risk reduction strategies for common NCDs and periodontal disease is a recent development. The lack of awareness regarding this concept has been identified as a significant barrier to its implementation, coupled with challenges such as time constraints, concerns about the sustainability of long-term risk reduction strategies, and the need for extended resources. Moreover, the existing strict specialization within healthcare disciplines and the lack of interdisciplinary coordination pose additional obstacles to the effective execution of CRFA. The subthemes encompass: (i) Lack of awareness (ii) Time constraints (iii) Sustainability (iv) Long-term outcomes or no outcomes (v) Lack of resources (vi) Lack of interdisciplinary coordination and strict specialization.MP2 said ‘One is that among us practitioners, we do not give due significance to the link between oral health and systemic health. There are no awareness programs as far as I know. The emphasis is less’.

GP5 said, ‘They (medical practitioners) don’t have time to peep into the oral cavity to say you have caries, go to a dentist or say you have diabetes and there is a chance to develop periodontal disease. Such opportunities are less.’

The results of following the risk reduction strategies may take a long time to manifest, and sometimes the outcomes are not as significant as what the patient would have expected. This results in a spiralling of the patient’s attitude and a failure of further follow-up.

‘In long term, the patients may become uncooperative, and patients will not be willing for a follow-up, they will go for things that have cost-benefit’MP1.

The lack of resources, manpower and facilities to deliver the care act as significant barriers to implementation of CRFA.

MP3 has stated, ‘Cost is a problem, social acceptance is a problem, policy makers and political involvement are a problem, lack of communication between communities…In the western countries, like UK, they have NHS care, we don’t have that in India and patients hence don’t go for any care if they feel it is unnecessary’.

i) Content validation

The total number of questions included in the dental and medical questionnaires using the deductive approach was 28 and 24 respectively. After discussion, one question was eliminated from both the medical and dental questionnaires as it had a similar connotation to a previous question. Content validation of each scale was performed by five experts to ensure content relevance, representativeness, and technical quality. The KAP questionnaire was reduced to 26 questions for dental practitioners after content validation. Item reduction was performed to 22 for the questionnaire for medical practitioners after eliminating 1 question. A few questions were rephrased based on the suggestions given by the expert panel prior to administering the questionnaire for test-retest reliability assessment. The details of content validation are given in Table  2 .

ii) Test-re test reliability

The scoring of items was done, and the data was utilized to assess the reliability of the questionnaire. 21 questions in dental and medical questionnaires were subjected to test-retest reliability assessment. Five questions in the dental questionnaire were option questions, leading questions, or open-ended questions (Eg: Are you a periodontist) and one question in the medical questionnaire was open ended, hence they were not subjected to test-retest reliability. The unweighted Kappa coefficient was used to assess the reliability of the items with binary responses (Table  3 ). The intraclass correlation coefficient (ICC) was used for assessing the questions in the attitude category with categorical variables (Table  3 ). Based on the test-retest reliability assessment, three questions from the dental questionnaire and two questions from the medical questionnaire were eliminated.

iii)Psychometric evaluation of questionnaire

The 20-item medical and 23-item dental KAP questionnaires (including the open-ended and leading questions) were administered to 450 medical and dental practitioners, and responses were obtained from 225 samples in the medical stream and 307 in the dental stream.

In the medical KAP questionnaire, four items from the knowledge domain and one item each from the attitude and practice domain were eliminated owing to the high difficulty statistic. One item each from the knowledge and practice domain was retained considering the importance of the items, even though they had a higher difficulty range. After item reduction using item response theory, 14 items (including the open-ended question) remained in the final questionnaire for medical practitioners. The KMO sampling adequacy and test of sphericity for the domains of knowledge, attitude, and practice were found to be in an acceptable range. Internal consistency measured using Cronbach’s alpha improved from 0.471 to 0.658 for the attitude domain after item deletion. For knowledge and practice, the Cronbach’s alpha after item deletion was reported to be 0.553 and 0.727, respectively.

The 23-item questionnaire was reduced to 19 items with the elimination of 3 items from THE knowledge domain and single item from attitude domain. Two items with poor scores of difficulty were deemed to be important in the questionnaire and were not eliminated. After item reduction, a total of 14 items remained in the final questionnaire in addition to the five leading/option questions. The KMO sampling adequacy and test of sphericity for the domains of knowledge, attitude, and practice were found to be in acceptable range. Internal consistency measured using Cronbach’s alpha was found to be slightly reliable in case of the attitude domain (0.459). While for knowledge and practice domain internal consistency was within the acceptable range (Knowledge 0.634, Practice 0.513) after item deletion.

Multivariate logistic regression was attempted between the parameters such as age, gender, qualification, experience, type of service, location, and number of patients seen per day and the knowledge, attitude, and practice regarding CRFA for both medical and dental practitioners, and no significant results were obtained for both medical and dental practitioners. (The details of the psychometric evaluation of the questionnaire and the characteristics of the study population are given in supplementary file 1)

iv) standardization of scores

For the south Indian population, the 14 item questionnaire scores were standardized (Table  4 ).

The validated questionnaires for medical and dental practitioners are given in supplementary file 2. For the medical KAP questionnaire, scores below 14 indicated low KAP, scores between 15 and 18 indicated medium, and scores greater than 18 indicated good knowledge, attitude, and practice of CRFA. For dental practitioners, scores less than 16 were reported to be low KAP; scores 16 to 19 indicated medium level; and scores greater than 20 indicated a good level of knowledge, attitude, and practice regarding CRFA.

Total KAP amongst the medical practitioners who participated in the present study was 21.84 ± 2.87 and that of dental practitioners was 22.82 ± 3.21. Both values indicated a high level of KAP amongst the participants regarding CRFA.

Meta-integration

Eight overarching themes emerged in the meta integration of the qualitative and quantitative data (Fig.  1 ). The themes that had a confirmatory fit as assessed from both the quantitative and qualitative aspects of the study include (i) awareness of common risk factors for NCDs including periodontal diseases, (ii) neglect of dental status while assessing general health, (iii) awareness of effect of systemic diseases on oral health, (iv) awareness of risk factor reduction and improvement of NCD status. Contradictory observations from the quantitative and qualitative arms of the study resulted in a discordant fit in the following themes: (i) regular follow-up of periodontal health of patients with NCDs (ii) awareness regarding need for referral for periodontal examination and management in patients with NCDs (iii) awareness of perio-systemic interlink. The qualitative arm of the study explained the theme ‘Reasons for lack of referral to dental practitioners by medical practitioners’ and provided reasons such a reduced emphasis on oral health with a lack of awareness regarding the same amongst the practitioners, resource and time constraints that prevent the medical practitioners from looking into the overall health of the patient apart from the presenting complaint, overspecialization of the medical field with focus only on the specific field of specialization, to state a few.

Awareness of common risk factors for NCDs including periodontal diseases

NCDs and periodontal disease pose substantial societal burdens in terms of economic costs and years lost to ill-health, disability, or premature death [ 34 ]. Various factors, including social, demographic, environmental, behavioural, and personal elements, predispose individuals to major NCDs and oral diseases [ 5 ]. The CRFA addresses these shared risk factors, allowing the regulation of a few risk factors to exert a significant impact on controlling multiple chronic conditions [ 5 ]. This study has successfully developed and validated a questionnaire with satisfactory content validity and reliability to assess the knowledge, attitude, and behavior regarding CRFA for managing NCDs and periodontal disease.

To the best of our knowledge, this is the first study to create a suitable questionnaire for this purpose, incorporating a qualitative component to comprehend potential pathways and barriers to CRFA implementation. All retained questionnaire items demonstrated discrimination and difficulty parameters within acceptable ranges [ 20 ]. The KAP questionnaire exhibited acceptable internal consistency, validating its effectiveness for assessing CRFA-related KAP.

A crucial finding is the lack of understanding among medical and dental practitioners regarding common risk factors for NCDs and periodontal disease, hindering the implementation of CRFA. Literature that demonstrates the presence of shared risk factors between periodontal disease and other non-communicable diseases has, perchance, not been extensively explored by the health-care community. Almeida et al., in their systematic review, showed that the inflammatory mediators CRP and IL-6 had a significant association with both periodontitis and atherosclerosis [ 35 ]. A study by Arregoces et al. [ 36 ] showed an increase in ultrasensitive CRP (usCRP) in acute myocardial infarction (AMI), diabetes and periodontal disease. abdominal obesity [ 37 ] and insulin resistance [ 38 ] are proven to be contributing risk factors for metabolic syndrome and periodontal disease. The risk for CVD and periodontal disease is related to poor glycemic control, dyslipidemia, and chronic inflammatory state [ 39 , 40 , 41 ]. Smoking has been proven as a risk factor for periodontal disease, hypertension, diabetes, and metabolic syndrome through several studies [ 42 , 43 , 44 , 45 ]. Holmlund et al. have demonstrated the association between immunoglobulin G levels against P gingivalis and the risk for AMI and periodontal disease [ 46 ]. The presence of Aggregatibacter actinomycetemcomitans (Aa) is shown to be a risk factor for Coronary Artery Disease (CAD) and periodontal diseasecx [ 47 ]. The role of stress and depression as risk factors for CVD and periodontal disease has been investigated and recognized [ 48 ].

Apart from the lack of sufficient knowledge regarding the shared risk factors between periodontal disease and NCDs, there are additional barriers to the implementation of CRFA for the management of periodontal disease and NCDs. Barriers include time and resource constraints, oral health neglect in general health assessments, insufficient recognition of the need for oral health care referral for NCD patients, and limited acknowledgment of the perio-systemic interlink. However, the integration of medical and dental care is not impossible, and efforts such as creating awareness, education programs, mass media campaigns, and efficient referral systems are advocated by healthcare professionals.

The Health Resources and Services Administration (HRSA) has explained initiatives for incorporating oral health into primary medical care practice and training primary health care professionals in oral health assessment and clinical competencies [ 49 ]. The combination of preventive dental care with general health care practice can help reduce duplication of care modalities and expenses incurred. Six levels of integration, with the evolution of the key elements involved in the integration, from communication to physical proximity to practice change, have been described [ 50 ]. Communication is the key element in the first and second levels of integration in which there is minimal collaboration and basic collaboration at a distance respectively. Basic collaboration onsite and close collaboration onsite with some system integration form the third and fourth levels of integration, in which physical proximity is the key element. The fifth and sixth levels of integration include practice change, in which there is close collaboration with an integrated practice and full collaboration with a merged, integrated practice [ 50 ].

This research indicates that while the presence of shared risk factors among NCDs is acknowledged, medical practitioners often overlook the link between oral health and systemic health. Addressing this gap in healthcare practice involves providing basic oral health care training as an integral part of general health education.

In India, the checklist for early detection of NCDs, which is used in community based NCD surveillance, takes into consideration risk factors such as age of patient, smoking, alcohol consumption, measurement of waist, physical activities, and family history of NCDs [ 51 ]. These risk factors are similar to the risk factors for periodontal disease [ 13 ]. Thus, the risk factor surveillance may be extended to include periodontal disease as well. The primary healthcare teams can be trained in strategies to reduce or modify the risk factors associated with systemic diseases and oral diseases. The methods to assess the efficiency of the integrated practice in the primary health care setting include the calculation of the percentage of patients assessed using the surveillance tool, to the percentage of staff satisfied with the referral process [ 52 ]. Research conducted in Saudi Arabia showed that the availability of an appropriate source of oral health knowledge was significantly associated with increased odds of inter-disciplinary practice [ 53 ]. Regular patient reviews and examinations, along with the reinforcement of risk reduction strategies, can be achieved through the application of knowledge regarding shared risk factors, facilitating the efficient integration of medical and dental care.

This combined mixed-methods study has the limitation that the quantitative aspect was primarily conducted through online Google Forms, which were sent only to the medical and dental practitioners who are registered in the databases that were utilized in the study, and hence the representativeness of the sample may be compromised. However, given the study’s design, which provides insights into the perspectives of healthcare professionals in various fields, the results offer a valuable reflection of the KAP regarding CRFA among medical and dental practitioners.

The questionnaire derived from the quantitative segment of this study stands as a straightforward and effective tool for evaluating KAP related to the CRFA concerning both oral and general health. In alignment with the ongoing global efforts to enhance oral health strategies, CRFA emerges as a promising approach for seamlessly integrating medical and dental care. The qualitative aspect of this study showed that to foster this integration, key recommendations include raising awareness about the interconnectedness of oral and systemic conditions, addressing constraints related to time and resources, and establishing robust referral systems between medical and dental practitioners. These measures collectively aim to establish a unified and integrated medical-dental care system.

Data availability

The datasets generated and analysed during the current study are available from the corresponding author on reasonable request.The mixed-method study received ethical approvals from the Institutional Review Board of Amrita Institute of Medical Sciences, Kochi, with the reference IRB-AIMS-2020-165, and the Kasturba Medical College and Kasturba Hospital Institutional Ethics Committee, under the reference IEC-664/2020 and informed consent was obtained from the participants.

WHO, Non-Communicable, Diseases. Published. 2018. Accessed December 11, 2019. https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases

Jeong NJ, Park E, Del Pobil AP. Effects of behavioral risk factors and social-environmental factors on non-communicable diseases in South Korea: A National Survey Approach. Int J Environ Res Public Health. 2021;18(2):1–11. https://doi.org/10.3390/IJERPH18020612 .

Article   Google Scholar  

Azzolino D, Passarelli PC, De Angelis P, Piccirillo GB, D’addona A, Cesari M. Poor oral health as a determinant of malnutrition and sarcopenia. Nutrients. 2019;11(12). https://doi.org/10.3390/nu11122898 .

Uy SNMR, Deng K, Fok CTC, Fok MR, Pelekos G, Tonetti MS. Food intake, masticatory function, tooth mobility, loss of posterior support, and diminished quality of life are associated with more advanced periodontitis stage diagnosis. J Clin Periodontol. 2022;49(3):240–50. https://doi.org/10.1111/JCPE.13588 .

Article   PubMed   Google Scholar  

Puzhankara L, Janakiram C. Common risk factor Approach to Limit Noncommunicable diseases and Periodontal Disease—the Molecular and Cellular basis: a narrative review. J Int Soc Prev Community Dent. 2021;11(5):490. https://doi.org/10.4103/JISPCD.JISPCD_109_21 .

Article   PubMed   PubMed Central   Google Scholar  

Janakiram C, Taha F, Joseph J, Ramanarayanan V. Assessment of Common Risk Factors Between Oral Diseases and Non-communicable Diseases in a Hospital-based Population in Kerala, IndiaA Cross-sectional Study. JOURNAL OF CLINICAL AND DIAGNOSTIC RESEARCH . Published online 2019. https://doi.org/10.7860/jcdr/2019/38182.12712

Otto M. Teeth: The Story of Beauty, Inequality, and the Struggle for Oral Health in America: 9781620971444: Medicine & Health Science Books @ Amazon.Com .; 2017. Accessed October 21, 2020. https://www.amazon.com/Teeth-Beauty-Inequality-Struggle-America/dp/1620971445

Simon L. Overcoming historical separation between oral and General Health Care: Interprofessional Collaboration for Promoting Health Equity. AMA J Ethics. 2016;18(9):941–9. https://doi.org/10.1001/journalofethics.2016.18.9.pfor1-1609 .

Health at a glance 2023 . OECD; 2023. https://doi.org/10.1787/7a7afb35-en

Nobelika Aa, Simon A. An observational study to find the patterns of out-of-pocket expenditure for oral healthcare among sanitary workers in Coimbatore, India. J Indian Association Public Health Dentistry. 2020;18(2):143. https://doi.org/10.4103/jiaphd.jiaphd_78_19 .

Masood M, Sheiham A, Bernabé E. Household Expenditure for Dental Care in Low and Middle Income Countries. Milgrom PM, ed. PLoS One . 2015;10(4):e0123075. https://doi.org/10.1371/journal.pone.0123075

Jeffcoat MK, Jeffcoat RL, Gladowski PA, Bramson JB, Blum JJ. Impact of periodontal therapy on general health: evidence from insurance data for five systemic conditions. Am J Prev Med. 2014;47(2):166–74.

Sheiham A, Watt RG. The common risk factor approach: a rational basis for promoting oral health. Community Dent Oral Epidemiol. 2000;28(6):399–406. https://doi.org/10.1034/J.1600-0528.2000.028006399.X .

Article   CAS   PubMed   Google Scholar  

Heath B, Wise Romero P, Reynolds K. A Review and Proposed Standard Framework for Levels of Integrated Healthcare. ; 2013.

Santhosh Kumar C, Somasundara S. Common Risk Factor Approach: Finding Common Ground for Better Health Outcomes . Vol 4. Online; 2015. Accessed March 21, 2020. www.ijcmr.com.

Heilmann A, Sheiham A, Watt R, Jordan R. Common risk factor Approach – An Integrated Population- and Evidence-Based Approach for Reducing Social Inequalities in oral health. Das Gesundheitswesen. 2015;78(10):672–7. https://doi.org/10.1055/s-0035-1548933 .

Benzian H, Guarnizo-Herreño CC, Kearns C, Muriithi MW, Watt RG. The WHO global strategy for oral health: an opportunity for bold action. Lancet. 2021;398(10296):192–4. https://doi.org/10.1016/S0140-6736(21)01404-5 .

da Rocha JM, Chaves VR, Urbanetz AA, dos Baldissera RS, Rösing CK. Obstetricians’ knowledge of periodontal disease as a potential risk factor for preterm delivery and low birth weight. Braz Oral Res. 2011;25(3):248–54. https://doi.org/10.1590/S1806-83242011000300010 .

Khan F, Ayub A, Kibria Z, Knowledge. Attitude and Practice about Oral Health among General Population of Peshawar | Journal of the Dow University of Health Sciences (JDUHS). Journal of the Dow University of Health Sciences . 2013;7(3):117–121. Accessed March 23, 2022. https://jduhs.com/index.php/jduhs/article/view/1541/0

Selvaraj S, Naing NN, Wan-arfah N, Karobari MI. Development and Validation of Oral Health Knowledge, attitude and Behavior Questionnaire among Indian adults. Medicina (B Aires) . Published Online 2022:1–11.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa .

Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE Guide 131. Med Teach. 2020;42(8):846–54. https://doi.org/10.1080/0142159X.2020.1755030 .

Obulareddy VT, Nagarakanti S, Chava VK. Knowledge, attitudes, and practice behaviors of medical specialists for the relationship between diabetes and periodontal disease: a questionnaire survey. J Family Med Prim Care. 2018;7:175–8.

Akram Z, Abduljabbar T, Hanif A, Khan A, Vohra F. Attitude and knowledge of family medicine practitioners towards the association between periodontal disease and obesity. Niger J Clin Pract. 2017;20(5):595–9. https://doi.org/10.4103/1119-3077.197017 .

Suadad AL-DB, AL-Daboony S, Knowledge. Attitude and practices towards Noncommunicable Disease Risk factors among medical staff. Glob J Med Res. 2016;16(3).

Megha V, Shashikanth H, Rajesh KS, Arun K. Periodontal Medicine: Assessment of Awareness Among Medical Professionals. 2015;3(1):1–6.

Umeizudike KA, Iwuala SO, Ozoh OB, Ayanbadejo PO, Fasanmade OA. Association between periodontal diseases and systemic illnesses: a survey among internal medicine residents in Nigeria. Saudi Dent J. 2016;28(1):24–30. https://doi.org/10.1016/j.sdentj.2015.03.005 .

Prakash S, Krithi N, Lakshminarayan N. Knowledge and willingness to adopt Chairside Screening for Medical conditions by practicing. J Dent Sci Oral Rehab. 2016;7(June):57–61.

Google Scholar  

Shimpi N, Glurich I, Panny A, Acharya A. Knowledgeability, attitude, and practice behaviors of primary care providers toward managing patients’ oral health care in medical practice: Wisconsin statewide survey. J Am Dent Assoc. 2019;150(10):863–72. https://doi.org/10.1016/j.adaj.2019.05.020 .

Boateng GO, Neilands TB, Frongillo EA. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research : A Primer. 2018;6(June):1–18. https://doi.org/10.3389/fpubh.2018.00149

Taber KS. The Use of Cronbach’s alpha when developing and Reporting Research Instruments in Science Education. Res Sci Educ. 2018;48(6):1273–96. https://doi.org/10.1007/S11165-016-9602-2/TABLES/1 .

Lamprianou I, Athanasou JA. Percentile Ranks and Standard Scores. In: Teacher’s Guide to Educational Assessment . Sense Publishers; 2009:305–308.

Ministry of Health and Family Welfare. Update on Prevention and control of non-communicable diseases (NCDs). Press Information Bureau.

Wolf TG, Cagetti MG, Fisher Jmarcus, Seeberger GK, Campus G. Non-communicable Diseases and Oral Health: An Overview. 2021;2(September):1–6. https://doi.org/10.3389/froh.2021.725460

Almeidaa APCPSC, Fagundesa NCF, Maia LC, Lima RR. Is there an Association between Periodontitis and atherosclerosis in adults? A systematic review. Curr Vasc Pharmacol. 2018;16:569–82. https://doi.org/10.2174/1570161115666170830141852 .

Article   CAS   Google Scholar  

Arregoces FE, Uriza CL, Velosa J, Beatriz M, Camargo F, Morales AR. Relation between ultra-sensitive C-reactive protein, diabetes and periodontal disease in patients with and without myocardial infarction. 2014;58(4). https://doi.org/10.1590/0004-2730000002899

Andriankaja OM, Sreenivasa S, Dunford R, Denardin E. Association between metabolic syndrome and periodontal disease. Published Online 2010:252–9. https://doi.org/10.1111/j.1834-7819.2010.01231.x

Benguigui C, Bongard V, Ruidavets J-B, Chamontin B, Sixou M, Ferrie`res JAJ. Metabolic syndrome, insulin resistance, and periodontitis : a cross-sectional study in a middle-aged French population. J Clin Periodontol 2010. 2010;37:601–8. https://doi.org/10.1111/j.1600-051X.2010.01571.x .

Dhir S, Kumar V. Are cardiovascular risk parameters and glycemic levels associated with periodontitis in type 2 diabetes patients ? A clinical study. Indian Heart J Published Online 2017:2–4. https://doi.org/10.1016/j.ihj.2017.10.002

Hanioka T, Morita M, Yamamoto T, et al. Smoking and periodontal microorganisms. Japanese Dent Sci Rev. 2019;55(1):88–94. https://doi.org/10.1016/j.jdsr.2019.03.002 .

Toyoko Morita Y, Yamazaki C, Fujiharu, Takanori Ishii MS, Norihide Nishinoue Y, Sasaki K, Nakai HT, Takayuki Kawato MM. Association between the Duration of Periodontitis and increased cardiometabolic risk factors. Metab Syndr Relat Disord. 2016;14(10):475–82. https://doi.org/10.1089/met.2016.0018 .

Górska R, Dembowska E, Konopka TP, Wysokińska-miszczuk J, Pietruska M, Ganowicz E. Correlation between the state of periodontal tissues and selected risk factors for periodontitis and myocardial infarction. Published Online. 2017. https://doi.org/10.17219/acem/74652 .

Gupta S, Maharjan A, Dhami B, Amgain P, Katwal S, Shukla A. Status of Tobacco Smoking and Diabetes with Periodontal Disease. 2018;56(213):818–24. https://doi.org/10.31729/jnma.3610

Hong M, Kim HY, Seok H et al. Prevalence and risk factors of periodontitis among adults with or without diabetes mellitus. Published Online 2016:910–9.

Koo HS, Hong SM. Prevalence and risk factors for Periodontitis among patients with metabolic syndrome. Metab Syndr Relat Disord. 2018;16(7):375–81. https://doi.org/10.1089/met.2018.0003 .

Holmlund A, Hedin M, Pussinen PJ, Lerner UH, Lind L. Porphyromonas gingivalis (pg) a possible link between impaired oral health and acute myocardial infarction. Int J Cardiol. 2011;148(2):148–53. https://doi.org/10.1016/j.ijcard.2009.10.034 .

Mantyla P, Buhlin K, Paju S, Persson GR, Nieminen MS, Sinisalo JPPJ. Subgingival Aggregatibacter actinomycetemcomitans associates with the risk of coronary artery disease. J Clin Periodontology. 2013;40:583–90. https://doi.org/10.1111/jcpe.12098 .

Kjellström B, Gustafsson A, Nordendal E, Norhammar A, Nygren Å, Näsman P. Symptoms of depression and their relation to myocardial infarction and periodontitis. Published online 2016. https://doi.org/10.1177/1474515116686462

White paper: Integrating oral health with primary health care. Association of state and territorial dental directors (ASTDD). Published December 2016. Accessed October 22. 2020. https://www.astdd.org/www/docs/integrating-oral-health-with-primary-health-care.pdf

SAMHSA-HRSA Center for Integrated Health Solutions (CIHS) | SAMHSA. Accessed October 26. 2020. https://www.samhsa.gov/integrated-health-solutions

Puzhankara L, Janakiram C. Medical-Dental Integration-Achieving Equity in Periodontal and General Healthcare in the Indian scenario. J Int Soc Prev Community Dent. 2021;11(4):359. https://doi.org/10.4103/JISPCD.JISPCD_75_21 .

Hummel J, Phillips Kathryn E, Holt Bre HC. Oral Health: An Essential Component of Primary Care White Paper .; 2015.

Gaffar B, Farooqi FA, Nazir MA, et al. Oral health-related interdisciplinary practices among healthcare professionals in Saudi Arabia: does integrated care exist? BMC Oral Health. 2022;22(1):1–12. https://doi.org/10.1186/s12903-022-02113-5 .

Download references

Acknowledgements

The authors acknowledge the participants of the study.

The authors have not received any source of funding for this study.

Author information

Authors and affiliations.

Department of Periodontology, Manipal College of Dental Sciences, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Lakshmi Puzhankara

Department of Public Health Dentistry, Amrita School of Dentistry, Amrita Vishwa Vidyapeetham, Kochi, Kerala, India

Vineetha Karuveettil & Chandrashekar Janakiram

Department of Public Health Dentistry, Manipal College of Dental Sciences, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Ramprasad Vasthare

Department of Periodontics, Amrita School of Dentistry, Amrita Vishwa Vidyapeetham, Kochi, Kerala, India

Sowmya Srinivasan

Formerly affiliated with Kent County Council, Maidstone, UK

Angel Fenol

You can also search for this author in PubMed   Google Scholar

Contributions

LP: conception and design, acquisition of data and interpretation of data, drafting the article, final approval of the version to be published; VK: conception and design, acquisition of data, analysis and interpretation of data, drafting the article, final approval of the version to be published; LP and VK contributed equally for the preparation of the manuscript; CJ: conception and design, analysis and interpretation of data, revising article critically, final approval of the version to be published; RV: Design, interpretation of data, revising article critically, final approval of the version to be published; SS: Design, interpretation of data, revising article critically, final approval of the version to be published; AF: Design, interpretation of data, revising article critically, final approval of the version to be published.

Corresponding author

Correspondence to Chandrashekar Janakiram .

Ethics declarations

Ethics approval and consent to participate.

The mixed-method study received ethical approvals from the Institutional Review Board of Amrita Institute of Medical Sciences, Kochi, with the reference IRB-AIMS-2020-165, and the Kasturba Medical College and Kasturba Hospital Institutional Ethics Committee, under the reference IEC-664/2020 and informed consent was obtained from the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Puzhankara, L., Karuveettil, V., Janakiram, C. et al. Exploring medical and dental practitioner perspectives and developing a knowledge attitude and practice (KAP) evaluation tool for the common risk factor approach in managing non-communicable and periodontal diseases. BMC Oral Health 24 , 1017 (2024). https://doi.org/10.1186/s12903-024-04772-y

Download citation

Received : 02 May 2024

Accepted : 20 August 2024

Published : 30 August 2024

DOI : https://doi.org/10.1186/s12903-024-04772-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Healthcare coalitions
• Health Services Research
• Medical-dental integration
• Non-communicable diseases
• Periodontal diseases
• Risk factors

BMC Oral Health

ISSN: 1472-6831

• General enquiries: [email protected]