quantitative research rigour

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

3.7 Quantitative Rigour

The extent to which the researchers strive to improve the quality of their study is referred to as rigour. Rigour is accomplished in quantitative research by measuring validity and reliability. 55 These concepts affect the quality of findings and their applicability to broader populations.

Validity refers to the accuracy of a measure. It is the extent to which a study or test accurately measures what it sets out to measure. There are three main types of validity – content, construct and criterion validity.

• Content validity: Content validity examines whether the instrument adequately covers all aspects of the content that it should with respect to the variable under investigation. 56 This type of validity can be assessed through expert judgment and by examining the coverage of items or questions in measure. 56 Face validity is a subset of content validity in which experts are consulted to determine if a measurement tool accurately captures what it is supposed to measure. 56 There are multiple methods for testing content validity – content validity index (CVI) and content validity ratio (CVR). CVI is calculated as the number of experts giving a rating of “very relevant” for each item divided by the total number of experts. Values range from 0 to 1, with items having a CVI score > 0.79 relevant; between 0.70 and 0.79, the item needs revisions, and if the value is below 0.70, the item is eliminated. 57 CVR varies between 1 and −1; a higher score indicates greater agreement among panel members. CVR is calculated as  (Ne – N/2)/(N/2), where Ne is the number of panellists indicating an item as “essential” and N is the total number of panelists. 57 A study by Mousazadeh et al. 2017 investigated the content, face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. 58 To ensure face validity, the questionnaire was given to 25 female adolescents, a psychologist and three nurses, who were required to evaluate the items with respect to problems, ambiguity, relativity, proper terms and grammar, and understandability. For content validity, 15 experts in psychology and nursing were asked to assess the qualitative content validity. To determine the quantitative content validity, the content validity index and content validity ratio were calculated. 58
• Construct validity: A construct is an idea or theoretical concept based on empirical observations that are not directly measurable. An example of a construct could be physical functioning or social anxiety. Thus construct validity determines whether an instrument measures the underlying construct of interest and discriminates it from other related constructs. 55 It is important and expresses the confidence that a particular construct is valid. 55 This type of validity can be assessed using factor analysis or other statistical techniques. For example, Pinar, Rukiye 2005 , evaluated the reliability and construct validity of the SF-36 in Turkish cancer patients. 59 The SF-36 is widely used to measure the quality of life or health status in sick and healthy populations. Principal components factor analysis with varimax rotation confirmed the presence of the seven domains in the SF-36: in the SF-36: physical functioning, role limitations due to physical and emotional problems, mental health, general health perception, bodily pain, social functioning, and vitality. It was concluded that the Turkish version of the SF-36 was a suitable instrument that could be employed in cancer research in Turkey. 59
• Criterion validity: Criterion validity is the relationship between an instrument score and some external criterion. This criterion is considered the “gold standard” and has to be a widely accepted measure that shares the same characteristics as the assessment tool. 55 Determining the validity of a new diagnostic test requires two principal factors – sensitivity and specificity. 60   Sensitivity refers to the probability of detecting those with the disease, while specificity refers to the probability of the test correctly identifying those without the disease. 60 For example, the reverse transcriptase polymerase chain reaction (RT PCR) is the gold standard for testing COVID-19; its results are available at the earliest several hours to days after testing. Rapid antigen tests are diagnostic tools that can be used at the point of care, and the results can be obtained within 30 minutes). 61, 62 Therefore, the validity of these rapid antigen tests was determined against the gold standard. 61, 62 Two published articles that assessed the validity of the rapid antigen test reported sensitivity of 71.43% and 78.3% and specificity of 99.68% and 99.5%, respectively. 61, 62 Thus indicating that the tests were less effective in identifying those who have the disease but highly effective in identifying those who do not have the disease. While it is important to assess the accuracy of the instruments used, it is also imperative to determine if the measure and findings are reliable.

Reliability

Reliability refers to the consistency of a measure. It is the ability of a measure or tests to reproduce a consistent result over time and across different observers. 55 A reliable measurement tool produces consistent results, even when different observers administer the test or when the test is conducted on different occasions. 55, 5 6 Reliability can be assessed by examining test-retest reliability, inter-rater reliability, and internal consistency.

• Test-retest reliability: Test-retest reliability refers to the degree of consistency between the outcomes of the same test or measure taken by the same participants at varying times. It estimates the consistency of measurement repetition. The intraclass correlation coefficient (ICC) is often used to determine test-retest reliability. 56 For example, a study may be conducted to evaluate the reliability of a new tool for measuring pain and might administer the tool to a group of patients at two different time points and compare the results. If the results are consistent across the two-time points, this would indicate that the tool has good test-retest reliability. However, it is important to note that the reliability reduces when the time between administration of the test is extended or too long. An adequate time span between tests should range from 10 to 14 days. 56 The article by Pinar, Rukiye 2005 , demonstrated this by assessing a test–retest stability using intraclass correlation coefficient-ICC. The retest procedure was conducted two weeks after the first test as two weeks was considered to be the optimum re-test interval. 59 This would be sufficiently long for participants to forget their initial responses but not too long that most health domains would change. 59
• Inter-observer (between observers) reliability: is also known as i nter-rater reliability, and it is the level of agreement between two or more observers on the results of an instrument or test. It is the most popular method of determining if two things are equivalent. 55, 56 For example, a study may be conducted to evaluate the reliability of a new tool for measuring depression. This will involve two different raters or observers independently scoring the same patient on the tool and comparing the results. If the results are consistent across the two raters, this would indicate that the tool has excellent inter-rater reliability. The Kappa coefficient is a measure used to assess the agreement between the raters. 56 It can have a maximum value of 1.00; the higher the value, the greater the concordance between the raters. 56
• Internal consistency: Internal consistency refers to the extent to which different items or questions in a test or questionnaire are consistent with one another. It is also known as homogeneity, which indicates whether each component of an instrument measures the same characteristics. 55 This type of reliability can be assessed by calculating Cronbach’s alpha (α) coefficient, which measures the correlation between different items or questions. Cronbach α is expressed as a number between 0 and 1, and a reliability score of 0.7 or above is considered acceptable. 55 For example, Pinar, Rukiye 2005 reported that reliability evaluations of the SF-36 were based on the internal consistency test (Cronbach’s α coefficient). The results showed that Cronbach’s α coefficient for the eight subscales of the SF-36 ranged between 0.79 and 0.90, confirming the internal consistency of the subscales. 59

Now you have an understanding of the quantitative methodology. Use the Padlet below to write a research question that can be answered quantitatively.

An Introduction to Research Methods for Undergraduate Health Profession Students Copyright © 2023 by Faith Alele and Bunmi Malau-Aduli is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Rigour in quantitative research

Affiliation.

• 1 Anglia Ruskin University, Chelmsford, England.
• PMID: 26198528
• DOI: 10.7748/ns.29.47.43.e8820

This article which forms part of the research series addresses scientific rigour in quantitative research. It explores the basis and use of quantitative research and the nature of scientific rigour. It examines how the reader may determine whether quantitative research results are accurate, the questions that should be asked to determine accuracy and the checklists that may be used in this process. Quantitative research has advantages in nursing, since it can provide numerical data to help answer questions encountered in everyday practice.

Keywords: Diagnostic studies; observational studies; quantitative research; randomised controlled trials; rigour; systematic reviews; validity.

Publication types

• Biomedical Research / methods*
• Biomedical Research / standards*
• Biomedical Research / trends

• Cancer Nursing Practice
• Emergency Nurse
• Evidence-Based Nursing
• Learning Disability Practice
• Mental Health Practice
• Nurse Researcher
• Nursing Children and Young People
• Nursing Management
• Nursing Older People
• Nursing Standard
• Primary Health Care
• RCN Nursing Awards
• Nursing Live
• Nursing Careers and Job Fairs
• CPD webinars on-demand
• --> Advanced -->

• Clinical articles
• CPD articles
• CPD Quizzes
• Expert advice
• Clinical placements
• Study skills
• Clinical skills
• University life
• Person-centred care
• Career advice
• Revalidation

Art & Science Previous     Next

Rigour in quantitative research, leica sarah claydon senior lecturer, allied and public health, anglia ruskin university, chelmsford, england.

This article which forms part of the research series addresses scientific rigour in quantitative research. It explores the basis and use of quantitative research and the nature of scientific rigour. It examines how the reader may determine whether quantitative research results are accurate, the questions that should be asked to determine accuracy and the checklists that may be used in this process. Quantitative research has advantages in nursing, since it can provide numerical data to help answer questions encountered in everyday practice.

Nursing Standard . 29, 47, 43-48. doi: 10.7748/ns.29.47.43.e8820

[email protected]

All articles are subject to external double-blind peer review and checked for plagiarism using automated software.

Received: 30 January 2014

Accepted: 12 June 2014

Diagnostic studies - observational studies - quantitative research - randomised controlled trials - rigour - systematic reviews - validity

User not found

Want to read more?

Already have access log in, 3-month trial offer for £5.25/month.

• Unlimited access to all 10 RCNi Journals
• RCNi Learning featuring over 175 modules to easily earn CPD time
• NMC-compliant RCNi Revalidation Portfolio to stay on track with your progress
• Personalised newsletters tailored to your interests
• A customisable dashboard with over 200 topics

Alternatively, you can purchase access to this article for the next seven days. Buy now

Are you a student? Our student subscription has content especially for you. Find out more

22 July 2015 / Vol 29 issue 47

TABLE OF CONTENTS

DIGITAL EDITION

• LATEST ISSUE
• SIGN UP FOR E-ALERT
• WRITE FOR US
• PERMISSIONS

Share article: Rigour in quantitative research

We use cookies on this site to enhance your user experience.

By clicking any link on this page you are giving your consent for us to set cookies.

• Account Logins

What We Offer

With a comprehensive suite of qualitative and quantitative capabilities and 55 years of experience in the industry, Sago powers insights through adaptive solutions.

• Recruitment
• Communities
• Methodify® Automated research
• QualBoard® Digital Discussions
• QualMeeting® Digital Interviews
• Global Qualitative
• Global Quantitative
• In-Person Facilities
• Healthcare Solutions
• Research Consulting
• Europe Solutions
• Neuromarketing Tools
• Trial & Jury Consulting

Who We Serve

Form deeper customer connections and make the process of answering your business questions easier. Sago delivers unparalleled access to the audiences you need through adaptive solutions and a consultative approach.

• Consumer Packaged Goods
• Financial Services
• Media Technology
• Medical Device Manufacturing
• Marketing Research

With a 55-year legacy of impact, Sago has proven we have what it takes to be a long-standing industry leader and partner. We continually advance our range of expertise to provide our clients with the highest level of confidence.​

• Global Offices
• Partnerships & Certifications
• News & Media
• Researcher Events

Sago Secures Spot in the 2024 GRIT Business & Innovation Report

Sago Announces Leadership Transition: Schlesinger Resumes CEO Role

Efficient Qualitative Analysis with Query AI

Drop into your new favorite insights rabbit hole and explore content created by the leading minds in market research.

• Case Studies
• Knowledge Kit

Capturing Qual Insights in Real-Time: Analyzing Debate Responses with QualBoard

The Deciders, October: 2020 Non-Voters in Swing States

• Partner with us
• Join our panel

The Significance of Validity and Reliability in Quantitative Research

• Resources , Blog

Key Takeaways:

• Types of validity to consider during quantitative research include internal, external, construct, and statistical
• Types of reliability that apply to quantitative research include test re-test, inter-rater, internal consistency, and parallel forms
• There are numerous challenges to achieving validity and reliability in quantitative research, but the right techniques can help overcome them

Quantitative research is used to investigate and analyze data to draw meaningful conclusions. Validity and reliability are two critical concepts in quantitative analysis that ensure the accuracy and consistency of the research results. Validity refers to the extent to which the research measures what it intends to measure, while reliability refers to the consistency and reproducibility of the research results over time. Ensuring validity and reliability is crucial in conducting high-quality research, as it increases confidence in the findings and conclusions drawn from the data.

This article aims to provide an in-depth analysis of the significance of validity and reliability in quantitative research. It will explore the different types of validity and reliability, their interrelationships, and the associated challenges and limitations.

In this Article:

The role of validity in quantitative research, the role of reliability in quantitative research, validity and reliability: how they differ and interrelate, challenges and limitations of ensuring validity and reliability, overcoming challenges and limitations to achieve validity and reliability, explore trusted quantitative solutions.

Take the guesswork out of your quant research with solutions that put validity and reliability first. Discover Sago’s quantitative solutions.

Request a consultation

Validity is crucial in maintaining the credibility and reliability of quantitative research outcomes. Therefore, it is critical to establish that the variables being measured in a study align with the research objectives and accurately reflect the phenomenon being investigated.

Several types of validity apply to various study designs; let’s take a deeper look at each one below:

Internal validity is concerned with the extent to which a study establishes a causal relationship between the independent and dependent variables. In other words, internal validity determines whether the changes observed in the conditional variable result from changes in the independent variable or some other factor.

External validity refers to the degree to which the findings of a study can be generalized to other populations and contexts. External validity helps ensure the results of a study are not limited to the specific people or context in which the study was conducted.

Construct validity refers to the degree to which a research study accurately measures the theoretical construct it intends to measure. Construct validity helps provide alignment between the study’s measures and the theoretical concept it aims to investigate.

Finally, statistical validity refers to the accuracy of the statistical tests used to analyze the data. Establishing statistical validity provides confidence that the conclusions drawn from the data are reliable and accurate.

To safeguard the validity of a study, researchers must carefully design their research methodology, select appropriate measures, and control for extraneous variables that may impact the results. Validity is especially crucial in fields such as medicine, where inaccurate research findings can have severe consequences for patients and healthcare practices.

Ensuring the consistency and reproducibility of research outcomes over time is crucial in quantitative research, and this is where the concept of reliability comes into play. Reliability is vital to building trust in the research findings and their ability to be replicated in diverse contexts.

Similar to validity, multiple types of reliability are pertinent to different research designs. Let’s take a closer look at each of these types of reliability below:

Test-retest reliability refers to the consistency of the results obtained when the same test is administered to the same group of participants at different times. This type of reliability is essential when researchers need to administer the same test multiple times to assess changes in behavior or attitudes over time.

Inter-rater reliability refers to the results’ consistency when different raters or observers monitor the same behavior or phenomenon. This type of reliability is vital when researchers are required to rely on different individuals to rate or observe the same behavior or phenomenon.

Internal consistency reliability refers to the degree to which the items or questions in a test or questionnaire measure the same construct. This type of reliability is important in studies where researchers use multiple items or questions to assess a particular construct, such as knowledge or quality of life.

Lastly, parallel forms reliability refers to the consistency of the results obtained when two different versions of the same test are administered to the same group of participants. This type of reliability is important when researchers administer different versions of the same test to assess the consistency of the results.

Reliability in research is like the accuracy and consistency of a medical test. Just as a reliable medical test produces consistent and accurate results that physicians can trust to make informed decisions about patient care, a highly reliable study produces consistent and precise findings that researchers can trust to make knowledgeable conclusions about a particular phenomenon. To ensure reliability in a study, researchers must carefully select appropriate measures and establish protocols for administering the measures consistently. They must also take steps to control for extraneous variables that may impact the results.

Validity and reliability are two critical concepts in quantitative research that significantly determine the quality of research studies. While both terms are often used interchangeably, they refer to different aspects of research. Validity is the extent to which a research study measures what it claims to measure without being affected by extraneous factors or bias. In contrast, reliability is the degree to which the research results are consistent and stable over time and across different samples , methods, and evaluators.

Designing a research study that is both valid and reliable is essential for producing high-quality and trustworthy research findings. Finding this balance requires significant expertise, skill, and attention to detail. Ultimately, the goal is to produce research findings that are valid and reliable but also impactful and influential for the organization requesting them. Achieving this level of excellence requires a deep understanding of the nuances and complexities of research methodology and a commitment to excellence and rigor in all aspects of the research process.

Ensuring validity and reliability in quantitative research is not without its challenges. Some of the factors to consider include:

1. Measuring Complex Constructs or Variables One of the main challenges is the difficulty in accurately measuring complex constructs or variables. For instance, measuring constructs such as intelligence or personality can be complicated due to their multi-dimensional nature, and it can be challenging to capture all aspects accurately.

2. Limitations of Data Collection Instruments In addition, the measures or instruments used to collect data can also be limited in their sensitivity or specificity. This can impact the study’s validity and reliability, as accurate and precise measures can lead to incorrect conclusions and unreliable results. For example, a scale that measures depression but does not include all relevant symptoms may not accurately capture the construct being studied.

3. Sources of Error and Bias in Data Collection The data collection process itself can introduce sources of error or bias, which can impact the validity and reliability of the study. For instance, measurement errors can occur due to the limitations of the measuring instrument or human error during data collection. In addition, response bias can arise when participants provide socially desirable answers, while sampling bias can occur when the sample is not representative of the studied population.

4. The Complexity of Achieving Meaningful and Accurate Research Findings There are also some limitations to validity and reliability in research studies. For example, achieving internal validity by controlling for extraneous variables may only sometimes ensure external validity or the ability to generalize findings to other populations or settings. This can be a limitation for researchers who wish to apply their findings to a larger population or different contexts.

Additionally, while reliability is essential for producing consistent and reproducible results, it does not guarantee the accuracy or truth of the findings. This means that even if a study has reliable results, it may still need to be revised in terms of accuracy. These limitations remind us that research is a complex process, and achieving validity and reliability is just one part of the giant puzzle of producing accurate and meaningful research.

Researchers can adopt various measures and techniques to overcome the challenges and limitations in ensuring validity and reliability in research studies.

One such approach is to use multiple measures or instruments to assess the same construct. In addition, various steps can help identify commonalities and differences across measures, thereby providing a more comprehensive understanding of the construct being studied.

Inter-rater reliability checks can also be conducted to ensure different raters or observers consistently interpret and rate the same data. This can reduce measurement errors and improve the reliability of the results. Additionally, data-cleaning techniques can be used to identify and remove any outliers or errors in the data.

Finally, researchers can use appropriate statistical methods to assess the validity and reliability of their measures. For example, factor analysis identifies the underlying factors contributing to the construct being studied, while test-retest reliability helps evaluate the consistency of results over time. By adopting these measures and techniques, researchers can crease t their findings’ overall quality and usefulness.

The backbone of any quantitative research lies in the validity and reliability of the data collected. These factors ensure the data accurately reflects the intended research objectives and is consistent and reproducible. By carefully balancing the interrelationship between validity and reliability and using appropriate techniques to overcome challenges, researchers protect the credibility and impact of their work. This is essential in producing high-quality research that can withstand scrutiny and drive progress.

Are you seeking a reliable and valid way to collect, analyze, and report your quantitative data? Sago’s comprehensive quantitative solutions provide you the peace of mind to conduct research and draw meaningful conclusions.

Don’t Settle for Subpar Results

Work with a trusted quantitative research partner to deliver quantitative research you can count on. Book a consultation with our team to get started.

Five Essential Steps for Data Integrity

Q&A: Uncovering the Market Research Reality

Swing Voter Project, October 2024: Pennsylvania

5 Steps for Embracing DEIB in Market Research

Real Insights in Real-Time: A New Era in Qualitative Research with AI

The Deciders, September 2024: Young Male Voters

Swing Voter Project Georgia: September 2024

Take a deep dive into your favorite market research topics

How can we help support you and your research needs?

BEFORE YOU GO

Have you considered how to harness AI in your research process? Check out our on-demand webinar for everything you need to know