Introduction

The nature of qualitative research is such that the data collected from human subjects can be deemed subjective. In order for the data collected, analyzed, and reported to be considered trustworthy, valid and comprehensive, the researcher must ensure that there are measures of validity and reliability throughout the research process. These include prolonged engagement, triangulation, structured codebooks, purposeful sampling, negative case analysis, peer debriefing, refutation analysis and cross checking, establishing procedures for recording field notes, and using computer software.

Ensuring Validity

Validity refers to a measure of trustworthiness which must measure how accurately a method, instrument, framework, or technique measures the phenomenon being studied.[1]

1. Triangulation – there are 4 specific strategies for triangulation that can be used to show validity in qualitative research.
   
   1. Theory triangulation – the use of multiple theories, frameworks, or concepts when examining a situation or phenomenon.
   2. Data triangulation – the use of multiple data sources in a study.
   3. Methods triangulation – the use of various methods for data collection in a study.
   4. Investigator triangulation – the participation of two or more researchers in the same study to ensure that there is consistency in the observations and conclusions.

2. Prolonged Engagement – Qualitative researchers should spend prolonged periods of time, often 12 to 18 months, in the field conducting research to ensure that the effects of having a researcher within the study setting are mitigated to the best of the researcher’s ability. This also allows the researcher to check their own biases and perceptions of the phenomenon, increasing reflexivity. The ability to show that a sufficient amount of time was spent in the field collecting data is necessary in ensuring validity.

3. Structured Codebooks – Using a codebook that is properly sorted and organized with meaningful codes and linked categories is important for increasing validity. Beginning with a codebook and developing it as the transcripts are coded and analyzed helps achieve a greater construct validity. Checking interrater reliability when more than two researchers are coding ensures that the codes in the codebook are being interpreted similarly by the researcher, reduces researcher bias, and increases validity.

4. Purposeful Sampling – Researchers should be clear about why and how they select the samples for their population. Ensuring that enough respondents are involved in the study and that the participants chosen provide a comprehensive understanding of the phenomenon is critical for reproduction of findings.

5. Negative Case Analysis – Researchers must examine the analysis until it accounts for all of the findings in all of the cases, looking for contrasting cases, and increasing the number of cases if there are unresolved concerns.

6. Peer Debriefing – Having peers serve as a guide and provide feedback and questions that would be helpful to the study. Peer feedback on interpretations and concerns helps detach researchers from the field and indicates that the researcher acted on critical reflections and feedback.

Ensuring Reliability

Reliability refers to extent to which research data, analysis, and findings have the same or similar results for multiple participants.[2]

1. Refutation Analysis/Cross-Checking – Exploring the contradictions and differences between the individual cases and using multiple team members to confirm observations, transcriptions, and interpretations help build reliability.

2. Use of Computer Software – The use of computer software designed to assist with organization of data, data analysis, and data synthesis are helpful for managing qualitative data and provide researchers the ability to develop rigorous plans for data coding and analysis while also allowing for collaborative work which can increase reliability.

3. Establish Recording Procedures for Field Notes – Researchers can establish reliability by ensuring a strict procedure for taking field notes when collecting data. Having separate categories of field notes for verbatim accounts, expanded accounts of events that are written up immediately after field sessions, reflective notes on ideas, emotions, concerns, etc., and a running record for analysis and interpretations that are performed while in the field.[3]

How to Assess Reliability

There are multiple steps that a researcher can take to assess reliability in qualitative research:[4]

1. Determine whether the research questions are clear and that all elements of the research design are fitting for the research questions
2. Describe the researcher’s role and status within the research setting
3. Clearly specify all theoretical frameworks, paradigms, and analytical constructs
4. Assert that findings show meaningful parallels across the literature
5. Ensure that data are collected in the proper settings, on time, and with the correct participants
6. Maintain that all data collection field sites have similar data collection protocols
7. Conduct coding checks using inter-rater reliability to show adequate agreement in analysis
8. Provide data quality checks if appropriate
9. Develop a system of peer or colleague review

Conclusion

In summary, qualitative research must have reliability and validity checks to ensure that the researcher is providing a trustworthy and comprehensive account of the data. The use of multiple theories, frameworks, or concepts, multiple data sources, multiple methods, or multiple researchers in the same study when examining a situation or phenomenon is one method of ensuring validity. Qualitative researchers spending prolonged periods of time in the field conducting research to ensure that the effects of having a researcher within the study setting are mitigated to the best of the researcher’s ability. Using a codebook that is properly sorted and organized with meaningful codes and linked categories is important for increasing validity. Being clear about why and how they select the samples for their population and ensuring that enough respondents are involved in the study and that the participants chosen provide a comprehensive understanding of the phenomenon is critical for reproduction of findings. Examining the analysis until it accounts for all of the findings in all of the cases, looking for contrasting cases, and increasing the number of cases if there are unresolved concerns is another way to ensure validity. Peer feedback on interpretations and concerns helps detach researchers from the field and indicates that the researcher acted on critical reflections and feedback. Researchers can establish reliability by ensuring a strict procedure for taking field notes when collecting data. Exploring the contradictions and differences between the individual cases and using multiple team members to confirm observations, transcriptions, and interpretations also helps build reliability.

Take Away

Validity and reliability are key aspects of qualitative research design and methods to ensure these aspects should be considered in all stages of the research design process.

[1] Golafshani, N. (2003). Understanding reliability and validity in qualitative research. The qualitative report, 8(4), 597-607.

[2] Franklin, C., & Ballan, M. (2001). Reliability and validity in qualitative research. The handbook of social work research methods, 4(273-292).

[3] Coleman, P. (2022). Validity and reliability within qualitative research for the caring sciences. International Journal of Caring Sciences, 14(3), 2041-2045.

[4] Brink, H. I. (1993). Validity and reliability in qualitative research. Curationis, 16(2), 35-38.