Reliability and Validity Academic Research

Reliability and Validity

Inter-rater reliability

Reliability and Validity in Research – This is a statistical concept in the field of research whereby a particular phenomenon is being evaluated or rated by various raters. It is, therefore, the extent or degree to which there is an agreement in the rating scores amongst the multiple raters, which brings about homogeneity and unanimity amongst these raters. To measure inter-rater reliability, it entails taking the total number of ratings in a particular judgment conducted, as well as the counting the accumulative ratings are done in the rating exercise. The total number of agreements is then divided by the total number of ratings and converted into a percentage to give the inter-rater reliability. McHugh (2012) provides a good example of how inter-rater reliability is calculated by reviewing the various methods that have been stipulated by scholars previously.

Test-retest reliability

This is also another reliability aspect. Test-retest reliability is the extent or degree to which results obtained from a particular test (which is similar) and consistent over time. In test-retest reliability, a similar test is administered to the same people in two or more instances and then the results are evaluated. To measure the test-retest reliability, there are two primary formulas applied. The first formula, which is better applied in instances where two tests were conducted in the Pearson Correlation formula that tests how well two sets of data correlate.

The other method is intraclass correlation formula that is applicable where more than two tests were administered. These formulas help calculate the test-retest coefficients that range between 0 and 1. In his article on validity and reliability in social science research, Drost (2011) provides the various reliability and validity aspects and gives detailed examples of the test-retest reliability measurement.

Face validity

Face validity, which is also referred to as the logical validity, entails the extent or degree to which an evaluation or investigation intuitively seems to quantify or measure the variable or rather the theory that it is objectively meant to measure. This, therefore, means that face validity is when a specific evaluation or assessment tool does what it is meant to do to provide results. To measure face validity, one can engage in the assessment of the concepts or ideas to be measured against the theoretical and practical applications.

Predictive validity

This is the measure of how accurate or effective a given value from a research study is and can be used in the future or rather to predict future patterns in the field studied. In their research on the predictive validity of public examinations (Obioma & Salau, 2007) use the predictive validity aspect to predict how the performance of students in public examinations will affect their future academic performances in the university and college level.

Concurrent reliability and validity

This entails the degree to which current test results relate to results from a previous test. For instance, if in the measurement of an individual’s IQ test are taken at two varied intervals, concurrent validity is measured through comparing on how closely similar are these results from the two tests. A good example of research that has employed the use of concurrent validity is the research done by (Tamanini et al., 2004) on the Portuguese king’s health test performed on women after stress. The researchers indicate how this test is applied and measured by using it as their primary test in their research.

Addressing the issues of reliability and validity

On most qualitative researchers, the nature of the data is more important to the researcher than the other descriptive elements of the research. This, however, does not rule out the need for conciseness in the descriptive sections. Reliability in research entails the concerns the stability, consistency of the data as well as homogeneous repeatability of the results if several tests are done (LoBiondo-Wood & Haber 2014). On the other hand, validity entails the accuracy and integrity of the data or results collected from the various tests that a researcher performs. Various researchers address these issues of validity and reliability in different ways, based on the purpose and the kind of research they carry out.

The authors, Obioma & Salau, (2007), go down to research on the effects of public examinations on the future academic performance of students. The focus here, therefore, is more on the data validation to ensure that their conclusions, as well as the outcomes of the results, have the required accuracy and integrity to validate their arguments. The two authors and researchers have applied the aspects of predictive and concurrent validity in their research. In regards to the use of predictive validity, this is where their research question is based on.

Reliability and Validity Research
Reliability and Validity in Research

They have made sure that the data or the arguments that they bring forth as substantially valid and convincing to attain the objective of predicting the future academic performances of the children who undertake the public examinations that are governed by the various bodies in the country. They have however not applied any reliability aspects in their research. At least not anyone that can be easily identified.

In the book by Drost, he has touched on both aspects; validity and reliability. In this book, he has not presented it in a research form but rather brought it out to the readers in the form of a review of both aspects of research, but on the dimension of social sciences. For instance, she has covered the various instances of both validity and reliability, by providing real-life examples and the various methods that can be used to measure the respective instances of both aspects. She approaches the concepts of validity and reliability from a general perspective whereby she accounts for the reasons as to why researchers, especially in education and social sciences, should adopt a culture of ensuring validity and reliability in their results. He explains the various instances of reliability and provides formulas and tools that can be effectively applied to measure these instances. She also provides the various elements that can impact the level of validity and reliability of data or results in research.

In conclusion, the concepts of validity and reliability are important in research. The researcher from various fields should adopt a culture of achieving these concepts in the results they obtain during their research. As Drost argues it, strong support for the validity and the reliability of research not only makes the research highly validated or otherwise believed in but also limits the possible critiques that the research may face. It fills the gaps that may be identifiable in the research. A researcher should be able to understand the various instances of both reliability and validity as well as know when it is appropriate to apply what instance in the research.

References

McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3), 276-282.

Drost, E. A. (2011). Validity and reliability of social science research. Education Research and perspectives, 38(1), 105.

Obioma, G., & Salau, M. (2007). The predictive validity of public examinations: A case study of Nigeria. Nigerian Educational Research & Development Council (NERDC) Abuja.

Tamanini, J. T., Dambros, M., D’ancona, C. A., Palma, P. C., Botega, N. J., Rios, L. A., & Netto Jr, N. R. (2004). Concurrent validity, internal consistency and responsiveness of the Portuguese version of the King’s Health Questionnaire (KHQ) in women after stress urinary incontinence surgery. International Braz j Urol, 30(6), 479-486.

LoBiondo-Wood, G., & Haber, J. (2014). Reliability and validity. G. LoBiondo-Wood & J. Haber. Nursing research. Methods and critical appraisal for evidence-based practice, 289-309.

Reliability and Validity Relevant Blog Posts

University Dissertation Topics

Approaches for Research Dissertations

If you enjoyed reading this post on reliability and validity, I would be very grateful if you could help spread this knowledge by emailing this post to a friend, or sharing it on Twitter or Facebook. Thank you.