Research Validity

Validity is one of the important measures of the quality of a study. It refers to the degree to which the results obtained by the study are accurate or truthful. There are four main types of validity: (1) statistical conclusion validity, (2) construct validity, (3) internal validity, and (4) external validity.

Validity 12-9

  1. Statistical Conclusion Validity

This refers to the degree to which the conclusions that the study makes regarding the relationship between the study variables are accurate and reasonable. Two types of errors can compromise the statistical conclusion validity of the study. The first (type1 error) entails findings a statistical relationship between variable where none exists. The second (type II error) entails failing to establish a relationship between variables where one exists. The following are some of the factors that may threaten the statistical validity of the study:

a. Inadequate sample: Some relationships requires large number of cases to observe. Having small samples reduces the statistical power of the study to deduce such relationships leading to type II error. This threat can be minimized by defining the effect size of the relationship that you would like to study when designing your sampling plan.

b. Wrong alpha value: Relationship between variables is often tested by comparing its p-value to an alpha value that is arbitrarily set by the researcher. Setting a low p-value may lead to type 1 error while setting a high p-value may lead to type II error. In most academic studies, the alpha value is set at 0.05.

c. Violating assumptions of statistical tests: Each statistical test has its assumptions. For instance, parametric test such as t test, F test, and ANOVA assumes that the data on the dependent variable is normally distributed. Using these tests with data that does not satisfy the assumption leads to erroneous results. This threat can be minimized by conducting exploratory test before the primary analysis of data.

  1. Construct Validity

This is the extent to which the operationalization (instrumentation) used in a given study accurately and adequately captures that construct that need to be measured. For instance, if the study seeks to examine students’ knowledge in Geometry, the researcher must design a test that evaluates all aspects of geometry. The test must also be administered in a way that will bring out an accurate verdict regarding the student’s knowledge: Common threats to construct validity include:

a. Inadequate operationalization: This threat occurs when the operationalization does not capture the construct that study intended to investigate. In the Geometry example, Construct validity can be lost if the test does not accurately represent the geometry concept. Similarly, construct validity can be diminished if the test does not cover all the sub-topics of Geometry.

b. Social threats to construct validity: Validity is not determined by the instrument only, but also by how the instrument is administered. For instance, if the test is not administered in a good environment, students may fail, not because they lack knowledge of the subject, but because of anxiety. In a research context, this threat can be minimized by guaranteeing privacy, confidentiality, and anonymity.

  1. Internal validity

This is the extent to which the study can accurately conclude that the relationship observed between the independent and dependent variable are of a causal nature. To realize this, the researcher must prove that (1) a statistically significant relationship exists between the variables, (2) the variation in the independent variable (cause) comes before the variation in the dependent variable (effect), and (3) there is no other plausible explanation for the change observed in the dependent variable. Common threats to internal validity include: history, maturation, self-selection, and differential attrition of participants.

a. History: This refers to events that occur during the course of the study that have an impact on the dependent variable. For instance, increasing the compensation package of employees during a study that seeks to examine the effect of a training program on employees’ job performance may affect the internal validity of the study. This is because the increase in compensation package also has the potential of enhancing job performance; hence, the researcher will not be able to determine whether the increase in performance was due to the training program or the pay increment. This threat can be minimized by constantly watching out for historical events.

b. Maturation: This refers to changes that occur to the dependent variable due to normal developmental processes. For instance, in the study described in (a) above, employee job performance may improve overtime due to staff becoming more experienced and not necessarily as result of the training program. This threat can be minimized by ensuring that the data on the dependent variable is taken within a short time after the training program is completed. The researcher should also avoid developing a training program that requires a lengthy period to implement.

c. Self-selection: This occurs when participants are not assigned into the intervention and control groups using random methods. Self-selection also occurs in observational studies when participants are allowed to decide entirely for themselves whether to participate in the study or not. Self-selection reduces internal validity by creating biased sample. For instance, a study examining the sexual-behavior of individuals in a given population may end up with a sample that does not reflect the true characteristic of the population if it allows participants to decide whether to participate or not. This is because most people with sexual behaviors that deviate from societal norms are likely to opt out. This bias can be minimized by guaranteeing the highest level of anonymity, privacy, and confidentiality to potential participants.

d. Differential attrition of participants: This threat occurs when participants with certain characteristics exit the study is large numbers. For instance, a study examining the effect of a new method of teaching biology may not yield valid results if more students who are poor in this subject drop out of the study than high performing students.

  1. External validity

This is the extent to which the results of the study can be generalized to other people, places, settings, and time. Due to resource constraints, studies are often done using a small section of the population with the hope that the findings realize will apply to the entire population. There are several factors that threaten the external validity of the study. These factors include: bias in the sample selection process, attrition bias, and conducting the study is peculiar settings.

References

Christensen, L., Johnson, B., & Turner, L. (2014). Research Methods, Design, and Analysis, 12th Edition. Upper Saddle River, NJ: Pearson Education.

Hanneman, R., Kposowa, A., & Riddle, M. (2013). Basic Statistics for Social Research. San Francisco, CA; Jossey-Bass.

Vogt, P. (2007). Quantitative Research Methods for Professionals in Education and Other Fields. Boston, MA; Allyn & Bacon.

 

 

Advertisements