Provided by James R. Martin, Ph.D., CMA
Professor Emeritus, University of South Florida
Research Methods Main Page
The concept of validity appears to be a simple idea until one looks at the literature on the subject. The purpose of this section is to summarize the approaches and terminology used by various authors when discussing validity.
The following material is from my summary of Buckley, J. W., M. H. Buckley and H. Chiang. 1976. Research Methodology & Business Decisions. National Association of Accountants. (Summary).
Experimental Design and Control Criteria
Internal validity - Relates to the following questions:
1. Is the research design appropriate?
2. Does the design include all the important factors and relationships?
3. Have the independent variables been controlled to prevent bias?
4. Has randomization been used when possible to reduce systematic bias?
External validity - Relates to the problem of generalization. Can the findings be applied to other cases on a broad basis?
Broader Framework for Selecting and Evaluating Methodology
The authors say validity has been stressed to the exclusion of other important criteria and propose a broader framework.
1. Holism - ability to comprehend the problem in totality.
2. Precision - degree of refinement of a problem for objective measurement.
3. Internal validity - ability to control the variables in the problem.
4. External validity - extent to which the findings are applicable to other similar situations.
5. Quantitative - ability to define or analyze problems in quantitative terms.
6. Qualitative - ability to define and or analyze the problem in qualitative terms.
7. Relevance - value of the findings to significant areas of need and application.
8. Skill transfer - extent to which it is difficult to train and supervise research assistants.
Birnberg, J. G., M. D. Shields and S. M. Young. 1990. The case for multiple methods in empirical management accounting research (With an Illustration from Budget Setting). Journal of Management Accounting Research (Fall): 33-66.
These authors indicate that there are two key concepts of validity, internal and external, but add two other concepts. These concepts include:
1. Statistical conclusion validity - This concept refers to the correlation between the independent and dependent variables, x and y.
2. Construct validity - Relates to whether an operational definition of a construct is a valid measure of the construct.
3. External validity - Refers to whether the relationships in a study can be generalized to a broader array of groups, settings and times.
4. Internal validity - Whether changes in the dependent variable were caused by changes in the independent variable or variables. Whether unambiguous conclusions can be drawn.
The following are some notes from Dipboye, R. L., C. S. Smith and W. C. Howell. 1994. Understanding Industrial and Organizational Psychology: An Integrated Approach. Harcourt Brace College Publishers.
These authors discuss reliability and validity in relation to measurements.
Reliability refers to the extent that a measure is relatively free of random error and is consistent in the numbers assigned to objects or events.
Validity refers to whether the number obtained truly reflects what the user intended to measure. Validity requires reliability. According to these authors, validity is not a property of the measure, but instead refers to the truthfulness of the inferences that are drawn from the measure.
They discuss the Trinitarian model of validity and the Unitary concept indicating that the Unitary concept is more appropriate.
The Trinitarian Model of Validity
This model includes three concepts:
1. Criterion related validity - Does the measure allow correct predictions?
2. Content validity - Does the measure provide a good sample of the domain of behaviors that it is intended to measure?
3. Construct validity - Does it measure what it is supposed to measure?
Construct validity requires three steps including:
1. Define the construct.
2. Examine the relationships among observable variables that are specified as part of the construct.
3. Examine the relationships with different constructs.
The Unitary Concept of Validity
The unitary concept of validity stresses that validity is a unitary concept and should not be subdivided.
_______________________________________________
Related summaries:
Birnberg, J. G., M. D. Shields and S. M. Young. 1990.The case for multiple methods in empirical management accounting research (With an Illustration from Budget Setting). Journal of Management Accounting Research (2): 33-66. (Summary).
Buckley, J. W., M. H. Buckley and H. Chiang. 1976. Research Methodology & Business Decisions. National Association of Accountants. (Summary).
Christensen, C. M. and M. E. Raynor. 2003. Why hard-nosed executives should care about management theory. Harvard Business Review (September): 67-74. (Summary).
Miller, E. L. 1992. Questions That Matter: An Invitation To Philosophy, Third Edition. McGraw-Hill, Inc. (See Chapter 2: A Little Logic)