Summary by Anita Reed
Ph.D. Program in Accounting
University of South Florida, Spring 2002
Budgeting Main Page | Research Methods Main Page
The authors are motivated by published recommendations of other researchers espousing the use of various research methods. The authors’ purpose is to articulate the usefulness of multiple research methods in order to compensate for the inherent weaknesses within each method and to provide a more complete understanding of research phenomena by examination from different perspectives. The choice of a specific research method is guided by the instant research question, the current state of knowledge regarding the phenomenon and the feasibility of conducting a particular method. The use of different methods at various stages of a research stream is viewed as a circular, iterative process (see Figure 1 on pg. 53), wherein the results of subsequent studies reinforce, augment and triangulate the findings from prior studies, or raise new questions to be addressed in subsequent studies. Two variations of the multiple method approach are suggested: 1) using more than one method when conducting a single study and 2) using differing methods to examine a phenomenon by conducting a coordinated series of studies.
Empirical research in management accounting is typically either basic research, conducted to describe, explain and predict management accounting phenomena or applied research, conducted to evaluate a particular design or implementation of an accounting system with the goal of policy recommendation or design improvement.
The authors have defined the following three types of empirical research method (p. 34):
Field research-occurs in natural settings that are not created for the sole or primary purpose of conducting research. Three primary types of field research are case studies, field studies and field experiments.
Experiment – involves the manipulation of the independent variables and the observation of their effects on the dependent variables. True experiments require random assignment of sampling units to treatments. Quasi-experiments do not require random assignment to treatment groups. Laboratory experiments are differentiated from field experiments by the nature of the setting, with a laboratory experiment conducted in an artificially created setting designed for research control while a field experiment is conducted in a natural setting designed for other purposes.
Survey – employs a standardized approach in order to collect information from sampling units to make inferences about the population. Typically conducted through the mail, telephone or personal interview.
The authors compare the three research methods using five dimensions, three derived from Runkel and McGrath (1972): actors, behaviors and contexts; and two additional dimensions: researcher and research application. The comparison illustrates the strengths and weaknesses inherent in each of the research methods. The comparison is summarized in Table 1 (p.36-37), reproduced below.
| TABLE 1: Comparison of Empirical Research Methods | |||
| Field Research | Laboratory Experiments | Surveys | |
| Research Application: |
|||
| Primary purpose | Exploration | Test (Causal) theories | Exploration |
| Secondary Purpose | Description | Test cross-sectional theories | |
| Validity: | |||
| Statistical Conclusion | Low | High | Medium |
| Internal | Low | High | Low to medium |
| Construct | Low to high | Low to High | Low to high |
| External | Low to high | Low to medium | Medium |
| Control | Low | High | Low to medium |
| Researcher: | |||
| Demand Effects | Low to high | Low to high | Low to high |
| Researcher Expectancy Effects |
Low to high | Low to high | Low to high |
| Actors: | |||
| Volunteers | Volunteers or draftees via a request from a superior | Volunteers or draftees from classes or business | Volunteers or draftees via a request |
| Incentive to Participate | Desire to help Request from a superior Result feedback |
Class requirement Class credit Financial compensation Performance based compensation |
Desire to help Request from another Result feedback Nominal fixed compensation |
| Sampling method | Convenience or random sample Typically small sample size |
Convenience or random sample Sample size varies |
Convenience or random sample Typically large sample size |
| Evaluation Apprehension | Low to high | Low to high | Low |
| Observing and Recording of Behavior: | |||
| Methods | Subjective self-report Trace Visible observer Hidden observer Archival record Public record |
Subjective self-report Visible observer Hidden observer |
Subjective self-report |
| Reactivity Effects | Low to high | Low to high | Low to high |
| Context: | |||
| Artificial vs. Natural | Complete Natural setting Incomplete Natural setting Natural setting disturbed by researcher |
Artificial Simulated firm or market |
Incomplete natural setting |
| Experimental Realism | Low to high | Low to high |
Low to high |
| Mundane Realism | High | Low to medium | Medium |
The four validity criteria are defined (p. 38-39):
Internal validity-assessment of whether the changes in the dependent variable are caused by changes in the independent variable. Statistical validity is a necessary condition for internal validity.
External validity-whether the observed causal relations in one study can be generalized to or across groups, settings and times. Construct validity and internal validity are necessary conditions for external validity.
Statistical conclusion validity-whether the independent and dependent variables covary, given the statistical evidence. A necessary condition for internal validity.
Construct validity-whether a particular operational definition of a construct is a valid measure of the construct. Assessed by measuring convergence of various measures and divergence of various measures.
The researcher may inadvertently introduce two behaviors in participants: demand effects, in which participants behave in a certain way because they believe it is required of them or expectancy effects, in which participants behave in certain ways because they believe it is what the researcher expects or wants them to do.
Some of the considerations related to Actors, or participants, are concerned with the usefulness of students as appropriate surrogates for other groups. Key considerations include the nature of the task and the ability of the surrogate to perform the task. The authors (p. 45) also point out two reasons why the representativeness of the sampling units may not be a factor (Lynch, 1982): 1) test of theory are stronger if there is small error variance, which can occur when a homogeneous sample is used and 2) if a theory is universal it should apply to any and all people who satisfy its boundary conditions.
The six methods of recording data are (p. 48):
Subjective reports – an actor is aware of being observed and recorded.
Trace – record of an actor’s behavior when the actor is not aware the record will be used for research purposes
Visible observer – actors know that a researcher is observing and recording their behavior
Hidden observer – actors are not aware that a researcher is observing and recording their behavior
Public records – written by someone other than the researcher, actors are aware that their behavior is being recorded
Archival records – written by someone other than the researcher, actors are not aware that their behavior is being observed and recorded or may be aware that the behavior is being observed and recorded but not have an expectation that the record will be used for research purposes.
Research context may vary significantly and is an important concept in its effect on the validity of research results. Two elements of context are defined (p. 49-50):
Experimental realism – increases when the research context engages or involves the participant fully in the research. The goal is to increase the probability that the research context captures the intended essence of the theoretical variables
Mundane realism – whether the research context attempts to include superficially relevant features of the "natural world".
The authors illustrate the usefulness of the multiple method approach by an examination of selective literature on budget setting, an important area of management accounting research. They summarize this illustration as follows (p. 57-58):
"The catalyst for the investigation of budget setting was non-accounting research on participative decision making…initial accounting studies were primarily done in the field and documented the nature of budget setting. Surveys followed that expanded the sample of firms and the situations observed. Laboratory studies were then used to test for causal relations among key variables identified by the other methods. Finally, field studies were used to reexamine some of the original issues, but with insights gained form both the laboratory and survey studies."
This demonstrates the circular nature of research and the way in which each research method is informed by and supports the other methods. The use of multiple methods contributed to the extension of the research stream.
The authors believe that significant changes in organizations will result in the near future and advocate the use of multiple research methods in the examination of the impact of these organizational changes on management accounting.
_____________________________________________
Related summaries:
Ahrens, T. and J. F. Dent. 1998. Accounting and organizations: Realizing the richness of field research. Journal of Management Accounting Research (10): 1-39. (Summary).
Boehm, V. R. 1980. Research in the "real world" - A conceptual model. Personnel Psychology (33): 495-504. (Comparison of academic and practitioner research models).
Buckley, J. W., M. H. Buckley and H. Chiang. 1976. Research Methodology & Business Decisions. National Association of Accountants. (Summary).
Chan, Y. L. and B. E. Lynn. 1991. Performance evaluation and the analytic hierarchy process. Journal of Management Accounting Research (3): 57-87. (Summary).
Chenhall, R. H. and K. Langfield-Smith. 1998. The relationship between strategic priorities, management techniques and management accounting: An empirical investigation using a systems approach. Accounting, Organizations and Society 23(3): 243-264. (Summary).
Covaleski, M. A., M. W. Dirsmith and S. Samuel. 1996. Managerial accounting research: The contributions of organizational and sociological theories. Journal of Management Accounting Research (8): 1-35. (Summary).
Foster, G. and S. M. Young. 1997. Frontiers of management accounting research. Journal of Management Accounting Research (9): 63-77. (Summary).
Hopwood, A. G. 2002. If only there were simple solutions, but there aren't: Some reflections on Zimmerman's critique of empirical management accounting research. The European Accounting Review 11(4): 777-785. (Summary).
Ittner, C. D. and D. F. Larcker. 2001. Assessing empirical research in managerial accounting: A value-based management perspective. Journal of Accounting and Economics (32): 349-410. (Summary).
Ittner, C. D. and D. F. Larcker. 2002. Empirical managerial accounting research: Are we just describing management consulting practice? The European Accounting Review 11(4): 787-794. (Summary).
Jonsson, S. 1998. Relate management accounting research to managerial work! Accounting, Organizations and Society 23(4): 411-434. (Summary).
Jonsson, S. and N. B. Macintosh. 1997. CATS, RATS, and EARS: Making the case for ethnographic accounting research. Accounting, Organizations and Society 22(3-4): 367-386. (Summary).
Kaplan, R. S. 1983. Measuring manufacturing performance: A new challenge for managerial accounting research. The Accounting Review (October): 686-705.(Summary).
Kaplan, R. S. 1993. Research opportunities in management accounting. Journal of Management Accounting Research (5): 1-14. (Summary).
Kaplan, R. S. 1998. Innovation action research: Creating new management theory and practice. Journal of Management Accounting Research (10): 89-118. (Summary).
Kasanen, E., K. Lukka and A. Siitonen. 1993. The constructive approach in management accounting. Journal of Management Accounting Research (5): 243-264. (Summary).
Luft, J. and M. D. Shields. 2002. Zimmerman's contentious conjectures: Describing the present and prescribing the future of empirical management accounting research. The European Accounting Review 11(4): 795-803. (Summary).
Luft, J. and M. D. Shields. 2003. Mapping management accounting: Graphics and guidelines for theory-consistent empirical research. Accounting, Organizations and Society 28(2-3): 169-249. (Summary).
Luft, J. L. 1997. Long-term change in management accounting: Perspectives from historical research. Journal of Management Accounting Research (9): 163-197. (Summary).
Lukka, K. and J. Mouritsen. 2002. Homogeneity or heterogeneity of research in management accounting? The European Accounting Review 11(4): 805-811. (Summary).
Martin, J. R. Not dated. The concept of validity. Management And Accounting Web. Validity Notes
Mock, T. J. 1972. A decision tree approach to the methodological decision process. The Accounting Review (October): 826-829. (Note).
Reiter, S. A. 1994. Beyond economic man: Lessons for behavioral research in accounting. Behavioral Research in Accounting (6) Supplement: 163-185. (Summary).
Shields, M. D. 1997. Research in management accounting by North Americans in the 1990s. Journal of Management Accounting Research (9): 3-61. (Summary).
Snead, K. C. 1991. An application of expectancy theory to examine managers' motivation to utilize a decision support system. Abstract. Journal of Management Accounting Research (3): 213-222. (Summary).
Tongtharadol, V., J. H. Reneau and S. G. West. 1991. Factors influencing supervisor's responses to subordinate's poor performance: An attributional analysis. Journal of Management Accounting Research (3): 194-212. Used a field experiment. (Summary).
Zimmerman, J. L. 2001. Conjectures regarding empirical managerial accounting research. Journal of Accounting and Economics (32): 411-427. (Summary).