Study Guide by James R. Martin, Ph.D., CMA
Professor Emeritus, University of South Florida
Research Methods Main Page | Decision Theory Main Page
Preface
This monograph is part of a larger project developed jointly by the National Association of Accountants and the Society of Industrial Accountants of Canada. It deals with research methodology and research on business decisions. The study is intended for practitioners as well as academics and can serve as a primer on methodology. The purpose of Chapter 1 is to provide a framework for finding and solving research problems. Chapter 2 extends the framework to address the criteria for choosing and evaluating different research methodologies. Chapter 3 is dedicated to business decision problems and Chapter 4 provides a summary of eleven examples of research on decision making.
Chapter 1 - A Framework For Methodology
Need for a Framework
This chapter begins with some questions that are difficult to answer to the satisfaction of everyone, such as the following.
What is research?
Who is a researcher?
Where do research problems originate?
How should the researcher go about solving his or her problem?
Where should the researcher go for information?
How does the researcher know if and when the problem is solved?
This monograph provides a framework to aid in designing and conducting research in a systemic way.
A Framework for Research Methodology
The authors define research methodology as "the strategy or architectural design by which the researcher maps out an approach to problem-finding or problem-solving." Their framework has six parts as indicated below. Parts I and II are related to problem finding, while parts III through VI relate to problem solving.
I. Problem Finding
Formal Approaches:
Prior research - Inductive fact finding that leads to generating a theory. Deductive relates to testing a theory.
Analog method - Using knowledge gained in one problem area to formulate a research question in a related area.
Renovation - Replacing defective components of a theory. This requires training in structural analysis, i.e., the ability to diagnose a system to find the defective part.
Dialectic method - Developing counter-plans for challenging, refining or deposing an existing theory. Price level adjustments to financial statements is mentioned as an example.
Extrapolation - Extending stable trends into the future and posing questions related to predicted outcomes.
Teratological analysis - Similar to extrapolation, but involves unstable trends, or unexpected futures. Radical what if questions related to contingences.
Morphology - A method of formalizing combinational possibilities in complex problems.
Decomposition - Breaking problems into component parts
Dichotomy - separating a problem into questions that can be answered yes or no. Examples include flow charting, network analysis, algorithms, and computer programming.
Aggregation - Aggregate research findings and apply them to more complex problems. Example - Applying input-output analysis, utility theory and motivational theory
to measuring managerial performance.
Informal Approaches:
Conjecture - Based on intuition or hunch.
Phenomenology - Research problems created by a phenomenon, e.g., computers, conglomerates, environmentalism, etc.
Consensus - Group agreement that there is a problem. The FASB approach is given as an example.
Experiential - Research to find ways to avoid a reoccurrence of a problem, e.g., lawsuits, liquidity problem, customer complaints
II. Research Problem
What is a research problem? A research problem:
1. must be defined properly,
2. must be posed in solvable terms,
3. must be logically connected to the environment from which it was drawn so that the solution can be applied to that environment.
4. has been screened against existing body of knowledge, i.e., not been solved,
5. has a potential significant contribution.
Examples of deficiencies in defining the research problem include:
1. Mislabeling the problem. Confusing descriptive with normative. Descriptive questions relate to empirical research, observing what is. Normative questions involve opinion research, i.e., what should be. Applying past or future problems to the present. Using students to study the behavior of business people.
2. Defining a problem that is unsolvable such as how businessmen solve problems.
III. Research Mode
A researcher needs to know which mode they are in to understand the consequences of the methods.
Induction: the process of generating theory - from specific facts to generalizations.
Relevant questions search for truth, e.g., which, where, who, why, whether, how, and what. Avoid one's own theories or opinions.
Deduction: the process of testing theory- testing a hypothesis against theory - from general to specific.
Relevant questions include; will questions such as will it work?, is questions such as is it a good idea?, set-response questions such as how do people feel?, task-response questions such as did he do a good job, and if questions such as will this occur if we do that?
Research Strategies, Domains and Techniques
Research methodology includes a set of strategies, domains and techniques used in generating or testing theory or problem solving.
IV. Research Strategy
Research strategy refers to the nature of the data, the process of obtaining the data and the way we go about generating or testing theory. Research strategies include: opinion, empirical, archival and analytic.
Opinion - Includes views, judgments or appraisals of individuals or groups. There is no direct observation of the facts. Techniques Include questionnaires, opinion polls, interviews, Delphi (for groups where opinions are stated, if no consensus, then circulate and include invitation to revise, continue until a consensus is reached or appears infeasible) and brainstorming.
Empirical - Based on observation or experience. Direct access to the facts. The researcher either experiences the phenomena or is an eye-witness to the events. Domains include:
Case - where there is no attempt to structure the research, i.e., no experiments, design or control, field and laboratory.
Field - includes experimental design, but no control. Has a structure and method of collecting and analyzing data. Time and motion studies given as an example.
Laboratory - Includes experimental design and control. Simulation is an example.
The authors support their definition of empirical research strategy with the following comments. "The term empirical has acquired an ambiguity in the accounting literature. In its broadest sense it has come to mean the use of "real world" data in the application of research, and as such would apply to opinion or recorded data which purports to be of real-world origin. But this fails to distinguish between first and second-hand evidence, or between impressions and reality. Because these distinctions are important, as we will see in the course of this study, we favor a stricter construction of the term. Our definition, we believe, is substantiated widely in the methodology of science literature." (See p. 24).
Archival - Based on recorded facts, but no direct observation of the facts. Domains include:
Primary - original documents, e.g., invoices, purchase orders.
Secondary - journals, ledgers, financial reports.
Physical - fingerprints, footprints, dog tracks. Techniques include scanning, observation, sampling and content analysis, i.e., a formal technique for evaluating written or oral communication. Erosion and accretion measures are used for research in the physical domain.
Analytic - Based on the internal logic of the researcher.
Inductive argument - from specific to general, generalizing from incomplete data or drawing conclusions from inadequate evidence. Rules are: agreement, difference, joint agreement and difference, residues, and concomitant variations (i.e., correlated).
Deductive argument - from general to specific. Syllogism - a form of reasoning where a conclusion is drawn from two premises. Formal techniques include: mathematical logic, mathematical modeling, formal organization techniques (flowcharting, network analysis, decision trees, algorithms, and heuristics). Informal techniques include: philosophical argument, the scenario, the dialectic (fully developed argument and counter argument), the dichotomous method (series of yes and no questions) and the Teratological method (formulating hypotheses beyond the limits of rationality).
Summary: Problem - Mode - Strategy - Domain - Techniques
These concepts are summarized in the illustration below adapted from Exhibit 1 in the monograph.
What is Research?
Conceptually, research is defined as the orderly search for truth. Operationally, research is:
1. an orderly investigation of a defined problem,
2. using an appropriate scientific method or methods,
3. to gather adequate and representative evidence,
4. producing conclusions drawn from logical reasoning without bias,
5. that can be validated,
6. and yield general principles or laws that may be applied in similar conditions.
Artifacts of research form a symbolic and rational system of inquiry and the language of research. These artifacts include facts, experiences, data, concepts, constructs, hypotheses, conjectures, principles and laws. These terms are defined on page 29.
Steps in the Scientific Method
The following steps are designed to achieve reliability and validity.
1. Recognition that knowledge stems from observations.
2. Define the problem. Why and what is to be achieved.
3. Formulate a research plan including appropriate strategies, domains and techniques. The purpose of the plan
should be directed towards hypothesis construction (induction) or hypothesis testing (deduction).
4. Conduct inquiry in accordance with the plan.
5. State the outcome in explicit terms that may support or refute the hypothesis.
6. State the conclusions with sufficient support and clarity to establish what was done, what was found, and what significance
the findings may have, including the relationships to the work of others in terms of similarities and differences.
Who is a Researcher?
A researcher is anyone who uses the scientific method.
Research and Creativity
Research does not eliminate the need for creativity. Research supports, but does not replace the intuition and judgment of decision makers.
Chapter 2 - Criteria for Selecting A Research Methodology
A flowchart for selecting a methodology is provided on page 34. It maps out the options and can be used to classify research projects.
Strengths and Deficiencies of Different Research Methods
| Method | Strengths |
| Opinion | Suited for research on attitudes, futures research, can use large samples, easiest method to use, and lends itself to many types of data analysis. Herzberg's work on hygiene factors and motivators is mentioned as an example. See the Herzberg summary for an illustration similar to Exhibit 6. |
| Empirical | Based on facts, not opinions. Best suited to analyzing actual behavior, fact finding and seeking reality. Gets the researcher involved. Case and field studies provide a rich context for research. Laboratory studies provide the most control. Can use sophisticated techniques such as gaming and simulation. |
| Archival | Best suited for analysis of data, in documents, historical analysis, gathering hard evidence, extrapolation of trends, etc. Ability to access a large quantity of hard, often factual information. |
| Analytic | There is no need to search for additional data and analytic research is not limited by existing data. It provides the broadest scope for imagination and creativity. Best suited for the use of logic, philosophy, operation research techniques. |
| Method | Deficiencies |
| Opinion | Opinion is not fact, perceptions differ from reality. Methodology is subject to bias in the survey instrument, and in biases in the way people respond to questions. Opinions are unstable over time, and group opinion is difficult to capture and analyze. See the Ethics topic for more on bias. |
| Empirical | In general, only a small number of situations can be studied in detail and there is limited ability to generalize from these isolated cases. It is restricted to the present and is the most time consuming method. Case research lacks experimental design and control. Field research has experimental design, but lacks control (See 8 steps below). Laboratory has experimental design and control, but observations may be deficient for several reasons (pp. 38-39). |
| Archival | Selective depositing - only certain things are recorded, e.g., the accomplishments of white males. Selective survival - much is lost, unpublished, revised etc. Selective retrieval - subject to bias and sampling errors. Filling in the gaps - editorializing. Biases of researchers. Skill deficiencies of the researcher (anachronism, false periodization, interminability, chronicism and staticism (p. 41.) |
| Analytic | The most abused strategy and the most difficult to criticize. Requires a first rate mental ability that is rare. Can more readily be used to mislead. Often sloppy. It is subject to logical errors (see below), problems of semantics, etc. Temptation to focus on trivial and irrelevant problems. Can only create theory, never proof. |
Field Research - Eight Steps (p. 38)
1. Recognize and identify problems.
2. Explore the problem - gathering information and defining the problems specifics.
3. Formulate the research objectives.
4. Estimate the costs.
5. Define the population.
6. Select the sampling design.
7. Design the research implement.
8. Plan and execute the field operations.
Examples of poor logic in accounting Analytic research (p. 43)
1. Building a strong argument on a weak premise.
2. Jumping from a few instances to generalizations.
3. Jumping from a generalization to a specific instance.
4. Ignoring other possible causes.
5. Ignoring other possible effects.
6. Circular reasoning.
7. Using an illogical train of thought.
Experimental Design and Control Criteria (p. 46)
Internal validity - Relates to the following questions:
1. Is the research design appropriate?
2. Does the design include all the important factors and relationships?
3. Have the independent variables been controlled to prevent bias?
4. Has randomization been used when possible to reduce systematic bias?
External validity - Relates to the problem of generalization. Can the findings be applied to other cases on a broad basis?
Broader Framework for Selecting and Evaluating Methodology
The authors say validity has been stressed to the exclusion of other important criteria and propose a broader framework.
1. Holism - ability to comprehend the problem in totality.
2. Precision - degree of refinement of a problem for objective measurement.
3. Internal validity - ability to control the variables in the problem.
4. External validity - extent to which the findings are applicable to other similar situations.
5. Quantitative - ability to define or analyze problems in quantitative terms.
6. Qualitative - ability to define and or analyze the problem in qualitative terms.
7. Relevance - value of the findings to significant areas of need and application.
8. Skill transfer - extent to which it is difficult to train and supervise research assistants.
The Need For Balance
The authors argue that we should use eclectic methodology. The practice of law is an example where all strategies are used to build a case such as opinions, eye-witness reports, lab evidence, library research, logic, philosophy, and cross-examination. A narrow focus limits the problems that we can study and excludes many problems including research on decision-making.
Research in accounting has been biased in favor of theoretical essays and testing hypotheses. We need more inductive research aimed at generating theory. We need to include greater attention to abstract problems such as decision processes that could yield benefits for the improvement of management practices.
Chapter 3 - The Scientific Study and Classification of Business Decision Problems
The authors make a distinction between the science of decision-making (Praxeology) and the art of decision-making. The art of decision-making cannot be studied or learned. The science of decision-making can be studied and learned, as any science by applying deductive and inductive analysis.
Exhibit 8 provides a summary (based on the work of Murdick and Ross) showing the development of scientific management from behavioral theory (human relations and social system) to management theory (experimental analysis and functional analysis) to management science (decision analysis, quantitative analysis and technology). The chapter includes a discussion of the benefits from each of the areas of research and provides a classification scheme for business decision problems.
Benefits of Behavioral Research
In this section the authors discuss the work of several authors including Hawthorne, Seashore, Van Zelst, Fiedler, Zander, Maslow, Mayo, Vroom, and Likert, etc. The work of many of these authors has helped us understand the behavior of groups. Group control often neutralizes company control systems. Intra-group competition can have disastrous effects on group cohesiveness, while inter-group competition has the opposite effect. Formal systems are viewed by employees as propaganda. The grapevine is relied on for information. Organizations have unique cultural, ideological, sociological, attitudinal and technological environments.
Maslow's needs hierarchy has been useful in the area of motivation. Vroom's work (Performance = ability x motivation) revealed that motivation is more important for performance than ability. Likert's work on leadership produced four styles including: exploitive, benevolent, consultative and participative leaders.
Benefits of Management Theory Research
Fayol and others (e.g., Mooney and Reiley, Gulick,Terry, Koontz and O'Donnell) provided and expanded the functional approach to management. For example, Fayol indicated that management functions include: planning, organizing, staffing, directing and control. Harvard developed and promoted the experiential approach using case studies.
Benefits of Management Science Research
Developments in this area include decision analysis, quantitative analysis and management technology.
Barnard promoted the science of decision making. Simon refuted the concept of the rational economic man, i.e., that a decision maker can define the problem accurately, search for all possible solutions, assess the benefits of each and choose the optimal solution.
Simon introduced the concepts of:
Bounded rationality - people have fragmented knowledge of consequences.
Means-ends analysis - defines a sequence of decision points with a limited set of alternatives.
Satisficing - people seek satisfactory rather than optimum solutions due to mental limitations and information overload. Miller and Haire's work supports.
The concurrency of phases - phases in the decision process run concurrently, rather than is fixed steps. Goals constantly change.
Quantitative methods developments have been continuous and include time and motion study, the Gantt chart, PERT, CPM, price level models, game theory, decision trees, program planning and budgeting systems (PPBS).
Management Science researchers have considered problems in many areas including inventory, allocation, queuing, sequencing, routing, replacement, competition, and search.
Classification of Business Decision Problems
A classification scheme appears in Exhibit 10 on page 60. The classifications include:
Cataloging - identification only.
Grouping - Dichotomous - classify into one of two categories or multiple categories.
Dimensioning - Attributes of two or more problems are juxtaposed to produce a contrasting effect.
Ordering - Ordinal - establishes rank order.
Interval - determines distance of intervals and levels of complexity.
Ratio - items positioned in relation to scales.
Systemizing - Decomposing - breaking a problem into parts.
Network - connect elements of a system by cause.
Static reciprocity - mapping (static) cause and effect relationships.
Dynamic - mapping (dynamic) cause and effect relationships.
A Typology for Business Problems
Business problems are shown in a matrix of combinations related to the quality of data (objective or subjective) and the nature of the relationships among the decision variables (simple and complex). A flow chart classifying problems into these four categories (subjective-simple, subjective-complex, objective-simple and objective-complex) is presented in exhibit 21.
This chapter includes many examples and illustrations not mentioned in this brief summary.
Chapter 4 - Concluding Remarks and Application
This chapter includes a summary of eleven examples of research on decision-making (Exhibit 25) and a flowchart for classifying methodology (Exhibit 26). The flowchart below was adapted from Exhibit 26.
__________________________________________________
Related summaries:
Atkinson, A. A. and W. Shaffir. 1998. Standards for field research in management accounting. Journal of Management Accounting Research (10): 41-68. (Summary).
Birnberg, J. G., M. D. Shields and S. M. Young. 1990.The case for multiple methods in empirical management accounting research (With an Illustration from Budget Setting). Journal of Management Accounting Research (2): 33-66. (Summary).
Boehm, V. R. 1980. Research in the "real world" - A conceptual model. Personnel Psychology (33): 495-504. (Comparison of academic and practitioner research models).
Ittner, C. D. and D. F. Larcker. 2001. Assessing empirical research in managerial accounting: A value-based management perspective. Journal of Accounting and Economics (32): 349-410.
Jonsson, S. and N. B. Macintosh. 1997. CATS, RATS, and EARS: Making the case for ethnographic accounting research. Accounting, Organizations and Society 22(3-4): 367-386. (Summary).
Kaplan, R. S. 1998. Innovation action research: Creating new management theory and practice. Journal of Management Accounting Research (10): 89-118. (Summary).
Kasanen, E., K. Lukka and A. Siitonen. 1993. The constructive approach in management accounting. Journal of Management Accounting Research (5): 243-264. (Summary).
Luft, J. and M. D. Shields. 2003. Mapping management accounting: Graphics and guidelines for theory-consistent empirical research. Accounting, Organizations and Society 28(2-3): 169-249. (Summary).
Martin, J. R. Not dated. The concept of validity. Management And Accounting Web. ValidityNotes
Mock, T. J. 1972. A decision tree approach to the methodological decision process. The Accounting Review (October): 826-829. (Note).
Reiter, S. A. 1994. Beyond economic man: Lessons for behavioral research in accounting. Behavioral Research in Accounting (6) Supplement: 163-185. (Summary).
Shields, M. D. 1997. Research in management accounting by North Americans in the 1990s. Journal of Management Accounting Research (9): 3-61. (Summary).
Williams, P. F. 1987. The legitimate concern with fairness. Accounting Organizations and Society 12(2): 169-189. (Summary).
Zimmerman, J. L. 2001. Conjectures regarding empirical managerial accounting research. Journal of Accounting and Economics (32): 411-427. (Summary).
