)V1ESI_} RETURNING MATERIALS:

P1ace in book drop to
LJBRARJES remove this checkout from
1--:y--L your record. FINES will

 

 

 

be charged if book #15
returned after the date
stamped below.

 

Maw A39

 

 

THE CONSTRUCTION OF AN EXPERT SYSTEM TO
MAKE MATERIALITY JUDGMENTS

BY

Paul John Steinbart

A DISSERTATION

Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Accounting

1984

'0

‘1.

\j ‘3
\J<
/{9

ABSTRACT

THE CONSTRUCTION OF AN EXPERT SYSTEM TO
MAKE MATERIALITY JUDGMENTS

BY

Paul John Steinbart

Previous research on materiality has focused on the
judgments that are made when evaluating audit evidence. How-
ever, auditors also make materiality judgments to help plan
the nature, timing, and extent of the audit procedures that
will be used to provide audit evidenceu These judgments have
a direct effect on audit efficiency and effectiveness. This
study examines how auditors make these preliminary judgments
of materiality.

Traditional techniques used to study auditor judgments
(eqy, the lens model) only measure the relationship between
decision inputs and outputs; they do not explain how those
inputs are used to make the judgment. This study constructs
an expert system, called AUDITPLANNER, which is capable of
actually making materiality judgments. Analysis of AUDIT-
PLANNERfis decision rules reveals how various input factors
are used to make materiality judgments.

Analysis of AUDITPLANNER/s decision rules indicates
that the preliminary materiality judgment is influenced by
(l) the characteristics of the company being audited, (2)
the perceived needs of the users of the financial state-
ments, and (3) the degree of risk associated with the audit

and the auditor’s own attitudes toward such risk.

ACKNOWLEDGEMENTS

I would like to take this opportunity to thank those
who helped make this dissertation possible. First, I want to
thank God for giving me the talents to pursue this task. My
committee chairman, WillianIEL McCarthy, provided insightful
criticisnl throughout tflma entire :research. process. My
readers, Alvin A”.Arens and George Stockman, both deserve
thanks for their helpful and timely advice. Thomas H. Carr
provided helpful comments on a preliminary version of the
work; Graham Gal joined in frequent and fruitful discussions
about the methodology. Bob Byers agreed to permit the use of
the computer facilities at Jackson Community College; this
research would not have been possible without his assist-
ance. The Deloiite, Haskins & Sells Foundation provided the
financial support which enabled me to pursue this project.
Finally, I want to thank my wife, Linda, and daughter,
Stephanie, for their constant support, encourage-ment, and
understanding. Their love was fundamental to the success of
this undertaking.

ii

TABLE OF CONTENTS

LIST OF TABLES O O O O O O O O O O O O O O O O O 0
LIST OF FIGURES . . . . . . . . . . . . . . . . . .

INTRODUCTION 0 O O O O O O O O O O O O O O O O O 0

CHAPTER

I. PREVIOUS RESEARCH ON MATERIALITY . . . . . .
Types of Materiality Judgments . . . . . .
Determinants of Materiality Judgments . .

Judgments of Accounting Materiality . .

Judgments of Auditing Materiality . . .
Implications of Prior Research Findings .

Types of Cases Used Previously . . . . .

Role of Circumstantial Variables . . . .

II. THE USE OF EXPERT SYSTEMS TO STUDY

DECISION MAKING BEHAVIOR . . . . . . . . . .

The Production System Architecture . . . .
Production Memory . . . . . . . . . . .
Inference Engine . . . . . . . . . . . .

Forward-chaining . . . . . . . . . . .
Backward-chaining . . . . . . . . . .
Working Memories . . . . . . . . . . . .

The Information Processing Paradigm . . .
Effect of the Task Environment . . . . .
Structural Characteristics of the
Problem Solver . . . . . . . . . . . . .

Heuristic search . . . . . . . . . . .
Reasoning by manipulating symbols . .
The Problem Solver’s Knowledge . . . . .
The nature of expertise: theory . . .
The nature of expertise: empirical
evidence . . . . . . . . . . . . . . .
Summary . . . . . . . . . . . . . . . .

Expert Systems as Psychological Theories .
The Production System Architecture as
a Theory of Cognitive Behavior . . . . .
AUDITPLANNER as a Theory of Professional
Judgment . . . . . . . . . . . . . . . .

iii

vi.
vii.

20

20
21
22
22
23
25
26
28

30
31
33
35
36

38
40
41
41

45

CHAPTER

III. RESEARCH METHOD . . . . . . . . . . . . . . . 46
Research Tool: EMYCIN . . . . . . . . . . . 46

Reasons for Choosing EMYCIN . . . . . . . 48

Previous successful use of EMYCIN . . . 48

Similarity of the original task for
which EMYCIN was designed and

materiality judgments . . . . . . . . 49
Formal task analysis of judgments of
auditing materialiy . . . . . . . . . . Sl
EMYCIN’S satisfaction of the requirements
of the formal task analysis . . . . . . 53
Features of EMYCIN . . . . . . . . . . . . 54
Search strategy . . . . . . . . . . . . 54
Representation of knowledge . . . . . . 57
Reasoning under uncertainty . . . . . . 58
Summary . . . . . . . . . . . . . . . . . 62
Construction of Prototype System . . . . . . 62
Choice of Subjects . . . . . . . . . . . . 63
Lack of consensus across firms . . . . . 63
Lack of consensus within firms . . . . . 66
Implications . . . . . . . . . . . . . . 67
Subject selection . . . . . . . . . . . 69
Source of Knowledge for Prototype . . . . 70
Refinement of the System . . . . . . . . . . 7l
Explanation of Reasoning During a
Consultation . . . . . . . . . . . . . . . 72
Explanation After a Consultation is Over . 73
Summary . . . . . . . . . . . . . . . . . 74
Testing AUDITPLANNER . . . . . . . . . . . . 74
Subjects . . . . . . . . . . . . . . . . . 78
Test Cases . . . . . . . . . . . . . . . . 79
Procedure . . . . . . . . . . . . . . . . 79
Results . . . . . . . . . . . . . . . 81
Competence of AUDITPLANNER . . . . . . . 82
Reasonableness of model . . . . . . . . 82
Usefulness of AUDITPLANNER . . . . . . . 85
Ease of use and enjoyment . . . . . . . 86
Adequacy of question-answering
facilities . . . . . . . . . . . . . . . 86
Summary . . . . . . . . . . . . . . . . 86
CHAPTER
IV. ANALYSIS OF THE SYSTEM . . . . . . . . . . . . 88
The Decision Setting . . . . . . . . . . . . 88
The Judgment Model . . . . . . . . . . . . . 91
Overview . . . . . . . . . . . . . . . . 91
Identification of the Materiality Base . . 99
Determination of the type of entity . . 100
Identification of users’ interests . . . 104

Choosing the materiality base . . . . . 106

iv

CHAPTER
IV. (continued)

Choosing a Percentage Rate . . . . . . .
Calculation of the Materiality Level . .
Tentative level too small . . . . . .
Tentative level too large . . . . . .
Special Situations . . . . . . . . . . .
Audits of nonprofit organizations . .
Audits of financial institutions . . .
Summary . . . . . . . . . . . .

The Nature of the Materiality Judgment
Process . . . . . . . . . . . . . . . . .

V. CONCLUSION AND SUGGESTED EXTENSIONS . . . .

Research Contribution . . . . . . . .
Possible Future Extensions to the Work . .
Further Development of AUDITPLANNER . .
Broadening AUDITPLANNER’s domain of
expertise . . . . . . . . . . . . . .
Integrating with other audit judgments
Comparative Studies . . . . . . . . . .
Experimental Studies . . . . . . . . . .
Conclusion . . . . . . . . . . . . . . . .

APPENDIX I Example of AUDITPLANNER’S explanation

capabilities during a consultation . . .

APPENDIX II Example of AUDITPLANNER’S explanation
capabilities after a consultation . . .
APPENDIX III Sample session with AUDITPLANNER . . .
APPENDIX IV Evaluation questionnaire . . . . . . .
BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . .
General References . . . . . . . . . . . . . . . .

109
112
113
115
116
116
117
118

119
126

126
130
130

130
131
131
131
132

133

135
137
139
141
149

TABLE 1.
TABLE 2.
TABLE 3.

LIST OF TABLES

Characteristics of test clients . . . . . . . 80
Responses to test questionnaire . . . . . . . 83
Effect of qualitative factors on judgments

of auditing materiality . . . . . . . . . . . 120

vi.

FIGURE
FIGURE

FIGURE
FIGURE
FIGURE
FIGURE
FIGURE

1.
2.

3.
4.
5.
6.
7.

LIST OF FIGURES

Theory of Problem Solving Behavior
Role of EMYCIN in Construction of an

EXPERT SYSTEM . . . . . . .

Causal Model of Auditing Materiality .

Overview of AUDITPLANNER’S Judgment Model
Calculation of Overall Materiality Level

Identification of Materiality Base .

Calculation of Materiality .

vii.

27

47
89
92
93
94
95

INTRODUCTION

The concept of materiality is the cornerstone of
auditing. The auditorks opinion that an enterprise’s finan-
cial statements "present fairly" the results of operations
implies that those statements are not materially misstated
[AICPA, 1983, para. 3]. In planning the audit, a materiality
judgment is made to help determine the extent of testing
needed to provide sufficient evidence upon which to base an
opinion. Another materiality judgment is made to evaluate
the results of those tests. No wonder that Statement on
Auditing Standards (SAS)IML 1 states that "the concept of
materiality is inherent in the work of the independent
auditor" [AICPA, 1979, para. 150.04].

The importance of the concept of materiality is also
apparent from its inclusion on the Financial Accounting
Standards Boardfs (EASE) original agenda. The FASB issued a
Discussion Memorandum [FASB, 1975] on the topic in an effort
to develop some guidelines for making materiality judgments.
That effort ended, however, with the FASB’sidecision that
such guidelines were not feasible.

The Board’s present position is that no general
standards of materiality could be formulated to take
into account all the considerations that enter into

an experienced human judgment [FASB, 1980, para.
131].

2

SAS No. 47 describes the resulting state of affairs
and summarizes the reason the reasoning behind the FASB’s
decision:

The auditorfis consideration of materiality is a
matter of professional judgment and is influenced by
his perception of the needs of a reasonable person
who will rely on the financial statements. [Those
needs] . . . are recognized in the discussion of
materiality [by the FASB, cited above] . . . that
discussion recognizes that materiality judgments are
made in light of surrounding circumstances and
necessarily involve both quantitative and qualita-
tive considerations [AICPA, 1983, para. 6].

Accounting researchers have long been interested in
how materiality judgments are made. Holstrum and Messier
[1982] review previous empirical research on materiality and
conclude that:

Definitive comprehensive implications for audit

practice or policy formulation are difficult to de-

cipher from the materiality research to date [p659].
They point out that previous research has not examined the
effect of the type of entity and its industry classification
on materiality judgments, nor has there been an attempt to
measure the sensitivity of those judgments to the nature of
the event or item for which the judgment is being made.

Moreover, previous research has concentrated on the
materiality judgments that are made when evaluating audit
evidence. However, auditors also make materiality judgments
in the planning stage of the audit to help determine the
extent of audit tests and procedures. The lack of attention
paid to this type of materiality judgment reflects the

general paucity'of.researchcnuthe auditorfisinitial plan-

ning process. Felix and Kinney [1982] state the need for

such research:

while the amount of research on the auditor’s deci-
sion processes is growing, the planning decision
processes are not being included in this growth.
While the complexities of the setting present con-
siderable difficulties, effective planning of the
audit is critical to efficient auditing, and
research on the planning processes should be
encouraged [p.255].

This dissertation seeks to model how materiality judg-
ments are made in the planning stage of the audit. The goal
is to explain how circumstantial factors, such as the nature
of the company being audited, influence those judgments.

Techniques from.tflu2 field of artificial intelligence
will be used to express the model in the form of a computer
program known.as a rule-based expert systenu Such systems
are particularly well-suited to modeling tasks with charac-
teristics such as those found in judgments of materiality.

When intelligent behavior consists (n3 numerous
specialized responses to widely varying and largely
unpredictable situations, the antecdent - consequent
structure of [rule-based expert systems] isolates
and represents the appropriate logic of such data-
directed behavior in a natural way [Waterman and
Hayes-Roth, 1978, p.22].

The expert systems nmmhodology has been successfully
applied to the study of a wide range of tasks in accounting
and auditing. Such systems have been built to plan for indi-
vidual estate taxes [Michaelsen, 1982], to determine the
collectibility of delinquent trade accounts receivables
[Dungan, 1983; Dungan and Chandler, 1983], to evaluate the
causes of fluctuations identified during the process of

analytical review [Braun and Chandler, 1982], to analyze EDP

controls in an advanced computer system [Hansen and Messier,

4
1983], to evaluate the quality of internal controls [Ga1,
1984], and to decide on the appropriateness of issuing a
"subject to" audit opinion [Dillard, Mutchler, and Rama-
krishna, 1983].

The construction of an expert system involves the
detailed study of the decision making behavior of an expert
in order to identify the basis for that behavior. It is,
therefore, inherently descriptive research. In fact, one of
the principal motivations for using this methodology is to
obtaixla better unnderstanding of current decision making
practices.

The aim here [in building an expert system] is thus
not simply to build a program that exhibits a cer-
tain specified behavior, but £9 use the program con-
struction process itself as a w_y of explicating
knowledge in the field, and_ to use the program text
as a medium —of expression of many forms of knowledge

about the task and its solution [Davis and Lenat,
1980, p.471].

 

 

 

 

 

 

 

 

 

 

Descriptive research on current methods of decision
making is important.

The ultimate goalcflfhuman information processing
research in accounting is to improve decision mak-
ing. Before decision making can be improved, how-
ever, it is useful to evaluate the current quality
of decision making, and before decision quality can
be evaluated, decision making must be understood
[Ashton, 1982, p.vii].

The remainder of this dissertation is organized as
follows. Chapter I reviews previous research on materiality.
Chapter II discusses the expert system methodology for
studying decision making behavior. Chapter III describes the

procedure to be used in building the expert system. Chapter

IV analyzes the resulting system, examining the rules

5
contained therein in order to demonstrate how various
circumstantial factors affect materiality judgments. Chapter

V summarizes the findings and explores possible directions

for future research.

CHAPTER I

PREVIOUS RESEARCH ON MATERIALITY

This chapter reviews the results of previous empirical
research on materiality. Most of that research has examined
materiality in relation to the evaluation of audit evidence.
However, materiality considerations also affect the planning
of the audit. Consequently, the first part of this chapter
discusses the relationship between these two judgments. The
second part then examines the research findings on each type
of judgment. The final part of the chapter discusses the

implications of those results for the current research.

Types pf Materiality Judgments

 

Statement on Auditing Standards (SAS) No. 47 describes
two different times during the course of the audit when the
auditor needs to consider materiality.

The auditor should consider . . . materiality
both in (a) planning the audit and designing audit-
ing procedures and (b) evaluating whether the finan-
cial statements taken as a whole are presented
fairly in conformity with generally accepted
accounting principles [AICPA, 1983, para. 8].
The purpose of the initial consideration is to ensure that
the auditor plans the extent of audit tests so as to provide
adequate empirical evidence upon which to base an opinion on

the fairness of presentation of the financial statements. In

6

7
the second instance, the auditor considers materiality in
order to determine the effect on that fairness of presenta-
tion of any errors that may have been uncovered by those
audit tests.

Theoretical discussions of materiality distinguish
between these two judgments, referring to the former as
involving "auditing materiality" and the latter as involving
"accounting materiality" [CICA, 1965; Leslie, 1977; Thomas
and Krogstad, 1978]. These terms will be adopted here. How-
ever, it is important to recognize that they do not refer to
two different concepts, but only to two different instances
of that concept.

In fact, judgments of auditing materiality may be
thought of as pro-forma judgments of accounting materiality.
Both are concerned with the point at which the amount of
errors destroys fairness of presentation: the former makes
this determination prior to performing any audit tests, the
latter does so after testing is completed. This common con-
cern makes it likely that there are many similarities
between the two judgments, so that research on one may pro-
vide useful insights into the nature of the other. However,
because the two judgments are made at<different points in
time, they are based on different sets of information and
are likely to differ in their conclusions. SAS No. 47 recog-
nizes this difference:

Assuming, theoretically, that the auditor};
judgment about materiality at the planning stage was

based on the same information available to him at
the evaluation stage, materiality for planning and

8

evaluation purposes would be the same. However, it
ordinarily is not feasible . . . to anticipate all
of the circumstances that may ultimately influence
his judgment . . . Thus, his preliminary judgment
about materiality ordinarily will differ from his
judgment about materiality used in evaluating the
audit findings [AICPA, 1983, para. 15].

Consequently, research is needed on both types of
materiality judgments. The next section reviews the results

of prior research on each judgment.

Determinants pf Materiality Judgments

 

 

Research on materiality has sought to identify the
factors which are used to make materiality judgments and to
measure their relative influence on those judgments. Those
studies which addressed the issue of accounting materiality

are reviewed first.

Judgments pf Accounting Materiality

 

 

Discussions of materiality in the authoritative liter-
ature emphasize that materiality judgments are primarily
quantitative in nature, being concerned with whether the
item in question is large enough to destroy the fairness of
presentation of the financial statements. However, those
discussions also point out that:ftijsthe relative, rather
than the absolute, size of the item that matters. Moreover,
qualitative considerations, particularly the nature of the
item, also influence those judgments.

the answer to that question [is the item material?]
will usually be affected by the nature of the item;
items too small to be thought material if they
result from routine transactions may be considered

material if they arise in abnormal circumstances
[FASB, 1980, para. 123].

9
The results of empirical research on judgments of
accounting materiality tend to support the discussions in
the authoritative literature. Studies by Ewer [1975],
Pattillc>and Siebel [1974], and Pattillo [1975, 1976] used
questionnaire cases in which both the characteristics of the
firm and the nature of the item upon which the materiality
judgment was to be based were varied. The series of studies
by Pattillo involved over 700 subjects, and included
auditors, bankers, financial analysts, and fincancial execu-
tives. Participants were asked to determine the materiality
of an error in terms of its effect on the fairness of
presentation of the financial statenmnnxh Pattillo [1976]
summarized the findings of those studies:
The ’rule of thundf of 5% to 10% of net income is
presently widely used in practice as an overall
materiality criterflmm The participants demonstrated
this criterion to be the primary basis for their
initial determination of an item.s materiality. How-
ever, the criterion was freguently supplemented by

other guantitative criteria and was modified when
called for by the circumstances existing in the

judgment situations [Pattillo, 1976, gull, emphasis
added].

 

 

 

 

 

 

 

The absolute size of the item and its affect on the
earnings trend were two of the other quantitative factors
that were consistently ranked high in importance by all
participants. However, the nature of the itenl(eng., whether
it was a contingency, an extraordinary item, etc.) was
consistently ranked by all participants across all cases as
being the single most important factor influencing their
judgments of materiality.

Boatsman and Robertson [1974], Firth [1979], Hofstedt

10

and Hughes [1977], Messier [1979], and Moriarity and Barron
[1976] conducted experimental studies in which several fin-
ancial characteristics of the firm were systematically
varied in order to determine the effects of those character-
istics on materiality judgments. These studies all found
that the single most important determinant of materiality
was the size of the item in relation to net income. The size
of the firm [Firth, 1979; Moriarity and Barron, 1976] and
the effect of the item on the trend in earnings [Messier,
1982] were quantitative factors of secondary importance.

Most of these experimental studies, however, did not
vary the nature of the judgment item. The one exception was
the study by Boatsman and Robertson. That study used three
types of items: (1) a gain or loss on the sale of noncurrent
assets, (2) a change in accounting principle, and (3) a
future uncertainty. The nature of the item was found to be
significantly related to judgments of its materiality.

In fact, inclusion of this variable in the judgment
model markedly improved the model’s predictive accuracy.
Boatsman.andeobertson reported that a simple model based
solely on the size of the item expressed as a percentage of
income could only accurately predict 65 percent of the
subjects’ judgments and erred on the side of underdisclo-
sure. However, when two additional variables were added to
the model, one to represent the nature of the item, and the
other to represent the degree of risk in the audit, the

resulting model had a predictive accuracy of 84 percent.

11
Statistical analysis indicated that most of the improvement
was due to the variable representing the nature of the item.

An archival study by Frishkoff [1970] reported similar
results about the effect of the nature of the item on judg-
ments of materiality. Frishkoff examined annual reports in
an effort to find which variables were useful predictors of
a qualified audit opinion. He reported that a discriminant
function based solely on the item’s effect on net income
only classified nine percent of the casescnia.better than
chance basis. The inclusion of two additional variables, one
to represent the size of the client and the other to repre-
sent the nature of the item, resulted in a classification
accuracy of 91 percent.

Besides the nature of the item, Boatsman and Robertson
also found that the level of risk perceived in the audit was
significantly related to materiality judgments. Ward [1976]
examined auditors’ perceptions of the consequences of fail-
ing to find an error that affected the amount of net income.
There was a large degree of diversity across auditors, with
considerable disagreement for those cases in which the
effect of the error was to reduce net income. Newton [1977]
examined auditors’ attitudes towards risk and how the degree
of uncertainty about an item’s resolution affected judgments
of materiality. 55 percent of the auditors were risk-averse,
while 34 percent were classified as being risk-seeking. In
addition, the degree of uncertainty associated with the item

appeared to influence judgments of its materiality.

12

In summary, previous research indicates that although
the size of an item in relation to net income is probably
the most important quantitative determinant of accounting
materiality, other qualitative factors representing the
circumstances in which the judgment is made are also impor-
tant. In particular, both the nature of the item and the
auditor’s perception of the personal risk associated with
the audit (should material errors go undetected) appear to

influence judgments of accounting materiality.

Judgments pf Auditing Materiality

 

 

Only two studies examined the materiality judgments
that are made when planning the extent of audit tests and
procedures. Cushing, Searfoss, and Randall [1979] asked
auditors to estimate the overall level of materiality for an
audit and then tested a model designed to allocate that
estimate among various financial statement accounts. The
study did not specifically address the issue of how auditors
make such estimates, but it did find that auditors expressed
a high degree of confidence in the accuracw'of those judg-
ments.

Moriarity and Barron [1979] conducted an experiment
which did attempt to determine how judgments of auditing
materiality are made. Partners of a major public accounting
firm were given summarized financial statements for thirty
hypothetical companies and asked to establish the overall
materiality level that should be used to plan audit tests

and procedures. Five financial variables were manipulated:

13
net income, total assets, the debt-to-equity ratio, the
number of shares outstanding, and the trend in earnings.
(Total assets was varied by using financial statements that
were multiples of one another).

The results indicated that net income was the most
important factor affecting the judgments of four of the five
partners, with total assets being most important for the
fifth. Total assets was second in importance for two part-
ners, while the earnings trend was second in importance for
the other three. In addition, there was evidence of a break-
even effect: the importance of net income declined as it
approached the breakeven level.

Moriarity and Barron did not provide the participants
with any background qualitative information about the firms
for which the materiality judgments were to be made. In
post-experimental debriefings, the participants complained
about the lack of such information.

Most participants indicated that they would be more
familiar with the operations of their clients, the
type of management, and management objectives“ Thus,
some of the participants said they would like to
have known what industry we were dealing with, to
whom the audit report would be distributed, and what
kinds of audit problems had been experienced in the
past [Moriarity and Barron, 1979, pp.129-130].

Discussions in iflua authoritative literature indicate
that such information is important.

an amount that is material to the financial state-
ments of one entity may not be material to the fin-
ancial statements of another entity of a different
size or nature. Also, what is material to the finan-

cial statements of a particular entity might change
from one period to another [AICPA, 1983, para. 5].

14

For example, in an enterprise with few, but large,
accounts receivable, the accounts individually are
more important and the possibility of material error
is greater than in another enterprise that has a
great number of small accounts aggregating the same
total. In industrial and merchandinsing enterprises,
inventories are usually of great importance to both
financial position and results of Operations and
accordingly may require relatively more attention by
the auditor than would inventories of a public util-
ity company [AICPA, 1979, para. 150.04].

A study by Gibbins and Wolf [1982] provides some empi-
rical support for the importance of such factors in making
judgments of auditing materiality. Auditors from six public
accounting firms were askedtxarank:various environmental
factors in terms of their importance in affecting the con-
duct of an audit at various stages in the audit process.
Although the study addressed the conduct of the audit as a
whole, and not materiality, the relationship between plan-
ning and materiality noted in SAS No. 47 suggests that the
findings for the planning stage of the audit may apply to
auditing materiality as well.

the nature, timing, and extent of planning -- and
thus of the considerations of audit risk and
materiality -- vary with the size and complexity of
the entity, the auditor’s experience with the
entity, and his knowledge of the entity’s business
[AICPA, 1983, para. 11].

Gibbins and Wolf found that the following qualitative
factors were important in the planning stage of the audit:
(1) the service needs of the client, (2) information from
prior years/(audit programs and file notes, (3) plans for
the sale or major financing of the client, and (4) the

nature of the client’s business.

In summary, the results of the three studies reviewed

15
in this section tend to indicate that there are similarities
between the two types of materiality judgments. The primary
financial variable used in both judgments appears to be the
amount of net income. That factor is supplemented, however,
by other qualitative factors. In the case of accounting
materiality, the most important qualitative factor is infor-
mation about the nature of the judgment item. In the case of
auditing materiality, because those judgments are estimates
of what size errors would be material it is likely that
information about the nature of the company should be of

primary importance.

Implications pf Prior Research Findings

 

 

Two aspects of previous research have important impli-
cations for this dissertation:(l) the types of cases used
to study materiality judgments, and (2) the role of circum-

stantial variables in those judgments.

Types pf Cases Used Previously

 

A major problem with previous experimental research on
materiality judgments is the abstract nature of the cases
that were used. Many of the qualitative factors mentioned in
both the results of questionnaire studies and in discussions
in the authoritative literature as being important determin-
ants of materiality flag. the nature of the item, the nature
of the companyis business) were either omitted entirely or
were presented in a summary manner. The quantitative factors

used in the cases were also typically highly summarized. The

16
following example, taken from the Hofstedt and Hughes study
is typical. The case deals with a loss from the writedown of
a subsidiary, and participants were asked to state the prob-
ability that they would disclose it as an extraordinary item
because it was material.
Case A:
The loss is 5% of operating income, 5% of parent
investments, and 10% of subsidiary book value
[Hofstedt and Hughes, 1977, p.388, format modified].
The use of such predigested and highly aggregated
cues may produce behavior different from what would be
observed outside the laboratory [Ebbesen and Konecni, 1980].
Indeed, in gmmtrexperimental debriefings, participants in
many of the studies complained about the abstract nature of
the cases and indicated that they would have liked to have
had additional information to use in making their judgments.
Newton [1977] described the nature of this information:
All participants claimed that they needed more
information because of the many factors which merit
consideration in materiality decisions. Typically,
questions were asked concerning the firm’s balance
sheet, environment (industry and economic condi-
tions), history, management, accounting policies,
previous materiality decisions, etc. [p.106].

Clearly, future research needs to find ways to use more

realistic cases. One advantage of the use of expert systems

is that their refinement entails using them to solve real

examples of the problem being studied. Thus, all the infor-

mation normally available is included in the research study.

17

Role pf Circumstantial Variables

 

The results of previous research indicate that two
circumstantial factors, the nature of the item and the level
of perceived risk, both influence judgments of accounting
materiality. The results of the Gibbins and Wolf study and
the discussions in the authoritative literature lead one to
suspect that qualitative information about the company being
audited plays a similarly important role in judgments of
auditing materialityu.After all, if the nature of a known
error affects judgments about the materiality of that error,
then information about the nature of the company and the
users of its financial statements should affect judgments
about the level of errors that, if found, would be material.

However, although there is some evidence concerning
which factors influence materiality judgments, little is
known about ghy those factors are importantq and hgg they
enter the judgment process. As Carroll [1980] explains, to
acquire such knowledge requires more than merely relating
decision variables to the final decisions; it requires exam-
ining the processes leading up to the decision:

The decision analyst is misdirected by the impor-
tance of the moment when the decision maker identi-
fies a selection. We are seduced by language and
common sense into believing that the choice ig the
decision. Yet . . . the choice is the end product of
the decision, the moment when we see the pigeon in
the magician’s hand. The decision is the process of
arriving at a choice, the process by which the
pigeon got into the magician’s hand [p.69]

Researchers in the fields of both artificial intelli-

gence and cognitive psychology argue that traditional

l8
statistical modeling methods are inappropriate for building
models which really explain how judgments are made.

The problem, of course, is that statistical methods
are not good models of the actual reasoning process,
nor were they designed to be. . . . [they are] for
the most part, ’shallow’, one-step techniques which
capture little of the ongoing process actually used
by expert problem solvers in the domain [Davis,
Buchanan, and Shortliffe, 1977, p.39].

the output of a quantitative mechanism, be it numer-
ical, statistical, analog, or physical (nonsym-
bolic), is too structureless and uninformative to
permit further analysis. Number-like magnitudes can
form the basis of decisions for immediate action,

. . . but each is a ’dead end’ so far as further un-
derstanding and planning is concerned, for each is
an evaluation and not a summary. A number cannot
reflect the considerations that formed Lt.

This does not mean that people do not, or even
that they should not, use such methods. But because
of the block they present to further contemplation,
we can predict that they will tend to be focused in
what we might call terminal activities. In large
measure, these activities may be just the activities
most easily seen behavioristically and this might
account in part for the traditional attraction of
such models to workers in the behavioristic tradi-
tion. The danger is that theories based upon them --
response probabilities, subjective probabilities,
reinforcement schedule parameters -- are not likely
to be able to account for sophisticated cognitive
activities. As psychological theories they are very
likely to be wrong [Minsky, 1975, p.275].

 

 

In place of traditional statistical models, the use of
models based on the production system architecture is urged
in those situations where the goal of the research is to
understand hp! the decision was made.

A final advantage of [production systems] is their
ability to represent the role of the environment in
governing the [subject/s] behavior in a way that a
more conventional process model cannot. For a [pro-
duction system] presents the set of possible actions
that the subject can take together with the basis on
which he decides between them, whereas a flow- chart
or algorithm states only the outcome of that
decision [Young, 1978, p. 397, emphasis added].

 

 

 

 

 

 

 

19
The expert system to be built in this dissertation is
based on the production system architecture. The next chap-
ter describes the nature of that architecture and discusses

its use as a means of studying decision making behavior.

CHAPTER II

THE USE OF EXPERT SYSTEMS TO STUDY

DECISION MAKING BEHAVIOR

The production system architecture has been used to
build rule-based expert systems which perform a wide variety
of tasks. Examples inlcude systems which diagnose infectious
diseases [Shortliffe, 1976], analyze the molecular structure
of chemical compounds [Buchanan, Sutherland, and Feigenbaum,
1969, 1970], prospect for mineral ores [Gaschnig, 1982], and
diagnose faults in computer systems [Hartley, 1979].

This chapter begins by describing the essential fea-
tures of the production system architecture underlying such
systems. Then the theoretical paradigm underlying the use of
that architecture to study decision making behavior is pre-
sented. The final section of the chapter discusses the
status of expert systems as psychological theories of exper-

tise in the task domain being modeled.

The Production System Architecture

 

A production system consists of three components: (1)
a production memory or knowledge base, (2) an executive or
inference engine, and (3) one or more working memories
[Feigenbaum, 1979; Hayes-Roth, Waterman, and Lenat, 1978;
Newell, 1973, 19803]. The following sections discuss these

20

21

three components in more detail.

Production Memory

 

Production systems get their name because they repre-
sent knowledge about relationships among domain variables in
th form of conditional rules known as productions. Each rule
is of the form: situation => action.iflmaleft-hand side of
the rule consists of a set of clauses which describe the
situation in which that rule applies; the right-hand side
describes the actions or inferences that occur when the rule
is executed or "fired".'Three of the rules found in AUDIT-
PLANNER (the name given.to the expert system built in this
dissertation) are presented below:

Rule 1 IF: 1) the client is a public entity, and

2) there is no significant concern about
the liquidity or solvency of the client
THEN: it is assumed that the principal
external users of the financial state-
ments are primarily interested in
information about the results of current
operations.

Rule 2 IF: 1) the principal external users of the
financial statements are primarily
interested in the results of current
operations, and

2) net income is above the break-even
point

THEN: the materiality judgment should be

22
based on net income.
Rule 3 IF: 1) the basis for making the materiality
judgment is known, and
2) the percentage rate for making the
materiality judgment is known
THEN: the materiality level equals the
product of the percentage rate times
the base used to make the materiality
judgment.
In the next section, these rules will be used to illustrate

the various control strategies for production systems.

Inference Engine

 

The inference engine is the control strategy for guid-
ing the selection and execution of particular rules.‘This
control strategy can be described as a recognize-act cycle.
This strategy can be implemented as either a forward- or a

backward-chaining reasoning process.

Forward-chaining. In a forward-chaining or data-

 

directed strategy each recognize-act cycle begins by compar-
ing the facts that.describe the current state of the world
to the situation part of each production rule. Each rule
which has its conditions for firing satisfied is placed into
a conflict resolution set. After the entire set of pro-
duction rules has been so examined, one of those in that set
is selected and fired. The method of selection is called the

conflict resolution rule. The firing of the rule constitutes

23

the action part of the cycle. An inference is made, and the
statezof the world is changed. If that change has resulted
in the goal being attained, the system stops and reports
success. If not, the entire cycle begins again with every
rule being compared to the new set of facts describing the
world. Lfru>rules have their conditions for firing satis-
fied on a particular cycle, the system stops and reports
failure.

The following example, using the three rules presented
earlier, illustrates how this forward-chaining process leads
to a particular decision. When information is obtained that
the client is a public entity and that there is no concern
about liquidity or solvency,1nflral would fire.Its firing
would add to working memory the assertion that the users of
the financial statements are primarily interested in the
results of current operations. This would satisfy the first
premise clause of rule 2. Upon receipt of information that
net income is above the break-even point, rule 2 would fire.
Its firing would result in the assertion that the materi-
ality judgment should be based on net income. At this point,
clause 1 of rule 3 would be satisfied; as soon clause 2 is
satisfied, rule 3 would fire and the materiality judgment

would be made.

Backward-chaining. Sula backward-chainingcn:goal-

 

directed reasoning strategy each recognize-act cycle begins
by examining the action part of the production rules in the

knowledge base.(kflqrthose rules whose firing will attain

24

the current goal are checked to see if their situation part
is satisfied. If any such rules are found, they are fired
and the system stops and reports success. If the rules that
will attain the current goal do not have their situation
part satisfied, then a new set of sub-goals for satisfying
those conditions is established. Each rule in the production
memory is then checked to see if its firing would alter the
current state of the world and attain the current sub-goals.
Those rules are then checked again to see if they can fire.
This backward-chaining continues until either some rule is
finally found which can fire or no such rule is found. In
the former case, firing that rule leads to a chain of
firings that eventually attains the goal; in the latter
case, the system stops and reports failure.

The set of three rules presented earlier can be used to
illustrate this backward-chaining reasoning process. The
system begins with the goal of making a judgment of materi-
ality. Scanning the action parts of the three rules reveals
that firing rule 3 would satisfy this goal. Consequently,
the premises of rule 3 would be matched against the contents
of working memory to ascertain whether they are satisfied by
the facts known about the current situation. Neither premise
is likely to be satisfied at the start of a consultation, so
the verification of each of rule 3’s premise clauses would
become the new subgoals. The system would then scan the
action parts of the remaining rules to see if the firing of

any of them would satisfy the current subgoals. This would

25

reveal that firing rule 2 would establish the appropriate
base to be used for making the materiality judgment. Both of
rule 2’s premises would then be checked to see if they were
satisfied by the facts and assertions stored in working
memory. The process would eventually lead the system to try
to establish the premises for rule 1 so that the first
premise of rule 2 could be satisfied.

It should be clear that with either type of control
strategy the production system architecture exhibits behav-
ior that is responsive to the facts of the case at hand.
Although the set of production rules is fixed, the particu-
lar set of rules that fires on any session, and the order in
which they fire, depends upon the facts that describe the

current consultation.

Working Memories

 

The working memories function as a scratchpad to keep
track of the goals being sought and the current state of
progress toward their attainment. The set of facts that
describes the world and which is compared to the situation
part of each rule is contained in these working memories, as
are any sub-goals that have been created.

Now that the basic features of the production system
architecture have been described, it is time to discuss the
theoretical paradigm underlying the use of that architecture

as a means of studying decision making behavior.

26

The Information Processing Paradigm

 

The information processing paradigm focuses attention
on the internal cognitive processes that underly observable
behavior. Young [1978] contrasts this approach to the func-
tional or behaviorist paradigm:

Unlike a functional approach which tends to ask
questions about the effect of various controllable
factors on certain gross measures of the subject’s
performance. .. the information processing
approach prefers to ask questions directly about p93
the [subject] is carrying out the task, seeking an
answer in terms of the psychological processes and
representations that underlie his behavior [pu360].

 

Newell and Sinmnfs [1972] study of one basic cognitive
activity -- problem solving -- led to the development of
many of the essential ideas in the information processing
paradigm. Their theory of human problem solving behavior is
illustrated in Figure l.

The figure shows that the problem solver forms an
internal representation of the task that reflects how he or
she perceives the task. It is this subjective interpretation
of the task, rather than the objective statement of the
problem, that governs all subsequent cognitive activity. The
problem solver possesses a store of methods for solving var-
ious tasks and also a store of factual knowledge. The basic
cognitive activity consists of a search through these stores
for knowledge that can be applied to solve the problem.

Thus, problem solving behavior is basically a function
of three variables: (1) the task environment, (2) some basic
structural characteristics of the problem solver, and (3)

the specific knowledge possessed by the problem solver. The

27

 

TASK ENVIRONMENT

Problem SIJlg‘IllCnK

 

Affect

cmimnmcm P11081131 SOLVER

 

 

 

 

Internal Representation r—+—%

 

Change
rcprcsc HI :I l i( m

 

 

Sch't‘l
mvllmd

 

 

 

 

[mt-mu!
gencldl
[;!1«_)\A'|1'(]g1‘

Mrllmd
store

 

 

 

 

 

 

 

 

 

Nulc: lhc cyc 4 indium that input rcpxcwnlatnui
|\ not umlcx umlml Hf inputting pmu‘ss

 

 

 

Figure 14 Theory of Problem Solving Behavior

Sourcez' Newell and Simon (1972. p.89)

28
role of each of these elements is discussed in more detail

below.

Effect pg the Task Environment

 

The effect of the task environment on decision making
behavior has long been recognized [Brunswik, 1955a,l955b].
The task environment includes both the structural properties
of the task and the specific content of the task [Einhorn
and Hogarth, 1981; Payne, 1982]. Normative theories get
their generality by focusing on the former and ignoring the
latter. But Newell and Simon’s theory of problem solving
indicates that it is the problem solver’s subjective percep-
tion of the task that governs behavior. A great deal of evi-
dence suggests that the content of the task plays an impor-
tant role in shaping this subjective representation. Thus,
task content may have an even greater effect on behavior
than do the formal properties of the task.

For example, Einhorn and Hogarth [1982] found that
changes in the composition of the set of alternative hypo-
theses that did not alter the objective probability of that
set did produce a change in its subjective probability. In
one experiment, they asked subjects to decide what foreign
language was spoken by a group of robbers. Subjects in one
condition were told that four eyewitnesses said that they
had heard the thieves speaking in German and four others had
said that it was Italianu Subjects in the other condition
were told that four eyewitnesses said it was German, two

said it was French, one said Spanish, and one Italian. Note

29

that in both conditions one-half of the eyewitnesses said
that they had heard the robbers speaking in German. Yet, the
subjects in the second condition were were much more certain
than were the subjects in the first condition that the lan-
guage was German.

Changes in task content can directly affect the level
of performance. Adelman [1981] found that realistic task
content facilitated subjects’ ability to learn from outcome
feedback. He had subjects predict grade point averages. When
given only abstract cues (e.g., Cue 1, Cue 2) outcome feed-
back was not very helpful. But when the cues were correctly
labeled (e.g., expectations for academic achievement) sub-
jects were able to use outcome feedback to markedly improve
their performance. Similarly, subjects"performance on a
task involving syllogistic reasoning markedly improved when
the syllogisms included realistic causal relationships with
which the subjects were familiar [Cox and Griggs, 1982; Hoch
and Tschirgi, 1983].

Task content can also hide formal task properties, so
that subjects cannot apply previously learned strategies.
The ease with which problems that have identical formal
properties can be solved varies with the particular content
of the task [Newell and Simon, 1972; Hayes and Simon, 1977].

Changes in task content induced by altering the way in
which the problem is worded can.even leadiuba.reversal in
preferences [Kahneman and Tversky, 1979; Tversky and

Kahneman, 1981]. Apparently, small changes in wording

30

dramatically alter the meaning of the choice. Tversky and
Kahneman [1981] provide a graphic illustration. They asked
subjects to choose one of two strategies for dealing with a
flu epidemic that was expected to kill 600 people if nothing
was done. One wording of the choice was as follows: Option
one will save 200 peOple, while option two has a one-third
chance of saving all 600 but a two-thirds chance of saving
no one.‘The other wording was: Option one will result in the
deaths of 400 people, while option two has a one-third
chance of no deaths but a two-thirds chance of 600 deaths.
Many subjects reversed their choices when.the‘wording was
changed.

In summary, the content of the task appears to signif-
icantly affect the way in which the task is perceived, and
thus the ultimate behavior of decision makers. Since subtle
changes in the way in which a task is described can produce
major changes in behavior, it is important that the setting
of a task be as realistic as possible. A major advantage of
using expert systems to study decision making behavior is
that subjects are permitted to act in naturally occuring
decision settings. The means by which this is achieved is
explained 1J1 Chapter III’s discussion of tflua research

methodology.

Structural Characteristics 9; the Problem Solver

 

 

The structural characteristics of the problem solver
include: the size and nature of memory, the speed with which

memory can be accessed to either retrieve old knowledge or

31
to store new knowledge, and the nature of the basic cogni-
tive processes underlying behavior. Two facets of the latter
are of particular importance:(l) heuristic search as the
fundamental cognitive activity in problem solving, and (2)
the use of symbols to represent knowledge and the manipula-

tion of those symbols as the basic means of reasoning.

Heuristic search. Simon [1980] stated that heuris-

 

tic search is the principal mechanism underlying intelligent
problem solving behavior in both humans and computers“ To
say that search is heuristic means that it is guided by
knowledge about the particular task under consideration and
uses that knowledge to quickly focus in on the heart of the
problem. This is in contrast to a "blind", exhaustive
search wherein each possible action that can be taken is
tried until either one works or none do.

One reason for heuristic search is that for many
problems there does not exist any formal algorithm that is
guaranteed to produce a solution. Often there may not even
be a clearly defined criterion. Such problems are called
ill-structured or ill-defined problems. Judgments of materi-
ality are an example of an ill-structured problem. In con-
trast, inventory control is a well-structured problem with a
known algorithm (the EOQ formula) for its solution.

The problem solver is not left totally in the dark
when attempting to solve an ill-structured problenn Often
there is a good idea of what the important subgoals are, but

no fixed method for achieving them. For example, an auditor

32

may know that the determination of materiality involves
applying some percentage to a base. There just does not
exist any general formula that can be applied to all
clients. Instead, the auditor relies upon experience gained
from prior audits to determine what the appropriate base and
percentage is for this particular client. The production
system architecture underlying expert systems is well-suited
to capturing this type of conditional, pattern-directed
behavior.

Even when some general algorithm does exist, time con-
straints may preclude its use and make an heuristic search
necessaryujRaphael [1976] gives a good example of this in
the context of solving cryptarithmetic problems such as the
following:

BEST
: ELDE
MASER

The goal is to assign a unique digit to each letter so
that the equation is true. A blind, exhaustive search of all
possible combinations is certain to produce the answer, but
such a search would require considering 1,814,400 possible
assignments of digits to letters. This approach is not only
inefficient, it is clearly not very intelligent either.
Basic knowledge of arithmetic can be used to dramatically
reduce the size of the search space. For example, it is
obvious that the letter M must be assigned the digit one
becauseethat.is the largest carry that can be generated by

adding two single digits. That fact alone reduces the search

33
space by a factor of ten. (The answer to the problem is:
M=1, A=0, B=9, D=8, S=6, T=7, R=2, and E=5).

In summary, problem solving involves searching for a
way to solve the problem. To be effective, the search must
utilize knowledge about the domain acquired previously. The
use of that knowledge makes the search heuristic. The
production system architecture is well-suited for exhibiting

this type of behavior.

Reasoning by manipulating symbols. A basic tenet of

 

 

the information processing paradigm is that concepts are
represented by symbols and that reasoning involves the mani-
pulation of those symbols [Lachman, Lachman, and Butter-
field, 1979]. Symbols are not merely tokens that label some
concept; rather, they contain the essence of the concept and
permit that essential meaning to be.accessed whenever the
symbol is processed [Newell, 1980b]. For example, "Cash" is
a symbol, and whenever the problem solver processes that
symbol its meaning (e4L, that it is a monetary current
asset, is easily misappropriated, eth is available and
guides the processing. Task context influences behavior by
indicating which of these meanings is most relevant for the
given situation. Thus, when doing foreign currency transla-
tion the fact that Cash is a monetary current asset is most
salient and is retrieved first from memory. On the other
hand, when evaluating the quality of internal controls, the
susceptibility of cash to theft becomes more relevant.

Symbols can be grouped into symbol structures in order

34
to represent more complex ideas. For example, the following
set of symbols represents the current ratio: (Quotient
Current-Assets Current-Liabilities). New symbols can be
created, in turn, to represent these structures (e.g.,
Current-Ratio). In this way, the problem solver can react to
new situations and "learn".

Human behavior is characterized by its interruptibil-
ity and temporal correlations. That is, work on a task can
be temporarily suspended and resumed later without having to
start over from the beginning (within limits, of course).
Most behavior is also purposive; actions are taken in order
to accomplish something definite. Such behavior is said to
be goal-directed.flfimzexistence of goals accounts for the
temporal correlations of behavior: a sequence of actions has
a.<xnmnon purpose. Goals are also necessary for interrupti-
bility: they serve as reminders of what was being sought
after.

Goals can be built directly into the system or repre-
sented by means of symbols. Decision flowcharts are an
exampleeof the former approach: the problem with adopting
that approachiis that it limits the ability to adequately
respond to minor changes in the task:

Knowledge about the particular managerial situation
is implicit, not explicit, in a decision flow chart
... because the reasons for a particular branch-
ing are not available to the program, in general it
cannot make even simple deductions about thenn Thus,
unless the situation matches exactly a series of
branches in the flow chart, the program is helpless.
Its lack of underlying knowledge prevents it from

adjusting its approach to a variation of the mana-
gerial problem [Gorry and Krumland, 1983, p.215].

35

Representing goals in1flu2fornlof symbols overcomes
these problems. New symbols can be created to represent the
features in the task that differ from the "average" task,
and that difference can itself be represented symbolically.
A new subgoal then is to find some method of resolving that
difference. Consequently, Newell [1980b] argues that the
ability to manipulate symbols is a prerequisite for intelli-

gent behavior.

The Problem Solver’s Knowledge

 

Symbols are used to represent the problem solver’s
knowledge, the third major determinant of behavior in Newell
and Sinmnfis theory. In Figure 1, that knowledge was seen to
consist of a store of problem solving methods and general
factual knowledgeu.Researchers in.artificial intelligence
have long debated about the nature of that knowledge. In
particular, they have argued over whether expertise was due
to some powerful, general reasoning techniques or whether it
resulted from a large store of specific knowledge.

The neat view of AI [artificial intelligence]
assumes that a few edegant principles underlie all
the manifestations of human intelligence. Discovery
of those principles would provide the magic key to
the workings of the mind. flfimascruffy View is that
intelligence is a kludge: people have so many ad hoc
approaches to so many different activities that_ no
universal principles can be found. . . Every appli-
cation requiresailarge amount.of domain-specific
knowledge that cannot be shared with any other
application [Sowa, 1983, p.23].

 

 

Anecdotal evidence abounds to support the notion that
expertise is based on acquiring domain-specific knowledge.

Professionals typically specialize 111 some particular

36
subpart of their field in order to become recognized as
experts (e.g., EDP auditors, tax accountants, divorce
lawyers). In fact, it has been argued that professional
judgment in auditing consists of such heuristic rules-of-
thumb, or norms. Those rules are acquired through experience
and capture the commonalities across classes of audits:

[The absence of such commonalities] would imply that
every audit situation must be assessed in isolation
and without reference to any other past assessments
of materiality. If this were true, then it would be
impossible to acquire any judgment through experi-
ence» Such a View of completely unrelated audit sit-
uations seems unreasonable. Every decision on
materiality must surely be based on both (a) the
auditcu“s understanding of what the general level of
significance should normally be considered to be,
and (b) his assessment of any special conditions
creating unusual significance in the situation in
question. The second factor can hardly operate with-
out the first. . . . in making any such judgment the
auditor must start, at least intuitively, from some
norm [CICA, 1965, p.5].

There are solid theoretical arguments and empirical
evidence in support of this View that expertise is charac-
terized by the possession of domain-specific knowledge, and
that that knowledge is represented in the form of heuristic

rules. Each of these sources of support is discussed next.

The nature 9; expertise: theory. Card, Moran, and

 

 

Newell [1983] studied the acquisition of skills in computer-
ized text-editing. They concluded that the development of
expertise consisted of the acquisition of domain-specific
knowledge to guide the search for a method to solve the
task:

With little control knowledge, the problem solver
will wander about the problem space in search of a

37

goal state. . . but with training or experience in
doing the problem, the problem solver will acquire
knowledge for guiding the search and making it more
efficienta.. the problem solver may eventually
build up enough search control knowledge so that he
goes straight to the goal. . . as his search control
knowledge increases, he becomes more expert; and his
behavior changes from problem solving to cognitive
skill [Card, Moran, and Newell, 1983, p.365].

This view of the development of expertise is similar
to that found in Andersonfis [1982] theory of skill acquisi-
tion. Anderson postulated that skill involved the procedur-
alization of factual knowledge. That is, with experience the
factual knowledge of a domain becomes embodied in procedures
which can be used to perform the skill. Instead of having to
think about what tochaixxa particular situation, behavior
(with practice) becomes automatic. Anderson claims that this
is the reason that the verbal protocols of subjects often
become less verbose with practice.

Anderson goes on to argue that the refinement of
skills involves combining a sequence of simple productions
that achieve some action into one larger production; this
process is called composition. For example, the first time
that a neW'friend/S‘belephone number is dialed there is a
great deal of conscious attention directed to the dialing of
each digit. With practice, however, the number can be dialed
while carrying on a conversation with someone else. Again,
Anderson turns to evidence from verbal protocols. He asked
subjects to solve two-column proofs in geometry.lussub-

jects’skill increased with practice, the steps that were

verbalized became larger; instead of explaining every little

38
detail, they began to refer to the rule or principle that
was being applied.

Anderson [1982] and Card, Moran, and Newell [1983]
both explain that the development of expertise is based on
acquiring knowledge about the task through repeated prac-
tice. Practice reveals important relationships among vari-
ables in the domain” Knowledge of those relationships makes

it possible to get to the heart of the matter more quickly.

The nature 2E expertie: empirical evidence The

 

 

 

results of several empirical studies support the notion that
expertise is based on domain-specific knowledge. Chi, Felto-
vich, and Glaser [1981] studied how experts and novices
solved physics problems“ The main difference was that the
experts possessed and effectively used a large body of
knowledge about physics principles which the novices did not
yet possess. The verbal protocols of the experts contained
explicit references not only to the rules that were useful
for solving the problem, but also to the reasons for their
applicability in the examples. No such references were found
in the protocols of novices.

Hunter [1962] studied the ability of the British math-
ematician, A. C. Aitken, to perform prodigious feats of
mental arithmetic. He found that Aitken simply knew a lot of
facts about the relationships between various numbers.
Aitken also knew when certain rules were appropriate. For
example, when asked to decimalize a number, he immediately

thought of rules about factoring; when asked to find the

39
root of a number, different rules came to mind.

Chase and Simon [1973] tested the ability of novice
and expert chess players to recall facts from memory. Chess
boards were set up based on either a random assignment of
pieces or on positions from actual tournament play. Subjects
were allowed to look at each board for five seconds and were
then asked to reconstruct the positions of the pieces. There
was no difference between novices and experts in their abil-
ity to recall the random boards, but experts did much better
than novices for the realistic boards. The former result
indicates that there is no significant difference between
expert and novice chess players.in their ability'to recall
facts from their episodic (personal) memory. Therefore, the
superior ability of the experts to reconstruct the realistic
scenarios must be due to domain-specific knowledge about the
relationships between chess pieces that permitted them to
store and recall larger "chunks" of the scenarios. With the
random boards, experts could not use that knowledge and had
to simply try to memorize the positions of the pieces.

Empirical research on expert systems has led to simi-
lar conclusions about the nature of expertise. The construc-
tion of expert systems has involved extensive examination of
the problem solving behavior of experts in a variety of task
domains. That body of empirical observation has led to a
theorycnfexpertise basedcnithe following two tenets:(l)
expertise results from the possession of a large amount of

domain-specific knowledge rather than from any superiority

40
in general reasoning abilities, and (2) the most important
part of that domain-specific knowledge consists of heuristic
rules of thumb that contain knowledge about relationships
among domain variables [Brachman et a1., 1983; Feigenbaum,
1979; Hayes-Roth, Waterman, and Lenat, 1983; Stefik et al”
1983a]. It is this store of heuristic knowledge which under-
lies experts’ ability to quickly focus in on the important
aspects of a problem and which enables them to deal with

incomplete and inaccurate data.

Summary

Lenat [1977] summarizes the theory of cognitive behav-
ior that has been discussed in this chapter:

The View we choose is that Man is a symbolic infor-
mation processor. The theory is that sophisticated
cognitive tasks can be cast as searches or explor-
ations, and that each human possesses (and effi-
ciently accesses) a large body of informal rules of
thumb (heuristics) which constrain his search
[p.1093].

 

 

 

Decision making is seen as being a conditional process. The
choice of what information to acquire next and the evalua-
tion of that information are both influenced by the informa-
tion previously acquired. Researchers in artificial intelli-
gence (e.g., Hayes—Roth, Waterman, and Lenat [1983]) and in
accounting [Gibbins, 1984] argue that traditional mathemat-
ical techniques such as regression cannot be used to model
such conditional behavior.(Regression equations, for exam-
ple, imply that all relevant information is considered
simultaneously). Consequently, researchers in both cognitive

psychology and artificial intelligence have turned to the

41
use of the production system architecture to model this type
of conditional behavior. The next section discusses the
status oftfluaexpert systems builtcnuthat architecture as

theories of the expertise being modeled.

Expert Systems EE Psychological Theories

 

 

The production system architecture has been used by
cognitive psychologists to model a variety of intermediate
level tasks such as subtraction [Young and.CYShea, 1981] and
speech understanding [Newell, 1980a]. In contrast, research
in artificial intelligence has used that same architecture
to build systems that perform tasks typically performed by
experts (84L, medical diagnosis).‘This section discusses
the status of both types of systems as being psychological
theories of the task being modeled.

The Production System Architecture
§§.3 Theory 9: Cognitive Behavior

 

 

Newell and Simon [1972], who developed the essential
features of the production system architecture, argued that
itaccurately modeled the way the mind works:

We confesstxaa strong premonition that the actual
organization of human programs closely resembles the
production system organization.. . . We cannot yet
prove the correctness of this judgment, and we sus-
pect that the ultimate verification may depend on
this organizationfis proving relatively satisfactory
in many different small ways, no one of them deci-
sive [pp.803-804].

Subsequent research in both cognitive psychology and
artificial intelligence has used the production system

architecture to successfully model a wide variety of

42
cognitive behavior. One of the strengths of the production
system architecture is that it can be used to implement
theories at several different levels of abstraction. Young
[1979] lists three levels that are commonly used:

Firstly, with the least commitment in the architec-
ture, is what one might call the language level.
Here, PSs [production systems] are used.because of
their suitability as a language for expressing cer-
tain kinds of computations. The content of the PS
may express a psychological theory, but the notation
is chosen on grounds of its computational conve-
nience.. .. Secondly one can find a number of
studies which operate at what one might call the
rule level. At this level there is a commitment to
the psychological reality of rules, iJe. to the idea
that the cognitive ’program’ is structured as a PS,
and also perhaps to some aspects of the conflict
resolution. But this commitment does not necessarily
extend to the details of the architecture adopted.
. . . Lastly there is the immediate processor level,
at which the PS architecture is itself intended as a
theory of the structure of human cognitive pro-
cesses. The allowable actions on the righthand side
of the rules, for example, are taken to be the ’ele-
mentary information processes7 of the cognitive sys-
tem [p.43].

 

 

 

Research in cognitive psychology has tended to operate
at the rule level and the immediate processor level [e.gr,
Anderson, [1982,1983]; Newell, [1980a]; Young, [1978]; Young
and.CVShea, [1981]). Such work has led to discussions of the
mappings between features of the production system architec-
ture and the human mind. The method by which the control
processes direct behavior has been likened to the role
played by consciousness in directing attention [Ueckert,
1980]. The production memory stores the systemfis knowledge
and,therefore, is seen as filling the same role asthat of
human long-term memory. The role of the working memories in

holding the facts currently under consideration has been

43
compared to the role played by short-term memory [Newell,
1970, 1980a; Young, 1979].

Research in artificial intelligence, on the other
hand, has tended to operate at the language level or the
rule level. Certain features of expert systems violate well-
established properties of the human mind in order to attain
high levels of performance. For example, most expert systems
have extremely large working memories, even though the capa-
city of human short-term memory appears to be quite limited
[Miller, 1956]. In addition, expert systems frequently
differ from pure production systems in that they provide a
means of marking which rules have been used so that the line
of reasoning that was followed can be explained. This fea-
ture contributes to the usefulness of expert systems as
practical tools and also facilitates their use to study why
decisions are made in a particular manner.

The foregoing discussion should not be taken as imply-
ing that researchers in artificial intelligence are totally
unconcerned with the correspondence between expert systems
and human problem solving behavior. On the contrary, a dis-
tinction is made between computer programs which can perform
complex tasks but do so in a manner which does not even
remotely resemble that used by human experts and those pro-
grams which attempt to more faithfully reflect human problem
solving behavior. The former have been labeled "intelligent
artifacts" [Feigenbaum, 1979].

In order to be considered an expert system“ a CXJmputeI

44

program must meet these two criteria: (1) the program must
reason by manipulating symbols, and (2) the program must use
heuristic search [Brachman et al., 1983]. Thus, special
purpose programs, such as those that solve differential
equations by means of numerical analysis, are not expert
systems because they do not reason symbolically. Likewise,
chess-playing programs are not generally classified as
expert systems (in spite of the fact that they have achieved
levels of performance rivaling that of master players). The
reason is that most of those programs rely on an exhaustive
search of all possible moves, while there is evidence (Chase
and Simon, [1973]) that human chess experts only consider a
few moves but look at the ramifications of that move for
many levels ahead.

There is no "correct" level of abstraction at which a
theory should be cast; instead, the appropriate choice is
dictated by both the nature of the task being studied and
the motivations of the researcher. Pylyshyn [1981] and
Newell [1982] both discuss a level of abstraction at which
behavior is described in terms of the goals, beliefs, and
intentions that give rise to observed actions. At this level
of abstraction, behavior is viewed as being goal-directed
and purposive. The focus is on understanding the logic
underlying the performance of some task; there is no need to
deal with the mechanics of that process (e.g., the speed at
which facts can be recalled, the length of time they can be

remembered, etc.).

45

AUDITPLANNER E§.§ Theory 9:
Professional Judgment

 

 

AUDITPLANNER, the expert system designed to make judg-
ments of auditing materiality, operates at what Young [1979]
refered to as the-rule level. There is a definite commitment
to the psychological reality of rules and to the suitability
of the production system architecture for explicitly illus-
trating how those rules interact to produce behavior. No
claim is made, however, that the mechanics underlying AUDIT-
PLANNER’s behavior accurately reflect the psychological
processes underlying.auditors’behavior.Rather,timefocus
is on the content of the knowledge constituting professional
judgment, and on illustrating how that knowledge results in
particular decisions.

In other words, it is assumed that auditors possess
heuristic rules-of—thumb and that the contents of those
rules represent the essence of professional judgment. Those
rules explain the relationships between various characteris-
tics of the client being audited and the proper level of
materiality to use when planning the extent of audit proce-
dures. Indeed, the existence of such rules or norms as the
basis of professional judgment has been advanced before (the
earlier quotation from the CICA on pn36). The next chapter
discusses the process of acquiring such rules and implement-

ing them in an expert system.

CHAPTER III

RESEARCH METHOD

This chapter describes how AUDITPLANNER, an expert
system.for making judgments of auditing materiality, was
built. The chapter is divided into four parts. Part one
discusses EMYCIN, the software tool used to help build
AUDITPLANNER, and explains the reasons for its selection.
Part two describes how the initial prototype version of
AUDITPLANNER was built. Part three discusses the refinement
of AUDITPLANNER. Part four describes the procedure for tes-

ting the finished system.

Research Tool: EMYCIN

 

Research on expert systems has led to the development
of several software tools that are designed to facilitate
the construction of such systems. With the use of these
tools, researchers in fields outside of artificial intelli-
gence can now build expert systems as a means of studying
decision making behavior in a wide variety of task domains.
These software tools provide the basic functions needed for
managing and manipulating the knowledge base of expertise.
Consequently, by using such tools researchers in applied
fields of study can concentrate their research effort on
identifying and studying the expert’s knowledge, rather than

46

47

on the programming aspects of expert system construction.

This dissertation makes use of one such tool, EMYCIN
[Van Melle et al., 1981]. Figure 2 below illustrates the
role played by EMYCIN in the development of AUDITPLANNER. It
shows that AUDITPLANNER consists of two basic components:
(1) a knowledge base consisting of production rules which
represent the auditing expertise used to make judgments of
materiality, and (2) EMYCIN, which contains both the book-
keeping functions used to create and maintain the knowledge
base and also the inference engine used to manipulate those
rules. Thus, the research effort in this dissertation con-
sists of identifying the knowledge used to make materiality
judgments, expressing that knowledge in the form of produc-

tion rules, and then analyzing the content of those rules.

 

AUDITPLANNER

 

EMYCIN

- bookkeeping functions
- inference engine

 

 

 

 

KNOWLEDGE BASE

 

 

 

 

Figure 2. Role of EMYCIN in Construction of an EXPERT
SYSTEM

48
The reasons for choosing EMYCIN as the software tool
to be used to help build AUDITPLANNER are discussed next.
Following that discussion, the important features of EMYCIN

are described in more detail.

Reasons for Choosing EMYCIN

 

There were three reasons for choosing EMYCIN as the
software tool to be used in this dissertation: (l) EMYCIN
has been used successfully to build expert systems in a wide
variety of task domains, including accounting, (2) the basic
features of the task for which EMYCIN was originally created
are similar to those found in making judgments of auditing
materiality, and.(3) EMYCIN contains the functions that a
formal task analysis indicated must be present in an expert

system designed to make judgments of auditing materiality.

Previous successful use 2E EMYCIN. Van Melle et a1.

 

[1981] report that EMYCIN has been used successfully to
build expert systems that perform tasks in a wide variety of
domains, including the following: (1) to diagnose lung dis-
eases, (2) to interpret data about the geological character-
istics of oil wells, (3) to recommend when structural engi-
neers should use a costly and complex computer program for
structural analysis, and (4) to identify the causes of
telecommunications system failures. EMYCIN has also been
used in accounting to develop an expert system that performs
estate tax planning for individuals [Michaelsen, 1982]. The

evidence indicates that EMYCIN is a very versatile software

49

tool for building rule-based expert systems.

 

Similarity g; the original task for which EMYCIN was

 

designed and materiality judgments. A second reason for

 

choosing EMYCIN was that the task for which it was origin-
ally designed bears many similarities to the characteristics
of making judgments of materialityuIEMYCIN.istflmedomain-
independent core of the expert system called MYCIN [Short-
liffe, 1976]. In fact, EMYCIN stands for Essential MYQEN.
MYCIN is an expert system designed to diagnose infectious
blood diseases; EMYCIN is MYCIN stripped of its knowledge
base of medical expertise.

One can draw an analogy between the diagnosis of a
patient by a physician and the examination of a firm’s fin-
ancial statements by an auditor. The physician asks the
patient about the presence and nature of any symptoms. Then
the physician performs a series of tests to provide evidence
upon which to base an opinion about the patient’s state of
health. If those tests turn up evidence of a disease, the
physician then attempts to determine the cause of that evi-
dence, possibly by performing additional tests. The appro-
priate therapy depends upon the seriousness of the disease;
in some cases surgery is required, in other cases a simple
presecription drug is sufficient.

Similarly, the auditor begins the examination of a
firnVs financial statements by inquiring about any important
events that have occurred and about any future plans that

the client may have. The auditor then designs a set of tests

50

and procedures that will provide evidence upon which to base
an opinion about the fairness of presentation of the finan-
cial statements. If those tests and procedures uncover evi-
dence of errors, the auditor then tries to determine the
cause of those errors. The appropriate response depends upon
the seriousness of the error. In some cases, the auditor may
have to give an adverse opinion on the financial statements,
while in other cases the item may not even require footnote
disclosure.

This analogy between the process of making medical
diagnoses and materiality judgments is further supported by
the similarities in the state of knowledge about each type
of judgment. Shortliffe [1976] describes the characteristics
of medical diagnosis:

It is unfortunately the case that most human disease
states are not sufficiently well understood to be
characterized by well-defined mathematical formulae.
Even causal relationships are seldom understood.
MYCIN is a program that attempts to use AI [Artifi-
cial Intelligence] techniques to model decision mak-
ing in ill-defined areas such as these..After all,
experts g9 reach decisions when such medical prob-
lems arise, and they can usually offer theoretical
arguments for making the judgments that they do. Our
goal has been to capture such judgmental knowledge
and to create a program that uses the information
effectively'and.in a*way that is‘acceptable to the
physician for whom it is designed [pn32].

Similarly, the causal relationships behind materiality
judgments are not understood well enough yet to permit the
establishment of any general quantitative guidelines. Never-
theless, auditors routinely make such judgments and often

document the rationale underlying those judgments in their

working papers.

51

Formal task analysis bf judgments bf auditing materi-

 

 

 

EliEY- A formal analysis of the task of making judgments of
auditing materiality was undertaken in order to reveal the
requirements that an expert system must meet in order to
make those judgments. This analysis was based initially on a
review of discussions about auditing materiality in the
authoritative literature. Those readings were later supple-
mented by discussions of materiality in the audit manuals of
public accounting firms and by interviews with members of
several of those firms.

The first standard of fieldwork requires auditors to
adequately plan the audit engagement. lui important part of
such planning involves determining the extent and nature of
the audit tests and procedures that need to be performed.
Both SAS Nos. 22 and 47 require auditors to consider their
judgments of materiality when making that determination.

SAS No. 22 discusses the types of information upon
which these planning judgments are based. That information
includes knowledge about (1) the client/s type of business,
capital structure and form of organization, (2) the industry
in which the client operates, (3) the general economic
conditions facing the client,anui(4) the intended purpose
of the financial reports being audited.

SAS No. 47 points out that these judgments are based
on the auditor’s perception of the needs of the users of the
financial statements. It also notes that the judgments are

based on uncertain information that is subject to revision.

52

In some situations, the auditor considers
materiality for planning purposes before the finan-
cial statements to be examined are prepared. In
other situations, his planning takes place after the
financial statements under examination have been
prepared, but he may be aware that they require sig-
nificant modification. In both types of situations,
th auditor’s preliminary judgment about materiality
might be based on the entity”s annualized interim
financial statements or financial statements of one
or more prior annual periods, as long as he gives
recognition to the effects of major changes in the
entityks circumstances (for example, ea significant
merger) and relevant changes in the economy as a
whole or the industry in which the entity operates
[AICPA, 1983, para. 14].

The end result of the auditor’s consideration of
materiality is an estimate of the level of errors which, if
found, would diminish the fairness of presentation of the
financial statements.SAS No.47’states thatsnxfllan esti-
mate may or may not be explicitly quantified. Discussions of
auditing materiality in the audit manuals of public account-
ing firms indicate that those judgments are usually quanti-
fied explicitly whenever statistical sampling is going to
play a major role in the audit. In such cases, the materi-
ality judgment is one factor that is used to set the preci-
sion level of audit tests. That precision level is then used
to help establish required sample sizes.

In summary, the task analysis indicates that judgments
of auditing materiality require the auditor to interpret a
great deal of qualititative and quantitative information
about the client;in¢order'to determine the likely needs of
the users of the client/s financial statements. Some of the

information used to make this judgment is incomplete and

subject to revision (e4L, interim financial statements),

53
and some of it is inherently uncertain (e.g., the perceived
level of risk associated with the audit). Consequently, an
expert system designed to make judgments of auditing materi-
ality must be able to handle information possessing these
characteristics. Further, because the judgments inherently
involve important assumptions (egn, about the needs of the
users of the statements), the system should be able to
explain the basis for its interpretation of the data and the
nature of any assumptions that it made. Indeed, most public
accounting firms require such documentation of the materi-

ality judgments made when planning the audit.

EMYCINWS satisfaction RE the reguirements 2i the for-

 

 

 

mal task analysis. EMYCIN, as mentioned earlier, was

 

designed to perform diagnostic tasks. Stefik et a1. [1983]
discuss the properties of the various tasks performed by
experts. They point out that a major component of diagnosis
is the interpretation of data. They then describe the
features that an expert system must possess in order to
interpret data.

The main requirement of the task is to find
consistent and correct interpretations of the data.
It is often important that analysis systems be rigo-
rously complete, that is, that they consider the
possible interpretations systematically and discard
candidates only when there is enough evidence to
rule them out. The key problem is that data are
often noisy and errorful; that is, data values may
be missing, erroneous, or extraneous. (1) This means
that interpreters must cope with partial informa-
tion. (2) For any given problem the data may seem
contradictory, so the interpreter must be able to
hypothesize which data are believable. (3) When the
data are unreliable, the interpretation will also be
unreliable. Thus for credibility it is important to

54
identify where information is uncertain or incom-
plete and where assumptions have been made. (4)
Since reasoning chains can be long and complicated,
it is helpful to be able to explain how the inter-
pretation is supported by the evidence [pn83].

EMYCIN contains functions to deal with partial infor-
mation, to reason under uncertainty, and to explain the line
of reasoning that led to its recommendations. Thus, EMYCIN
appears to be a logical choice as the tool to be used to
build an expert system that makes judgments of auditing
materiality. The next section describes the features found

in EMYCIN which enable expert systems based on it to perform

the functions listed above.

Features 2i EMYCIN

This section describes some of the structural features
of EMYCIN. The search strategy in EMYCINWs inference engine,
which ensures that its interpretation of the data is rigor-
ously complete, is discussed first. The methods used to rep-
resent the knowledge used by that strategy are discussed
next. Following that, EMYCINWs provisions for dealing with
uncertainty are described. The facilities which allow EMYCIN
to explain the line of reasoning that it followed to make
its recommendations are discussed later, in the section that

discusses the refinement of the system.

Search strategy. EMYCIN-based expert systems follow

 

a goal-directed or backward-chaining search strategy. Thus,
AUDITPLANNER begins with the goal of making a judgment of

materiality that will be used to plan the scope of the

55

audit. Those rules which, if fired, could conclude about the
materiality level have their premises examined to see if
they are satisfied by the data known about the client. If
the data required to determine whether or not those premises
are satisified is not known, then AUDITPLANNER establishes a
new goal of obtaining that data. This causes it to examine
the conclusions of all the rules in its knowledge base to
see which can satisfy the new subgoals. In this way, a tree
of goals is created which represents the line of reasoning
used to make the materiality judgment.

The process of creating new subgoals and looking for
rules which can satisfy those subgoals continues until one
of three things happens. First, a rule is found which has
its premises satisfied by the data already known about the
client. In this case, that rule fires and the conclusions it
makes enables the next higher subgoal to be satisfied and so
on up the tree until a materiality judgment is made. The
second alternative is that a rule is found and it is known
that its premises are not satisfied by the known about the
client. In that case, the tree of goals cannot be satisfied
and AUDITPLANNER fails to determine a materiality level. The
third option is that no additional rules can be found that
conclude about the current subgoals. In that case, AUDIT-
PLANNER asks the user a question in order to determine
whether the premises of the rule it last considered are
satisfied. These questions typically involve objective data

such as the amounts of various financial statement accounts

56
or the type of business of the client.

Thus, AUDITPLANNER interacts with the user as would a
consultant.Tfimabackward~chaining search strategy ensures
that the questions asked of the user are focused, because
the sequence of questions follows the logical structure of
the goal tree. In contrast, a forward-chaining search stra-
tegy often produces a sequence of questions that reflects
more of a "shotgun" approach, because the answer to one
question may satisfy the premises of a rule about a quite
unrelated variable and thereby trigger a question about an
entirely different topic.

EMYCIN’s use of backward-chaining can not only be
defended on the basis of its effects on the interaction with
the user, but also in terms of overall performance. Aiello
[1983] compared the efficiency and effectiveness of several
different search control strategies by building different
versions of the same expert system. There were no signifi-
cant differences between the forward- and backward-chaining
strategies in terms of (1) the number of rules considered,
(2) the number of rules fired, or (3) the amount of data
examined. However, the backward-chaining strategy produced a
line of reasoning that users judged to be easier to follow
and understand.

The reader may question whether the use of a backward-
chaining strategy reflects the process used by auditors to
make materiality judgments. The discussion in Chapter 2

indicated that (1) there is some evidence that for tasks

57
involving deliberate planning thataibackward-chaining or
goal-directed strategy is probably followed, and (2) the
purpose of constructing AUDITPLANNER is bbb to try to model
the search process underlying judgments of auditing materi-
ality, but rather to model the role played by various situ-
ational factors in making those judgments. In other words,
the focus is on the content of the system, as reflected in

the production rules, and not on its architecture.

Representation 2E knowledge. EMYCIN-based systems

 

 

represent knowledge about a task domain in the form of
production rules. Such rules serve four distinct purposes :
(l) to make inferences, (2)tx>establish default values of
certain parameters, (3) to control the ordering of subtasks,
and (4) to describe the contextual setting in which some
goal is appropriate.

EMYCIN uses its backward-chaining search strategy to
implement functions 1, 3, and 4 described above. The rules
involved are called consequent rules, because their premises
are checked only if the actions accompanying their being
fired help to satisfy the current goal being pursued. The
second function, establishing default values, is implemented
by the use of antecedent rules. These rules fire in a
forward-chaining manner whenever data is obtained which
satisfies their premises. They are used to make definitional
conclusions and to avoid asking redundant questions. For
example, AUDITPLANNER contains an antecedent rule which

states that if the client’s main line of business falls into

58
one of several categories hag., manufacturing or retail
sales), then the client is definitely not a nonprofit organ-
ization.‘This rule prevents AUDITPLANNER from asking whether
General Motors is a nonprofit entity.

EMYCIN-based systems store factual knowledge about the
subject of a consultation in the form of fact triples such
as the following: (CLIENT TYPE-OF-ENTITY PUBLIC). The first
element is termed the context; it serves to associate facts
about an object in the domain. In this case, it states that
the fact represented here pertains to the client being
audited.(If the client is a nonprofit organization, some
attributes may only apply to certain funds or fund types; in
that case, the context variable would be the name of the
relevant fund). The second element of the triple is the name
of the attribute, in the example, the type of entity that
the client is. The third element represents the value of
that attribute; in this case, the client is a public entity.
These fact triples are formed in response to user answers to

questions and to the firing of production rules.

Reasoning under uncertainty. The task analysis indi-

 

cated that some of the information upon which judgments of
auditing materiality are based is inherently uncertain. For
example, the amounts of the items in interim financial
statements are used to estimate what the final, annual
amounts of those items will be. EMYCIN-based expert systems
deal with this type of reasoning by assigning certainty

factors.(CFs) to the facts about the consultation and also

59
to the rules used to manipulate those facts.

Shortliffe and Buchanan [1975] explain the mathematics
underlying the manipulation of CFs. CFs represent subjective
degrees of belief and are defined to be the difference
between the degree of belief in some assertion, given the
available evidence, and the degree of disbelief in that
assertion given that same evidence.‘This definition is simi-
lar to Einhorn and Hogarth%;[l982, 1984] concept of the net
strength of a hypothesis being the difference between the
weight of the evidence in favor of and against that hypo-
thesis.

CFs range in value from -1.0, representing absolute
certainty that the assertion is not true, to +1.0, which
represents absolute certainty that it is true. As mentioned
above, CFs are associated with both fact triples and with
the actions of rules. Thus, a CF of .9 attached to the fact
triple (CLIENT TYPE-OF-ENTITY PUBLIC) means that there is a
high degree of belief (but not absolute certainty) that the
client is a public entity. A CF of 1.0 attached to the
action of a rule means that whenever that rule’s premises
are satisfied the specified inference can be made with
absolute certainty.

A rule is fired in EMYCIN-based systems only if the
absolute value of the CF associated with its situation part
is at least equal tc>.2. The calculation of the CF of the
situation part of a rule depends upon the relationship among

the clauses that make up that situation. For example,

60
AUDITPLANNER contains the following rules:

Rule 40 IF: 1) the client has publicly-traded debt
or equity securities, or

2) the client has restrictive debt
covenants that are measured in
terms of results of current
operations

THEN: it is definite that the client is a
public entity. (1.0)
Rule 11 IF: 1) the client is a public entity, and

2) there is no concern about liquidity
or solvency

THEN: it can be assumed that the users of
the financial statements are primarily
interested in the results of current
operations. (1.0)

The first rule is a disjunctive rule: that is, it can
fire if either of its premises are satisfied. Therefore, the
CF attached to the assertion that is made whenlthe rule is
fired equals the product of the maximum CF of the premises
and the CF of the action. In this case, if the CF of premise
l was .7 and the CF of premise 2 was .9, the CF of the
resulting assertion would be:.9 (the maximum of the two
premises) times 1.0 (the CF of the rule/s action).

The second rule, on the other hand, is a conjunctive
rule: that is, both of its premises must be satisfied in
order to fire the rule. In this case, the value of the
resulting assertion equals the value of the minimum CF
attached to either premise times the CF of the rule. Thus,

if the CF of premise l was .7 and that of premise 2 was .9,

then the CF of the assertion would be .7 times lJL This

61

ensures that the degree of belief attached to an assertion
cannot exceed the degree of belief in the weakest link
supporting that assertion.

To ensure that the CF of an assertion never exceeds
1.0 in absolute value, EMYCIN uses the following method for
dealing with the situation where several rules fire and make
the same conclusion. The first rule to fire attaches a CF
with the assertion as described above. Any additional rules
which fire, however, serve only to reduce any remaining
uncertainty. For example, if the first rule makes an asser-
tion with a CF of AL and another rule fires which makes the
same assertion with a CF of .7, the combined CF after both
rules have fired is .8 + (.7 x (l- .8)) = .94.

EMYCIN-based systems will attempt to apply all rules
that conclude about a given assertion unless the firing of a
rule would lead to absolute certainty (CF = +1.0 or -1.0) in
the assertion, in which case only that rule is fired. Thus,
EMYCIN-based systems meet the requirement that interpreta-
tion systems rigorously examine all possible values of a
parameter. Moreover, the mechanics of the CF calculation
permit EMYCIN-based systems to deal with incomplete data in
an effective manner that results in only a gradual degrada-
tion in the quality of performance. If some of the informa-
tion discussed in the previous example was missing, so that
only one rule could fire, the system would still be able to
make a conclusion about the type of entity that the client

is (albeit with less certainty than if kxnfli rules had

62

fired).

Summary

The use of a software tool such as EMYCIN enables the
accounting researcher to use the expert system methodology
to study decision making behavior without having to devote
an inordinate amount of time to programming. EMYCIN has been
used successfully to build expert systems in a wide variety
of task domains. The task analysis indicated that judgments
of auditing materiality require auditors to interpret a
large body of uncertain information about a client. EMYCIN
contains the procedures necessary for a rule-based expert
system to perform tasks of that type. The next part of this
chapter describes how the initial prototype version of

AUDITPLANNER was built.

Construction 9E Prototype System

 

 

The process of acquiring expertise and encoding it
into the knowledge base of an expert system is referred to
as "knowledge acquisition" [Buchanan et al., 1983]. EMYCIN
permits most of this knowledge acquisition to be done inter-
actively with an expert. The expert uses the current version
of the system to perform the task, notes the areas where
there are gaps or flaws in the system’s logic, and suggests
changes or additions to the knowledge base to correct those
problems. This interactive process presupposes, however, the
existence of a working version of the system. This section

describes how the initial prototype version was built.

63

Choice 9E Subjects

A major constraint on the choice of subjects for this
dissertation is that EMYCIN, as well as other software tools
available for building expert systems, cannot accomodate
conflicting rules based on different judgment models. There
is considerable evidence that there is a lack of consensus
among auditors from different accounting firms for a wide
range of judgments, including materiality. Moreover, several
studies indicate that this lack of consensus also exists
among auditors from the same accounting firm. Because of
this apparent lack of consensus, and also due to the time
required to build and refine an expert system, AUDITPLANNER
is an expert system reflecting the rules-of-thumb used by an
audit partner of a single accounting firm. The remainder of
this section reviews the evidence of a lack of consensus
among auditors and also examines the role of individual-

specific models in the study of decision making behavior.

Lack 9E consensus across firms. Holstrum’s [1981]

 

review of empirical research on consensus in a wide range of
audit judgments lead him to conclude that there is a lack of
consensus among auditors for many of those judgments.
In general, the most crucial aspect oftfluaauditor
judgment research to date is the lack of consensus
among auditors in typical judgments made in the
audit process [Holstrum, 1981, pp.3l-32].
Previous research on materiality indicates that there

is a lack of consensus on these judgments. Pattillo [1976]

reports that the level of consensus among auditors regarding

64
the materiality of various errors was quite low, and in fact
was lower than that for any other group in his study which
included bankers, financial analysts, and financial execu-
tives. Firth [1979] also measured the consensus of auditors
in comparison to bankers, financial analysts, and chief
accountants and similarly found the lowest degree of consen-
sus among auditors. In fact, he reported that there was no
consensus among auditors in about one-third of the cases.
Lewis [1980] found significant differences among auditors
from four different public accounting firms in terms of the
threshhold at which they would disclose a contingent liabil-
ity arising from a lawsuit.
One possible cause of the lack of consensus in these
studies is that they all confounded the decision about the
materiality of the item with the decision about the proper
method of its disclosureu Indeed, in its Discussion Memoran-
dum (n1 Materiality, tflua FASB explicitly distinguishes
between these two judgments:
The determination of the materiality of an item,
transaction, or situation is just one step in the
financial accounting and reporting process. For
example, in considering the disclosure of litiga-
tion, a determination that a suit is material is
only part of the process. A judgment must also be
made as to the type and extent of disclosure to be
made in the circumstances, considering the degree of
materiality involved [FASB, 1975, para. 169].

Thus, it is possible that the reported lack of consensus

found in the studies cited above may be due to (l) a lack of

consensus about the materiality of the item , (2) a lack of

consensus over the proper method of disclosing that item,

65
given that it is material, or (3) both of the previous
reasons.

A study by Ward [1976] provides some support for the
second explanation. He asked auditors to rank the importance
of twenty factors representing legal and technical aspects
of the audit environment and found a statistically signifi-
cant level of agreement on those rankings. However, he also
found a marked lack of consensus about the impact of an
error on the auditor’s legal liability.

On the other hand, Messier [1982] did ask auditors to
make separate decisions about the materiality of a writedown
of inventory and its proper method of disclosure. He found a
lack of consensus among auditors from "Big-8" and "non Big-
8" accouting firms. Mayper [1982] asked auditors to evaluate
the materiality of of various internal control weaknesses.
He reported that the level of consensus across auditing
firms was only 45 percent.

The evidence for a lack oflconsensus across auditing
firms regarding judgments of materiality should not be sur-
prising. Einhorn [1974] points out that differences in back-
ground and training can lead to a lack of consensus among
experts. Each accounting firm has its own audit methodology
which is reflected in its audit manuals and in its training
programs. The Gibbins and Wolf [1982] survey found that one
factor which significantly affected all phases of the audit
was the firm’s general audit methodology. Cushing and

Loebbecke [1983] read the audit manuals of twelve public

66
accounting firms and classified each firm into one of four
categories reflecting the degree of structure reflected in
their written materials. The author of this study also
noticed distinct differences in the discussions of auditing
materiality in the audit manuals of ten different accounting

firms.

Lack of consensus within firms. Few studies have

 

 

examined the degree of consensus on audit judgments among
auditors from the same accounting firm. However, two studies
do report a lack of such consensus. The first is a study by
Mock and Turner [1979] about auditors’ievaluations of the
quality of internal control and their subsequent choice of
sample sizes for substantive tests. Although all auditors
were members of the same accounting firm, there were consid-
erable differences in their choices of sample sizes.

More important, the lone previous empirical study of
auditing materiality [Moriarity and Barron, 1979] found.a
lack of consensus in those judgments among partners from the
same accounting firm. Only two partners were in agreement on
a majority of the cases.

The differences found in the Moriarity and Barron
study probably reflect significant differences in the back-
ground of the participants. Some of the partners had had
experience primarily with profit-oriented firms, while
others had dealt mainly with nonprofit organizations. Post-
experimental interviews indicated that these two groups had

distinct differences in the types of problems they looked

67
for in an audit. Moreover, only two of the partners had had
much experience in statistical sampling applications where
this type of materiality judgment.is more frequently made
and explicitly quantified. In fact, the lack of such experi-

ence caused a sixth subject to refuse to perform the task.

Implications. It appears that there are significant

 

differences in the audit judgments made by members of
different accounting firms. There is also some evidence that
there is a similar lack of consensus in the judgments of
auditing materiality made by members of the same accounting
firm. This evidence should not be surprising. A basic tenet
of human information processing research istflmnzthe task
environment typically contains many items of information
which are redundant in nature; this redundancy, and the
adaptive nature of human behavior can produce a variety of
equally useful strategies for solving a particular problem.

We must expect to find different systems (even of

the same species) using quite different strategies

to perform the same task.]2an1not aware that any

theorems have been proved about the uniqueness of

good, or even best, strategies. Thus, we must expect

to find strategy differences not only between sys-

tems at different skill levels, but even between

experts [Simon, 1980, p.42].

The construction of an expert system which reflects
one strategy for making judgments of auditing materiality
may play an important role in understanding how those judg-
ments are made. Simon [1980] argues that progress in under-

standing problem solving behavior depends upon developing a

taxonomy of the alternative strategies for solving a given

68
task:

research on the performance of adaptive systems must
take on a taxonomic, and even a sociological aspect.
We have a great deal to learn about the variety of
strategies, and we should neither disdain nor shirk
the painstaking, sometimes pedestrian, tasks of
describing that variety. That substrate 2E descrip-
tion lfi ii necessary b9 BE BE the taxonomic sub-
strate has been b9 modern biology [p.42, emphasis
added].

 

 

 

 

 

 

This dissertation represents one step toward develop-
ing such a taxonomy for the task of making materiality judg-
ments. Moreover, Dukes [1965] points out that studies of an
individual subject in which the researcher is focusing on
how a particular problem is solved have played a major role
in psychological studies of behavior.

Problem-centered research on only one subject may,
by clarifying questions, defining variables, and
indicating approaches, make substantial contribu-
tions to the study of behavior. Besides answering a
specific question, it may (Ebbinghaus’ work, 1885,
being a classic example) provide important ground-
work for the theorists [Dukes, 1965, p.78].

The construction of AUDITPLANNER will be based on the
materiality judgments of an expert auditor in many different
situations. Previous research, as pointed out in Chapter
One, either held the situation in which materiality judg—
ments were to be made constant, or else only varied them
slightly. The use of the expert systems methodology: on the
other hand, permits for more complex and realistic decision
settings. Such variation of situations is essential if the
goal is to explain the role of that factor in making materi-

ality judgments. In this regard, this dissertation should

add to our knowledge of the materiality judgment process.

69

Subject selection. Eleven public accounting firms

 

with.offices.in Detroit were contacted in order to find a
subject for this study. Ten of those firms agreed to provide
either their entire audit manual or those portions which
described their approach toxnaking materiality judgments.
Reading those manuals indicated that there were considerable
differences across firms in terms<mf(l) how judgments of
auditing materiality were made, and (2) how those judgments
fit into the planning stage of the audit.

Interviews were arranged with managers or partners of
five firms whose manuals indicated that there was some
degree of structure in the firm’s approach to making those
judgments, but at the same time the judgment.had not been
reduced to a general formula applied across all clients
(Elliott [1983] provides an example of the use of such a
formula by one major firm).

The interviews indicated that one of the five firms
routinely made judgments of auditing materiality as part of
the process of establishing the precision levels for audit
tests. That firm’s audit manuals contained extensive discus-
sions of how those judgments were to be made, but did not
set forth a general formula that applied to all clients.
That firm was therefore chosen to serve as the subject of
this research.

The next step involved securing the cooperation of an
expert who would provide the rules that would be included in

AUDITPLANNER’s knowledge base. The sensitive nature of the

70
materiality judgment process meant that the subjectfis
cooperation could only be obtained upon a guarantee of
anonymity. However, some general demographic information is
permitted. The subject is a partner in a major international
public accounting firm. The firm would probably fall into
one of the two more structured categories in Cushing and
Loebbecke’s [1983] taxonomy. The subject has had a broad
range of experience in auditing both profit—oriented and
nonprofit organizations, with particular emphasis on finance

companies and health care firms.

Source 9: Knowlegge for Prototype

 

The initial set of rules included in AUDITPLANNER’S
knowledge base came from the discussions of materiality in
the audit manuals of the firm to which the subject belonged.
Those manuals contain the firnfis basic approach to making
such judgments and should reflect some of the techniques
used by the subject to make those judgments.

Textbook knowledge has been successfully incorporated
in EMYCIN-based systems which perform medical diagnosis [Van
Melle et al., 1981] and plan for individual estate taxes
[Michaelsen, 1982]. In fact, a large proportion of the rules
in the latter system were taken from directly from the tax
codeu.Moreovery because textbook knowledge underlies many
domains of expertise, a major topic of current research in
expert systems focuses on the development of text under-
standing systems which can acquire this knowledge directly

by reading the textbooks themselves [Buchanan et al., 1983].

71

Many of the "rules" or guidelines contained in the
audit manual were quite vague. Consequently, an initial
interview with the subject was arranged in order to clarify
the meaning of those rules.

The final result was a knowledge base which contained
enough rules to enable the system to make fairly accurate
materiality judgments for clients with characteristics that
matched those of the "textbook cases" found in the audit
manual. This was sufficient, however, to begin in earnest
the knowledge acquisition process. The expert could use the
prototype system to make materiality judgments for actual
clients, and in the process begin to fill in rules which
reflected the professional judgment acquired through experi-
ence. The details of this process are described in the next

section on the refinement process.

Refinement b; the System

 

 

The refinement stage constitutes the majority of the
time spent in developing an expert system. It is an itera-
tive process in which the expert uses the existing version
of the system and suggests changes that need to be to it.
The system builder then makes the recommended changes and
arranges for another session with the expert. The expert
then uses the revised version to make additional judgments
for actual clients, and may suggest additional changes. This
process took five months to construct AUDITPLANNER.

The question-answering and explanation capabilities of

EMCYIN greatly facilitated this process. At any time during

72
or after a consultation, the expert could examine the basis

for AUDITPLANNER’S recommendations.

Explanation b: Reasoning During 3 Consultation

 

 

 

EMYCIN provides functions which allow the expert to
interrupt the consultation at any point and examine the
logic being followed. In response to any question that the
system asks, the subject can refuse to answer and instead
type in the word."whym. An example of this interaction is
included in Appendix I; following that transcript may be
helpful in understanding the following description of the
process.

AUDITPLANNER responds to the word "why" by printing an
interpretation of what it thinks the user is asking, and
then displays an English translation of the current decision
rule which generated the most recent question the system
asked. The user may then either continue the consultation or
pursue the system’s line of reasoning further. To do the
latter, the user simply types "why" again. Notice that
AUDITPLANNER correctly interprets the meaning of the second
"why".

It is also possible to ask the system how it arrived
at any conclusion it mentions when it explains why it is
asking some question. To do this, the user simply types "how
Inn", where ndlrefers to the particular conclusion to be
explained. As the transcript indicates, AUDITPLANNERE:
initial response is to simply state which rule was used.

Although this may make sense to the system builder, the user

73
typically has to type in "how" again, in order to get an

English translation of that rule.

Explanation After 3 Consultation lfi Over

 

 

In addition to querying the system about its line-of-
reasoning during a consultation, the user can also examine
that line-of—reasoning after the consultation is over and
the system has displayed its recommendations. This can be
done in one of two ways.

The user can use the systenVs REVIEW feature to look
at each conclusion the system madeu Should the user disagree
with any conclusion, the cause of the error can be tracked
down by the use of the structured "how" and "why" questions.
This process can be guided by either the user or by the sys-
tem.

The structured REVIEW procedure only works for conclu-
sions with which the user disagrees. A different interaction
technique is used to examine the basis for conclusions with
which the user agrees. This is known as the Explanation or
QA mode, an example of which is presented in Appendix II.

As that transcript shows, the user can ask relatively
open-ended questions about how the system made various con-
clusions. EMYCIN provides functions which enable the system
builder to create a domain-specific dictionary which is used

to answer these type of questions.

74
Summary

The ability to track down the rules that led to cer-
tain conclusions is very helpful in increasing the systemfs
overall performance. The process often elicited additional
rules from the expert to deal with special exceptions that
may be present in the particular consultation being run that
were not adequately dealt with by more general rules. With-
out these question-answering and explanation features, the
refinement of the system would have been subject to the
expert’s ability to generate (l) the general rules which
failed in a specific situation, and (2) the appropriate
methods for dealing with that situation.

This iterative refinement process continued until the
expert was satisfied that AUDITPLANNER was performing satis-
factorily on the set of cases used to deve10p the system
(which occurred after five months of development). At that
time, AUDITPLANNER’s performance was formally tested and the
contents of its knowledge base analyzed. The procedures for

this testing are described in the next section.

Testing AUDITPLANNER

 

Davis.and Lenat [1980] point out that the refinement
of an expert system is a never-ending process, because it is
always possible to improve performance by adding more rules
either to cover additional facets of the task or to handle
changes that have taken place in the nature of the task
itself. Nevertheless, there must be some point at which this

iterative testing and refinement of the system is suspended,

75
its performance is measured, and the contents of its know-
ledge base are examined and analyzed. Indeed, the major
motivation behind the construction of AUDITPLANNER is the
analysis of the heuristic rules used to make materiality
judgments.

The previous section stated that the refinement of
AUDITPLANNER would be suspended when the system reached an
acceptable level of performance. However, defining "accept-
able performance" in making materiality judgments is a prob-
lem. It is not possible to simply compare the system’s
judgments to some objective standard and state that when the
system attains a certain percentage of correct answers, then
it is evidencing expertise.bk>such objective standard of
materiality exists.

How, then, can expertise be tested? There are no
generally accepted methods for evaluating the expertise of
auditors. Although auditors must pass an examination in
order to become certified, that test is not used later to
evaluate their expertise. This problem is not unique to
accounting; it is faced in many fields of professional
endeavor. Needless to say, the absence of any simple means
of testing and evaluating human expertise makes the evalua-
tion of expert systems difficult. Nevertheless, guidelines
do exist for evaluating expert systems.

Gaschnig et a1. [1983] point out that the appropriate
method for evaluating the performance of an expert system

depends upon its stage of development. Systems in the early

76
stages should be evaluated in terms of the correctness of
their general line of reasoning, while systems that are
approaching the stage of commercial release must undergo
more extensive testing of the "correctness" (or at least the
"acceptability") of their recommendations.

Gaschnig et a1. explain why they feel that systems in
the early stages of development should not be evaluated only
in terms of the "correctness" of their recommendations:

One problem with this emphasis on final performance
is that it fails to take into account the micro-
structure of problem-solving behavior, which can be
extremely important in permitting the extrapolation
from representative instances of behavior to make

judgments concerning overall competence. Evaluators
want to be convinced that the system is consistentLy

getting the right answers for the—Fight reasons
[p.252, emphasis added].

 

 

 

 

Gaschnig et a1. then describe a method of evaluating
expert systems which will provide such assurance:

the expert system needs to be exercised within a
wide-ranging series of test situations aimed at dis-
covering ways to make the system fail. The experts
engaged in evaluating system performance must have
full access to all aspects of behavior, so that they
can push and probe, looking for weaknesses and defi-
ciencies. This would seem b9 rule out blinded, com-
parative studies ii 32 appropriate framework for
expert-system evaluation, 3b least Eb the early
bbgggb 12 the development life cycle [p.252,
emphasis addedII

 

 

 

 

 

 

 

Consequently, the evaluation of AUDITPLANNER consists
of two separate procedures. First, the expert auditor who
served as the source of the expertise embedded in the system
was asked to determine whether the system is adequately per-
forming on the types of clients which have been used to test

it during the refinement stage. Although this evaluation is

77
subjective, it is similar to the manner in which the subject
evaluates the performance and expertise of junior auditors.

Once the expert felt that the system was performing
adequately'on the test cases, a more formal evaluation by
other experts was then made.‘This evaluation entailed using
the systenlto make materiality judgments for clients that
differ in a number of ways from those used to develop the
system. The purpose of the testing was to discover the
limits of the systenfis expertise. The focus is on identify-
ing areas where additional rules are needed if the system is
ever to be useful as a practical tool.

The use of these "outside experts" provides two other
benefits. First, it provides a general measure of the degree
generality of the problem solving strategy of the system.
The question is whether the rules used by one expert to make
materiality judgments are considered reasonable by other
experts. Second, it permits an assessment of the potential
usefulness of the system. In other words, do the experts see
any use for AUDITPLANNER?

It is important to stress that this evaluation by the
"outside experts" is not used to determine whether the
system is finished. Rather, it is intended solely to provide
additional data for analyzing the rules contained in the
knowledge base. It is worthwhile to reiterate at this point
the underlying purpose of this research: to increase our
understanding of how various environmental factors affect

materiality judgments. As Gaschnig et a1. [1983] point out,

78
it is not necessary to have a fully-developed, commercially
usable system in order to accomplish such a purpose:

Once a system begins generating performance, it
becomes an important part of the laboratory appara-
tus available to the knowledge engineer and cogni-
tive analyst to gain fresh insights into the domain
of expertise for which that system was built. The
true goal 9__f evaluation should not b_e Lo show how
well _a system doaes what i_t was designed Lo Lo b__Lut
rather, Lo gain agreater appreciation of the pro-
cessL structure, and limits 2L expertise. This sys-
tem can later be parlayed into new levels of expert
performance in successive system developments
[p.252, emphasis added].

 

 

 

 

 

 

 

 

The remainder of this section discusses the methods

and results of the testing of AUDITPLANNER.

Subjects

Three managers and three seniors in the same public
accounting firm as that of the expert subject participated
in the evaluation of AUDITPLANNER. Their cooperation was
secured by means of an office memo written by the partner
who served as the subject for this research.

The managers ranged in experience from six to nine
years, with a mean of eight years. The seniors had from
three and one-half to four and one-half years experience,
with a mean of four years. The seniors had at least one and
one-half years experience in making judgments of auditing
materiality.

The subjects also had fairly diverse backgrounds, with
areascfifspecial expertise including insurance companies,

savings and loans, and closely-held businesses.

79

Test Cases

 

Each participant was asked to bring the working papers
for two clients to the evaluation session. The clients that
were used in the evaluation were not a random sample;
instead, they were chosen in order to maximize the breadth
of situations over which AUDITPLANNER would be tested. This
was necessary in order to obtain an accurate assessment of
its overall level of competency and also in order to obtain
additional information about the generality of the knowledge
contained in the system. In addition, some of the test
clients had characteristics not present in any of the
clients used to develop AUDITPLANNER.flHmareason for this
was to provide another test of the system’s competency:
would it at least take a logical approach to new situations?
The exact charac-teristics of the test clients are listed in

Table 1.

Procedure

 

Each evaluation session lasted one hour, during which
time the participants used AUDITPLANNER to make materiality
judgments for their clients. After collecting demographic
information about the participant, the author provided
instructions on the use of AUDITPLANNER. The participant
used AUDITPLANNER to make a materiality judgment for one of
the test clients, compared that judgment to the actual
materiality level used for that client, and evaluated the
quality of AUDITPLANNER’S judgment. This process was then

repeated for the other test client.

TABLE 1. Characteristics of test clients

CLIENT

Machine tool manufacturer

Machine tool manufacturer

Insurance company
Restaurant

Automobile dealership
School district

Boy Scout Council
Computer manufacturer
Retail supermarket

Retail supermarket

Common carrier - trucking

Common carrier - trucking

Microcomputer retailer

FEATURES

involved in a major
acquisition

a subsidiary of a
foreign parent

a subsidiary

profitable

nonprofit organization

nonprofit organization

suffered a loss for
the current year

subject of litigation
and made some major
acquisitions

subject of an inquiry
by a regulatory agency

private entity under-
going incorporation

81
Upon completion of the one hour session, each partici-
pant was givenLa questionnaire with which to evaluate the
overall performance of AUDITPLANNER, as well as several
aspects of its usefulness. Participants completed the ques-
tionnaire in their own offices and returned it to the author
later that day. Appendix IV contains a copy of the question-

naire.

Results

AUDITPLANNER was used tornakelnateriality judgments
for 13 clients. (The last session lasted two hours and used
three test clients). Eight of those recommendations were
judged as being acceptable. In general, AUDITPLANNER usually
was more conservative than the auditors; that is, the
materiality levels it recommended were lower than those
recommended by the participants. The possible causes for
this conservatism are discussed in the next chapter’s analy-
sis of the system’s materiality judgment model.

The test sessions were useful as another form of
system refinement, as additional rules were suggested for
several new types of clients. Discussions with the principal
subject indicated that these rules should be added to the
system because they reflected expertise in areas outside of
the subject’s prior experiences.

Overall, the six managers and seniors said that they
felt that AUDITPLANNER approached the materiality judgment
in a reasonable and logical manner. Moreover, all six indi-

cated that they would like to be able to use such a tool if

82
it ever became available.
Table 22 presents a frequency distribution of the
responses to the questionnaire evaluation of AUDITPLANNER.

The remainder of this section discusses those results.

Competence _o_f AUDITPLANNER. Questions 7 and 22

 

 

addressed.the issue of AUDITPLANNERksgeneral competence.
Ffivtaof’the six participants disagreed with the statement
that AUDITPLANNER was not competent, and four of the six
agreed with the statement that it was competent. The one
subject who felt that AUDITPLANNER was not competent was
also the one subject who indicated that he would approach
the materiality judgment in a different manner than did
AUDITPLANNER. Discussions during the sessions indicated that
the subject had a consistently higher materiality threshhold
than did AUDITPLANNER.

Three of the subjects indicated that they would accept
AUDITPLANNER’S recommendations about the materiality level
to be used in planning the audit (question 8). Taken
together, the responses to these three questions indicate
that the evaluators felt that AUDITPLANNER showed evidence
of a basic level of confidence in making judgments of audit-
ing materiality, but that they were not yet ready to accept

its recommendations in every situation.

Reasonableness of model. Questions 24, 3, 15, and 19

 

addressed the issue of the generality and reasonableness of

AUDITPLANNERTS approach to making materiality judgments. The

83

TABLE 2. Responses to test questionnaire

 

QUESTION TOPIC s5 5 g 2 s9
7. AUDITPLANNER not competent O l 0 3 2
22. AUDITPLANNER is competent 0 4 2 0 O
8. Would accept AUDITPLANNER’S
recommendations 0 3 1 2 O
24. Would approach judgment
in different manner than did
AUDITPLANNER 0 1 0 5 0
3. AUDITPLANNER asked irrelevant
questions 0 0 0 3 3
15. AUDITPLANNER’S logic hard to
follow 0 0 0 3 3
l9. AUDITPLANNER’S logic easy to
follow 2 4 0 0 0
4. AUDITPLANNER would be useful
as a training device 2 3 l 0 0
18. Would not want subordinates
to use AUDITPLANNER as a training
device 0 0 1 4 1
5. AUDITPLANNER would be useful as
a decision aid (see note 1) l 4 0 0 O
6. Would want to use AUDITPLANNER
as a decision aid 0 6 0 O O
14. Would permit subordinates to
use AUDITPLANNER as a decision aid 0 5 0 O 1
ll. AUDITPLANNER would be more useful
as a training device than as a
decision aid 0 0 4 2 O

12. No conceivable use for AUDITPLANNER O O 0 4 2

21. Flow of dialogue easy to follow 2 4 0 0 O

84

TABLE 2 (continued)

§Aé§2§2
1. Easy to use 2 4 O 0 O
25. AUDITPLANNER jumped around from
topic to topic 0 0 0 5 1
13. Hard to use 0 O O 4 2
9. Did not enjoy using AUDITPLANNER 0 0 0 4 2
20. Enjoyed using AUDITPLANNER 1 4 l 0 0
10. AUDITPLANNER was too slow 0 O O 3 3
2. poor help facilities 0 0 2 3 l
17. inadequate question-answering
capabilities 0 l l 4 0
23. HOW and WHY features useful 2 l 3 O 0
16. Used question-answering features
extensively O 2 3 0 1
Key:
SA = Strongly Agree
A = Agree
N = Don’t know or neutral
D = Disagree
SD = Strongly Disagree
note 1 - one person did not answer question 5. His

responses to the other questions about usefulness were:
as a training device - agree
want to use as decision aid - agree
more useful for training than as decision aid - neutral
no use at all - disagree
permit subordinates to use as a decision aid - agree
would not let subordinates for training - disagree

85
responses indicate that the evaluators agreed with the logic
of the model. Five of the six evaluators indicated that they
approached judgments of auditing materiality in the same way
that AUDITPLANNER did. All Six indicated that AUDITPLANNER
did not ask irrelevant questions; and all six agreed that it

was easy to follow the system’s line-of-reasoning.

Usefulness of AUDITPLANNER. An important question in

 

 

building an expert system is whether there is any potential
use for it other than as a method for studying decision
making behavior; Questions 4, 5,Eh,ll, 12, 14, and 18 all
addressed this issue of usefulness. The responses to those
questions support the comments made during the sessions that
the evaluators would like to use a tool like AUDITPLANNER.
All six evaluators agreed with the statement that they would
like to use AUDITPLANNER as a decision aid. All agreed that
it would be useful as a decision aid (question 5) and five
of the six indicated that they would permit their subordin-
ates to use it as a decision aid.

Five of the evaluators indicated that they thought
that AUDITPLANNER would be useful as a training device. They
also indicated that they would not mind having their subord-
inates use it as a training device. The responses to
question 11 (whether AUDITPLANNER would be more useful as a
training device than as a decision aid) were inconclusisve,
although there was a slight bias in favor of its use as a

decision aid.

86

Ease of use and enjoyment. Several questions also

 

 

addressed the evaluators’ general attitude towards the sys-
tem in terms of ease of use and enjoyment (questions 1, 9,
10, 13, 20, 21, and 25). The responses were quite favorable.
Of particular interest is the uniformly favorable reaction
to the system’s sequencing of questions and flow of dialogue
(questions 21 and 25% All six evaluators agreed that the
system asked questions in a natural order that was easy to
follow and understand. This reaction is in agreement with
the findings by Aiello [1983] that users found the backward-

chaining control strategy easy to understand.

Adequacy of question-answering facilities. Responses

 

to questions 17 and 23 indicate that the evaluators felt
that AUDITPLANNER/s question-answering capabilities were
quite helpful. However, during the sessions most evaluators
made only limited use of those facilities, preferring to
just answer the systenVs questions and then examining its
recommendations. This subjective impression is supported by
the responses to question 16, which indicate that four of
the six evaluators felt that they did not make extensive use
of those facilities. Apparently, however, the little use

that was made generated a favorable response.

Summary. The six auditors who used AUDITPLANNER to
make materiality judgments for their own clients indicated
that they found the system easy and enjoyable to use. The

flow of dialogue was natural and easy to follow.

87

The evaluators indicated that they felt that AUDIT-
PLANNER exhibited a basic level of competence in making
judgments of auditing materialityu However, they were not
willing to always accept its recommendations. The major
source of disagreement centered on the proper materiality
threshhold to be used. Subsequent discussions indicated that
disagreements on this point are defensible differences of
opinion. Indeed, the evaluators did agree that the logic of
the model employed by AUDITPLANNER was reasonable and easy
to follow.

Finally, the evaluators were unanimous in their enthu-
siasm for the system and their belief that it would be
useful both as a training device and as a decision aid.
Several rules were suggested for dealing with clients out-
side the principal subjectfs primary areas of expertise and
a willingness to help in further development of the system
was expressed.

In conclusion, AUDITPLANNER was able to competently
and successfully make materiality judgments for clients with
which the partner who served as the subject of this research
had previous experience. The results of having six other
auditors from the same accounting firm use AUDITPLANNER to
make materiality judgments for their clients indicates that
the logic underlying AUDITPLANNER’s approach toward making
those judgments reflects the general approach used by that
accounting firm. The next chapter describes the nature of

that judgment process.

CHAPTER IV

ANALYSIS OF THE SYSTEM

This chapter examines the model of the materiality
judgment process that is represented in the production rules
of AUDITPLANNER. First, the setting in which those judgments
are made is described. This description explains the role
played by materiality in the audit planning process of one
public accounting firm. Then the method by which those judg-
ments are made is discussed. English translations of the
system’s rules are presented in order to illustrate exactly
how various qualitative factors enter into the materiality

judgment process.

The Decision Setting

 

Figure 3 illustrates the role played by judgments of
auditing materiality in the audit planning process of the
firm studied in this dissertation. The figure shows that
those judgments are used to set the precision level of the
audit tests. However, materiality is not the only factor
that affects the precision level. The clientfs service needs
and certain aspects of audit risk also influence the choice
of a precision level.

Figure 3 shows business risk as a factor that affects
materiality. Business risk represents risks to tjualauditor

88

89

Business Risk ;> Materiality

 

Client Expectations

Audit Risk A) Precision Level
/

 

\V
Sample Sizes

Figure 3. Causal Model of
Auditing Materiality

9O
hag., litigation) arising from the expression of an opinion
on the client’s financial statements. In response to such
risks the auditor may choose to reduce the materiality level
used to plan the extent of audit tests. Boatsman and Robert-
son [1974] found that business risk was a significant pre-
dictor of their subjects’ materiality judgments.

Figure 3 shows that the precision level is then used
to help determine sample sizes for audit tests. Audit risk
also affects the choice of sample sizes. Thus, Figure 3
illustrates the relationship between audit risk and materi-
ality that is discussed in SAS No. 47:

Audit risk and materiality, among other matters,
need to be considered together in determining the
nature, timing, and extent of auditing procedures
[AICPA, 1983, para. 1].

Audit risk is the risk that the auditor will conclude
that the financial statements are not materially misstated
when, in fact, material errors do exist. SAS No. 47 breaks
audit risk down into three components:(1) inherent risk,
which is the risk that material errors would occur in the
absence of a system of internal controls, (2) control risk,
which is the risk that any errors which do occur will not be
detected or prevented by the system of internal controls,
and (3) detection risk, which is the risk that the auditorks
procedures will fail to find any material errors that may
exist.

Figure 3 indicates that audit risk affects both the

precision level and the choice of sample sizes. The inherent

risk component of audit risk affects the precision level by

91

reducing that level for the amount of uncorrected errors
the auditor expects to find, based on previous audits of the
client. The control and detection risk components of audit
risk both directly influence the choice of sample sizes. The
level of control risk is derived from the assessment of the
quality of internal controls. That assessment is then used
to determine the level of detection risk that can be accep-
ted and still achieve the desired overall level of audit
risk. The actual sample sizes that will be used in audit
testing are then determined by means of a formula that uses
both the precision level and the level of detection risk.

In summary, judgments of auditing materiality are but
one factor used to establish the precision level of audit
tests. Those judgments of materiality are influenced by the
auditor’s assessment of business risk. Consequently, AUDIT-
PLANNER contains rules to deal with both materiality and the
evaluation of business risk. The remainder of this chapter
describes the logic of that judgment model.

The Judgment Model

 

Appendix III contains a sample consultation with
AUDITPLANNER. That transcript shows that AUDITPLANNER asks
the user for basic facts about the client being audited and
then uses those facts to recommend an overall materiality to

be used in planning audit tests.

Overview
Figures 4-7 illustrate AUDITPLANNERHS judgment model.

These figures are based on the data flow diagrams that are

Prior Year's Materiality
Levels

Financial Characteristics
of the Client

    
 
      
 
  
 
    

Nonfinancial Characteristics

of the Client
Future Plans of the Client#__.._————"':?

Nature of the Audit Engagement ‘,,r””””;7

Intended Uses of the Client's
inancial Statements

Calculate

  

Overall
Materiality
Level

Kitterlaiew Burirpnv

 

Figure 4. Overview of AUDITPLANNER'S
Judgment Model

93

Financial Characteristics
of the Client

‘\$$ Identify.

 

 

 

 

     
 

 

 
    

 

 
 

 

 

 

Nonfinancial Characteristics Materiality
of the Client >: Base
Future Plans of the Clientl,;?
Intended Use of the Client's
Financial Statements \_ Select
/' Percentage Percentage
Rate Rate
Nature of the Audityf/yyy,;7 2.0
Engagement
\/
\/

   

Prior Year‘s Materiality

Levels Calculation
of

Materiality

3.0

   

\/

 
    

Financial Characteristics
of the Client

  

\/

 

\ KQIIEIlenew finintpnv

Figure 5. Calculation of Overall
Materiality Level

94
Client's Industry

Form of Ownership““~“‘f>

Client's Capital Structure\
I

Nature of Debt Covenant549

 

 
    
   
 

 

Determination
of the
Type of Entity
1.1

  

Future Plans of Client4,r€7

Alllua
30 edfil

 

Identification
Financial Measures of of
Liquidity and Solvency \; User's Interests

 

/' 1.2

 

sisaiaiul
s‘iesn

Income from Continuing
Operation

yPaid-In-Capital ENE‘E‘7A
Prior Year's Inddm2--“7>

  
     
      
 

 

Choosing

 

 

the
Long Term Debt 9“~‘j% Materiality Materiality
Base Base 3

  

 

I
Current Assets _.———"' 3
Current Liabilities ,av" I

Figure 6. Identification of
Materiality Base

1.3 ’

 

95

   
 
     
   

 
 
  

 

Calculate
Materiality Base :> Tentative
Materiality
Percentage Rate \; Level
7 4.1

 

{sAaq

Airterxsiew enrieiual

 

\/

Materiality Base

  
 
 
      
   

  
 
 

Qgrrent Assets Test
Adequacy
Stockholder's Equity ‘ o
Materiality'
Level
4.2

Percentage Rate

\V vi, (,4,

Last Year's Materiality Level

 

‘\ Kitterxaiew Buritpnv

Figure 7. Calculation of Materiality

96
used in the structured analysis of information systems
[DeMarco, 1979]. The bubbles represent the processes or
decisions that are made during a consultation. The labeled
arcs represent the information that is used to make those
decisions.

Figures 4-7 represent the judgment model in increasing
levels of detail. Figure 4 provides a highly summarized pic-
ture of the entire process. It indicates that the following
information is used to calculate the overall materiality
level:

1. The materiality levels used in prior audits of the
client.

2. Various financial characteristics of the client.

3. Various nonfinancial characteristics of the client.
4. Future plans of the client.

5. The nature of the audit engagement.

6. The intended uses of the client’s financial
statements.

The financial characteristitmsof the client that are
used to calculate an overall materiality level include the
following:

1. Current assets

2. Current liabilities

3. Long-term debt

4. Paid-in capital

5. Retained earnings

6. Income from continuing operations

7. Prior years’ income

97

The nonfinancial characteristics include:

1. The industry of the client

2. The client’s capital structure

3. The nature of the client’s debt covenants

4. The nature of the client (is it a subsidiary)

The future plans of the client refer to any client
plans for the issuance of additional debt or equity securi-
ties. The nature of the audit engagement refers to whether
or not this is the initial audit of the client. The intended
uses of the client/s financial statements represent certain
types of business risks which affect the materiality calcu-
lation.

Figure 5 illustrates the major steps involved in this
calculation. Various financial and nonfinancial characteris-
tics of the client, together with information about the
client’s future plans, are used to select the appropriate
base for the materiality calculation (bubble 14H. Informa-
tion about the intended use of the financial statements and
the nature of the audit engagement is used to select the
percentage raterto use in calculating materiality (bubble
2.0). The percentage rate, the materiality base, prior
years’ materiality levels, and various financial character-
istics of the client are all used to make the actual materi-
ality calculation (bubble 3.0).

AUDITPLANNER uses the following production rule, known
as a goal rule, to ensure that each of these three major

activities is performed during the course of a consultation:

98

Rule _Q_(note: the numbers on rules are for reference
purposes only)

 

IF: l)Ii;is definite that the client isrunza non-
profit organization, and

2) Information has been gathered about the
appropriate base for the materiality calcu-
lation and whether there are any special
business risks which need to be considered
when planning the audit, and

3) An attempt has been made to deduce the
percentage rate used to determine the
materiality level, and

4) An attempt has been made to deduce the over-
all materiality level that will be used to
plan the extent of audit procedures and tests

THEN: It is definite that a judgment of auditing

materiality which will be used to plan the
extent of audit procedures has been made.

This rule is called a goal rule because its action
part satisfies the goal of the consultation: making a judg-
ment about the overall materiality level to be used in plan-
ning audit tests. (The reason for clause 1 is that the
procedure for dealing with nonprofit organizations differs
somewhat.fron1the procedure for profit-oriented entities.
Each fund or fund type of a nonprofit organization has its
own materiality level).

The attempt to satisfy the premises of rule 80 results
in the creation of a goal tree which orders the major sub-
tasks that have to be performed. First, AUDITPLANNER deter-
mines whether or not the client is a nonprofit organization.
If it is, then a different goal rule is tried. If not,

AUDITPLANNER then gathers background information about the

client in order to identify the appropriate base for the

99
materiality calculation.Next, information about any special
business risks is gathered and evaluated. That information
is used to select a threshold rate. The materiality base is
then multiplied by the threshold rate to derive an overall
materiality level.

The reader is reminded that this ordering of the
materiality judgment process was designed to produce a
consultation whose logic was easy to follow. No claim is
made that the subject necessarily follows the steps in the
sequence outlined above. For example, an auditor may assess
the risks associated with the audit prior to determining the
basis for the materiality calculation. Nevertheless, these
steps are all part of the process of making the materiality
judgment, and the rules used by AUDITPLANNER to perform
those steps do accurately reflect the way that the expert
subject makes those decisions.

The remainder of this section describes how each of
these three major decisions (represented by bubbles 1.0,

2.0, and 3.0 in Figure 5) is made.

Identification gf the Materiality Base

 

 

Figure 6 illustrates the steps involved in identifying
the appropriate base for the materiality calculation (bubble
1.0 in Figure 5). Three steps are involved. First, nonfinan-
cial client characteristics and the client’s future plans
are used to determine what type of entity the client is.
Then that decision and various financial measures of liquid-

ity and solvency are used to infer the primary interests of

100
the users of the client/s financial statements. Finally, the
inferred users’ interests and other financial characterist-

ics of the client are used to select the materiality base.

Determination f the type _f entity. Figure 6 shows

 

that the following information is used to determine what
type of entity the client is:
1. The client’s industry

2. Whether or not the client is a subsidiary
(form of ownership)

3. The client’s capital structure

4. The nature of any debt covenants

5. Future plans of the client
These factors are all mentioned in SAS No. 22 as being items
that should be considered when planning an audit. Subjects
in previous empirical studies also mentioned that they would
like to have such information upon which to base their
materiality judgments. However, neither the authoritative
literature nor previous empirical research has explained how
these factors influence materiality judgments. Figure 6
shows that they are used to help determine what type of
entity the client is, and this information in turn is used
to eventually select the appropriate base for calculating
the overall materiality level. Moreover, examination of the
rules used to determine the type of entity the client is
shows exactly Egg these qualitative factors influence
materiality.

AUDITPLANNER has seven rules which it uses to deter-

lOl
mine what type of entity the client is (bubble 1.1).

Rule 7

IF: 1) It is likely that the client may be a
private entity, and

2) The client is not filing with a regulatory
agency in preparation for the sale of its
securities in a public market, and

3) The client does not intend to "go public" in
the next two or three years

THEN: The client is a private entity.

ule 2

so

 

H

F: 1) It is likely that the client may be a
private entity, and

2) A: The client is filing with a regulatory
agency in preparation for the sale of
its securities in a public market, or

B: The client intends to go public in the
next two or three years

THEN: The client is a public entity.

 

 

Rule 4_

IF: 1) The client has publicly traded debt or
equity securities, or

2) The client has restrictive debt covenants

that are measured by or depend on periodic
financial statement amounts or ratios that
involve results of operations

THEN: The client is a public entity.

Rule _3

IF: The client’s main line of business or industry

classification is insurance
THEN: The client is a public entity.

Rule 5

 

IF: 1) The client does not have any publicly traded
debt or equity securities, and

THEN:

Rul

(D

 

 

2)

3)

l)

2)

3)

4)

5)

6)

3)

102

The client does not have any restrictive
debt covenants that are measured by or
depend on periodic financial statement
amounts or ratios that involve results of
operations, and

The client is not controlled by a public
entity

It is likely that the client may be a
private entity.

The client does not have any publicly traded
debt or equity securities, and

The client does not have any restrictive
debt covenants that are measured by or
depend upon periodic financial statement
amounts or ratios that involve results of
operations, and

The client is controlled by a public entity,
and

The client is a wholly-owned subsidiary, and

The principal external users of the client’s
financial statements are creditors or others
who are primarily interested in the client’s
financial position rather than in results of
Operations, and

The client’s parent considers it to be a
private subsidiary

It is likely that the client may be a
private entity.

The client does not have any publicly traded
debt or equity securities, and

The client does not have any debt covenants
which are measured by or depend on periodic
financial statement amounts or ratios that

involve results of operations, and

The client is controlled by a public entity,
and

103

4) A: The client is not a wholly-owned
subsidiary, or

B: The principal users of the client’s
financial statements are not creditors
or others who are more interested in the
client’s financial position than in
results of operations, or

C: The client’s parent does not consider it
to be a private subsidiary

THEN: The client is a public entity.

These rules represent the firnfs definitions of public
and private entities. Rules 40 and 56 define the clear-cut
situations. Rules 94 and 9S deal with the situation where
theeclient;is a subsidiary'of a public entity, and outline
the conditions that must be met in order for it to be con-
sidered a private entity. The transcript in Appendix III
indicates that the user is asked to supply the information
needed to establish the validity of the premises in these
seven rules.

Rule 53 is particularly worth noting, because it is an
example of the importance of information about the clientis
industry classification. The justification for this rule is
that the regulatory agencies which are among the major users
of insurance companies’ financial statements are interested
in their results of‘operations, regardless of the form of
ownership.

This rule illustrates one of the advantages of using
the expert systems methodology to study the materiality
judgment process. Rule 53 represents one way in which infor-

mation about the client’s industry can influence those

104
judgments. However, because there are many other variables
which intervene between this factor and the ultimate materi-
ality level, it is not likely to be a statistically signifi-
cant predictor of those judgments.

Gibbins and Wolf [1983] reported that plans for future
sales or financing by the client were an important factor
influencing the conduct of the audit. Rules 7 and 26 show
one way in which that factor affects materiality judgments.
If the auditor knows that a privately-held client is going
to become publicly-held in the near future, it is considered
to be a public entity. The rationale is that current owners
are likely to become shareholders and will have the same
interests as do the owners of publicly-held entities.

With these seven rules AUDITPLANNER is able to deter-
mine what type of entity the client is. Figure 6 shows that
this decision is then used as one of the factors that helps
to identify the primary interests of the users of the finan-
cial statements (bubble 1.2). The next section explains how

these interests are inferred.

Identification gf users’ interests. SAS No. 47 states

 

 

that " the auditor’s consideration of materiality . . . is
influenced by his perception of the needs of a reasonable
person who will rely<n1the financial statements "[AICPA,
1983, para. 6L.Figure 6 shows that information about the
type of entity the client is and financial characteristics
relating to measures of solvency and liquidity are used to

infer those needs. This decision is based on a model that

105

assumes that different classes of users (e4L, creditors,
investors, etc») have different needs and interests. Credi-
tors and owners are assumed to be more interested in mea-
sures of liquidity and solvency than in results of opera-
tions because the former more directly affect their claims.
On the other hand, investors are assumed to be more inter-
ested in results of operations than in measures of liquidity
or solvency, because the former more directly affect future
returns (dividends) on their investments.

Although several previous studies of materiality com-
pared auditors to various user groups (e4L, Boatsman and
Robertson [1974],Fdrth [1979]L,only Pattillo [1976] pro-
vides any data about the differences between user groups. He
reported that bankers used consistently higher materiality
thresholds than did financial analysts. This difference
existed across every decision setting that was used. These
results do indicate that there may be some differences in
the interests of different user groups.

AUDITPLANNER has three rules which it uses to infer
the primary interests of the users of the client’s financial

statements.

 

Rule 1_
IF: 1) The client is a public entity, and
2) There is not significant concern about the
client’s liquidity or solvency
THEN: It is definite that the principal external

users of the client’s financial statements
are primarily interested in the results of
current operations.

106

 

 

Rule 1_

IF: 1) The client is a private entity

THEN: It is definite that the principal external
users of the client’s financial statements
are primarily interested in the client’s
financial position.

Rule 42

IF: 1) The client is a public entity, and

2) There is significant concern about the

liquidity or solvency of the client

THEN: It is definite that the principal external

users of the client’s financial statements
are primarily interested in the client’s
financial position.

Rules 11 and 12 deal with the ordinary situation when
the client is profitable. Rule 42 deals with the situation
where a public entity’s continued viability is in question;
in this case, the interests of investors are assumed to
become the same as those of‘owners and creditors. Because
the judgment of auditing materiality precedes any detailed
analytical review, only a crude test of the client’s liquid-

ity or solvency is made by looking at the current ratio and

retained earnings.

Choosing the materiality base. Figure 6 indicates

 

 

that information about the interests of the users of the
client’s financial statements and various financial charac-
teristics of the client are used to select the base for the
materiality calculation (bubble 1.3). AUDITPLANNER has five

rules which are used to make this decision.

Rul

(D

 

H
m

THEN:

Rul

(D

 

H
”3

THEN:

50
C
....a
(D

 

H
'1]

THEN:

1)

2)

l)

2)

3)

4)

l)

2)

3)

107

The principal external users of the client’s
financial statements are primarily inter-
ested in the results of operations, and

Income from continuing operations is greater
than 0

It is definite that the materiality judgment
should be based on the amount of income from
continuing Operations.

The principal external users of the client’s
financial statements are primarily inter-
ested in the results of operations, and

Income from continuing Operations is less
than or equal to zero, and

Last year’s income from continuing opera-
tions is greater than 0, and

The number of times in the past three years
that income has been at or below the break-
even point is less than 2

It is definite that the materiality judgment
should be based on the trend in past
earnings.

The principal external users of the client’s
financial statements are primarily inter-
ested in the results of operations, and

Income from continuing operations is less
than or equal to 0, and

A: Last year’s income from continuing opera-
tions was less than or equal to 0, or

B: The number of times in the last three
years that income has been at or below
the breakeven level is greater than or
equal to 2

It is definite that the materiality judgment
should be based on the amount of owners- or
stockholder’s-equity

108

SD
C3
....1
(D
N

 

l) The principal external users of the client’s
financial statements are primarily inter-
ested in the client’s financial position,
and

H
"11

2) The amount of current assets is greater than
or equal to the amount of owners- or
stockholder’s-equity

THEN: It is definite that the materiality judgment
should be based on the amount of owners- or
stockholder’s-equity

w
(3
H
(D
b

 

H
u;

l) The principal external users of the client’s
financial statements are primarily inter-
ested in the client’s financial position,
and

2) The amount of current assets is less than
the amount of owner’s- or stockholder’s-
equity

THEN: It is definite that the materiality judgment

should be based on the amount of current
assets.

Rule 17 indicates that normally, investors are assumed
to be interested in the results of current operations. Rules
18 and 19 deal with breakeven situations. If current income
is at or below the breakeven level, a surrogate is used. If
the breakeven situation is unique (iJL, has not occurred
frequently in the past), rule 18 fires and materiality is
based on the trend in.past earnings. If,cn1the other hand,
the client has experienced several years of losses recently,
rule 19 fires and the materiality judgment is based on the
amount of stockholders equity.

These three rules provide one explanation of the

breakeven effect found by Moriarity and Barron [1979]. They

109

also account for the relatively low significance of the
trend in earnings as a predictor of materiality judgments.
Rules 18 and 19 indicate that this trend is only used as‘a
surrogate for current income when the breakeven situation is
relatively rare.]flfthe client has experienced several bad
years recently, attention shifts away from results of
current operations to measures of financial position.

Rules 23 and 24 deal with the materiality bases for
private entities. They indicate a risk-averse attitude where
the materiality base selected is the one that results in the
most conservative materiality level. This reflects the
general attitude toward riskcnfthe expert subject of this
study.

This completes the description of Figure 6. At this
point AUDITPLANNER has selected the base for the materiality
calculation. The next step is to choose the percentage rate
(threshold level) to be used with that base. This involves

the assessment of business risk.

Choosing 3 Percentage Rate

 

The causal model of the materiality judgment in Figure
3 indicated that business risk affected the calculation of
the overall materiality level. Figure 5 indicated that this
occurs through the effect of business risk on the choice of
a percentage or threshold rate for the materiality calcula-
tion,.AUDITPLANNER contains one general rule to guide the

collection of information about such risks.

110

71
C
H
(D

E

 

H
'13

Information has been gathered about

1. Whether the financial statements are going
to be used in connection with a public
offering of securities by the client, and

2. Whether the financial statements are being
used in connection with a transfer of
interests in the client, and

3. Whether the financial statements are being
used in connection with a contest for the
control of the client, and

4. Whether the financial statements are going
to be used to settle some litigation in
which the client is involved, and

5. Whether the financial statements are being
used in an inquiry by a regulatory agency,
and

6. Whether the client appears to be in
violation of any restrictive debt
covenants

THEN: The existence of any special business risks

that need to be considered when choosing a
percentage rate has been assessed.

This rule causes AUDITPLANNER to ask a series of
questions about the intended uses of the client’s financial
statements. The nature of these events may cause users’
materiality thresholds to be lower than otherwise. For
example, when the financial statements are being used in a
purchase of the client, the purchase price may be related to
the account balances in the financial statements. As a
result, users are going to be concerned about much smaller
errors than may otherwise be true.

Should any of the six questions be answered in the

affirmative, AUDITPLANNER asks additional questions to

lll
evaluate the degree of risk involved. For example, if the
client is being investigated by a regulatory agency, AUDIT-
PLANNER asks further questions about the reason for the
investigation.

AUDITPLANNER has ten rules which it uses to determine
the appropriate percentage rate to use in calculating the
overall materiality level. These rules reflect the expertks
views regarding the relative severity of different types of
situations representing business risk. Because business risk
refers to potential losses to the auditor, the expert who
participated in this study requested anonymity. Similar rea-
sons preclude the publication of the rules which relate
specific business risks to choices of materiality threshold
rates.

AUDITPLANNER uses the following rule for the situation
where there are no special business risks.

Rule 7

 

IF: 1) The financial statements are not going to be
used in connection with a public offering of
securities, and

2) The financial statements are not being used
in connection with a transfer of interests
in the client, and

3) The financial statements are not being used
in connection with a contest for the control
of the client, and

4) The financial statements are not being used
to settle some litigation involving the
client, and

5) The financial statements are not being used
in connection with an inquiry by a
regulatory agency, and

112
6) The client is not in violation of, and is
not likely to be found in violation of, any
restrictive debt covenants relating to the
results of continuing operations.

THEN: A percentage rate of 5 percent should be used
to calculate the overall materiality level.

Thus, this rule represents the expert’s default rule
for a "normal" audit. The expert indicated that the
accounting firnfs policy did not specify what this default
percentage rate had to be, but did recommend an appropriate
range in which might fall. Consequently, different auditors
in the firm may use different default rates, depending upon
their general attitude toward risk.

Figure 5 indicates that the nature of the engagement
also affects the choice of the percentage rate. The follow-
ing rule shows how this factor enters the judgment process.

Rule 1 1

 

IF: 1) This is the first time that we have audited
the client, and

2) A: This is the first time that the client
has ever been audited, or

B: The client’s previous auditors left
because of a dispute with the client

THEN: The materiality level needs to be reduced to
reflect the increased uncertainty associated
with an initial audit.

The next section discusses the rules which use the

percentage rate to calculate the overall materiality level.

Calculation 9f the Materiality Level

 

 

Figure 7 illustrates tun» AUDITPLANNER calculates the

materiality level to be used in planning the extent of audit

113
tests. The percentage rate representing the materiality
threshold is multiplied by the amount of the materiality
base. This produces a tentative materiality level. That
level is then tested to see whether it is either so small
that it would result inauiinordinate amount:of testing or
so large that most users would question its usefulness.

There are separate rules to deal with each situation.

Tentative level too small. The expert indicated that

 

too low of a materiality level would result in a level of
audit testing that would not be economically justified for
most clients. AUDITPLANNER has three rules for calculating
an alternative materiality level in such situations.

ul 0

:6
(D

 

H
"1:1

1) The materiality base used to calculate the
tentative materiality level is current
assets

THEN: Recalculate the materiality level using owners-
or stockholders-equity as the materiality base.

1

ON

Rule

 

IF 1) The materiality base used to calculate the
tentative materiality level is owners- or

stockholders-equity

THEN: Recalculate the materiality level using current
assets as the materiality base.

Rule 5

 

IF: 1) The materiality base used to calculate the
tentative materiality level is income from
continuing Operations, or

2) The materiality base used to calculate the
tentative materiality level is the trend in
past earnings

114

THEN: Recalculate the materiality level using owners-
or stockholders equity as the materiality base.

Rules 60 and 61 refer to the situation where the
materiality base is either owners-equity or current assets.
Earlier, rules 23 and 24 were shown to select the lower of
these two financial account balances as the materiality base
to use in calculating the tentative materiality level. Thus,
rules 60 and 61 serve to offset the effects of being too
conservative in the choice of a percentage rate.

Rule 65 fits into the general model of user interests
that was discussed earlier. If the materiality level relat-
ing to results of operations is too low, this means that
although the client has been profitable in the past, that
profitability was only marginal. Consequently, it is assumed
that in such situations investors are more interested in
measures of the client’s financial position than in results
of operations.

AUDITPLANNER has two other rules which check the ade-
quacy of the alternative materiality level.

Rule 9

 

IF: 1) The alternative materiality level is also
too low, and

2) A: The original materiality base was income
from continuing operations, or

B: The original materiality base was the
trend in past earnings, and

3) The alternative materiality level was calcu-
lated using owners- or stockholders-equity

THEN: Recalculate the materiality level using current
assets as the materiality base.

115
Rule 120

IF: 1) An attempt has been made to calculate an
alternative materiality level, and
2) The alternative materiality level is itself
too low
THEN: Set the overall materiality level at the
absolute minimum level that is economically
justifiable for any client.
These five rules illustrate how cost considerations
influence judgments of auditing materiality. They serve to
counterbalance the generally conservative approach taken by

AUDITPLANNER, yet do not deviate from the model of user

interests that is part of AUDITPLANNER’s logic.

Tentative level too large. The expert expressed a

 

belief that above a certain point users would regard the
absolute value of the materiality level as itself being
material. Two rules deal with this situation.

Rule 6

 

IF: 1) The tentative materiality level is too high
and

2) The tentative materiality level is greater
than or equal to the materiality level that
was used last year

THEN: Set the tentative materiality level to be equal
to last year’s materiality level plus 20 per-
cent of the difference between last year’s
level and the tentative materiality level.

Rule 7

 

H

F: l) The tentative materiality level is too high

2) The materiality level used last year is
greater than the tentative materiality
level calculated this year

THEN: Use the tentative materiality level to plan
this year’s audit.

116

These two rules once again reveal the conservative
risk attitude of the expert. If the client has a better year
than last year, the tentative materiality level will be
greater than thelnateriality level used last yearyiRule 66
serves to discount the effect of such "good news" events. On
the other hand, if the client had a worse year than last
year, the tentative materiality level is accepted. Thus,

rule 67 serves to accept "bad news" at face value.

Special Situations

 

 

At this point all of the bubbles in Figures 4-7 have
been explained. This section discusses two other situations
that have not yet been covered: (1) audits of nonprofit

organizations, and (2) audits of financial institutions.

Audits gf nonprofit organizations. AUDITPLANNER has a

 

 

different goal rule for the audits of nonprofit organiza-
tions. That rule serves to generate a materiality level
calculation for each major fund or fund type. The calcula-
tion, however, is still the result of multiplying the
materiality base by a percentage rate.

AUDITPLANNER has two rules for identifying the materi-
ality base of a fund or fund type.

Rule 2

 

IF: The fund balance is less than 20 percent of
the fund’s total assets

THEN: The materiality base is the total assets of the
fund.

117

Rule 3

 

The fund balance is greater than or equal to
20 percent of the fund’s total assets

H
"1

THEN: The materiality base is the general
expenditures of the fund.

These two rules reflect a model of the needs of the
users of the financial statements. Rule 73 assumes that the
users of the financial statements are primarily interested
in how the entity spends the money it has received. Rule 72,
however, assumes that if the fund/s future viability becomes
a matter of concern, that users’ interests shift to measures

of the fund’s financial position.

Audits gf financial institutions. Financial institu-

 

tions often have large portfolios (ME interest-bearing
investments.1fimasize of those portfolios is such that it
would not be economical to design tests of those portfolios
on the basis of the overall materiality level.'Therefore,
AUDITPLANNER contains rules which calculate a separate
materiality level for those portfolios.

Rule 7

 

IF: Stockholders-equity times the percentage
rate used to calculate the overall materi-
ality level is greater than the result of
dividing income from continuing operations
by the average rate of return on the
investment portfolio

THEN: The materiality level for the investment
portfolio is equal to the amount of income
from continuing operations divided by the
average rate of return on the investment
portfolio.

118

w
(3
H
(D
.b

 

H
'13

Stockholders-equity times the percentage
rate used to calculate the overall materi-
ality level is less than or equal to the
result of dividing income from continuing
operations by the average rate of return on
the investment portfolio

THEN: The materiality level for the investment

portfolio is equal to the amount of stock-
holders-equity times the percentage rate
used to calculate the overall materiality
level.

These rules again reflect a conservative bias. The
impact of the investment portfolio on the results of current
operations is compared to its impact on financial position,
and the separate materiality level is calculated for the
item which is affected most by the investment portfolio. The
existence of these rules is also another example of how the

client’s industry classification affects judgments of audit-

ing materiality.

Summary

This section described the model of the materiality
judgment process that is reflected in the rules used by
AUDITPLANNER to make those judgments. The calculation of the
overall materiality level was seen to be the product of
multiplying the materiality base deemed most relevant to the
users of the financial statements by a percentage rate which
reflects the degree of business risk associated with the
audit. The rules used to make these decisions were presented
in order to illustrate how various qualitative factors were

used to make the judgments” The effects of those factors are

ll9
summarized in Table 3.

This section has described the process of making judg-
ments of auditing materiality at a very detailed level. The
next section describes the materiality judgment process from
a more general perspective, focusing on what the detailed

analysis implies about the nature of that judgment process.

The Nature gf the Materiality Judgment Process

 

 

The purpose in building AUDITPLANNER was to obtain a
better understandingcflfthe materiality judgment process,
specifically the role of materiality in planning an audit.
The previous section presented the rules that reflect the
auditing expertise used to make those judgments. That analy-
sis indicated that the calculation of materiality involves
choosing an appropriate base for the calculation and then
multiplying that base by a percentage rate that represents
the appropriate threshhold given the intended use of the
financial statements.

These two major components of the materiality judgment
process have markedly different characteristics. The choice
of the appropriate base for the materiality calculation is a
function of the auditor’s perceptions of the needs and
interests of different classes of users of the financial
statements. Rules 11, 12, and 42 present the essence of the
model underlying those perceptions. These rules are based on
objective data about the client and represent the standard
policy of the accounting firm. Therefore, there is almost

total agreement among members of the firm concerning the

120

TABLE 3.Emfect of qualitative factors on judgments of
auditing materiality

QUALITATIVE FACTOR

 

Client’s industry

Capital structure:

lines of credit

debt

subordinated debt

shares outstanding
Type of entity (subsidiary)
Future plans of client
Events (e.g., major
acquisitions, new products)
Solvency/Liquidity measures
Profitability

Initial audit

Purpose and intended use
of the financial statements

Prior years’ materiality
levels

EFFECT
Choice of materiality base.
Need for additional materi-
ality levels.

Type of entity.

Choice of materiality base.

Choice of materiality base.
Materiality threshhold.

Choice of materiality base.

Choice of materiality base.
Choice of materiality base.
Materiality threshhold.

Materiality threshhold.

Overall materiality level.

121
appropriate base for the calculation of materiality.

The choice of the appropriate percentage rate to use
in calculating the overall materiality was seen to depend on
the auditom“s assessment of the degree of business risk pre-
sent for a particular audit engagement. The accounting firm
has a standard approach to determining whether any such
risks exist (see rule 47). However, the appropriate response
to those risks is determined by the individual auditor’s own
attitude towards risk. Thus, this component of the materi-
ality judgment process is highly idiosyncratic.

Exhibits I and II present the logic of the procedures
for choosing the appropriate materiality base and percentage
rate, respectively. Close examination of each Exhibit makes
the difference between the two component decisions readily
apparent. Exhibit I reflects a policy that can be uniformly
followed by all decision makers. Exhibit II highlights the
role of individual values in making materiality judgments.

This difference in the nature of the two major parts
of the materiality judgment process may provide an explana-
tion of the findings of a lack of consensus among auditors
regarding materiality. It is possible that auditors may
agree on the base for materiality judgments and on the types
of situations which represent a greater than normal degree
of business risk. However, because the response to those
risks is based on personal attitudes towards risk, there may
be disagreements about the appropriate percentage rate and

thus about the appropriate materiality level.

122

EXHIBIT 1. PROCEDURE FOR IDENTIFYING MATERIALITY BASE
(bubble 1.0, figure 5)

1. PROCEDURE FOR DETERMINING TYPE OF ENTITY
(bubble l.l, figure 6)

Select the policy which applies:

Case 1 (Client has publicly traded debt or equity
securities);
Client is a public entity.

Case 2 (Client has covenants on debt or lines of credit
that are related to results of operations);
Client is a public entity.

Case 3 (Client is not controlled by public entity);
If (client is not filing with regulatory agency
in preparation for sale of its securities in
a public market and does not intend to do so
in the next two or three years)
Then client is a private entity.
Otherwise,
Client is a public entity.

Case 4 (Client is controlled by a public entity);
Select the case which applies:

Case 4a (Client is not a wholly-owned subsidiary);
Client is a public entity.

Case 4b (Principal users of client’s financial
statements are interested in results of
current operations);

Client is a public entity.

Case 4c (Client’s parent considers it to be a
public entity);
Client is a public entity.

Case 4d
If (client is not filing with a regulatory

agency in preparation for the sale of its
securities in a public market and does not
intend to do so in the next two or three
years)

Then client is a private entity.

Otherwise,
Client is a public entity.

123
Exhibit 1. (continued)

2. PROCEDURE FOR DETERMINING PRIMARY INTEREST OF PRINCIPAL
EXTERNAL USERS OF CLIENT’S FINANCIAL STATEMENTS
(bubble 1.2, figure 6)

Select the case which applies:

Case 1 (client is a public entity)
If (there is no concern about the liquidity or
solvency of the client)
Then assume primary interest is in results of
current operations.

Otherwise,
Assume primary interest is in financial position.

Case 2 (client is private entity)
Assume primary interest is in financial position.

3. PROCEDURE FOR SELECTING MATERIALITY BASE
(bubble 1.3, figure 6)

Select the case which applies:

Case 1 (Primary interest is in results of current
operations)
If (income from continuing operations is above
the breakeven level),
Then materiality base is income from
continuing operations.
Otherwise,

If (last year’s income was above the break-
even point and there losses in no more
than one of the prior three years)

Then materiality base is the trend in
past earnings.

Otherwise,

Materiality base is stockholders equity.

Case 2 (Primary interest is in financial position)
If (current assets is greater than or equal to
stockholders equity)
Then materiality base is stockholders
equity.
Otherwise,
Materiality base is current assets.

124

EXHIBIT 2. PROCEDURE FOR SELECTING MATERIALITY THRESHOLD
(bubble 2.0, figure 5)

1. Determine if there are any intended uses of the client’s
financial statements which may indicate the presence of
special audit risks.

2. For each such risk identified in step 1, obtain
additional information to determine the seriousness of
the situation.

3. Choose percentage (threshold) rate.
Select the case which applies:

Case 1 (no special audit risks);
Use your default materiality threshold.

Case 2 (special audit risks exist);
Reduce default materiality threshold by an
amount appropriate to the level of risk, given
personal risk attitude.

125

Indeed, previous empirical research on materiality
judgments found that auditors had different attitudes toward
risk in general [Newton, 1977] and toward the business risk
associated with particular situations [Ward, 1976]. The
comments of the auditors who evaluated AUDITPLANNER also
support this possibility. The evaluators generally agreed
with each other and with AUDITPLANNER regarding the types of
situations which represented increased business risk. There
was also a general consensus that each of these situations
warranted a reduction in the percentage rate used to calcu-
late the overall materiality level. However, there was no
consensus about either (1) how much the percentage rate
should be reduced for a given situation, or (2) the appro-
priate default percentage rate to use when no special busi-
ness risks existed.

This concludes the analysis of the mechanics and
nature of the materiality judgment process. The next chapter
summarizes the findings, discusses the implications of this
research for future attempts to develop general standards
for making materiality judgments, and suggests possible

future extensions to this research.

CHAPTER V

CONCLUSIONS AND SUGGESTED EXTENSIONS

The previous four chapters presented the theoretical
justification for and the analysis of an expert system which
makes judgments of auditing materiality. This chapter begins
by summarizing the results of this research effort and its
contributions to accounting research. Then several sugges-
tions for possible future extensions to the work are

presented.

Research Contribution

 

Participants in previous empirical studies of the
materiality judgment process frequently complained about the
artificiality of the cases that were used, particularly the
absence of many qualitative environmental factors that serve
to put the decision into context. The primary purpose of
this dissertation was to use the expert systems methodology
to obtain a better understanding of how such factors affect
judgments of auditing materiality. The successful use of
that methodology is seen as the major contribution of this
research.

AUDITPLANNER, the expert system built in this disser-
tation, makes judgments of auditing materiality. The rules
which constitute its knowledge base describe how various

126

127
factors influence those judgments. The analysis in Chapter
IV indicated that the judgment process consists of two sepa-
rate decisions: (1) the identification of an appropriate
base for the materiality judgment, and (2) the selection of
a percentage (threshhold) rate to be applied to that base.

The first decision, identification of the materiality
base, depends upon the auditor’s perceptions of the needs of
the users of the financial statements. It is at this point
where many oftfluaqualitative environmental factors which
together describe the nature of the client (e.g., industry,
form of ownership, eth enter into the materiality judgment
process. The second decision, selection of a percentage
(threshhold) rate, depends upon the auditor’s perceptions of
the risk associated with the audit. That assessment relies
primarily on information about the intended use of the
financial statements being audited.

Although AUDITPLANNER is based on the judgment model
of one member of one public accounting firm, the analysis of
the logic of that model is consistent with the results of
previous research. Moriarity and Barron [1979] found that
net income was the primary basis for judgments of auditing
materialijqu Their study used only publicly-owned profit-
oriented companies. AUDITPLANNER/s rules likewise choose
income as the materiality base for such companies. Moriarity
and Barron also observed a break-even effect; AUDITPLANNER,
too, has rules which choose an alternative materiality base

whenever income is at or near the breakeven point.

128

Gibbins and Wolf [1982] reported that information
about the nature of the client, the client’s plans for
future financing, and prior years’ audit decisions all had a
significant effect on planning and designing an audit. These
factors also appear in AUDITPLANNER’s rules. The client’s
industry and form of ownership influence the choice of the
materiality base. Client plans for future financing affect
both the choice of a materiality base (by possibly changing
a client’s classification from that of a private to a public
entity) and also the percentage rate itself (due to the
increased level of risk when the financial statements are
going to be used in connection with the issuance of public
debt or equity securities). AUDITPLANNER also has a rule
which compares this year“s materiality level to that used
last year, in order to evaluate the reasonableness of the
former.

Finally, the logic of AUDITPLANNER’S model is also
consistent with the judgments of the other auditors who used
it to make materiality judgments for their clients.

The expert systems methodology does more than merely
indicate which factors significantly affect (i.e”. are
highly correlated with) the decision criterion. It also
explains hgg those factors influence the criterion. For
example, the analysis of AUDITPLANNER/s rules indicated that
the nature of any debt covenants affected the classification
of the client as either a public or a private entity. This,

in turn, affected the auditor’s perception of the primary

129
interests of the users of the financial statements, and thus
determined the choice of a materiality base. It is this
ability to explain at a very detailed level hgg various
factors affect a decision criterion which makes the expert
systems methodology well-suited to studying and understan-
ding decision making behavior.

Libby [1982] points out, however, that accountants
study decision making in order to find ways to improve the
quality of the decisions that are made. He lists three basic
options for improving decision making: (1) changing the
information used to make the decision, (2) teaching the
decision maker to change the way he or she makes decisions,
or (3) replacing the decision maker with a model of the
decision process.

The capability of expert systems to explain the basis
for making decisions means that they are useful not only as
a means of studying decision making, but also as a means of
improving decision quality. In particular, an expert system
can be used as either (1) a training device, to teach the
decision maker how to make a decision, or (2) as a decision
aid which the decision maker can use to help make the actual
decision. In fact, the auditors who evaluated AUDITPLANNER
expressed great interest in further developing the program
so that it could be used for both purposes. The next section
describes some of the possibilities for such further

development.

130

Possible Future Extensions E9 the Work

 

Future extensions of this dissertation fall into three
areas: (1) further development of AUDITPLANNER to enable its
use as either a decision aid or training device, (2) compar-
ative studies of the materiality judgment processes of
auditors from different accounting firms, and (3) experi-

mental studies of some of the implications drawn here.

Further Develppment pf AUDITPLANNER

 

 

As mentioned earlier, the auditors who evaluated
AUDITPLANNER expressed great interest in its further
development so that it could be used either as a training
device or as a decision aid. Such development could involve
either broadening its current domain of expertise in regard
to new types of clients, or integrating the judgments of
auditing materiality with other judgments that are made
during the course of an audit.

Broadening AUDITPLANNER/sldomain pf expertise. The

 

 

testing of AUDITPLANNER resulted in the acquisition of
additional rules to handle clients which were different from
those used to develop the system. One possible future exten-
sion would be to work with other experts to cover areas not
adequately covered at present (e.g., small clients). The
ultimate objective would be to have AUDITPLANNER contain
rules that would allow it to deal with almost every conceiv-

able type of company that might be audited.

131

Integrating with other audit judgments. One of the

 

 

auditors who evaluated AUDITPLANNER suggested that the
system be extended to work with the analytical review tech-
niques currently used by that firm. Another possibility is
to integrate AUDITPLANNER with an expert system which evalu-
ates internal control, so that the entire planning process
could be performed. Another extension would involve adding

rules for making judgments of accounting materiality.

Comparative Studies

 

AUDITPLANNER represents tflma materiality judgment
process of one auditor from one public accounting firm.
Future work could involve the construction of similar sys-
tems for other firms in order to develop a complete taxonomy
of materiality judgment models.(huxasuch.a taxonomy was
develOped, comparative analysis may suggest methods for
increasing comparability across accounting firms. Such a
taxonomy would also permit the development of general
theories of the materiality judgment process, playing a role

similar to that played by taxonomies in biology.

Experimental Studies

 

The analysis of AUDITPLANNER suggests numerous possi-
bilities for controlled experimental studies of the effect
of various background variables.(kmaobvious candidate.is
the comparison of materiality judgments for different types
of companies (exp, public versus private versus nonprofitL

Such studies would be an important step towards developing a

132
consensus about appropriate bases for judgments of auditing
materiality.Ahother major linecflfresearch would involve
the study of the needs of different types of users HLg.,

creditors versus investors).

Conclusion

 

This dissertation demonstrated time usefulness of
applying the expert system methodology to the study of
materiality, a topic of long-standing interest both to prac-
ticing accountants and to academic researchers. The ability
of this methodology to make explicit the role played by the
different factors used to make materiality judgments repre-
sents a major methodological advance. The favorable recep-
tion of the system by those who evaluated its performance
suggests that it may not only be useful as a research tool,

but that it may also be useful as a practical tool as well.

APPENDICES

133

Appendix 1
Example of AUDITPLANNER'S explanation
capabilities during a consultation

Below are some examples of the WHY and HOW questions that
can rue asked during the course of a consultation. The
question numbers are from an actual consultation.
3) Is this the first tame we have audited Example ?
** WHY

[i.e. WHY is it important to determine whether this is
the first time that we have audited CLIENT-3?]

[1.0] Whether this is the first time that we have audited
a client is one of the 5 standard parameters that identify
a client. This may be useful later in the consultation.
For example, it is relevant in determining whether there
is a need to reduce the calculated materiality level
due to the increased risk associated with an initial
audit of the client and is relevant in determining the
percentage rate used to determine the materiality level.

[back to question 3...]
** YES

(note: several questions asked by AUDITPLANNER are skipped
here, until the next time the user asks ”WHY“)

10) Is Example filing with a regulatory agency in
preparation for the sale of its securities in a
public market?

** WHY

[i.e. WHY is it important to determine whether CLIENT-4
is filiru; with a regulatory agency in preparation for the
sale of its securities in a public market?]

[2.0] This w ill aid in determining the type of entity
CLIENT-4 is.
It has already been established that
[2.1] it is su5pected that the type of entity Example
is is private.
Therefore, if
[2.2] Example is not filing with a regulatory agency
in preparation for the sale of its securities in
a public market, and

134

[2.3] Example does not intend to go public within
the next two or three years
then
it is definite (1.0) that the type of entity Example
is is private.
[RULE007]

[back to question 10...]
** WHY

[i.e. WHY is it important to determine the type of entity
CLIENT-4 is?]

[3.0] This will aid in determining what the principal
external users (Hi'the financial statements are primarily
interested in.
It has already been established that
[3.1] the type of entity Example is is private
Therefore
it is definite (1.0) that what the principal external
users of the financial statements are primarily
interested in is financial-position
[RULE012]

[back to question 10...]
** HOW 2.1

[i.e. HOW was it established that it is suspected that
the type of entity Example is is private?]

[2.4] RULE056 was used to conclude that the type of
entity Example is is private (.9).

Thus, it has been established that it is suspected that
the type of entity Example is is private.

[back to question 10...]
** HOW 2.4

[i.e. HOW was RULE056 used?]

It has already been established that
[4.1] Example does not have publicly traded debt
or equity securites, and
[4.2] Example does not have restrictive debt covenants
that are measured by or depund on periodic financial
statement amounts or ratios that involve results
of operations, and
[4.3] Example is not controlled by a public entity

Therefore
there is strongly suggestive evidence (.9) that
the type of entity Example is is private

[RULE056]

135

Appendix 11
Example of AUDITPLANNER'S explanation
capabilities after a consultation

Several types of questions which may be asked after a
consultation is over are presented below.

** WHAT TYPE OF ENTITY IS THE CLIENT?
The type of entity Example is is public

** WHY DID YOU ASK WHETHER THE CLIENT HAS ANY PUBLICLY

TRADED DEBT OR EQUITY SECURITIES?

RULEO4O used:

whether Example has publicly traded debt or equity
securities. This information also would have been used
in: 56, 94, 95. However, none of these succeeded in
the context of Example. If you would like an explanation
for why any of these rules failed, please enter their
numbers:

**

** PR 40

RULEO4O
[This rule applies to clients, and is tried in order to
find out about the type of entity the client is]

If: 1) The client has publicly traded debt or equity
securities, or
2) 1%”: client has restrictive debt covenants that
are measured by or depend on periodic financial
statement amounts or ratios that involve results
of Operations
Then: It is definite (1.0) that the type of entity the
client is is public.

**WHY DID YOU ASK QUESTION 13?
Question 13 was asked in order to find out whether the
financial statements are being used in connection with
a contest for the control of Example in an effort to
execute RULE047.

** WHAT RULES TELL YOU ABOUT THE MATERIALITY PERCENTAGE
RATE?
The following rules conclude about:
the percentage rate used to determine the materiality
level 21, 77, 78, 89, 114, 115, 116, 117, 118, 27
The parameters most frequently used in these rules are:
whether the financial statements are going to be used

136

in connection with a public offering of securities
whether the financial statements are being used in
connection with a transfer of interests in the client

Would you like an summary of ALL the parameters used by
these rules?
** YES

Parameter (rules with this parameter)
whether the financial statements are going to be used in
connection with a public offering of securities
(78, 89, 27)
whether the financial statements are being used in
connection with a transfer of interests in the client
(115, 116, 27)
whether the financial statements are being used in
connection with an inquiry by a regulatory agency
(77, 27)
whether the financial statements are being used in
connection with a contest for the control of the client
(114, 27)
whether"the financial statements are being used to settle
some litigation involving the client (117, 27)
whether the client has violated some of its debt covenants
(118, 27)
whether the regulatory agency conducting the inquiry is
the IRS (77)
whether the inquiry is a routine audit by the IRS (77)

whether this is the initial public offering of securities
by the client (78)

whether this is the first time that we have audited the

client (78)

whether the client is making a large offering of securities
(78)

whether the client is offering a warranty on the financial
amount included in the financial statements being audited,
or the sales price is directly dependent upon the amounts
stated in those statements (115)

Which of the rules do you wish to see?
**

137

Appendix 111
Sample session with AUDITPLANNER

Below is a sample session with AUDITPLANNER.

24-Oct-84 13:00:13
You will be asked a series of questions about a client
you are about to audit. 0n the basis of your answers to
those questions, a set of recommendations will be presented
about the apprOpriate materiality level(s) to be used in
planning the extent of audit procedures.
-------- CLIENT-2--------
1) What is the name of the client?
** EXAMPLE
2) What is Example '5 main line of business or industry
classification?
** MANUFACTURING
3) Is this the first time we have audited Example ?

** N0

4) Have there been any acquisitions which either singly
or in total amount to 10 percent or more of the
client's total assets?

** N0

5) Some of the questions that follow will ask you to
input dollar amounts from the client's financial
statements. Will your answers be in terms of
thousands of dollars?

** YES

6) Does Example have any publicly traded debt or
equity securities?

** YES

7) What do you project retained earnings to be?

** 7616

8) What is the amount of current assets?

** 5822

9) What is the amount of current liabilities?

** 3009

10) What do you estimate the annual income from
continuing Operations (before taxes) to be?

** 3786

11) Are the financial statements going to be used in

connection with a public offering of securities?
** N0
12) Are the financial statements being used in
connection with a transfer of interests in

Example ?

** N0

138

13) Are the financial statements being used in
connection with a contest for control of Example?

** N0

14) Are tfiua financial statements being used to settle
some litigation involving Example ?

** NO

15) Are the financial statements being used in
connecticni with an inquiry be a: regulatory agency?

** N0

16) How much long-term debt does Example have?

** 882

17) Does Example have any covenants on either its
long-term debt or existing lines-of—credit?

** YES

18) Does Example have any debt covenants which restrict
dividends to a percentage of either current earnings
or cash flow?

** N0

19) TM) any of Example '5 debt covenants require it to
maintain some level of earnings?

** NO

2N1) Does Example have any debt covenants which require
it to use part of its cash flow to retire its debt?

** NO

The overall materiality level to be used in planning
the extent of audit procedures is 189300.0 .

The reasons for the materiality judgment are as follows:

The calculation of the materiality level was based
on the use of income-from-continuing-Operations
as the base for the materiality calculation.

The materiality level was calculated using a rate
of .05.

139

APPENDIX IV

Evaluation questionnaire

Read each question and circle the response which best repre-
sents your feelings to that statement.

Where SA = Strongly Agree

A = Agree

N = Neutral

D = Disagree

SD = Strongly Disagree

1. I found AUDITPLANNER easy to use. SA A N D SD

2. AUDITPLANNER did not provide
adequate HELP facilities to make it
easy to use. SA A N D SD

3. AUDITPLANNER asked too many questions -
often about irrelevant information. SA A N D SD

4.1 think AUDITPLANNER would be useful
as a training device. SA A N D SD

5. I think AUDITPLANNER would be useful
as a decision aidwhen planning an
audit. SA A N D SD

6. I would like to use AUDITPLANNER as
a decision aid. SA A N D SD

7. AUDITPLANNER does not exhibit a basic
level of competencein making
materiality judgments. SA A N D SD

8. I would accept AUDITPLANNER’S
recommendations. SA A N D SD

9. I did not enjoy using AUDITPLANNER. SA A N D SD

10. AUDITPLANNER was too slow in making
its recommendations. SA A N D SD

IJ.AUDITPLANNERwould be more useful
as a training device than as a
decision aid. SA A N D SD

12. I can see no use for a program like
AUDITPLANNER. SA A N D SD

13. AUDITPLANNER was hard to use. SA A N D SD

140

14. I wouldnot mind my subordinates
using AUDITPLANNERas a

decision aid. SA
15. It was hard to follow
AUDITPLANNER’S flow of logic. SA

l6.Imadefrequent use of AUDITPLANNER’s
questionanswering facilities. SA

l7.AUDITPLANNER’s questionanswering
facilities did not adequately explain
what it was doing. SA

18. I would not want my subordinates to
use AUDITPLANNERas a training
device. SA

19. AUDITPLANNER’s line of reasoning
was logical and easy to follow. SA

20. I enjoyed using AUDITPLANNER. SA

21.The sequencing of questions in
AUDITPLANNER was easy to follow. SA

22. I think that AUDITPLANNER’s recommenda-
tionsand rationale were sound and
reflected professional competence. SA

23. AUDITPLANNER’sability to respond
to HOW and WHY questions was very
helpful. SA

24.Iwou1d not approach the planning
materiality decision the way that
AUDITPLANNER did . SA

25. AUDITPLANNER jumped around from
topic to topic. SA

A

D

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

SD

26. Please write down your overall impressions of AUDIT-
PLANNER. Discuss what you think it could best be used for,
needed improvements, potential role in an audit, etc. Use as
much space as you need, including the backs of the question-

naire pages.

BIBLIOGRAPHY

BI BL IOGRAPHY

Adelman, L. "The Influence of Formal, Substantive, and Con-
textual Task Properties (Hi the Relative Effectiveness
of Different Forms of Feedback in Multiple-Cue
Probability Learning Tasks". Organizational Behavior
and Human Performance (l981),27,pp.423-442.

 

 

Aiello, N. "A Comparative Study of Control Strategies for
Expert Systems: AGE Implementation of Three Variations
of PUFF," Proceedings of the National Conference on
Artificial Intelligence (August, 1983), pp. 1- 4.

 

 

 

American Institute of Certified Public Accountants, State-
menton Auditing Standards Number47:Audit Riskand
Materiality ip Conducting anAuditJAICPA, 1983).

 

 

 

 

Codification of Auditing Standards Numbers 1 to 23,
(Commerce Clearing House, 1979).

 

 

Anderson, J.R. "Acquisition of Cognitive Skill," Psycho-
logical Review (1982, 89:4): PP.369-406.

 

, The Architecture pf Cognition. (Harvard University
Press, 1983).

 

 

 

Ashton, R. H. Studies in Accounting Research #17: Human
Information Processing in Accounting. (American
Accounting'Association, 1982).

 

 

 

 

Boatsman, J.R. and J3C. Robertson, "Policy-Capturing on
Selected Materiality Judgments," The AccountingReview
(April,l974):PP.342-352.

 

Brachman, R.J., S. Amarel, C. Engelman, R.S. Engelmore, E.
A.Feigenbaum, and D.E. Wilkins, "What Are Expert
Systems?". in F. Hayes-Roth, D.A.‘Waterman, and D.B.
Lenat. (eds) Building» Expert §y§3§m§.(Addison-
Wesley,l983),pp.31-57.

 

Braun, H.M. and J.S. Chandler, "Development of an Expert
System to Assist Auditors in the Investigation of
Analytical Review Fluctuations", research proposal,
(University of Illinois, 1982).

141

142

Brunswik, E., "Representative Design and Probabilistic
Theory in a Functional Psychology," Psychological
Review, (May, 1955), pp. 193-217. (a)

 

"In Defense of Probabilistic Functionalism: A
Reply," Psychological Review, (May, 1955), pp. 236-242
(b)

 

 

Buchanan, B.G., D. Barstow, R. Bechtal, J. Bennett, w.
Clancey, C. Kulikowski, T. Mitchell, and D.A. Waterman,
"Constructing an Expert System," in F. Hayes-Roth, D.A.
Waterman, and D.B. Lenat (eds) Building Egert Systems
(Addison-Wesley,l983),pp.127—l67.

 

Buchanan, BJL, G.L. Sutherland, and E.A. Feigenbaum,
"Heuristic DENDRAL: A Program for generating explana-
tory hypotheses in organic chemistry,":h1EL Meltzer
and D. Michie (eds) Machine Intelligence 3, (Edinburgh
University Press,1969), pp.209-254.

 

"Rediscovering some problems of artificial
intelligence in the context of organic chemistryfl'in
B. Meltzer and D. Michie (eds) Machine Intelligence _5_.
(Edinburgh University Press,1970),pp.253-280.

 

 

Canadian Institute of Chartered Accountants, Materialipy ip
Auditing. (CICA,1965).

 

Card, SJL, TRP. Moran, and A. Newell, The Psychology pf
Human-Computer Interaction. (Erlbaum,l983).

 

 

 

Carroll, J.S. "Analyzing Decision Behavior: The Magician’s
Audience," in T58. Wallsten (ed) Cognitive Processes in
Choice and Decision Behavior. (Erlbaum,l980), pp.69-75.

 

 

 

Chase, W.G. and H.A. Simon, "The Mind’s Eye in Chess". in
VLG. Chase (ed) Visual Information Processing (Academic
Press, 1973), gmx215-281.

 

 

Chi, M.T.H., P.J. Feltovich, and R. Glaser, "Categorization
and Representation of Physics Problems by Experts and
Novices," Cognitive Science (l981),5,pp.121-152.

 

Cox, J.R. and R.A. Griggs, "The Effects of Experience on
Performance in Wason’s Selection Task," Memory and
Cognition (September,1982),lO,pp.496-502.

 

 

Cushing, B.E. and JJC Loebbecke, "Comparison of Audit
Methodologies of Large Accounting Firms," Working
Paper, University of Utah, 1983.

143

Cushing, B.E., D.G. Searfoss, and R.H. Randall, "Materiality
Allocation in Audit Planning: A Feasibility Study,"
Supplement 39 the Journal pf Accounting Research (1979)
pp.172-216.

 

 

 

Davis, R., Buchanan, B., and Shortliffe, E. "Production
Rules as a Representation for a Knowledge-Based Consul-
tation Program". Artificial Intelligence (February,
1977),8:Pp.15-45.

 

Davis, R. and.EL Lenat, Knowledge-Based Systems lg Artifi-
cial Intelligence. (McGraw-Hill,l980).

 

 

DeMarco, T. Structured Analysis and System Specification
(Prentice-Hall, 1979).

 

 

Dillard. ELFh, J.F. Mutchler, and K. Ramakrishna, "Knowledge
Based Expert Computer Systems for Audit Opinion Deci-
sions," research proposal, (Ohio State University,
1983).

Dukes, W.F”,"N = lfl'Psychological Bulletin,(l965, 64:1),
pp. 74-79.

 

Dungan, C.W. "A Model of an Audit Judgment in the Form of
an Expert System". Unpublished.EWLD. Dissertation, Uni-
versity of Illinois, 1983.

_, and JAL Chandler, "Analysis of Audit Judgment
Through an Expert System," Working Paper #982, Univer-
sity of Illinois, 1983.

 

Dyer, JxL. "Toward the Development of Objective Materiality
Norms," The Arthur Andersen Chronicle (October31975),
pp.38-49.

 

Ebbesen, ELB. and VZJ. Konecni, "On the External Validity of
Decision-Making Research:What Do We Know About Deci-
sions in the Real World?fl'iJIGLS. Wallsten (ed) Cogni-
tive Processes ip Choice and Decision Behavior.
(Erlbaum Press,1980)dKL21-45.

 

 

 

 

Einhorn, HJL "Expert Judgment: Some Necessary Conditions
and an Example,".Journal pf Applied Psychology(October,
l974),59:5,p.562-57l.

 

and RJL Hogarth, "Behavioral Decision Theory: Pro-
cesses of Judgment and Choice," Journal pf Accounting
Research (Spring,198l),pp.l-31.

 

, "A Theory of Diagnostic Inference:
I. Imagination and the Psychophysics of Evidence,"
Working Paper, (University of Chicago, 1982).

 

144

"A Theory of Diagnostic Inference:
Judging Causality," Working Paper, (University of
Chicago, 1984).

 

Elliott, RJL, "Unique Audit Methods: Peat Marwick Interna-
tional," Auditing: A Journal pf Practice é Theory,
(Spring, 1983), pp.1-12.

 

 

Feigenbaum, FLA. "Themes and Case Studies of Knowledge

 

 

 

Engineering," in D.lHichie (ed) Expert Systems 12 the
Microelectronic Age. (Edinburgh University Press,1979),
pp.3-25.

Felix, W.L.,Jr., and W.R.Kinney,Jr. "Research on the Audi-
tor"s Opinion Formulation Process: State of the Art,"
The Accounting Review (April,l982),pp.245-27l.

 

Financial Accounting Standards Board, Discusion Memorandum:
Ag analysis 9f issues related pg Criteria for Determin-
ing Materiality (FASB,1975).

 

 

 

 

, Statement 9: Financial
Accounting Concepts No. 2i Qualitative Characteristics
9: Accounting Information (FASB,1980).

 

 

 

 

 

 

Firth, M. "Consensus Views and Judgement Models in Materi-
ality Decisions," Accounting, Organizations and Society
(1979),4:4,pp.283-295.

 

Frishkoff, P. "An Empirical Investigation of the Concept of
Materiality in Accounting," Empirical Research 12
Accounting: Selected Studies,l970. Supplement £g_phg
Journal g Accounting Research,pp.ll6-129.

 

 

 

 

Gal,(L, "Using Auditor Knowledge to Formulate Data Model
Constraints: Expert Systems for Internal Control Evalu-
ation," Dissertation Proposal, (Michigan State Univer-
sity, 1984).

Gaschnig, J. ”Prospector: An expert system for mineral ex-
ploration," in D. Michie (ed) Introductory Readings ip
EXPERT SYSTEMS,(Gordon andBreach,1982):PP.47-64.

 

 

, P. Klahr, H. Pople, E. Shortliffe, and A. Terry,
"Evaluation of Expert Systems: Issues and Case Studies"
in F. Hayes-Roth, D.A. Waterman, and D.B.Lenat (eds)
Building Expert Systems. (Addison-Wesley,l983),pp.241-
280.

 

 

Gibbins, M., "Propositions about the Psychology of Profes-
sional Judgment in Public Accounting," Journal pf
Accounting Research, (Spring, 1984), pp. 103-125.

 

145

, and FhM. Wolf, "Auditors’ Subjective Decision
Enviroment - The Case of a Normal External Audit," The
Accounting Review (January, 1982), me105-124.

 

 

Gorry, GLA. and R.B. Krumland, "Artificial Intelligence
Research and Decision Support Systems," in JkL. Bennett
(ed) Building Decision Support Systems. (Addison-
Wesley,l983):PP.205-219.

 

Hansen, J.V. and W.F. Messier,Jr. "A Knowledge-Based Expert
System for Auditing Advanced Computer Systems," ARC
working Paper 83-5, Brigham Young University, 1983.

Hartley, RJL. "How Expert Should an Expert System Be?,"
Proceedings pf the 7th International Joint Conference
pp Artificial Intelligence (Stanford,1979),pp.862-867.

 

 

 

 

Hayes, JxR. and.PLA. Simon, "Psychological Differences among
Problem Isomorphs," in N.J. Castellan,Jr., D.B. Pisoni,
and G.R. Potts (eds) Cognitive Theory: Volumg g.
(Erlbaum,l977):PP.21-41.

 

Hayes-Roth, F., D.A. Waterman, and D.B. Lenat, "Principles
of Pattern-Directed Inference Systems":hiILA. Waterman
and F. Hayes-Roth (eds) Pattern-Directed Inference Sys-
tems (Academic Press,1978),pp-577-601.

 

"An Overview
of Expert Systems," in F. Hayes-Roth, D.A. Waterman and
DgB. Lenat (eds) Building Expert Systems. (Addison-
Wesley,l983),pp.3-29.

 

 

Hoch, S.J. and J.E. Tschirgi, "Cue Redundancy and Extra-
Logical Inferences in a Deductive Reasoning Task,"
Memory and Cognition (March,1983),ll,pp.200-209.

 

 

Hofstedt, TJL and GJL Hughes, "An Experimental Study of
the Judgment Element in Disclosure Decisions,"fmua
Accounting Review (April,l977),pp.379-395.

 

Holstrum, GJL, "Improving Auditor Judgment Through Decision
Modeling and Computer Assistance: Part I ’Consenus in
Audit Judgments’flh. Working Paper, (Deloitte, Haskins,
& Sells, 1981).

, and W.F. Messier, Jr. "A Review and Inte-
gration of Empirical Research on MaterialityyWAuditing:
A Journal pf Practice and Theory'(Fall,1982)dnx45-63

 

 

 

Hunter, IAM.L. "An Exceptional Talent for Calculative
Thinking". British Journal 9; Psychology ,l962,53:3,
pp.243-258.

 

 

146

Kahneman, D. and A. Tversky, "Prospect Theory: An Analysis
of Decision under Risk". Econometrica (March,l979),47,
pp.263-29l.

 

Lachman,!h, JJL Lachman, and EJL Butterfield, Cognitive
Psychology and Information Processing; Ag Introduction.
(Erlbaum,l979).

 

 

 

 

LenatJLB."The UbiquitycflfDiscoveryfl‘1977 Computers and
Thought Lecture in Proceedings 9f the Fifth Inter-
national Joint Conference gg Artificial hfixflligence:
Vol. 2, pp. 1093-1105.

 

 

 

 

 

Leslie, ILA. "Materiality in Auditing (Some of the Issues)fl'
Symposium on Auditing Research 1; (University of Illi-

nois,l977), pp. 83-128.

 

 

Lewis, BgL. "Expert Judgment in Auditing: An Expected
Utility Approach," Journgl p_f Accounting Research
(Autumn,l980), 18:2, pp. 594-602.

 

Libby, R. Accounting and ggman Information Processing:
Theory And Applications (Prentice-Hall, 1981).

 

 

 

 

Mayper, ILG. "Consensus of Auditors’ Materiality Judgments
of Internal Accounting Control Weaknesses," Journalgfi
Accounting Research (Autumn, l982),20:2:2,pp.773-783.

 

Messier, W.F., Jr. "The Effect of Experience and Firm Type
on Materiality/Disclosure Judgments," Working Paper,
University of Florida, 1982.

Michaelsen, RJL "A Knowledge-Based System for Individual
Income and Transfer Tax Planning". Unpublished PhJL
dissertation, University of Illinois,l982.

Miller, (LA., "The Magical Number Seven, Plus or Minus Two:
Some Limits on Our Capacity For Processing Information"
Psychological Review, (March, 1956), pp. 81-97.

 

Minsky,lw. "A Framework for Representing Knowledge," in
P.H. Winston (ed) The Psychology Q; Computer Vision.
(McGraw-Hill,l975):pp.211-277.

 

 

Mock, TuJ., and Turner, J.L., "The Effect of Changes in
Internal Controls on Audit Programs," Behavioral Exper-
iments Ag Accounting £1, (Ohio State University, 1979),
pp. 277-302.

 

 

Moriarity» S. andeJL Barron, "Modeling the Materiality
Judgments of Audit Partners," Journal gfi Accounting
Research (Autumn,l976),pp.320-341.

 

147

"A Judgment Based Definition of
Materiality," Supplement 39 the Journal pf Accounting
Research (1979).pP.ll4-l35.

 

 

 

 

Newell, A. "Remarks on the Relationship Between Artificial
Intelligence and Cognitive Psychologyfl'iJIILB. Banerji
and FLD. Mesarovic (eds) Theoretical Approaches pg Non-
Numerical Problem Solving: Proceedings 9; Egg 1y Sys-
tems §ymposium g; Case Western Reserve University.
(Springer Verlag,1970)dxx363-400.

 

 

 

 

 

"Production Systems: Models of Control Structures/'
iJi‘WJL Chase (ed) Visual Information Processing.
(Academic Press,1973),pp.463-526.

 

"Harpy, Production Systems, and Human Cognition," in
ILA. Cole (ed) Perception and Production 9: Fluent
Speech. (Erlbaum,l980):Pp.289-380.

 

"Physical Symbol Systems," Cognitive Science (1980),
4’pp0135-183o

 

______r "The Knowledge Level," Artificial Intelligence (Jan-
uary,l982),l8,pp.87-127.

 

 

_ and HLA. Simon Human Problem Solvigg (Prentice-
Hall,l972).

 

Newton,.LJL "The Risk Factor in Materiality Decisions,"
The Accounting Review (January,1977),pp.97-108.

 

Pattillo, J.W. "Materiality: The(former1y) elusive stan-
dardfl'Financial Executive (August,l975),pp.20-27.

 

The Concept pf Materiality ig Financial Reportigg
Volume A Financial Executives Research Foundation,l976.

 

 

 

 

and JxD. Siebel, "Factors Affecting the Materiality
Judgment," CPA Journal (July,l974),pp.39-44.

 

Payne, J.W. "Contingent Decision Behaviorfl'Psychological
Bulletin (1982),92:2, pp.382-402.

 

Pylyshyn, Z.W. "Psychological explanations and knowledge-
dependent processesfl'Cbggition,(l981),10,pp.267-274.

 

Raphael, B. The Thinking Computer: Mind Inside Matter.
(WJL Freeman,l976).

 

Shortliffe, E.H.Computer-BasedMedical Consultation: MYCIN.
(American Elsevier,1976).

 

148

, and Buchanan, BJL "A Model of Inexact
Reasoning in Medicine". Mathegatical Biosciences
(1975),23, pp. 351-379.

 

 

 

 

Simon,.HJL "Cognitive Science: The Newest Science of the
Artificial," Cognitive Science (1980),4,pp.3-46.

 

Sowa, J.F. Conceptual Structures: Information Processing 1g
Mind and Machine. (Addison-Wesley,l984).

 

 

Stefik, M., J. Aikins, R. Balzer, J. Benoit, L. Birnbaum,
F. Hayes-Roth, and E. Sacerdoti, "Basic Concepts for
Building Expert Systems," in FuHayes-Roth, ILA. water-
man, and ILB. Lenat (eds) Building Expert Systems.
(Addison-Wesley, 1983), gmL59-86.(a)

 

 

Thomas, C.W. and.TJL Krogstad, "Materiality guidance for
auditors". Journal g£ Accountancy (February,l979),
pp.74-77.

 

Tversky, A., and D. Kahneman, "The Framing of Decisions and
the Psychology of Choiceu" Science (January 13,1981),
211, pp. 453-458.

Ueckert, H. "Cognitive Production Systems: Toward a Compre-
hensive Theory on Mental Functioning," in F. Klix and
J. Hoffman (eds) Cognition and gggggy. (North-
Holland,l980),pp.33-39.

 

Van Melle, 9L, AHC. Scott, J.S. Bennett, and M. Peairs,
The Emycin Manual Stanford: Department of Computer
Science, Stanford University,1981.

 

 

Ward, B.H. "An Investigation of the Materiality Construct
in Auditing," Journal 9: Accounting Reseanfli(Spring,
1976).pp.138-152.

 

Waterman, ILA. and F3 Hayes-Roth, "An Overview of Pattern-
Directed Inference Systems," in DJL waterman and F.
Hayes-Roth (eds) Pattern-Directed Inference Systems.
(Academic Press,1978),PP.3-22.

 

Young, R.M. "Strategies and the Structure of a Cognitive
Skill," in G. Underwood (ed) Strategies gf Information
Processing. (Academic Press,1978),pp.357-401.

 

 

 

"Production Systems for Modelling Human Cognition,"
in D. Michie (ed) Expert Systems lg the Microelectronic
Age. (Edinburgh University Press,1979),pp.35-45.

 

 

and T. O’Shea, "Errors in Children’s Subtraction,"
Cognitive Science (l981),5,pp.153-l77.

 

149

General References

 

 

Bramer, MJL "A survey and critical review of expert systems
research," in D. Michie (ed) Introductory Readings _i_g
EXPERT SYSTEMS (Gordon and Breach, 1982), pp.3-29.

 

 

Buchanan, EAL "New research on expert systems". leJ.E.
Hayes, D. Michie, Y-H Pao (eds) Machine Intelligence,10
(Halsted Press, 1982), PP-3-39.

 

Davis, R. "Knowledge Acquisition in Rule-Based Systems -
Knowledge About Representations as a Basis for System
Construction and Maintenance". in ILA. Waterman and F.
Hayes-Roth (eds) Pattern-Directed Inference Systems
(Academic Press, 1978), pP.99-134.

 

, and King, J. "An Overview of Production Systems,"
in ELW. Elcock and D. Michie (eds) Machine Intelli-
gence g (Halsted Press, 1977), pp.300-332.

 

 

Duda, RJL, Hart, PJL, Nilsson, NHL, and Sutherland, GJL
"Semantic Network Representations in Rule-Based Infer-
ence Systems,":UiILA. Waterman and F. Hayes-Roth (eds)
Pattern-Directed Inference Systems (Academic Press,
1978). pp.203-221.

 

Goldstein, I.eumlS. Papert, "Artificial Intelligence and
the Study of Knowledge," Cognitive Science (l977,1:1),

 

 

pp.84-123.

Longuet-Higgins, TLC. "Artificial Intelligence - a new
theoretical psychology?," Cognition (l981),10,pp.197-
200.

Nau, D. "Expert Computer Systems," Computer (February,l983),
pp.63-85.

Newell,)L "A Theoretical Exploration of Mechanisms for

Coding the Stimulus". in A.W. Melton and E. Martin(eds)
Coding Processes _ig Human Memory (V.H. Winston, 1972),
pp.373-434.

 

 

Quinlan, J.R., "Fundamentals of the knowledge engineering
problem". in D. Michie (ed) Introductogy Readings lg
EXPERT SYSTEMS (Gordon and Breach,l982), pPu33-46.

 

 

Ringle, NL "Psychological Studies and Artificial Intelli-
gence". A; Magazine (Winter/Spring, 1983), pp.37-43.

Rosenshein, SHT. "The Production Systenu Architecture and
Abstraction". in D.A. Waterman and F. Hayes-Roth (eds)
Pattern-Directed Inference Systems (Academic Press,
1978): PP.525-538.

 

150‘

Rychener, M.D.aumIA. Newell, "An Instructable Production
System: Basic Design Issuesfl'jJiILA.Waterman and F.
Hayes-Roth (eds) Pattern-Directed Inference Systems.
(Academic Press, 1978), pp.135-153.

Simon, ILA., "Search and Reasoning in Problem Solving,"
Artificial Intelligence (March, 1983),21, pp.7-29.

Sloman, A. "Interactions Between Philosophy and Artificial
Intelligence: The Role of Intuition and Non-Logical
Reasoning in Intelligence". Artificial Intelligence
(Winter, l97l),2, pp.209-225.

Stefik, M., J. Aikins, R. Balzer, J. Beloit, L. Birnbaum,
F. Hayes-Roth, and E. Sacerdoti "The Organization of
Expert Systems: A Tutorial," Artificial Intelligence
(March, l982),18, gle35-l73.

"The Architecture of Expert
Systems," in F. Hayes-Roth, D.A. Waterman, and D.B.
Lenat (eds) Building Expert Systems (Addison-Wesley,
1983): pp. 89-126. (b)

 

Winston, P.H. Artificial Intelligence. (Addison-Wesley,
1977).