...:......... .....- 515...... ..,.....,._.....:.... ..,.....r.:........ ............... .... a... ..... ..., ...... .....x. ...... 1.3.2.... .-.... ...,.....:.. ........... ...... . , .-.... .....- . .11.... I... . .~.....‘_ . ...... ...-...: ......1 ........_v.... :2. 9.... ......v....... ...... ...f.......... 1.1;... ..Z.....Z... .......1...:§...... ... .: . 2.1.212... . .....I...:....;: ......1....... ..........v. 5.1.3.1. .;...... , ,...v.....vyur.p_ .......r.....r».. ...... ...: ..3... :y....x...:v... . ,5:r.. ....x: ......i; ....i. ...}... ....v.....:n_ ...... . ....r: .,x...(.... .r...... ...v........... .t..._.....,x..;......rv....... ...}...y, ..u, v..L‘c... »v.._...—, .1... .....r...av.a..‘.:. ... .H..T.....,z.s.: ...-... 2...... 1.2.2.33...» ..1......... 1... v3.2... .321: .s....... 5.2:... . .... ...........v...o... 2.2.1.1....‘3 .... e...2.......r... ....y......‘. ......z... \..\....,+x...... ..3. y. ...,....9.-. .....vex....._......... . ......L... . . .....x: .~ .L .......e...e.y_.._.a ...9 ...}..ln. .s.......... ...“.f I.....v’:.ll I-li .. .....2. «...»... 1. vHHWItrwlr. ....r. 3...... ......)v...v.r...»._. ... .. .... ;. ......C .r» .2... ...: .....uuu . . . .............. ... ..L. . , 2 Aorv.a:oll... ’ LIBRA n V V MlChlgaE'z 51.3552; ‘ :.. Lhfiflcrfity {ml '1 ' I. This is to certify tii thesis entitled I Illillllllfllillllllflllll 31 T ”Milli!!! 92 uu ORGANIZATION AND RETRIEVAL OF MULTI-ATTRIBUTE ENTITIES IN LONG TERM MEMORY presented by Stuart Winston Thomas, Jr. has been accepted towards fulfillment of the requirements for _Eh..D.__degreein_E§lO_0_Lgh 0 y MEW a Major professor _ Date____)_Nfi- '2 H74 0-7 639 mindset. ABSTRACT ORGANIZATION AND RETRIEVAL OF MULTI-ATTRIBUTE ENTITIES IN LONG TERM MEMORY By Stuart Winston Thomas, Jr. The study investigated a long term memory task which required subjects to memorize the six attributes identi- fying each of 20 spies, and then to respond to questions calling for the names of spies possessing either conjunc- tive or disjunctive combinations of one to five attributes. The attributes describing the spies were chosen from six binary dimensions so that each attribute from a given dimension characterized 10 spies, and the attributes from the six dimensions were assigned to the spies indepen- dently. The 10 college age female subjects were paid for their participation in three sessions conducted on consecutive days. In Session I all subjects memorized the roster of spies. In Session II the subjects responded to 1&4 questions specifying varying combinations of attri- butes. Session III was a repetition of Session II. Half the subjects were asked to name all relevant spies, and half were asked to name only two relevant spies in each response. Five a priori models were proposed for the subjects' Stuart Winston Thomas, Jr. behavior: (1) a sequential list model in which the spies are processed in seriatum: (2) an articulated list model in which spies are processed in seriatum with the aid of links between spies possessing a common attribute: (3) a direct activation model in which the likelihood of a spy's being named is governed by the number of attributes he has in common with the question; (A) an attribute entry model in which the attributes specified in the question are processed in seriatum; and (5) a subroster model in which spies are processed in seriatum within subrosters of spies possessing one or more attributes in common. Analysis was performed on the Session III data of individual subjects. Six subjects were found to have divided the roster into four subrosters, each subroster containing five spies of the same sex and nationality. The latencies of responses to conjunctive questions for these six subjects all conformed to the subroster model: The more attributes specified in the question, the slower the response: and for conjunctive questions of a fixed number of attributes, questions specifying sex and nationality were answered faster than questions specifying either sex or nationality, but not both; and questions specifying neither sex nor nationality were answered slowest. Furthermore, the three whole answer subjects with sex/nationality subrosters were almost perfectly consistent in answering disjunctive questions by naming Stuart Winston Thomas, Jr. relevant spies from each subroster in turn--also in con- formity with the subroster model. However, the two answer subjects varied in their responses to the disjunctive questions. One used her subrosters to retrieve spies who were conjunctively relevant for the disjunctive questions; the second used two favorite pairs to answer many of the disjunctive questions; and the third searched a short list of spies with technical backgrounds to find spies who possessed the first attribute specified in the disjunctive questions. Of the remaining two answer subjects, one had a sub- roster of 10 veterans and a subroster of 10 trainees. Her conjunctive responses conformed to the subroster model but there was no evidence that she used her subrosters to answer the disjunctive questions. The other two answer subject divided the roster into four subrosters on the basis of nationality and background, and she noted that back- ground and specialty were correlated within nationality. However, her data did not conform to any of the models. One of the remaining whole answer subjects divided the spies into four subrosters chosen arbitrarily, i.e., without regard to the spies' attributes. However, her data showed some evidence of organization within her arbitrary subrosters. The other whole answer subject processed the roster in alphabetical order in conformity with the sequential list model after she found sex/nationality Stuart Winston Thomas, Jr. subrosters unworkable. The major features of the data were clearly consistent with the subroster model. Discussion explored issues of individual differences and experimental control. ORGANIZATION AND RETRIEVAL OF MULTI-ATTRIBUTE ENTITIES IN LONG TERM MEMORY By Stuart Winston Thomas, Jr. A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Department of Psychology 1974 ACKNOWLEDGMENTS I want to express my appreciation to Dr. Gordon Wood and Dr. Ray Frankmann for their helpful comments and observations on the research reported here. Similarly, I would like to thank Dr. John Vinsonhaler for his help, and more particularly for the time he has spent as my mentor in information science. Dr. Vinson- haler is an example of what intellect and diligence can accomplish together. A Most especially, I want to thank Dr. John E. Hunter for the considerable investment of himself in my educa- tion. Not only was Dr. Hunter my principle instructor in quantitative psychology, but he also was a source of great moral support throughout graduate school. Furthermore, his help with this dissertation was invaluable. I hope I can maintain the high standards of intellectual effort and hard work that Dr. Hunter teaches by his own example. Finally, I want to acknowledge my wife's contribution. She encouraged me when I needed encouragement and prodded me when it seemed necessary. She also did the typing and much of the busywork associated with finishing a disser- tation. ii TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES INTRODUCTION The Organization of Free Recall Probe Recognition Alternatives to Sternberg's Model A sequential, self-terminating model A direct access model A parallel processing model The modes of information processing The Spy Retrieval Task Models The The The The The for the Spy Retrieval Task sequential list model articulated list model sequential attribute-entry model direct activation model subroster model Differentiation of the models METHOD The Spy Retrieval Task The Spy Roster iii Page vii xi H\O\}\10\Nl-‘ 1n 15 16 18 20 24 25 28 30 30 30 iv The Questions Question blocks Disjunctive versus conjunctive questions Number of attributes Number of prepotent attributes Treatments Subjects Procedure Overview Acquisition of the roster (Session I) Retrieval from the roster (Sessions II and III) Data Recording and Transcription RESULTS: THE SUBJECTS WHO SUBROSTERED USING SEX AND NATIONALITY Data presentation for the subjects who subrostered on sex and nationality Intersubject experimental variation Disjunctive Responses of the Whole Answer Subjects Pilot Subject CT Subject's self-report Ordinal analysis of CT's data Summary of the ordinal analysis of CT's data Subject JL (A Whole Answer Subject) Subject's self-report Ordinal analysis of JL's responses to disjunctive questions The order of selecting subrosters Search order within subrosters Summary of ordinal data analysis for JL Page 33 33 33 36 36 37 38 38 38 39 no 42 ”5 #6 1+7 48 as» Le 50 58 59 59 59 63 65 65 v Page Subject SM (A Whole Answer Subject) 67 Subject's self-report 67 Distribution of string lengths 67 Search order across subrosters 68 Search order within subrosters 70 Summary of ordinal characteristics of SM's disjunctive responses 7b Disjunctive Responses of the Two-Answer Subjects 75 Subject JK (A Two-Answer Subject) 76 Subject's self-report 76 Analysis of JK's data 76 Subject CM (A Two Answer Subject) 85 Subject's self-report 85 Analysis of CM's data 85 Summary of analysis of disjunctive questions for CM 94 Subject NA (A Two-Answer Subject) 95 Subject's self-report 95 Analysis of NA's data 95 Summary for NA's disjunctive responses 105 Analysis of Latency Data from Conjunctive Responses of the Subjects Who Subrostered on Sex and Nationality 105 An alternative explanation 113 RESULTS: THE SUBJECTS WHO DID NOT SUBROSTER ON SEX AND NATIONALITY 127 Subject SP (A Whole Answer Subject) 127 Subject's self-report 127 Analysis of SP's data 127 vi AH: A Two-Answer Subject Who Subrostered on Experience Subject's self-report The "all spies" question The latency data for the conjunctive responses Analysis of responses to disjunctive questions Summary for AH DK: The Two-Answer Subject Who Subrostered on Nationality and Background Subject's self-report The responses to the disjunctive questions Responses to conjunctive questions Summary for Subject DK CP: The Whole Answer Subject with Random Subrosters Subject's self-report The responses to the disjunctive questions Responses to the conjunctive questions Summary for subject CP DISCUSSION The Models Individual Differences Summary and Conclusions LIST OF REFERENCES Page 132 132 133 133 136 142 1&2 142 144 152 152 154 154 154 158 163 16h 166 168 170 172 Table Table Table Table Table Table Table Table Table Table Table Table Table 10 ll 12 13 LIST OF TABLES Summary of implications of five information processing models The roster of spies Questions listed in order of presentation Frequency of response strings for each subroster in CT's responses to 56 disjunctive questions Classification of response string frequencies by attribute and string length for CT's responses to 96 disjunctive questions Subroster precedence matrix for CT's responses to 56 disjunctive questions Classification of response string frequencies in JL's eight responses to the "all spies" request by attribute and string length Classification of response string frequencies by attribute and string length for responses to 96 disjunctive questions Subroster precedence matrix for JL's responses to non-marker questions in Session III Precedence matrices for the spies within JL's four subrosters Response string frequencies for SM's responses to Session III disjunctive requests, classified by attribute and string length Frequencies of subroster search orders in SM's Session III responses to disjunctive questions Subroster precedence matrices for SM's Session III responses to block requests vii Page 29 31 34 52 54 56 6O 62 64 66 69 71 72 Table Table Table Table Table Table Table Table Table Table Table Table Table Table 14 19 20 21 22 23 2h 25 26 27 viii Distribution of attribute string lengths in JK's eight responses to the "all spies" question Subject JK's responses to Session III disjunctive questions Frequency distribution of attribute agreements in JK's responses to Session III disjunctive questions Frequency distribution of JK's conjunctively perfect responses to Session III disjunctive questions classified by total attributes and prepotent attributes Subject CM's responses to Session III disjunctive questions CM's responses to Session III disjunctive questions calling for spies who are not-disguisable, female, English, or assassins CM's responses to Session III disjunctive questions omitting those calling for females (except those including both female and not-disguisable), English spies, and assassins Distribution of attribute strings in NA's eight responses to the "all spies" question Subject NA's responses to Session III disjunctive questions Spy usage frequencies by NA in Session III disjunctive questions Block characteristics of NA's Session III responses to disjunctive questions Frequencies of attribute matches in NA's responses to Session III disjunctive questions Analysis of variance on latency data for responses to conjunctive questions in Session III Expected number of spies examined to find 2nd relevant spy after 1st relevant spy is encountered Page 77 80 82 84 86 88 92 96 98 99 102 104 112 123 Table Table Table Table Table Table Table Table Table Table Table Table Table Table 29 3O 31 32 33 34 35 36 37 38 39 4O 41 ix Response intervals between first and second responses: Observed intervals and intervals predicted by Models I - V. Slope, intercept, and observed/predicted correlation for each model String distributions in AH's 8 responses to the "all spies" question Mean intervals in seconds between AH's first and second responses to Session III conjunctive questions The 2 scores for AH's responses to Session III disjunctive questions Number of question/answer attribute matches in AH's responses to Session III disjunctive questions Observed and expected frequency of veteran pairs, mixed pairs, and trainee pairs in AH's responses to Session III disjunctive questions Subject DK's responses to 64 Session III disjunctive questions Proportions of spies in DK's responses in Session III which match attributes specified in disjunctive questions Frequency with which spies named in response to Session III disjunctive questions match one or more attributes specified in the question Disjunctive questions DK answered with high frequency pairs in Session III Latency data for DK's responses to Session III conjunctive questions Distributions of attribute strings in CP's responses to Session III disjunctive questions Pairs and triplets of spies occurring in CP's responses to Session III disjunctive questions Latency data for subject CP's responses to Session III conjunctive questions Page 125 134 135 138 139 141 145 146 147 150 153 155 157 159 X Table 42 Time interval between first and second Page responses expressed as a proportion of the first response latency for Session III conjunctive questions 161 Table 43 Number of relevant spies omitted in responses to all Session III conjunctive questions 162 Figure Figure Figure Figure Figure Figure LIST OF FIGURES First response latencies for the whole answer subjects First response latencies for the two- answer subjects Second response latencies for whole answer subjects Second response latencies for two- answer subjects Mean alphabetical order indexes for SP's responses to disjunctive questions in Session II and Session III The relationship between intername intervals in seconds and number of intervening spies in the alphabetical roster. xi Page 109 110 115 116 129 131 Introduction The research reported in the following pages investi- gates the way that people handle the storage and retrieval of information having a dimensional character. That is, the set of information to be remembered and used derives from a set of entities which may be characterized on a set of dimensions. Each dimension consists of two or more attri- butes, and an entity's position on any given dimension is determined by the attribute from that dimension which characterizes the entity. If there are n dimensions relevant for a set of entities, each entity is characterized by n attributes. In the present study the entities are twenty spies who are characterized by their positions on six dimensions which are binary. That is, each dimension con- sists of only two attributes, e.g., sex (male, female), nationality (English, German), etc. Hunt (1962) noted that this type of information can be represented as a table of twenty rows (one row for each spy with the row stub being the spy's name) and six columns (where each column represents one of the six dimensions). The table entry at the intersection of a given row and column would be the attribute for the spy corresponding to the row and the dimension corresponding to the column. Suppose human information processors were asked to name 1 2 those spies possessing a given set of attributes. How might they do it? They might try processing each spy in turn and reporting relevant spies as they are encountered. This would be equivalent to entering the table one row at a time. To use this strategy is to recall and check a list of attributes for each spy. On the other hand, they might approach the problem the other way--by recalling and processing the list of spies who have a given attribute. This would be equivalent to entering the table one column at a time, refining at each step a set of potentially relevant spies from which the relevant spies are finally selected. Or might they do something else? The present research was planned to help answer this question. Questions about organization and retrieval of informa- tion are not new questions. They are being investigated currently along several lines. Two such lines of research are particularly relevant to this work: the work on the organization of free recall and the work on the "probe recognition" task relating reaction time to list length. .Ihe Organization of Free Recall Investigation of the organization of human memory is One of the oldest areas of research in psychology. The tilassic work was largely based on the idea of the association between two ideas. That is, memory was pictured as a vast network of connections between pairs of ideas such that tkinking of one idea brings the other to mind. This attempt ‘bo represent all the structure of memory in terms of binary 3 contiguities has long been known to be inadequate. Even philosophers such as John Stuart Mill were able to criticize it rather effectively. However this criticism remained mostly a matter of lip service until quite recently. The most seminal criticism of the classical position is that generated by Miller (1956). Miller noted the incon- gruity of two well-known facts about human memory: On the one hand, the number of isolated pieces of information stored in a typical brain doubtless runs into the billions. On the other hand, the number of items that can be considered at one time is only about seven. How then can we access such a huge total memory within the constraints imposed by such a small memory span? Miller's answer was taken from his long study of language: He suggested that information was "chunked". Thus if we must consider the behavior of the many dogs that live where we do, then we cease to think in terms of Fido and Prince and Rex, etc., and we think in terms of the one conceptual entity "the dogs in our neighbor- hood". This in turn can be merged with other ideas to form a chunk such as "dogs raised in the city" or simply "dogs". On a formal level he noted that such chunking constitutes a major portion of the structure of language where subordinate concepts are chunked under superordinate concepts. Miller, Galanter, and Pribram (1960) also gave a rather full discussion of the relationship between this analysis of the structure of memory and a similar analysis of the structure of organizations and behavior sequences made by Simon (1945). 4 The first influential expermental work on the concept of hierarchical structure in memory was done by Mandler (1967). He first allowed subjects to group the stimuli in a free recall task into categories. He then looked at the relation between the number of items recalled and the number of categories used by the subject in grouping the items. He found that the greater the number of categories used, the greater the number of words recalled by the subject. In the decade since then, dozens of studies have been done to extend Mandler's work. Most have found that categorical structure improves memory. This work has been ably reviewed by Wood (1972). The present work represents a departure from this work in two major ways: First, most existing studies have used either numbers or familiar words as the stimuli for the free recalland the actual dimensionality of both stimulus sets is quite unknown. This has introduced considerable uncer- tainty into the existing results and has greatly reduced the level of control of memory processes and has shrouded the main processes in a maze of ad hoc guesswork. In the present study the stimuli are spies which are created within the experiment. Both the dimensions and the identity of each spy are specified in a "dossier" given to the subject. Thus there should be no idiosyncratic dimensional structure brought into the study by the subject. Second, most existing studies deal with only the most transitory of memory structures. Subjects are shown a short 5 list of words and are givenanywhere from one to fifteen minutes to become familiar with the list (using various techniques such as free study, memory drums, etc.), and are never asked to do any more with the list than simply recall the elements on it. This differs greatly from most impor~ tant contexts for memory in ordinary life which involve much more extensive experience with complex information sets and which require the subject to make extensive complicated computations from his store of knowledge. Either of these factors might well evoke processes which bear little resemblance to those studied in the usual laboratory study. The present research seeks to avoid this in three ways: First, the set of information to be learned was much larger and more complicated than is usually the case (i.e. six attributes for each of 20 spies as contrasted with a list of 25 or 50 simple nouns). Second, the subjects were given much more experience with the information than is usually the case (two hours the first day, 10 or 15 minutes of relearning each of the second and third days as contrasted with 10 minutes total in the typical study). Third, the subjects were required to do much more than simply recall the spy-attribute lists which formed the core of the learning: they were also asked four hours' worth of complicated conjunctive and disjunctive questions about the set of spies. In all these ways, the present work is intended to be closer to the complexity of life as we know it as subjects in our own world. 6 The next section reviews another line of investigation which is relevant to the present work: the study of the probe recognition task. This task is particularly relevant here because investigators are studying models of the information processing which are applicable in the present context of retrieval of information having a dimensional character. Probe Recognition In a series of studies Sternberg (1966, 1967, 1969) had subjects decide whether a test stimulus (probe) was an element of a previously learned list of items. Presentation of the probe started a timer which was stopped when the subject pressed one of two keys to indicate whether the search was positive (the probe was a list item) or negative (the probe was mg} a list item). Sternberg found that reaction times (RTs) were an increasing linear function of list length for both positive and negative searches. This result held whether the subjects had many trials with a single memorized list or whether the subjects learned a new list for each trial. Sternberg argued that some average time, say 3 milli- seconds, was required for each comparison of the probe with a list item. Thus, as list length increased RT increased. For negative trials, 3 comparisons would be required for a list of g items, and this would require at milliseconds. That is, RT would be a linear function of list length with slope t. The experiments were designed so that positive 7 probes were selected equally often from each position in the (n+1) —T_ list. Thus, on the average comparisons would be necessary on positive trials with lists of length n. Conse- quently the time required for an average positive search (n +21) t milliseconds which is also a linear function of n with slope %. Hence, if subjects terminated would be their search and reported a positive result immediately upon encountering the probe in the list, the slope of the RT versus list length function for positive trials should be half the slope for negative trials. Instead, Sternberg found that the slopes for positive and negative trials were nearly identical. His interpretation was that subjects processed the lists exhaustively on positive trials rather than reporting the positive result immediately. Certainly exhaustive searching is within the realm of possibility. Subjects in these studies are typically college students who doubtless take multiple choice tests, and who have therefore learned to read all alternatives to a multiple-choice question before selecting their response. Nevertheless, it is somewhat surprising that subjects would continue to make useless probe/list-item comparisons after finding the item which matched the probe. What is not surprising is that researchers have sought alternative explanations for Sternberg's results. Alternatives to Sternberg's Model A sequential, self-terminating model. Theios, Smith, Haviland, Traupmann, and May (1973) observe that RT has repeatedly been observed to be a decreasing function of 8 stimulus probability. Furthermore by fixing probability of a positive trial across list lengths and selecting probes from each position in the lists equally often, Sternberg has completely confounded list length with stimulus probability. Theios et al. gathered reaction time data for a task similar to Sternberg's, but they varied list length and stimulus probability independently. They found that when their RT data was plotted as a function of list size they got virtually parallel linear functions for positive and negative trials--just as Sternberg did. However, when RT was plotted as a function of stimulus probability, it was found that RT decreased with increasing stimulus probability. It was also evident that when stimulus probability was held constant, RT increased with list size. Theios et al. then suggested that the error in Stern- berg's derivation of reaction times lay in his assumption that the list is always processed in the same order. They instead postulated that as the subject acquires experience in the task, he reorders the list of items so that the early items in the list are those with a high probability of being positive instances. Those which are likely to be negative instances then are left to be last on the list. With such a frequency-determined list order, Theios et al. then show that a serial, self-terminating model accounts quite well for their data. Searching is sequential and terminating. Having the high frequency stimuli at the top of the sequential list 9 accounts for the decreasing RT as stimulus probability increases. The observed increase in RT as function of list length is due to the fact that as list size increases, the typical position of a stimulus in the list will decrease, even if presentation probability is constant. Thus, the Theios et al. model accounts for Sternberg's results with a_terminating search procedure, and accounts for the effect of stimulus probability on RT as well. A direct access model. Corballis, Kirby, and Miller (1972) take a quite different approach. They note that in the studies where a subject learns a different list for each trial there should be no serial position effect if the exhaustive search hypothesis is correct. Yet they, as well as Corballis (1967) and Kirshner and Craik (1971), report that RT decreases with serial position of the items in the list. Sternberg (1969) reports relatively flat serial position curves, and notes that serial position effects would be most likely to occur when list items are presented rapidly and the probe occurs very shortly after list presentation. Corballis et al. offer a strength model to account for all these findings. They assume that for items such as numbers or familiar words, the subject has an internal set of representations of the items to which he has direct access. That is, given the item the subject has immediate access to the representation of the item. One part of the representation of the item is a "strength" for the item. This quantity has the property that it is increased by the 10 occurrence of the item as a stimulus and it has the property that it decays over time if the item does not occur as a stimulus. That is, the item strength is in fact an indirect indicator of how long it has been since the subject last saw (or heard) the item. Thus one method of deciding whether or not an item was on the learned list would be to check the current strength of the item against a criterion which is set high enough to eliminate the decayed strengths of stimuli which have not recently occurred. On the other hand, biases will creep into such a strategy as a function of the familiarity of the stimulus material because a very common stimulus may have such a strong residual strength that it exceeds the criterion set for uncommon stimuli which have recently occurred. It is these biases which Corballis etal. believe to be the explanation of a number of the findings. The model Corballis et al. suggest for their data calls for the subject to have access to an internal representation of the probe. The strength of the representation varies directly with the familiarity of the probe. If the list items are such familiar materials as letters or digits, then similar nonlist stimuli will have some residual strength. In this case the subject is assumed to have a distribution of strengths for list items and a distribution for nonlist items. A criterion is established on the strength continuum between the two distributions and all stimuli whose representations exceed the criterion are reported as positive. Reaction time is assumed to be a decreasing 11 function of the absolute difference between the criterion strength and the strength of a given probe--i.e., RT decreases as the subject becomes more sure his decision is correct. The strength of the items at the beginning of a list will have decayed more than the strength of those at the end of the list. This accounts for the serial position effect on RT. As list lengths increase, subjects compensate for the decreasing strength of items near the beginning of the lists by moving the criterion closer to the distribu- tion of strengths of non-list stimuli. This accounts for changes in RT as a function of list length. It would also imply an increase in the number of false positives with increasing list length, and this was confirmed in the data reported by Corballis et al. Corballis et al. contend that the above model can be extended to data where repeated probes are made of a well- memorized list. Since each item in the list is well rehearsed, within a given list all items should have the same strength: however, as list length increases the strength of each item should decline due to intraunit interference. Hence, RT should be an increasing function of list length, but no serial position effect would be expected--as Sternberg has reported. A parellel processing model. Shevell and Atkinson (1974) and Murdock (1971) have both proposed "parallel" processing models. Unfortunately the term parallel is used in two quite different senses in the two papers. By 12 parallel processing, Murdock means that the list items are processed simultaneously, but at differing rates which are determined by their serial position in the list. In Murdock's model, the time to respond for a negative probe is determined by the time to process the list item which is processed most slowly, and this time is shown to be a linear function of the list length. If searching for positive probes is assumed to be exhaustive, then processing for a positive probe is the same as for a negative probe. If searching for a positive probe is self-terminating, then Murdock's model indicates that response time is still a linear function of list length, and the slope averaged over all serial positions is approximately .97 times the slope for a negative probe. Thus, Murdock's model accounts for the linear, parallel function relating RT to list length and also accounts for serial position effects. Shevell and Atkinson (1974) present a series of models for list scanning where list elements are multi-attribute entities. Some of the models call for processing list elements in seriatum so that investigation of any given list element is completed before starting to investigate another list element. In contrast to the in seriatum processing, they propose what they designate as "parallel" processing: Each list item is examined with respect to one attribute at a time. When all items have been scanned with respect to the first attribute, those which possessed the first attribute of the probe stimulus are scanned with 13 respect to the second attribute of the probe stimulus, etc. This is parallel processing in the sense that at the end of scanning with respect to any given attribute, all remaining items are at the same stage of processing--i.e., are at parallel stages of processing. Shevell and Atkinson consider a number of different implementations of parallel processing, but all of their parallel processing models lead to linear relationships between the number of attribute comparisons necessary to complete a search and the length of the list, with slopes for positive and negative probes being equal or nearly so. The modes of information processing. The models for the probe recognition task each postulate a plausible mode of information processing: any one of them could be applied to a variety of tasks. It may be that one of the modes thus postulated is characteristic of most human information processors: it is not inconceivable, however, that one mode is characteristic of some individuals and another mode characteristic of others. It could even be the case that one individual can and will apply the different methods to different problems or at different times. At any rate, it would be of considerable interest to know that individuals aPply one or more of these techniques to information Processing problems. Unfortunately, in the context of the probe recognition 13ask it is difficult to demonstrate conclusively that a Intrticular subject is processing information in accord with 14 one model or another. The trouble is that each trial generates so little data. Consider, for example, the sequential processing models of Sternberg (1966) and of Theios et al. (1973). Although it is true that the authors differ on the question of whether searches are exhaustive (Sternberg) or terminating (Theios et al.). the crucial difference between the models is the order in which the list elements are processed on each trial. But the probe recognition task provides no evidence bearing directly on the order in which the stimuli are processed. Thus, the present study is concerned with devising and investigating a task which will afford a better opportunity to study information processing by individual subjects. It may be, of course, that no two subjects will process informa- tion in the same way. This in itself would be worth know- ing because it could save researchers from seeking a single process where perhaps not even a typical process, let alone a single process, exists. On the other hand, if the majority of individual subjects can be shown to process information in the same way, that mode of processing might be worth looking for in other contexts--such as probe recognition. The Spy Retrieval Task The task for subjects in the present study was to retrieve spies from a well memorized roster of 20 spies. Each spy in the roster was characterized by six binary attributes. Any given attribute was possessed by 10 of the 20 spies. The spies to be retrieved 15 were specified by the conjunction or disjunction of l to 5 attributes. The subjects responded by naming the relevant spies. For example, a typical search question would be one such as "Name all the spies who are Male or German or Disguisable" or "Name two spies who are Female and English and Assassins." Such a stimulus set offers a number of advantages. First, the spies are defined within the experiment so that extra-experimental associations should pose few problems. Second, because each spy's attributes are assigned, any association or classification of spies is likely to be made on the basis of attributes controlled by the experimenter. Third, because each spy is characterized by a set of attri- butes, it is possible for the subjects to process the roster either in parallel or sequentially. Fourth, "strengths" of spies for various retrieval specifications can be assessed in terms of numbers of matching attributes. Fifth, it is possible to investigate how subjects process conjunctive as opposed to disjunctive combinations of attributes, and to examine the effect of varying the number of attributes specified in conjunctive and disjunctive search specifi- cations. Models for the Spy Retrieval Task This section will offer five models of information processing plausible for the spy retrieval task. The processing each model specifies will be stated in sufficient detail to insure comprehension of the operation of each one, 16 but the mathematical implications of the models will not be explored in great detail. Rather, the implications of each will be considered in just sufficient detail to find characteristics of the response data which may be used to differentiate the models. The sequential list model. This model and the ones that follow hold that the search requests or questions are stored by the subject as a list of attributes. The attri- butes are stored along with an indication of whether the question is conjunctive (in which case a spy must possess all the specified attributes in order to be relevant) or disjunctive (in which case a spy must possess at least one of the specified attributes). In the sequential model the search for relevant spies proceeds through the roster in a fixed order. Each step in the search process begins with "retrieval of a spy", i.e. the spy's name and attributes become available in immediate memory. The spy's attributes are compared with the attributes specified in the question. If the spy is determined to be relevant his name is reported immediately: otherwise, the next step is initiated by retrieval of the next spy. The process terminates when the last spy in the fixed order roster has been retrieved and examined. Within the framework just given there is latitude for a variety of submodels. For example, one could allow the order in which spies are retrieved to be variable--perhaps as Theios et al. (1973) suggested--rather than fixed. 17 Similarly, one could specify that examination of a spy's attributes is exhaustive: alternatively, one could specify that processing terminates as soon as the spy is determined not to be relevant. If a subject were to process the questions in a manner consistent with this model, what would his data look like? First, it is clear that if the roster is processed in a fixed order and relevant spies are named as they are encountered, then the spies will always be named in the same order in responses to questions. Second, the time interval between names in responses to conjunctive questions should be an increasing function of the number of attributes specified in the question. There are two reasons for this. One is that as the number of attributes specified in the question increases, the number of attributes examined per spy will also increase. The other, and probably more important, reason is that as the number of specified attri- butes increases, the proportion of relevant spies decreases. This means that, on the average, more spies will have to be examined in order to find a relevant spy. Hence, the latency for the first response and the inter-response intervals should be an increasing function of the number of attributes in the conjunctive questions. This increase in the latencies, plus the spies being named in the same order to all questions, will serve to distinguish sequential list processors from subjects conforming to other models presented here. 18 The articulated list model. This model is essentially the same as the sequential list model except that the articulated list model assumes that each spy in the roster is linked to other spies which have the same attributes. In its most complete form, this model assumes that in addition to each attribute that a spy possesses there is stored a pointer to the next spy in the list who has the same attribute. Further, it is assumed that the pointers are such that given two pointers, the subject can tell which points to a related spy who is further down in the fixed order roster. The illustration below shows how such a list might be arranged. The first spy, Percy, who is an English male, has pointers to the fourth spy, Faith, who is also English, and to the second spy, Hans, who is also male. The pointers indicate that the next English spy is further down the list from Percy than is the next male spy. l. Percy English(4) male(2) 2. Hans German(B) male( ) 3. Gerda German( ) female(4) 4. Faith English( ) female( ) etc. The pointers simplify searching greatly by linking together all the spies possessing a given attribute. Thus, all the spies possessing a given attribute may be retrieved without accessing any spies who do not possess the given attribute. In this model conjunctive questions are processed with a minimum of time wasted examining irrelevant spies. The procedure for answering a conjunctive question is to start 19 at the top of the articulated list and seek a spy possessing one or more of the specified attributes. If the first such spy encountered is relevant, he is reported: otherwise he is covertly rejected. In either case, the pointers associated with the relevant attributes are checked. If they all point to the same spy, that spy is checked next. If they point to different spies, the spy furthest down the list is the next potentially relevant spy. Thus, in the four spy list given above, a search for English males would begin with Spy 1, Percy, who happens to be an English male. The next male spy is Spy 2 according to the pointer, but the next English spy is Spy 4. Since the spy must be both male and English, it is clear that Spy 2 is not relevant because he is not English. However, Spy 4 is known to be English, and could be male as well. Thus Spy 2 is bypassed in favor of Spy 4. What would happen as one increases the number of attri- butes specified in conjunctive questions? At each step one moves from the current spy to the next spy by following the pointer which indicates the spy furthest down the list from the current spy. That is, one moves down the list by the maximum of the numbers of steps indicated by the pointers. But as the number of attributes in the questions is increased, the number of pointers at each step increases, and the expected value of the maximum distance increases also. Thus, the more attributes specified in the question, the fewer spies will be examined on the average, so that increasing the number of attributes should decrease the latency of the first response and the subsequent intername intervals. V. 20 How are disjunctive questions handled by an articulated list processor? As with conjunctive questions, searching be- gins at the top of the list and continues a spy at a time until a spy is found possessing one of the attributes specified in the question. The subject then follows the pointers until he has named all spies bearing that parti- cular attribute. It is possible that some spies will be missed who do not possess the attribute just processed, but who do possess another attribute specified in the disjunctive question. Therefore, it is necessary to traverse the list in the manner just described for each attribute in order to find all relevant spies. However, care must be taken not to report a given spy more than once. In each traverse of the list after the first, the subject avoids this by checking each potentially relevant spy for the attributes previously processed. This means that the disjunctive responses of an articulated list processor will always begin with a string of ten consecutive spies possessing one of the attributes specified in the question. The string of ten spies will be followed by successively shorter strings of spies possessing the other specified attributes. The sequential attribute-entry model. This model has been called a parallel model by other writers (e.g., Shevell and Atkinson, 1974), but the present writer prefers not to use the term "parallel" because of the incorrect analogy with parallel processing in computers. What is meant by attribute-entry processing is that the subject sequentially 21 processes the entire list of spies with respect to a given attribute before considering another attribute. This is in contrast to the sequential processing suggested in the first model where the subject processes a given spy with respect to all his attributes before passing on to another spy. Consider how conjunctive questions are processed under this model. Suppose a conjunctive question called for spies who are male and English. The subject might begin with the attribute "male" and sequentially examine each spy in the roster, retaining in immediate memory the 10 spies who are male and discarding the 10 who are female. (Okada and Burrows, 1972, have shown that subjects can work with this many entities.) Next, the 10 males are examined in sequence to determine which are English: whenever an English spy is encountered he is reported immediately. The generalization to more dimensions is clear: conjunctive questions are answered by sequentially examining the spies and retaining for further examination those spies which possess one of the attributes specified in the question: the retained subset is then examined in order to select those spies which possess a second of two specified attributes, etc. This process continues until one reaches a semifinal set of spies, all of whom are known to possess all the specified attributes save one. The spies in this semifinal set are then processed and relevant spies are reported as they are encountered. If no relevant spies are found in this semifinal set, then the subject can report the search to be negative. 22 Since half the spies possess any given attribute, about half the spies in the semifinal set will prove to be relevant. If the size of the semifinal set is g, then on the average, one will examine 2 - E‘é—Z spies in order to find a relevant spy. Aslg decreases, this expected number of spies examined moves from 2 toward a minimum of 1. The size of the semifinal set, s, decreases if the number of attributes in a conjunctive question is increased. Thus, as the number of attributes in conjunctive questions increases, the number of spies examined in the semifinal set in order to find a relevant spy will decrease from 2 toward unity. Hence, the conjunctive responses of attribute entry processors will be marked by a gradual decrease in the inter- name time interval as the number of attributes in conjunctive questions increases. The latency of the first response to a conjunctive question however, will be a negatively accelerated increasing function of the number of attributes specified. If one attribute is specified then the entire roster is the semi- final set and the subject can begin reporting relevant spies as he encounters them. If two attributes are specified, all twenty spies must first be examined in order to produce a semifinal subset of ten. Only then can the subject report relevant spies as he finds them in the semifinal subset. Thus if two attributes are specified, at least 21 spies must be examined before the first relevant spy can be reported. If three attributes are specified, then this 23 effect is greatly magnified. The subject first processes the roster of 20 spies to obtain those 10 who have the first attribute. He must then examine these 10 spies for the second attribute in order to produce the semifinal set. Thus he must examine at least 20 + 10 + 1 = 31 spies before the first relevant spy is found. In summary, responses to conjunctive questions by attribute-entry processors will be characterized by first response latencies which are an increasing function of the number of attributes specified, and intername intervals which are a very gently decreasing function of the number of attributes. How are disjunctive questions handled? On the first pass, the subject checks all 20 spies for those who have the first attribute. Those who do are immediately reported. Those who do not have the first attribute are listed in immediate memory for further processing. This list of spies who do not have the first attribute is processed on the second pass as the subject looks for those who have the second attribute (and reports them) and lists those who do not for the third pass (if more than two attributes were specified). Hence disjunctive responses from an attribute- entry processor will begin with 10 consecutive spies who have in common one of the specified attributes, i.e. that attribute which the subject checks first. This string of 10 will be followed by a string of spies who share a second attribute specified in the question, and so on. It is true of disjunctive responses to both models that once a string 24 of spies is completed, no further spies who have the attribute which defined that string should appear as answers. Thus, a question calling for Germans, assassins, or veterans might be answered with a string of 10 German spies followed by a string of 5 assassins which is in turn followed by a string of 3 veteran spies. The direct activation model. The direct activation model is a classical strength model. The premise is that the likelihood of a spy being retrieved is proportional to the associative strength of that spy in response to that question. At the outset of answering questions all the spies have a certain residual strength because the spy roster is the set of entities being dealt with. Furthermore, the spies' strengths are probably not precisely equal because of such idiosyncratic background factors as one spy having the same name as a friend of the subject and there- fore being stronger or more likely to be retrieved than other spies. These differences will be ignored below. The question acts as a stimulus to which the spy names are potential responses. The strength of each such response is directly proportional to the strength of the association between the question and the given spy. This in turn is directly proportional to the number of attributes which the spy shares with the question. That is, the strength of a given spy name is proportional to the number of the specified attributes which that spy has. With the direct activation model as stated, spies gain 25 strength to the extent their attributes are the same as those specified in the questions: the spies lose strength to the extent their attributes are complementary to those specified. How are questions answered? Spies are retrieved in order of strength with the strongest retrieved first. As each spy is retrieved he is evaluated and reported if relevant. Retrieval and evaluation continue until a spy is reached who is irrelevant. At this point the search is assumed to be complete. Retrieval time should be inversely proportional to the strength of the spy. The response characteristics implied by the direct activation model are that spies will be retrieved in order beginning with those spies who have the greatest number of attributes in common with the question: furthermore, the spies who are retrieved first will be retrieved fastest. This will hold for conjunctive and disjunctive questions. The greater the number of attributes specified, the faster retrieval will be. Thus response time will decrease as the number of attributes specified in the question increases. This is true of both disjunctive and conjunctive questions. The subroster model. This model is similar to the sequential list model in positing sequential examination of spies with the relevance of one spy being determined before the next spy is examined. However, in the subroster model the roster of spies is partitioned into independently retrievable subrosters each of which is comprised of those 26 spies in the roster who possess a certain attribute or combination of attributes. For example, one might divide the roster into subrosters on the basis of nationality: this would result in one subroster of 10 English spies and one subroster of 10 German spies. These two subrosters might then be partitioned by sex, resulting in four sub- rosters of five spies each: English males, English females, German males, and German females. Note that every spy in any given subroster must possess all the attributes which define the subroster. Questions are answered by successive retrieval of the subrosters and sequential searching therein for relevant spies. In the case of disjunctive questions this will result in strings of names from the successive subrosters. If the subject had subrostered on sex and nationality, then the answers to a disjunctive question calling for spies who were males or Germans might consist of a string of male Germans, a string of female Germans, and a string of English males. How might such a subject answer conjunctive questions? In the same way: by successive subroster retrieval and sequential examination of the spies therein. Since spies must be examined individually one would expect response times for conjunctive questions to be an increasing function of the number of attributes specified, just as it was in the sequential list model. However, there is one important difference between the two models. When a subject has 27 subrostered, there will be some questions specifying the subroster attributes. If a subroster attribute is specified, then the subject can limit his search for conjunctively relevant spies to a subset of the subrosters. For example, if a subject subrosters on sex and nationality, he can answer a conjunctive question calling for German male assassins by referring only to the subroster containing the five German males. The other three subrosters are entirely irrelevant. Consider two other conjunctive questions specifying three attributes: (a) veteran disguisable assassins, and (b) veteran German assassins. Since half the spies in the roster possess any given attribute, about one spy in eight or, on the average, 2.5 spies out of 20 will be relevant for such a question. If the subject subrosters on sex and nationality, then for Question a, he must search through all four subrosters--all 20 spies--in order to find the 2.5 relevant spies. For Question b however, the 2.5 relevant spies will be found among the ten spies in the German male and German female subrosters. One can easily see that questions which identify subrosters will be answered more quickly than those which do not contain such attributes. This is because the proportion of relevant spies is greater in the former case than in the latter, and the greater the proportion of relevant spies, the more often the subject will encounter a relevant spy in the course of a sequential search. Thus, the responses of subjects who subrostered will 28 have two distinctive features: (a) responses to disjunctive questions will contain a string of names from each subroster in turn, and (b) of the conjunctive questions which specify a given number of attributes, those that contain attributes identifying subrosters will be answered more quickly than those which do not. Differentiation of the models. The foregoing discussion of the models of information processing in the spy retrieval task was by no means complete. However, the description of the models is sufficient to indicate response characteristics which may serve to identify the model a subject used as he answered questions. These character- istics of the models are presented in Table 1. The table has two columns: The left column shows the implications of each model for latency of the first response and the time interval between the first and second response. The right column shows the implications of each model for the order of reported spy names in response to disjunctive questions. Conjunctive questions (RT number of attributes) Sequential 29 Table 1 Summary of Implications of Five Information Processing Models Disjunctive questions (order in which spies are named) as a function of the Response 1: Fixed order across ques- Search Model increasing tions unrelated to attri- Response 2: butes specified in the increasing question Articulated Response 1: Be ins with string of 10 List Model decreasing spIes sharing some attri- Response 2: bute specified in ques- decreasing tion, followed by strings of decreasing length for other attributes. Order of strings may vary across questions, but order of spies within strings is fixed. Attribute Response 1: Same as articulated list Access Model increasing model Response 2: decreasing Direct Response 1: Spies named in order of Activation decreasing number of attributes they Model Response 2: possess in common with the decreasing question Subroster Response 1: Spies named in strings Model increasing, from successive subrosters but decreas- ing as function of number of subroster attri- butes Response 2: same as response one Method The Spy Retrieval Task The spy retrieval task permits an investigation of retrieval of well-learned materials from memory. Subjects memorize a roster of 20 spies, each of whom is characterized by six attributes. After the roster is memorized, subjects are asked to identify spies who possess specified combin- ations of attributes. The Spy Roster Table 2 provides the names of the 20 spies and their attributes. Each spy is characterized by his position on six dimensions, each of which is comprised of two attributes. The dimensions and their attributes are: Sex (male, female) Nationality (English, German) Specialty (assassin, burglar) Disguisability (disguisable, not disguisable) Background (political, technical) Status (veteran, trainee) On each dimension half the spies are characterized by one attribute and half by the other. The attributes were assigned to the spies independently so that, in general, knowledge that a spy possesses one attribute will provide no information on whether he possesses a second attribute. 3O 31 Table 2 The Roster of Spies Nation- Back- Disguis- Spy ality Sex Specialty ground Status ability 1. Anne Eng. f burglar tech. trainee disg. 2. Brian Eng. m assassin pol. trainee not-disg. 3. Colin Eng. m assassin tech. veteran disg. 4. Elsa Ger. f burglar 9 pol. veteran disg. 5. Erich Ger. m assassin tech. veteran disg. 6. Faith Eng. f assassin pol. veteran not-disg. 7. Gerda Ger. f assassin tech. veteran not-disg. 8. Hans Ger. m burglar pol. trainee not-disg. 9. Heidi Ger. f burglar tech. trainee not-disg. 10. Henry Eng. m assassin pol. trainee disg. 11. Klaus Ger. m burglar pol. veteran disg. 12. Liz Eng. f assassin pol. trainee disg. 13. Marlene Ger. f assassin tech. trainee disg. 14. Marta Ger. f burglar pol. veteran not-disg. 15. Otto Ger. m assassin tech. veteran not-disg. 16. Percy Eng. m burglar tech. trainee not-disg. 17. Sarah Eng. f burglar tech. trainee not-disg. 18. Victoria Eng. f burglar pol. veteran not-disg. 19. Werner Ger. m assassin pol. trainee disg. 20. Wilfred Eng. m burglar tech. veteran disg. 32 However, the names of the spies were chosen to reflect their sex and nationality so that the subjects effectively need learn only four attributes for each spy. A dossier was prepared for each spy. Each dossier consisted of a manila folder with a single-page biography inside. A summary slip typed parallel to the 3-inch edge of a 3x5 card was stapled to the front of the manila folder. The summary slips were similar to the following: NAME: KLAUS NATIONALITY: German SEX: Male SPECIALTY: Burglar BACKGROUND: Political STATUS: Veteran DISGUISABILITY:Disguisable The biographies had a standard format of three paragraphs. The first gave the spy's name, birthplace, and a sentence or two on his childhood. The latter part of this first para- graph indicated an unhappy childhood for all assassins and a happy childhood for all burglars. The second paragraph conveyed the spy's educational background plus information about any hobbies or employment which might be useful for assassins or burglars, or which might enhance one's ability to disguise one's self. The third paragraph contains remarks on the spy's tenure and experience as a spy and his ability to work in disguise. 33 The Questions The questions asked of the subjects are displayed in Table 3. As the table illustrates, the questions may be classified in several ways. Question blocks. The questions are organized in four blocks with 36 questions and 8 marker questions in each. Within each block the questions call for many combinations of attributes, but within a given block there is one attribute, the "block attribute", which appears in every question. The block attributes, in the order in which they appear, are "technical", "veteran", "political", and "trainee". Thus, all questions in the first block call for spies who have a technical background: all questions in the second block call for veteran spies, etc. Each block concludes with a set of "marker" questions which remain unchanged throughout. Each marker question calls for five spies. The attributes specified in the eight questions are, respectively, (1) male, (2) technical and trainee, (3) English, (4) political and veteran, (5) female, (6) political and trainee, (7) German, and (8) technical and veteran. The marker questions may be used to assess whether the subjects reordered the spy roster in the course of answering each block of questions. Disjunctive versus conjunctive questions. Each block of 36 questions includes four single-attribute questions. Of the remaining 32 questions, 16 are conjunctive and 16 are disjunctive. The conjunctive questions require the subjects 0\ U1 -F \d N p 1100*} TB gm TBD Egg TGAd ggmp TMD I: Questions 10. 11. 12. 13. 14. 15. 16. 17. 18. 10. 11. 12. 13. 14. 15. 16. 17. 18. 34 Table 3 Listed in Order of Presentation Block 1 TEF 19. TD 20. TA 21. TAD 22. TGA 23. TQFQ 24. TGMA 25. TEMAD 26. TGMAd 27. Block 2 Vd 19. XQM 20. VEM 21. XQMA 22. VFBd 23. ggmgg 24. VEMAD 25. VAD 26. fig 27. TC 2g TEB 33m TEBD Eggp TEFBD 1&1 VF Kg VGF Egg VGFd yggg VGFBd VEFBD 28. 29. 30. 31. 32. 33. 34. 35. 36. 28. 29. 30. 31. 32. 33. 34. 35. 36. TGF TAd TMAD TEFB TFd TGM VD Kg VAd VEMD VEMB VEB 35 Table 3 (cont'd.) Block 3 l. P 10. PB 19. E 28. F 2 . PG 11. 31:3 20. PA 29. _P_D 3 . 3g 12. PEA 21. PM 30. 313g :4. . PEP 13. £111; 22. PCB 31. PE 5 . 312 14. PGF 23. m 32. PAd 6 . PGFB 15. 39m 24. PEAd 33. 11:59 ’7 - M 16. PGBD 25. READ 34. PEFA 8 - d 17. W 26. PEMAd 35. 3&1]! 9 - 13;: 18. PGFBD 27. ppm 36. PBD Block 4 l - t 10. A 19. G 28. M 2 - 3g 11. tM 20. If: 29. td 3 - tF 12. 3g 21. tD 30. LB 4 - 1134 13. tEF 22. _t_EM 31. tAD 5 - tGM 14. 3g: 23. tFD 32. 3151 6 - 593g 15. tGMd 24. 35g 33. tMBd ’7 - “tEFD 16. Eggs 25. yFAD 34. EM 8 - Egg 17. tGMBd 26. W 35. th 9 - 1:13:11 18. 313313 27. tEFAD 36. 1:13p :Eisllflé- G = German T = Technical A = Assassin E = English P = Political B = Burglar M = Male V = Veteran D = Disguisable F = Female t = Trainee d = Not Disguisable D18 junctive questions are underlined. 36 to name spies possessing all attributes specified in the question: disjunctive questions may be answered with spies who possess fl gr; 2933 of the specific attributes. A conjunctive question calling for spies who are male assassins must be answered with spies who are M male an_d assassin, but a disjunctive question with the same attri- butes may be answered with spies who are male, spies who are assassins, or spies who are male assassins. As Table 3 indicates, each combination of attributes appearing in a conjunctive question in one block appears in a disjunctive question in another block. Number of attributes. Within each block, both conjunctive and disjunctive questions vary in specifying from one to five attributes. Number of prepotent attributes. Pilot data indicated that some subjects use sex and nationality to organize the Spy roster. The effect would be more pronounced with the pre— sent spy roster because in the pilot study the spies had ariii-I'lal names (Fox, Tiger, etc.) whereas spy names in the present study reflect both sex and nationality. It is FOSS ible that some subjects might use the a priori prepotent dimehsions, sex and nationality, to set up four subrosters. To deal with this possibility, each block of questions, both con finnctive and disjunctive, contains questions specifying 0 ' 1 . or 2 prepotent attributes. However, the prepotent attr‘ ibutes introduced some constraints in the composition of Clue'S‘tions, the most important being that it is not possible ‘ __—_——____— __1 37 to ask a question of more than three attributes without including an attribute from one of the prepotent dimensions. That is, because two dimensions-~background and experience-- provide block attributes and two more dimensions--sex and nationality--are prepotent dimensions, there remain only two dimensions from which attributes may be selected. Thus, one can form a three-attribute question including the block attribute plus one attribute from each of the two remaining dimensions, but to add a fourth attribute one must use one of the prepotent dimensions. Thus, the five-attribute questions all involve two prepotent attributes: half the four-attribute questions include one Prepotent attribute and half include two. The three- a‘t'tribute questions are equally divided among 0, 1, and 2 prepotent attributes. The two-attribute questions are selected so that half specify no prepotent attribute and half specify one: two-attribute questions with two prepotent attr ibutes were excluded because they would necessarily specify sex and nationality and might induce subjects to S‘113rcaster. Half the one-attribute questions involve a prepotent attribute and half do not. W There were two treatment groups. The whole-answer grOuI) consists of subjects required to name every spy having a. . Specified combination of properties. The other group, the W group, consists of subjects required to name only ‘tw\° spies having the indicated attributes. The two ¥——__ 38 treatments were included to assess the possibility that whole-answer subjects might feel forced to use a conserva— tive sequential roster—processing strategy in order to avoid onflssions. The two-answer subjects, on the other hand, znight feel free to use other retrieval strategies. The distinction between whole-answer and two-answer ssxibjects applies only to the four blocks of questions. The Inaxrker questions were the same for all subjects. Subjects Data reported here were gathered from six female under- égzraatiuates enrolled in psychology of personality courses at Michigan State University. These subjects were paid for tirieazir participation. In the following sections, they are referred to as subjects CP, AH, SP, JL, JK, and NA. Data is also reported for subjects designated as CT, 131(1. 53M, and CM. These subjects served voluntarily and with- out pay as pilot subjects. Subjects SM and CM are female employees of Michigan State University: subject CM is a de335’F-iartmental secretary and subject SM is a computer I32':‘<>1ga:‘ammer. Subject DK is a full-time undergraduate $3t11<3~ent and part-time staff member. All subjects except CT were under 25 years of age. P M Overview. Subjects were run individually. Each 8 . ubj ect partic1pated in one session per day for three 0 . 01-18 ecutive days. Sess1on I was devoted to acquisition of the roster of 20 spies: in Sessions II and III subjects ‘ 39 responded to a series of requests for names of spies with specified combinations of attributes, e.g., female or German spies, male assassins, etc. Sessions II and III began with a warm-up task which c:onsisted of a review of the roster of spies. The experi- nienter named the spies in random order: as each spy was riauned the subject listed his attributes. The subjects were aiJrlowed to refer to a list of dimensions (e.g., nationality, sex, background, etc.) while they were reciting a given spy ' s attributes . At the close of Sessions I and II, subjects were given 61 Jr~easoning problem to solve in order to divert attention fTr‘<>ln the experiment and reduce incorrect rehearsal of the 531>27' roster between sessions. The sessions are described in more detail below. Acquisition of the roster (Session I). Subjects V”’C33li"1—:ed individually. The subject was seated at a table c’j?3f‘£ering a 34“ x 36" work space which was empty except for EL ‘13Eaiblet and pencil. The following instructions were read -t<> 1bhe subject: This experiment is concerned with how people I am going to ask you to act as remember things. if you were in charge of a roster of espionage agents, or spies. You will need to be able to name spies who have certain characteristics. For example, if I asked you to name spies who are males and veterans, you would want to name (all/two) spies who are both male and veteran spies. On the other hand, if I asked you to name spies who are males or veterans, you would want to name (all/ two) spies who are either male, or veterans, or both male and veteran. Do you understand? ¥—_—_—_ 40 After answering any questions, the experimenter shuffled the dossiers and handed them to the subject. When the subject indicated that she was ready to review the roster, the experimenter took the dossiers, shuffled As each name them, and began calling the spies' names. was called the subject responded with the spy's nationality, background, specialty, disguisability, and status, referring as necessary to a list of these five dimensions to The subject insure that she identified all the attributes. was corrected immediately when she missed an attribute, and If the subject the dossier for that spy was set aside. erred on more than five spies, she was advised to study the dossiers some more, and another review was initiated after Otherwise, subjects were asked to “the additional study. redescribe any spies they had missed, plus five additional Spies whom they had described correctly before. When erry spy had been described correctly, the dossiers were put away and the subject was told: We will review the roster again tomorrow before you answer any questions. Now I have a problem for you to solve... theI‘eupon the subject was given one of the problems. The Retrieval from the roster (Sessions II and III). sub 5 set was seated facing the experimenter across the table whél‘e she had previously studied the dossiers. The session began with a review of the roster following the procedure described above for reviewing the roster at the end of Se'Ssion I. At the end of the review, the subject was 41 allowed to look at the dossiers for a few minutes if she ‘wished to do so. When she had finished, the subject was read the following instructions: Now I am going to ask you questions about the spies. Each question will call for names of (all/ two) of the spies with a given combination of characteristics. I will read the question to you and place a card on the desk with the question on it for you to look at. Remember, unless I say otherwise, I want you to name (all/two) spies with the specified characteristics. After any questions about the task were answered, the experimenter began requesting lists of spies having the combinations of characteristics described in Table 3. Coincident with the experimenter naming the last attribute Jirl 'the request, he placed a green 3 x 5 card on the table With the request typed in capital letters parallel to the .5F-:i;rich edge. The card was placed directly before the S1—11'Dject in such a way as to cover the card for the preceding recI«!_‘l...1est: the stack of cards was cleared away after each block (see Table 3) of requests. Table 3 lists all the re‘luests in the order in which they were presented to the Sub fl ects. 1ik1Q3 The same order was used for all subjects. When subject appeared to have finished responding, the experimenter said "Okay?" to insure that the subject was actually finished and then presented the next question. The sessions were tape recorded. At 30 to 40 minute int ervals, sessions were interrupted for about 5 minutes Whl’LILe the experimenter rewound a tape and inserted a new one - Following each such interruption, the subject was 42 asked to reanswer the request she was given just prior to the change of tape, and recording was resumed with the next request. When the subject had responded to the last request, she was asked to review the roster. As usual, the experimenter prompted her with the spies' names and she was allowed to refer to the list of dimensions. When the subject had described all 20 spies, she was given another reasoning problem to solve. Session III was a repetition of Session II except that no problem was given following the roster review. Instead, the subject was asked how she learned the roster, what kind 01? question (conjunctive or disjunctive) was harder to answer, what attributes were hardest to remember, and how She went about answering the questions. wRecording and Trangcription When each subject had finished Session III a transcript was prepared from her tape. The transcript included all u‘t‘terances, both relevant and extraneous, made by either the sub 3' ect or the experimenter. Any vocalization by the exp erimenter was transcribed in upper case letters whereas vocalizations by the subject were recorded in lower case :lEI13‘t2ers. Latency data were prepared with the assistance of a Digital Equipment Corporation PDP-8 computer which was speeially programmed to process input from a Teletype Model 33 terminal as follows: Depressing "e" on the , . 43 teletype set the PDP—8 clock to zero. The clock was read and the reading recorded on paper tape and on the teletype printer whenever the space bar on the teletype was depressed. The procedure for obtaining the response latencies was as follows. The tape recordings for the sessions were played back on the Dejur-Grundig Stenorette on which they were recorded, using a foot pedal to start, stop, and back the tape up. Before timing the responses, each question and its associated responses were read from the transcript and played from the recorder. This permitted verification 0f the transcript and helped avoid mistiming due to extraneous noises. When the transcript was verified for the Cinestion, the tape was backed up and the responses were timed. The clock was started by depressing the "e" key at the termination of the experimenter's reading of the question. At the onset of each response the spacing bar was depressed to record the current clock reading to the nearest hundredth second. In order to assess the accuracy of this procedure, the lat ency measurements were repeated for Questions I to 22 of the first timed subject. This provided 22 first response latencies (i.e., the time interval between termination of the experimenter's voice and onset of the first response) and ’44 intername intervals (i.e., the time between the onset of one response and the next). The median of the distr‘ ibutions of discrepancies between the first and second measurements were then expressed as a percentage of the ¥—_— 1 44 meagu latencies to the conjunctive questions. For first response latencies the median discrepancy was 3.8% of the mean first response time: the median discrepancy for intername intervals was 2.0% of the mean interval between the first and second responses. Thus, it seems unlikely that the manual timing procedure introduced serious distortion in the latency data. Results: The Subjects Who Subrostered Using Sex and Nationality At the conclusion of Session III all subjects were asked how they went about answering the questions. Of the five whole answer subjects, three reported learning the spies in four groups of five Spies each, with all spies in each group being of the same sex and nationality. These three 81113;jects said they processed a question by searching through tliee :relevant subrosters, examining each spy in turn, and reporting the relevant spies. A fourth subject reported learning the spies in groups <3f‘ iTive, but her groups were formed simply by dividing the rarldomly ordered stack of twenty dossiers into fourths vVithout any regard for characteristics the spies might have 111 <:onmmn. Her request processing-~as she described it--was QJJJi‘tee different from that of the three subjects who grouped the spies on the basis of common characteristics. The fifth whole answer subject reported learning the IUDS‘tEEr as a whole without any attempt to subdivide it. She Esai‘i she attempted to divide the spies by sex and nationality in Session II, and when that was "unsuccessful" she began processing the spies one at a time in alphabetical order. Based on their own reports, the whole answer subjects fa11~ into three categories: the three subjects who 45 IIIIIIIIIIIIl-.__l, ,.1141111444p111111144______4______l______1 1 .111 46 capitalized on sex and nationality as anticipated by the experimental design, and two subjects who used idiosyncratic strategies. The two answer subjects also differed in their approach to the task. Three of them subrostered on sex and nationality. A fourth two-answer subject divided the spies into two subrosters consisting of ten veterans and ten trainees. The fifth two-answer subject divided her spies into four subrosters of five spies on the basis of national- ity and background (technical or political). Furthermore, She noted the German political and English technical spies tended to be burglars (true of 8 spies out of 10), while the German technical and the English political spies tended to be assassins (also true in 8 cases out of 10). This chapter will present the data of those subjects who subrostered on sex and nationality and retained their SubPosters. The data for the subjects who subrostered idiosyncratically will be presented in the next chapter. Data presentation for the subjects who subrostered on \Sex and nationality. The whole answer subjects answered each question by naming all appropriate spies, whereas the two answer subjects gave only two answers to all questions except the marker questions. This makes a dramatic difference in the responses to the disjunctive questions where a single attribute identifies lO spies and two or more 8Lttr‘ibutes identify a minimum of 15 spies. Thus, the whole anSWer subjects provided a great deal more data than the two ‘ 47 answer subjects. Consequestly analytical strategies differed for the two groups and the data are most easily presented for each group separately. Data for the whole answer subjects are presented first. Before presenting the data, however, a brief digression is necessary to summarize some int ersubject differences in experimental procedure. Intersubject experimental variation. Two modifications were made in the set of requests during the course of data collection. First, in the interest of obtaining essentially a free recall of the spies in the roster, the request "name all the spies" was instituted as the final marker question after each question block. The all spies question was not made prior to the first block of requests in either Session II or Session III. Subjects CP, SP, and JL responded eight times to the "all spies" question but it was yet not part of the design for subjects SM or CT. The second modification was the deletion of ten dis junctive questions from each block of questions. The deleted questions consisted of all but two of the disjunctive clueStions involving sex or nationality. The two questions Q3 deleted were the four-attribute questions involving either sex or nationality, but not both. The disjunctive questions were deleted to shorten question-answering sessions which both SM and CP found very fatiguing. While SM merely repc‘I‘ted being very tired by the end of Session II, CP was ektr‘emely tired by the midpoint of Session II and was beginning to respond erratically. The session was 48 terminated at that point and CP finished Session II and Session IIIlwith the reduced set of requests the following day. Thus SM and CT responded to 64 disjunctive requests in each session: CP responded to a total of 44 disjunctive questions in Session II and 24 disjunctive questions in Session III, and all other subjects had the reduced set of 24 disjunctive questions in both sessions. The conditions applying for each whole-answer subject ar e summarized below. All spies Number disjunctive questions Subject questions Session II Session III 1 . CT NO 64 .. 2. SM NO 64 64 3 CP YES 44 24 4 . SP YES 24 24 5. JL YES 24 24 Since all data analysis was carried out within subjects, these variations in experimental procedure did not adversely E“Erect the results of the study. Mnctive Responses of the Whole Answer Subjects What can be said of the subjects who reported that they subrostered on sex and nationality? Did they in fact use nationality and sex to partition the roster? If so, how consistent were they in completing examination of the spies in one subroster before passing to another subroster? Were they consistent in the order in which they searched the 49 subrosters? Was the search order consistent within subrosters? These questions will be examined in turn for each of the whole answer subjects who reported subrostering. Pilot Subject CT The first whole-answer data presented will be the responses to disjunctive questions of the pilot subject CT. She participated in four sessions on four consecutive days, one session for each block of questions. As is usually the case with pilot subjects, some variation in procedure took place in the course of data collection. In the first session With CT the questions were given verbally, but not presented on cards: and eight disjunctive questions were omitted. Also, the sessions were not taped so her response latencies were not recorded. Nevertheless, there are marked similarities between the 0rd inal characteristics of CT's responses and the responses 0f the experimental subjects run later. Thus, examination 0f CT's data will serve to introduce the procedures used to analyze the ordinal characteristics of the whole-answer data. Anaolysis of the latency data will be described in a later Sect ion. Subject's self—report. The subject was asked how she leaI‘hed the Spy roster. She replied that she divided the Spies into four subrosters of five spies each so that all Spies in a given subroster were of the same sex and national- lty. Then she memorized the spies a subroster at a time. 50 CT went on to observe that this procedure proved useful when she started answering the questions because she could use the subrosters to conduct an orderly search, i.e., to insure that all the spies were examined without omission or repetition. This was particularly helpful with the dis junctive questions where the answers tended to be quite long, and furthermore, CT noted that many of the conjunctive questions were easy to answer because she could eliminate the spies belonging to one or more of the subrosters. Ordinal analysis of CT's data. Are the ordinal charac- teristics of CT's data consistent with her statements about how she answered the questions? If she used her subrosters as she stated, then certain regularities should appear in her responses. Furthermore, if she did use her subrosters, it will be of interest to determine whether she searched through the subrosters in a fixed order and whether the Spies were examined in a fixed order within the subrosters. The obvious expectation is that if CT answered the cll—lestions by searching through the subrosters one at a time, then her responses should consist of one to four strings of names with each string consisting of a contiguous series of 1flames from a given subroster. This tendency to produce i“trot-Etsubroster strings of names should be most obvious in the responses to those questions which afford CT maximal OpPOrtunity to impose her organization on her response. The conjunctive questions are unsatisfactory for this pun7‘Dose for two reasons. First, conjunctive questions ‘ ___ 51 specifying sex, nationality, or both confine answers to specific subrostersu-that is, the conjunctive questions will in some cases force subroster organization in CT's responses. Second, conjunctive questions specifying two or more attri- butes will generally have at most five relevant spies so that CT would have very little opportunity to respond with more than one or two names from any given subroster. On the other hand, disjunctive questions of two or more attributes typically identify at least fifteen spies selected from all four subrosters, allowing CT considerable latitude in organizing her response. If CT answers disjunctive questions strictly by search- ing through one subroster at a time, then, as noted Previously, each response to a disjunctive question should contain precisely one string of names from each subroster Containing relevant spies. However, if she organizes her sea.rch in some other fashion, then there is no reason to expect one string of names per subroster. Table 4 displays the frequency with which responses to the 56 disjunctive questions contained 1, 2, or 3 strings per subroster. It is obvious from the table that all 56 responses consisted of four strings of spies, one from each subroster. It is inc;- Onceivable that this pattern could occur in all 56 responses unless CT were organizing her search by subrosters. Another way to approach the question of organization is to Check CT's responses to the disjunctive questions for long strings of spies having a given attribute in common. ‘ 52 Table 4 Frequency of Response Strings for Each Subroster in CT's Responses to 56 Disjunctive Questions Number of Responses Having Subroster 1 String 2 Strings 2 Strings German female 56 0 0 German male 56 O 0 English female 56 0 0 English male 56 O O 53 For example, suppose CT processed her subrosters in a fixed order: English female (EF), English male (EM), German female (GF), and German male (GM). Then each response would consist of a string of up to ten English spies followed by a string of up to ten German spies; in addition there would be two strings of at most 5 female spies and two strings of at most 5 male spies. On the other hand, if assassins and burglars were irrelevant for CT's organization, there should be no tendency to produce long strings of assassins or burglars, so that CT's responses would be expected to c ontain many instances of strings of one, two, or three as sassins or burglars. In general, one would expect to find r elatively few strings for attributes CT uses to organize her responses, and that those few strings should be relatively long. Those attributes irrelevant for CT's organization should be characterized by a high incidence of Short strings. Table 5 summarizes CT's responses to disjunctive Que stions, displaying the string distributions for each a*"t‘tzrrfibute. The table clearly shows the organization by na-‘b ionality and sex inherent in CT's use of her subrosters. The predominance of long strings of same—nationality spies ( lengths 8, 9, and 10) in conjunction with the predominance or shorter strings of same-sex spies (lengths 1+ and 5) indicates that CT tended to process subrosters of a given ma"bionality in pairs, finishing one nationality before 8 tarting the other. All attributes other than sex and 54 a Table 5 Classification of Response String Frequencies by Attribute and String Length for CT's Responses to 96 Disjunctive Questions String gengths Attribute g, g 3 _ _ _ _ 9 l9 Tgtgl English 0 3 3 5 7 3 4 11 9 21 65 German 1 O O 2 2 3 7 l6 7 20 58 Female 0 4 9 25 31 l 2 3 3 12 90 Male 0 6 9 27 49 l l l 2 5 101 Technical 102 81 49 ll 4 O O O O O 247 Veteran 106 95 42 8 2 2 O O 0 O 255 Political 121 88 44 7 6 O O O O O 266 Trainee 148 42 62 14 l O l O O O 268 Burglar 163 122 21 2 1 o o o o o 309 Assassin 188 79 35 7 O O O O O O 309 Disguisable 159 99 18 5 8 o o o o o 289 Non-disg'ble 167 71 43 12 l O O O O O 294 55 nationality are characterized by many short strings indicating that the other attributes were not used in organizing the roster of spies. Further examination of the responses to the disjunctive questions confirms the expectation that CT tended to process the subrosters in an order which resulted in grouping the spies by nationality and by sex within nationality. This is evident from Table 6, a subroster precedence matrix. The entries in any given row indicate the number of times a name from the subroster corresponding to the row preceded a name from the subroster corresponding to the column. For example, in the answers to the disjunctive questions, there were 950 instances in which a name from the EF subroster preceded a name from the EM subroster whereas only 266 names from the EM subroster preceded names from the EF subroster. If CT were perfectly consistent in processing her subrosters in a specific order, then the precedence matrix could be arranged so that all entries below the diagonal were zero. The rows and columns of the precedence matrix in Table 6 are arranged in the order in which CT most consistently Processed her subrosters, and it is evident that the order 0f processing varied considerably from question to question. CT was more consistent in processing female spies before male (about 10 to 3) than in processing English spies before German (about 3 to 2). Thus, the picture that emerges is that CT tended to process the English subrosters as a pair and to process the German subrosters similarly. The German 56 Table 6 Subroster Precedence Matrix for CT's Responses to 56 Disjunctive Questions English German Female Male Female Male English female --- 950 749 718 English male 266 ~-- 634 632 German female 419 455 --- 987 German male 439 557 282 --- Note. Spies from the subrosters corresponding to rows were named before spies from the subrosters correspond— ing to columns when spies from both subrosters were named in the same response. 57 pair was nearly as likely to precede the English pair as vice versa, but within either nationality the female sub- roster was typically processed first. Was there some reason that sometimes English, and some- times German spies were processed first? Further examination of CT's responses indicated that she may have saved until lggt those subrosters containing spies of the nationality and/or sex specified in the question. That is, if sex or nationality is specified in the question, then all the Spies in these subrosters are relevant and such a subroster need merely be listed. The subrosters processed figgt were those containing spies who were ngt of the specified sex or nationality, i.e. subrosters in which each of the spies must be examined individually for relevance. The utility of organizing her search this way is that CT need examine individual spies in at most two, and often only one, subroster for any disjunctive question specifying sex and/or nationality. When the spies which must be individually examined are disposed of, CT needs not even remember the question; she merely names the spies in the remaining two or three subrosters. After the first few questions CT was quite consistent in responding to disjunctive questions in the manner just described: (1) There were 23 disjunctive questions specifying sex and nationality. The last 20 were answered beginning with spies from the subroster Of the complementary sex and nationality. (2) There were 17 questions which specified either sex or nationality; 58 in all 17 both subrosters of the complementary sex or nationality were processed first. (3) Finally, there were 16 questions which specified neither sex nor nationality. The first four were processed in the order EF, GF, GM, EM, and the last 12 were processed in the order EF, EM, GF, GM. Summary of the ordinal analysis of CT's data. CT des- cribed herself as having used sex and nationality to subdivide the spy roster into four subrosters, each contain— ing spies of a given sex and nationality. She said that this organization proved helpful in answering both the conjunctive and disjunctive questions. If she used her subrosters as she stated, her responses should be character— ized by relatively long strings of German, English, male, and female spies. Upon examination, it was found that each response to a disjunctive question consisted of precisely four strings of names, one string per subroster. Further examination revealed that she tended to produce strings of 8 to 10 consecutive German spies and English spies, strings of 4 or 5 male spies and female spies, and all other attributes appeared in strings predominantly of length 1 or 2. Also, it was found that the order in which CT processed her subrosters depended upon the attributes specified in the Question: If neither sex nor nationality were specified, CT tended to process subrosters in the order EF, EM, GF, GM; however, if sex, nationality, or both were specified, then CT processed the complementary subrosters first. All these features of CT's data are consistent with retention and use 59 of the subrosters in answering the disjunctive questions. Subject JL (a Whole Answer Subject) Subject's self-report. JL was asked how she learned the roster and how she answered the requests for lists of spies. Her answers were: (1) She subdivided the spy roster by sex, and within sex, by nationality to produce four subrosters of five spies each--(a) female Germans, (b) female English, (0) male Germans, and (d) male English spies. She learned one subroster at a time until she had mastered all four. (2) She responded to questions by searching through the relevant subrosters, evaluating each spy individually, and reporting any spies who met the criteria imposed by the ques- tions. To the extent JL's description of her processing activities is valid, one can expect certain regularities in her data; i.e., responses from each subroster in turn. Ordinal analysis of JL's responses to disjunctive questions. The question which imposes least constraint on the subjects is the request to name all the spies. Since all spies must be named, irrespective of their attributes, any tendency to respond consecutively with spies having one or more attributes in common should reflect grouping within the spy roster by JL. The "all spies" question was given JL at the end of each block of questions in Sessions II and III for a total of eight repetitions. Table 7 classifies the frequencies with Which JL responded to the "all spies" question with strings Tab 60 1e 7 Classification of Response String Frequencies in JL's Eight Responses to the "All Spies" Request Attribute Male Female German English Political Technical Assassin Burglar Non-disg'ble Disguisable Veteran Trainee by Attribute and String Length HA 0000 32 37 27 20 43 55 3o 17 IN 0000 21 ll 30 14 ll 22 27 O O Ix» I—‘KJJOO'QNOO MON H3” OOOOOi-‘OOOOOO String gengths .5. _ Z O O O O O 0 l6 0 0 l6 0 O O O O 7 O O O O O O O O O O O O O O O O O O O 0 00000 0000 O to 00000000 0000 [H o c> C) C» (m o CO 0000 61 of spies who shared a common attribute. The rows of the table correspond to the attributes characterizing the spies, and the columns indicate strings ranging from one to ten spies having a common attribute. The table shows that JL was perfectly consistent in grouping her spies by sex (one string of ten male and one string of ten female spies in each of the eight questions) and by nationality within sex (two strings of English spies and two strings of German spies in each of the eight questions). Thus, it seems reasonable to accept JL's statement that she organized her spies by sex and nationality. However, it might be the case that requests to name all 20 spies are qualitatively different from requests to name spies having certain combinations of attributes, and that JL processed the latter differently. Possibly naming all 20 spies forced JL to work with subgroupings in order to remember which spies had been named at any point in her response. Thus the evidence for subrostering found in responses summarized in Table 7 may be relevant only for "all spies" requests. Table 8 displays the distribution of string lengths for JL's responses to the 48 disjunctive requests. This table clearly indicates that the spies in JL's responses to the disa’unctive questions tended to be grouped by sex and by nationality within sex, as were the responses to the "all Spies" questions. Over 50% of the strings of male and fenuile names were of length 9 or 10; 92% of the strings of 62 Table 8 Classification of Response String Frequencies by Attribute and String Length for Responses to 96 Disjunctive Questions Attribute 1 2 2 3 .5. Q .7. 13. 2 19 Male 0 2 1 O 2 3 6 12 15 16 Female 0 1 2 O 1 4 8 14 13 14 German 5 6 20 38 43 1 O O 1 0 English 6 2 14 40 5o 0 o o 1 0 Political 157 87 28 8 2 O 1 O O 0 Technical 176 76 35 7 1 o o o o o Assassin 146 61 15 16 17 1 0 O O O Burglar 142 133 14 3 O O O O 0 O Disguisable 246 78 18 2 O O 0 0 O O Non—disg'ble 203 75 26 6 1 1 O 0 O 0 Veteran 186 87 22 5 1 O O O O Trainee 132 99 27 8 4 O O O O O 63 male or female responses were of length six or greater. Similarly, German and English spies tend to occur primarily in strings of length three to five (91%). By contrast, 80% or more of the strings for all other attributes were of length 1 or 2 (the modal length was 1 for each) even though these attributes appeared much more frequently in the disjunctive questions than attributes specifying sex or nationality. Hence, it appears that organization by sex and nationality was not confined to JL's responses to the "all spies" question. The order of selectingisubrosters. In addition to her consistency in organizing her responses by sex and national- ity within sex, JL was consistent in naming female spies before male spies, and German spies before English. This consistency is apparent from the subroster precedence matrix in Table 9. Since the rows and columns of the precedence matrix are in the hypothesized order, all non-zero entries below the upper-left/lower-right diagonal represent responses which were not in the expected order--e.g., there were eight instances in which a name from the English female subroster preceded a name from the German female subroster. Since the entries below the diagonal sum to 33, there were 33 instances in which JL named a spy out of order across the subrosters. A search of her data shows that 25 of these 33 inconsis— tencies occurred in JL's response to BEE particular question. All- told, inconsistencies occurred in only 5 out of 104 64 Table 9 Subroster Precedence Matrix for JL's Responses to Non-marker Questions in Session III German female English female German male English male German English Female Female --- 526 8 --.. 5 20 O 0 German Male 513 442 English Male 450 568 5u8 65 questions. In other words, in 99 of 104 questions, JL was perfectly consistent in reporting strings of names from the subrosters in the order German female, English female, German male, English male. Of the 104 requests, 16 were single attribute questions, 64 were conjunctive questions and 24 were disjunctive questions. It would appear that JL preferred this fixed order for processing the subrosters irrespective of the type of request or the number of attributes involved. Search order within subrosters. Precedence matrices may also be used to assess the extent to which JL processed spies in a fixed order within subrosters. In this context the rows and columns of the precedence matrices represent spies belonging to a given subroster: the entries in a given row indicate the number of times the spy corresponding to the row was named before the spy corresponding to the column when both appeared in the same response. Matrices for JL's four subrosters appear in Table 10. They indicate a marked tendency for JL to name spies in alphabetical order. Summary of ordinal data analysis for JL. The analyses just described indicate the following: (1) JL's responses to "all spies" and multi-element disjunctive questions were almost always organized hierarchically by nationality Within sex. That is, the responses tended to be a long string of female spies followed by a long string of male Spies: within strings of same-sex spies the German spies were named first, followed by the English spies. 66 Table 10 Precedence Matrices for the Spies within JL's Four Subrosters Subroster Erich Hans Klaus Otto Werner Erich - 25 31 29 30 German Hans 2 -- 28 19 28 male Klaus O 4 -- 15 25 Otto O 5 8 -- 21 Werner O 2 4 2 -— Elsa Gerda Heidi Marlene Marta Elsa -- 28 22 22 30 German Gerda 2 -- 22 18 30 female Heidi l 4 -- 13 15 Marlene O 8 9 -- 9 Marta 1 O 11 10 -- Brian Colin Henry Percy Wilfred Brian —- 24 31 3o 26 English Colin 2 —- 24 23 37 male Henry 0 1 -- 19 22 Percy 1 2 3 -- 25 Wilfred 2 1 4 6 -- Anne Faith Liz Sarah Victoria Anne -- 22 30 33 23 English Faith 3 -- 21 19 30 female Liz 3 ll -- 23 26 Sarah 1 6 6 -- 22 Victoria 2 3 3 4 ~- 67 (2) In cases where more than one subroster was used to answer a question, JL consistently accessed the subroster in the order (a) German females, (b) English females, (0) German males, (d) English males. (3) Within subrosters, JL tended to process the spies in alphabetical order. Thus, all the ordinal characteristics of JL's data are consistent with her statement that she organized the spies in subrosters and processed the subrosters in order, examining each spy in turn to determine his relevance for the given question. Subject SM (A Whole Answer Subject) Subject's self-report. SM stated that she learned the roster in subgroups defined by sex and nationality: that her search procedure capitalized on this organization for responding to conjunctive requests, and that she processed the rosters by sex within nationality when she responded to disjunctive questions. According to her report, SM's behavior closely resem- bles CT's and JL's behavior: SM also seems to be a subroster processer. Therefore SM's data were subjected to the same analytical procedures as JL's data. Distribution of string lengths. SM was the first whole answer subject, and was not given the "all spies" request as 'the last marker question after each block. Thus, the distri- tnztion of strings for each attribute was obtained only for heI‘responses to disjunctive questions with long answers. 68 However, unlike JL, SM responded to the complete set of 128 disjunctive questions. The distributions of string lengths for Session III are shown in Table 11. The table shows a clear tendency for names of German, English, male, and female spies to occur in long strings as would be expected if SM were responding from subrosters defined by sex and nationality. However, the distribution of strings is somewhat less striking than the distribution obtained from JL's data. Thus, a second analysis was carried out on SM's responses to the 64 disjunctive requests in Session III. All 64 disjunctive questions required at least one name from each subroster. Without exception, SM's responses to these questions consisted of four strings 2; names, one string corresponding tg each subroster. Search order across subrosters. Given this consistency in grouping names from within subrosters, why is it that SM's string distributions are less clear-cut than JL's? The reason is that SM did not process the subrosters in a fixed order; rather, the order of processing was related to the attributes specified in the question in a manner similar to that observed for subject CT. Whenever a disjunctive question specifies sex, nationality, or both, spies in subrosters defined by the specified sex and/or nationality are all relevant; searching for relevant spies is trivial in ENJCh subrosters. However, searching is non-trivial in the remaining subrosters because each spy must be examined in 'turuq, Whereas CT tended to process the trivial subrosters 69 Table 11 Response String Frequencies for SM's Responses to Session III Disjunctive Requests, Classified by Attribute and String Length String Length 6 7 Attribute _1_ g 3 3 5 __ __ §_ 9 German 29 23 7 20 35 2 13 17 14 English 36 17 19 21 62 2 6 17 6 Male 22 14 23 35 4O 3 4 11 11 Female 19 14 23 20 56 2 11 15 16 Political 280 166 79 29 15 O 2 O 0 Technical 273 246 45 20 11 2 O O O Assassin 362 225 58 7 5 O 0 0 l Burglar 401 167 68 19 3 1 1 O O Disguisable 260 161 72 31 23 2 O O O Non-disg'ble 279 228 63 21 6 1 O O 0 Veteran 136 189 80 21 23 12 2 1 O Trainee 143 169 127 20 17 2 l 1 O 32 3O 36 22 000 70 last, SM tended to process the trivial subrosters first. If there were no prepotent attributes in the question (i.e., there were no trivial subrosters) then SM usually processed the subrosters in the following order: (1) English male, (2) English female, (3) German female, (4) German male. Table 12 shows the consistency with which SM processed the subrosters in the order just described. In the one departure from this order, she still grouped the trivial and nontrivial subrosters, but she processed the nontrivial subrosters first. Search order within subrosters. Table 13 presents the within-subroster precedence matrices for SM's responses to all Session III questions except the marker questions. It is evident that, unlike JL, SM did not process subrosters in alphabetical order. Nor was there another fixed order for each subroster. The German females Egg; processed in a fixed order with relatively few exceptions--as was the case for each of JL's subrosters. Likewise, SM was quite consis- tent within the English male subroster except for Colin and Wilfred who preceded one another with approximately equal frequency. Since Colin and Wilfred are quite similar-—being both disguisable English male veterans with technical back- grounds-~it may be the case that Colin and Wilfred are stored as a unit or retrieved as a unit with one about as likely as the other to be attended to first. The same situation can be seen in the English female subroster. It is clear from the precedence matrix that SM 71 Table 12 Frequencies of Subroster Search Orders in SM's Session III Responses to Disjunctive Questions Number of Prepotent Fixed Order Trivial Attributes (EM,EFLGF,GM) First Other 0 16 O O l O 23 l 2 O 24 O 72 Table 13 Subroster Precedence Matrices for SM's Session III Responses to Block Requests Subroster Brian Percy HenryI Wilfred Colin Brian -- 54 62 I 56 53 English Percy 5 -- 54 61 54 male Henry 0 O -- I 52 49 Wilfred O O 3 ' -- 36 Colin 0 o 3 | 31 -- Sarah Anne Liz I Faith Victoria Sarah -- 59 35 | 49 52 English Anne 3 -- 35 I 46 48 female Liz 22 23 -- 55 52 Faith 1 6 O I -- 5 Victoria 1 O 6 6 -- Marlene Heidi I Marta Elsa Gerda Marlene -- 59 I 44 5O 58 German Heldl O -- 45 52 59 female Marta 0 6 I -- 57 55 Elsa O 1 3 -- 49 Gerda o 2 l o 8 -- Klaus Erich Otto I Hans Werner Klaus -- 48 51 I 34 34 German Erich 13 -- 59 32 33 male Otto 4 8 -- | 31 Hans 27 24 23 F _- 5% Werner 24 22 18 I 5 -- Note. Spies corresponding to rows were named before spies corresponding to columns when same response. both spies were named in the 73 consistently processed Sarah, Anne, and Liz before going on to Faith and Victoria. Sarah, Anne, and Liz are all English female trainees: Sarah and Anne are both burglars with technical backgrounds whereas Liz has a political background and is an assassin. Furthermore, it is clear that within this trio, Sarah and Anne were paired, with Sarah nearly always preceding Anne: Liz either preceded the Sarah-Anne pair (about 40% of the time) or followed the pair (60% of the time). Careful study of the precedence matrix for German males indicates that there are two groups of spies within which order of processing is fairly consistent, but either of which is likely to be processed first. One group is the veterans--Klaus, Erich, and Otto. The other group is the pair of trainees-~Hans and Werner. The veterans were processed first slightly more than half the time. It is evident that SM has imposed further structure within her subrosters. The precedence matrices in Table 13 have the rows and columns in the order in which spies were named with the greatest consistency. For the first three subrosters it is possible to draw a line between adjacent columns such that all spies to the left of the line are trainees and all to the right are veterans. In the case of the remaining subroster, the German males, a line between the columns for Otto and Hans separates the spies into veterans on the left and trainees on the right. Thus, the spies in each of SM's subrosters can be 74 separated into two groups-~veterans and trainees. Within each such group SM was fairly consistent in order of response. The single exception is the trio of English female trainees which itself was apparently subdivided into a pair of technical burglars and a single political assassin. Furthermore, in three of the four subrosters, SM was consistent in processing the group of trainees first: the exception is the German male subroster wherein the trainees were processed first about 40% of the time while the veterans were given priority the remaining 60% of the time. Summary of ordinal characteristics of SM's disjunctive responses. SM's data corroborated her statement that she subrostered on sex and nationality. There was strong support for her statement that she processed the subrosters by sex within nationality to produce responses to disjunc- tive questions. However, this was complicated by the fact that she always saved nontrivial subrosters until last so that her response string data were not so clear-cut as JL's. Within her subrosters SM imposed further structure: she split them into trainee spies and veterans. Within the veteran and trainee groupings she processed the spies in a consistent order. In three of the four subrosters she was consistent in processing trainees first and then veterans: however, in the German male subroster she processed the veterans first about 60% of the time. 75 Disjunctive Responses of the Two-Answer Subjects Each of the five two-answer subjects was asked how she had learned the roster and how she had answered the questions. All five subjects reported subdividing the roster into groups of spies so that all spies within a given subroster had at least one characteristic in common. Three of the two-answer subjects (JK, NA, and CM) grouped the spies by sex and nationality. Another subject (DK) also grouped spies by two characteristics, but she chose to use nationality and background (political or technical). Thus, four of the five two-answer subjects organized the spies in four subrosters of five spies each. The fifth subject divided the roster on the basis of status: this produced a subroster of ten veterans and a subroster of ten trainees. Data for the three subjects who subrostered on sex and nationality will be examined here: the two remaining two-answer subjects will be considered in a subsequent chapter. However, most of the analytical procedures which proved useful with the whole-answer subjects are not applicable to the two-answer data. Instead, analysis of this data will rely heavily upon examination of what spies were paired in the responses, how often pairs were repeated, how many differ- ent pairs were used, etc. One procedural Point is worth noting: Having to identify only two of the spies relevant for each question is much less demanding than having to name every spy who is 76 relevant. As one might expect, the two-answer subjects found the task much less tiring than did the whole-answer subjects. Hence, there was no need to delete some of the disjunctive requests in order to reduce fatigue in the two- answer subjects, so that all two-answer subjects responded to the complete set of disjunctive requests in Sessions II and III. One variation in experimental procedure occurred in the course of collecting the two-answer data--the addition of the "all spies" question. Subjects DK and CM were 223 asked the "all spies" question, but subjects AH, JK, and NA each responded eight times to the all spies question which was presented as the last of the marker questions following each of the four question blocks in Sessions II and III. Subject JK (A Two-Answer Subject) Subject's self-report. JK stated that she found it helpful to subdivide the roster during acquisition, but that she did not depend on the subrosters when she responded to questions. She was unable to describe precisely how she did respond to a question. She said that spies seemed to "occur" to her as each request was made, and that she recognized immediately whether the spies were relevant or not: that is, she was not aware of evaluating the spies for characteristics specified in the search request. Analysis of JK's data. The question which occurs first about JK is whether her spy roster retained the structure she imposed when she learned it. Table 14 shows the Distribution of Attribute String Lengths 77 Table 14 in JK's Eight Responses to the "All Spies" Question Attribute German English Male Female Political Technical Assassin Burglar Disguisable Non-disg'ble Veteran Trainee o c>lk4 O 37 31 26 34 20 26 39 IN \‘iOOOO 14 12 13 19 {T \7 O O O OM 10 14 12 o u) as :9 o H‘ A) C) <3 o c»! \A) String gengths OOOOOI—‘OOCDOO‘t O O 0 00000000 O 0 000000000 000 00000000 0 ho 00000 000000 |—‘ O O 0 DU”! 4:" CDUXIO OOO 78 distribution of attribute strings in her eight responses to the "all spies" question. The data in Table 14 indicate that spies were grouped by sex and nationality in all eight of her responses to the "all spies" question. Since the shortest observed string for either sex or nationality is five, she was perfectly consistent: she always responded with all the names from one subroster before going on to the next. Thus, her subrosters did remain intact throughout Sessions II and III. Since JK's subrosters remained intact, one might ask whether she might simply have been unaware of relying on the subrosters in responding to requests. One might infer that JK used her subrosters if her responses include a preponderance of intrasubroster pairs. How many intrasubroster pairs could be expected by chance? Note that the appropriate data are the responses to the disjunctive questions because the conjunctive questions often confine the responses to one or two sub- rosters. The simplest disjunctive question specifies two attributes; such a question may be answered by any two of at least 15 spies. As the number of specified attributes increases, so will the number of relevant spies. Thus, the minimum number of distinct relevant pairs for a disjunctive request is 105, the number of distinct pairs which may be formed from 15 spies. On the other hand, if all 20 spies were relevant for a given request, 10 pairs of spies could be formed from each 79 subroster, making a total of 40 relevant intrasubroster pairs. Thus, the maximum number of relevant intrasubroster pairs for a disjunctive question is 40. Dividing the maximum number of relevant intrasubroster pairs by the minimum number of spies relevant for any disjunctive question will provide a conservative estimate (that is, an over-estimate) of the probability of choosing an intrasubroster response at random. Thus, 40/105 is a conservative estimate of the probability of random selection of an intrasubroster pair from the set of spies relevant for any given disjunctive question. If selection of pairs is assumed to be independent from question to question, then the number of intrasubroster pairs selected randomly from the set of relevant spies is a binomial random variable. Since JK responded to 64 Session III disjunctive questions, the parameters of the binomial distribution descriptive of her responses are n = 64 and p = 40/105. The expected number of intrasubroster choices is 64p or 24.38 with a standard deviation of Vhp(l - p) or 3.89 pairs. JK's responses to the 64 Session III disjunctive questions are summarized in Table 15. The table indicates that 46 of JK's 64 responses were intrasubroster pairs. The z score for 46 intrasubroster pairs is 5.56; i.e., 46 responses are 5.56 standard deviations above the expected number of intrasubroster pairs formed by random selection. Using the normal approximation to the binomial distribution, 80 Table 15 Subject JK's Responses to Session III Disjunctive Questions Within Subroster Pairs German male English male English female Hans/Klaus (4) Brian/Henry (4) Faith/Liz (4) Otto/Erich (2) Colin/Henry (4) Liz/Anne (4) Otto/Klaus (2) Colin/Wilfred (2) Faith Sarah (2) Klaus Erich (l) Henry/Percy (2) Anne Faith (1) Hans Werner (1) Percy/Brian (l) Sarah/Anne (1) Percy/Wilfred (l) Sarah/Liz (1) Total 10 Percy/Colin (l) Victoria/Anne (1) German female Total 15 Total 14 Heidi/Marta (3) Gerda/Marta (2) Marlene/Marta (2) Total 7 Cross-Subroster Pairs Within nationality Within sex Other Otto/Gerda (l) Brian/Hans (l) Gerda/Percy (l) Elsa/Hans (l) Henry/Klaus (1) Marta/Percy (l) Heidi/Hans (l) Henry/Hans (1) Marta/Klaus (1) Percy Hans (1) Total 2 Liz/Brian (l) Otto Colin (1) Wilfred/Anne (l) Gerda/Liz (l) Elsa/Victoria (1) Total 6 Sarah/Heidi (l) Sarah/Marta (l) Faith/Marta (1) Total 10 Note. Frequencies are given in parentheses. 81 one finds that the probability of randomly selecting 46 intrasubroster pairs in 64 responses is less than 0.001. Thus, it is clear that JK produced many more intrasubroster pairs than could be expected by chance if she were randomly selecting pairs from among the set of relevant spies for each disjunctive question. This would suggest that JK used her subrosters in answering the Session III disjunctive questions. However, there is a possible alternative explanation for the high frequency of intrasubroster pairs. Table 16 shows the frequency with which spies named by JK in Session III possessed one or more attributes specified by disjunctive questions. A spy is relevant for a disjunc- tive question if he possesses even a single specified attribute. It is clear from the table that the spies named by JK far exceed this minimal criterion for disjunctive rel- evance. In fact, of the 128 spies she named, 64 met a conjunctive criterion for relevance (i.e., possession of all specified attributes) and another 51 were within one attribute of meeting a conjunctive criterion. The probability of this many conjunctive or near conjunctive agreements is very small if JK were seeking to satisfy the minimal disjunctive criterion. Rather, it is likely that JK was seeking spies which satisfied a conjunctive criterion even though the questions were disjunctive. If JK were to give conjunctive responses to all questions specifying two prepotent attributes, and as expected by chance, to respond with intrasubroster pairs to s the questions specifying one 82 Table 16 Frequency Distribution of Attribute Agreements in JK's Responses to Session III Disjunctive Questions 130%hgtgfiiggggg 1 Number 2f Matchigg Attribfites 5 2 8 (16) 24 (16) x x x 3 3 (12) 22 (24) 23 (12) x x 4 O (4) 3 (12) 16 (12) 13 (4) x 5 0(1) 0 (4) 7(6) 5 (4) 4 (1) Note. The number of agreements expected by chance if JK were selecting spies on the basis of one matching attribute are given in parentheses. 83 prepotent attribute and to i the questions which did not specify a prepotent attribute, then there would be 40 intra- subroster pairs in her 64 responses to the disjunctive questions. Thus, one may well ask whether JK's 46 intra- subroster responses are due to intrasubroster searching or due to use of a conjunctive search criterion. It is possible, of course, that JK used a conjunctive search criterion implemented by searching within subrosters. If JK were using a conjunctive criterion but not using her subrosters--i.e., weighting the prepotent attributes equally with other attributes, then the expected number of conjunc- tively perfect responses should depend only on the number of attributes in the question. However, if JK is using her subrosters, then the number of conjunctively perfect responses should increase as the number of prepotent attributes increases for questions having any fixed number of attributes. Table 17 displays the number of conjunc- tively perfect responses as a function of total attributes in the questions and of the number of prepotent attributes. Comparisons are possible for the 2-, 3-, and 4-attribute questions. In each case the number of conjunctively perfect responses increases as the number of prepotent attributes increases (total possible in each cell of the table is 16 responses). Thus, it appears that JK used her subrosters in implementing a conjunctive criterion for the disjunctive questions. 84 Table 17 Frequency Distribution of JK's Conjunctively Perfect Responses to Session III Disjunctive Questions Classified by Total Attributes and Prepotent Attributes Prepotent attributes Total attributes in the question in the question 2 3 4 5 o 11 5 - _ 1 13 7 5 - 2 - 11 8 4 Note. Maximum possible in each cell is 16 conjunctively perfect responses. 85 Subject CM (A Two Answer Subject) Subject's self-report. CM said that she learned the roster as four groups of spies, but she did not master one group before going on to the next. Rather, she studied the groups in turn until she was ready to review the roster with the experimenter. She stated that she defined the groups in terms of sex and nationality because that seemed a natural way to do it. Like JK, she felt that she did not rely heavily on the subrosters in answering questions: she said that she just tried to think of spies who fit the search specifications. Analysis of CM's data. Subject CM's responses to the Session III disjunctive questions are summarized in Table 18. They contrast sharply with JK's. JK used many different pairs (40) and used them sparingly--no pair occurred more than four times. CM, on the other hand, used relatively few pairs and used some of them extensively. Two pairs, Otto/ Gerda and Anne/Faith, were used 17 times each; together these pairs account for over half CM's answers. One is tempted to speculate that CM used the two favorite pairs for the disjunctive questions for which they were appropriate, rather than searching systematically through the subrosters she formed while learning the spies. It should be possible to relate CM's choice of these favorite pairs to the attributes occurring in the questions. 86 Table 18 Subject CM's Responses to Session III Disjunctive Questions Within Subroster Pairs German male German female English female Hans/Erich (7) Elsa/Gerda (3) Anne/Faith Klaus Otto (l) Heidi/Elsa (l) Faith/Sarah Hans Klaus (l) Heidi/Marta (l) Sarah/Liz Total 9 Total 5 Total English male (none) Cross-Subroster Pairs Same nationality Same sex Other Otto/Gerda (17) Anne/Elsa (2) Faith Hans Erich/Gerda (l) Hans Sarah Total 2 Sarah/Otto Total 18 Total Note. Frequency of usage is given in parentheses. (1 7) (5) (2) 24 AAA 0\ H NW VVV 87 The attributes shared by the members of the favorite pairs are: Gerda/Otto -- German, assassin, technical, veteran, non-disguisable Anne/Faith -- English, female One would expect CM to use these pairs in response to questions specifying one or more of the attributes which characterize them. Thus, Anne/Faith might be the predominant response to questions calling for English and/or female spies, and Otto/Gerda could be the preferred response to questions involving any of several attributes. Table 19 lists CM's responses to the questions calling for spies who are not-disguisable, English, female, or assassins. The first column contains Ell questions calling for not-disguisable spies; no other column contains a question involving that attribute. The second column contains all remaining questions calling for female spies: no column other than the first or second contains a question involving the attribute "female". The remaining questions calling for English spies are listed in the third column, and after the questions calling for English spies are considered, the last of the questions calling for assassins are listed in the fourth column. These 49 questions include all 17 of the questions answered with "Anne/Faith" and 15 of the 17 "Otto/Gerda" responses. Study of Table 19 yields the following observations. First, it is clear that most questions calling for English 88 Table 19 CM's Responses to Session III Disjunctive Questions Calling for Spies Who Are Not-disguisable, Female, English, (1) Not-disguisable (2) Female TED TFGD PBD PGMBD TD TEBD vAD tAD VGMAD tEMAU PD PMD TFGED PGMD PMBD VGAD TED tEAD Sarah/Faith Sarah/Faith Sarah/Faith Sarah/Faith Sarah/Faith Sarah/Liz Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Elsa/Gerda Erich/Hans Faith/Hans Sarah/Otto Anne/Faith DF VEF PEF PFD PFED tFGB PFAD tFGBD VEFBD TF TFG tFG PFEAD Anne/Faith Anne/Faith Anne/Faith Anne/Faith Anne/Faith Anne/Faith Anne/Faith Anne/Faith Anne/Faith Sarah/Liz Erich/Gerda Elsa/Gerda Otto/Gerda or Assassins (3) English VE Anne/Faith tE Anne/Faith tEA Anne/Faith VEB Anne/Faith tEMA Anne/Faith TEMD Anne/Faith TEMAD Anne/Faith tEM Hans/Erich TEM Hans/Erich VEMB Otto/Gerda VEBD Otto/Gerda (4) Assassins VGA VGMA tA TAD PAD TMAD VA Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Otto/Gerda Hans/Erich Faith/Hans 89 and/or female spies were answered with the pair Anne/Faith. In fact, "Anne/Faith" never appears as a response unless the question involves the attributes "female" and "English" either singly or in combination. Sixteen of the 17 occurrences of this pair are found in the columns of ques- tions calling for English or female spies: the single exception is the appearance of Anne/Faith in the column for not-disguisable questions. Even the latter question calls for spies who may be English as well as not—disguisable. Second, one notes that the pair Otto/Gerda is used primarily in response to questions in the not-disguisable and assassin columns. Further examination indicates that 10 of the 15 appearances of Otto/Gerda are for questions calling for assassins. Even in the column of questions calling for not-disguisable spies Otto/Gerda appears seven times, but four of the questions call for spies who may be assassins as well as being not-disguisable. With exceptions noted below, CM was more consistent in associating Otto/Gerda with questions calling for assassins than with any other type of question. Thus, CM's use of Anne/Faith and Otto/Gerda seems clear: Anne/Faith is the primary response to questions calling for female and/or English spies and Otto/Gerda is the primary response to questions calling for assassins. The reader may have noted that Otto/Gerda was not the response to eight of the questions calling for assassins. This is not due to inconsistency on CM's part, but rather 90 to one attribute dominating another. Five of the eight aforementioned questions call for spies who could be English or female as well as assassins. In each of the five cases, the answer was "Anne/Faith", indicating that CM responded to the attributes "English" or "female" rather than the attribute "assassin". That is, the former attri- butes dominated the latter. It should be noted that CM is not perfectly consistent in preferring "English" and "female" to "assassin" because Otto/Gerda was the response to a question calling for spies who could be English, assassins, or females: also, Otto/Gerda was the response to a ques- tion calling for English spies. However, in the majority of cases, English or female was the dominant attribute. If questions calling for English or female spies as well as assassins are eliminated, then CM's response to questions calling for assassins was Otto/Gerda in 8 out of 10 cases. Just as the attributes "English" and "female" dominate the attribute "assassin", so does "not-disguisable" appear to overshadow "female". This is indicated by the fact that while CM tended to answer questions calling for female spies with Anne/Faith, not once does she do so when questions call for spies who may be not-disguisable as well as female. There are four such questions: two are answered with Sarah/Faith, one with Otto/Gerda, and one with Sarah/Liz. The question which must now be dealt with is how CM answered the questions for which Anne/Faith and Otto/Gerda 91 were not the preferred responses. There are 27 such questions; the 12 not-disguisable questions not calling for assassins and 15 additional questions which were not listed in Table 19. The 27 questions are listed in Table 20. Column 1 contains all questions calling for not- disguisable spies. Column 2 lists all questions calling for burglars other than those which also call for not- disguisable spies. Column 3 lists the questions, other than those calling for burglars and/or not-disguisable spies, which call for German spies, etc. The figures underneath each column refer to attributes included in the questions in the column. They indicate the proportion of spies named in response to questions specifying a given attribute who actually possess the specified attribute. For example, there were 21 not-disguisable spies among the 24 spies named in response to questions calling for not-disguisable spies. The proportions related to the questions involving the attribute "not—disguisable" indicate that CM was consistent in naming not-disguisable spies when the disjunctive ques- tions called for them. However, when the questions called for not-disguisable spies, CM was not consistent in matching other attributes contained in the questions. Thus, it seems that "not-disguisable" is at the top of the dominance hierarchy containing the attributes found in the columns of Table 20. By inspection of the figures under the other 92 Table 20 CM's Responses to Session III Disjunctive Questions Omitting those Calling for Females (Except Those Including Both Female and Not-disguisable), English spies, and Assassins (l) Not-disguisable (2) Burglar TFD TFGD PBD Sarah/Faith Sarah/Faith Sarah/Faith PGMBD Sarah/Faith TD PD PMD Sarah/Faith Otto/Gerda Otto/Gerda TFGBD Otto/Gerda TFBD PGMD PMBD TBD 3' C) tn Ul Sarah/Liz Elsa/Gerda Erich/Hans Sarah/Otto 21/24 5/12 4/8 3/8 tGB Elsa/Gerda tGBD Anne/Elsa VB Heidi/Marta tB Hans/Sarah VBD Heidi/Elsa tBD Sarah/Hans B: 11/12 G: 3/4 M: -- D: 3/6 (4) Male TM Otto/Gerda PM Hans/Erich TMD Erich/Hans M: 5/6 D: 1/2 (3) German VG Klaus/Otto tG Erich/Klaus VGM Otto/Gerda PGM Erich/Hans (5) Disguisable TD Anne/Elsa PD Faith/Hans D: 2/4 93 columns of the table, it can be seen that the dominance hierarchy, from most to least dominant, is "not-disguisable", "burglar", "German", "male", and finally "disguisable". Thus, CM tended to answer questions calling for not- disguisable spies with spies who were not-disguisable: if the questions did not call for not-disguisable spies, but did call for burglars, then CM tended to name burglars: if questions called for Germans, but not burglars or not- disguisable spies, CM responded with Germans, etc. It is not clear how CM went about retrieving the spies which met her search criterion for the questions in Table 20. To some extent she continued to rely on favored pairs. Otto/ Gerda is the answer to five questions--three calling for not-disguisable spies, one calling for veterans or Germans, and one calling for technical spies. In each case, both Otto and Gerda possessed the indicated attributes. Sarah/Faith was used five times in response to ques- tions calling for not-disguisable spies and never occurred in response to any other type of question. Since both spies are English females, it is possible that this pair was named as a consequence of searching the English female subroster for not/disguisable spies. On the other hand, there is a third not-disguisable English female, Victoria, who was never named: this could indicate that Sarah/Faith was a unit like Otto/Gerda which CM used for not-disguisable questions. 94 However that may be, if the Sarah/Faith and Otto/Gerda responses are set aside, there remain 17 questions in Table 20 to which CM gave 11 intrasubroster responses. Calculations similar to those described in the discussion of JK's data indicate that the expected number of intra- subroster responses to occur by chance in 17 questions is 6.48 with SD equal 1.92. The z score for 11 intrasubroster responses is 2.35 (p<<.01). If Sarah/Faith is considered to result from intrasubroster searching, then there are 16 intrasubroster responses in 22 questions. The z scores for 16 intrasubroster responses is 3.48 (p'<.001). Thus, it seems likely that GM used her subrosters to answer some of the 27 questions listed in Table 20. Summary of analysis of digjunctive questions for CM. CM used an intrasubroster pair, Anne/Faith, and a cross- subroster pair, Otto/Gerda, 17 times each in answering 64 Session III disjunctive questions. Anne/Faith was the response to most questions calling for either English or female spies-~with the exception that CM tended to answer questions calling for spies who could be either female or not-disguisable with pairs of not-disguisable spies. Otto/ Gerda was the preferred response to questions calling for assassins unless the questions called for English or female spies as well. In the latter case, the response was typically Anne/Faith. After setting aside questions for which the above pairs were the preferred response, 27 questions remained. CM gave intrasubroster responses to the majority of these questions. 95 Subject NA (A Two-Answer Subject) Subject's self-rgport. When asked how she learned the roster, NA said I started out with Heidi and Marlene. I tried to memorize the outside green slip containing their status, etc., then I found it much easier if I read the folder which contained their background and personalities. When I knew that, it was easier to remember, since there were more things to distinguish them by.... That's how I did it and as I went along I tried to do it in groups of five. As I read each one I'd go back and test myself and try to remember what the one before it was.... Then I could repeat each group of five. When NA was asked if there were anything special about the groups of five, she said they were all English girls and German girls and then German men and English men. When asked how she answered the questions, NA said I'd think of which came to me quickest and then see what their personality was and if it was the right one. She noted, however, that Marlene and Heidi were exceptions because she knew them well as technical trainees, just as she remembered Colin and Gerda as veteran assassins. She also stated that the hardest questions were those which required her to look at both nationalities and both sexes, and that she had trouble with the political spies and the trainee spies. Analysis of NA's data. Table 21 displays the attribute string distributions for NA's eight responses to the "all spies" request. It is clear that NA referred to her sub- rosters in answering the "all spies" question. 96 Table 21 Distribution of Attribute Strings in NA's Eight Responses to the "All Spies" Question String Lengths Attribute 1 g 3 _ _ _ _ 2 .19 Male 0 1 O O 0 O l l 2 4 Female 2 O O O O O O 1 2 5 English 1 O O 4 7 1 O 0 1 1 German 2 O 2 2 12 O O O 0 0 Political 18 12 11 1 O O O O O 0 Technical 25 7 9 2 O O O O O O Assassin 24 11 5 3 O O O O O O Burglar 21 11 8 2 1 O O O O O Disguisable 25 17 5 O O O 0 O O O Not-disg'ble 29 11 8 1 o o o o o 0 Veteran 39 9 3 2 O O O O O O Trainee 31 18 4 O O O O O O O 97 NA's responses to the disjunctive questions are given in Table 22: her responses to the conjunctive questions will be examined in the next chapter. The table indicates that NA answered the 64 disjunctive questions with a total of 26 different spy pairs, of which 10 are intrasubroster pairs. These intrasubroster pairs account for 34, or two more than half, the answers. Does this mean that NA used her subrosters to answer the the disjunctive questions? After all, it was noted previously that one would expect only 26 intrasubroster pairs to occur by chance if NA were randomly choosing names from the lists of spies relevant for the 64 disjunctive questions. The problem is that 15 of the 34 intrasubroster pairs consist of repetitions of the pair Marlene/Heidi. Since NA identified this pair as being easily remembered, it may be that these 15 intrasubroster responses are not the result of intrasubroster searching, but rather represent NA's capitalizing on a pair of distinctive spies who happen to belong to the same subroster. If the occurrences of the pair Marlene/Heidi are discarded, then there are only 19 intrasubroster pairs in the remaining 49 answers. In 49 answers, one could expect about 19 intrasubroster pairs to occur by chance. Thus, the evidence favoring the use of the subrosters is weak. Table 23 presents data which suggest another possibility. The table indicates how often each spy was named in each block of questions (The reader will recall that the questions were arranged in four blocks: 64 98 Table 22 Subject NA's Responses to Session III Disjunctive Questions German male Hans/Otto (5) Hans/Klaus (l) 6 Total English female Sarah/Victoria (5 Victoria/Faith (1 Liz/Sarah (l 7 Total Within nationality Sarah/Percy (5) Gerda/Otto ( 2 ) Colin/Victoria (1) Colin/Sarah (l) Brian/Victoria (1) Total 10 Within Subroster Pairs German female Marlene/Heidi (15) Heidi/Gerda (2) Total 17 Cross-subroster Pairs Within sex Heidi Sarah (2) Elsa Victoria (1) Otto/Colin (l) Gerda/Victoria (l) Liz/Marta (l) Percy/Hans (1) Total 7 English male Henry/Colin ( Wilfred/Colin ( Percy/Henry ( Total Other Colin/Gerda ( Colin/Marta ( Otto/Victoria ( Werner/Vict'ra ( Anne/Hans ( HHHHO ) ) ) ) ) Total 13 99 Table 23 Spy Usage Frequencies by NA in Session III Disjunctive Questions % Use in Relevant 2 I 2 t. 22121.. Blocks 1. Heidi 9 o 3 Z 19* 84 2. Colin 2 22 2 o 16* 88 3. Marlene Z O 2 9 15* 87 4. Gerda 3 9 2 o 14* 86 5. Sarah ‘3 3 2 ‘5 14* 64 6. Victoria 3 3 2 4 12 42 7. Otto 2 2 4 2 9* f 33 8. Hans O O 9 2 8 100 9. Percy 1 l 3 a 7* 33 10. Henry 1 O 2 2 3 67 11. Liz o o 2 2 2 100 12. Marta O 2 2 O 2 100 13. Klaus o 9 2 o l 100 14. Werner 9 O O 2 l 100 15. Elsa l 9 9 O l 000 16. Brian 0 0 9 2 l 100 17. Wilfred 9 2 o o 1* 100 18. Anne 9 o o 2 1* 100 19. Faith 0 9 2 o l 100 20. Erich 9 9 o o 0* 000 Note. T = technical, V = veteran, P = political, and t = trainee. Underlined frequencies indicate blocks where spy possesses the block attribute. Asterisks designate technical spies. 100 questions calling for technical spies, 64 for veterans, 64 for political spies, and finally 64 consecutive questions calling for trainees.): it also indicates the total frequency with which each spy was named beginning with the most often named spy at the top of the table. The under- lined frequencies indicate the number of instances wherein a spy was named in those two blocks in which he matched the block attribute, and the column on the right gives the number of times a spy was named in the blocks in which he matched the block attribute in proportion to the total number of times he was named. The asterisks designate technical spies. Three features of the table are striking. First, one notes that NA used a small subset of the Spies for the majority of her responses to the disjunctive questions: in fact, the seven most often named spies account for 77% of the responses. Second, one notices that six of the seven most frequently named spies are technical spies. Overall, the technical spies account for 75% of the responses to these questions. Third, one observes that NA tended to name the spies in blocks where the spies matched the block attribute: of the 19 spies NA named, 13 were relevant for the block attribute over 80% of the times they were named. The foregoing suggests an hypothesis: NA answered most disjunctive questions by searching through a relatively short list of predominately technical spies for two spies who matched the block attribute. Both the search criterion 101 and the retrieval strategy seem reasonable. The block attribute was always the first attribute stated when a question was presented. Thus, NA could simplify the question to a single attribute and begin searching for answers as soon as she had registered the first attribute. Furthermore, the first block of questions NA answered in Sessions II and III called for technical spies. Since 34 of the questions were conjunctive, she had to search through the technical spies pretty thoroughly. Thus, at the end of the first block of questions, NA could be expected to have a list of technical Spies at her disposal. It is not surprising that she would continue to use this list. An implication of the above hypothesis is that NA should have experienced difficulty when she came to the block calling for political spies--and she did report that the political spies were difficult. The reason for the difficulty is that technical and political are complementary attributes. Thus, the list of spies she had used for the technical and veteran blocks would no longer serve her when she had to retrieve political spies. Table 24 lists the number of spies NA named in each block who possessed the block attribute. Over 80% of the spies named in each block matched the block attribute--except for the political block. In the political block, the proportion fell to 44%. This indicates that NA was consistent in retrieving spies who matched the block attribute, and that she experienced difficulty in the block of political questions. Table 24 102 Table 24 Block Characteristics of NA's Session III Responses to Disjunctive Questions £122.15. Technical Veteran Political Trainee No.Tech. No. Spies No.Technical Spies No.Intra- Matching Spies Named Matching Subroster Block Attr. in Block Block Pairs in Eiffij'iiijfi' fZ2§:'§§9§:— Attr. 16 Quest. a 27 .84 27 .84 27 L: 28 .88 28 .88 24 2 14 .44 18 .56 O 10 26 .81 23 .72 21 3 a Excluding the intrasubroster pair "Marlene/Heidi" 103 also indicates that even in the block of questions calling for political spies, over half of the spies NA named were technical spies. In the other three blocks, the technical spies were greatly predominant. Furthermore, in every block except the political block, over 86% of those spies who matched the block attribute were technical spies. What did NA do for answers in the political block since she couldn't rely on her list of technical spies? It appears that she returned to her subrosters. Excluding the pair Marlene/Heidi, NA had 10 intrasubroster pairs among her 16 answers to political questions: the next highest number of intrasubroster pairs (again excluding Marlene/Heidi) within a block is four-~in the technical block. There is convincing evidence that NA was responding to the first-stated (i.e. block) attribute. Did she include any other attributes in her search criterion? If she were responding to only one attribute per question, NA would nevertheless, purely by chance, sometimes give conjunctively perfect answers, answers within one attribute of being conjunctively perfect, etc. To assess whether NA had more than one criterial attribute for each question, the observed number of question/spy-attribute matches can be compared with the number of matches expected to occur by chance under the assumption that NA is seeking to match on only one attribute. These data are presented in Table 25. In each cell the expected number of matches is given in parentheses. As the reader can see, the fit is good in all but two or three cells. 104 Table 25 Frequencies of Attribute Matches in NA's Responses to Session III Disjunctive Questions NgfiAgfigigggfis 2 No. Matcging Attrigutes in Risponses 2 5 0 (l) 2 (4) 8 (6) 3 (4) 3 (1) 4 8 (4) 9 (12) 11 (12) 4 (4) - 3 9 (12) 26 (24) 13 (12) - - 2 15 (16) 17 (16) - - - Total 32 (33) 54 (56) 32 (30) 7 (8) 3 (1) Note. Expected frequencies assuming one criterial attribute are given in parentheses. 105 Summary for NA's disjunctive responses. It appears that NA's concern in answering disjunctive questions was to find spies who matched the first attribute stated upon presen- tation of the questions. She did not rely upon her subrosters. Rather, in the course of answering the first block of ques- tions, all of which called for technical spies, she developed a list of technical spies which she used to answer questions in subsequent blocks. This strategy worked well until NA was confronted with a block of questions calling for political spies. Since technical and political are complementary attributes in the spy roster, she could not rely upon the list of technical spies and was forced to return to the use of her subrosters for some of her responses. In the final block of questions, she again made heavy use of the list of technical spies. Analysis of Latency Data from Conjunctive Responses of the Subjgcts Who Subrostered on Sex and Nationality The analysis reported in the previous section indicated that whole-answer subjects CT, SM, and JL, and two—answer subjects CM, JK, and NA divided the spy roster into four subrosters containing spies of the same sex and nationality. This organization of the roster should greatly affect the response latencies to conjunctive questions of differing numbers of attributes and differing numbers of prepotent attributes. Half the spies in the twenty-spy roster possess any given attribute: that is, half the spies are English, half 106 are German, half are male, half are assassins, etc. This means that one spy in two would possess a given single attribute: one spy in four should possess a given pair of attributes, and so on. Thus, the more attributes that are specified in a conjunctive question, the smaller the proportion of spies which will be relevant and thus the greater the number of irrelevant spies who might be considered before a relevant spy is found. Hence, it might appear that adding attributes to conjunctive questions would increase the time required to find a spy possessing all the required attributes. However, it is not the case that adding an attribute to a conjunctive question will always be expected to increase the search time before a relevant spy is located: that depends on the attribute. Suppose answers were sought for a one-attribute question. One spy in two should be relevant. Now suppose the one-attribute question were changed to a two-attribute question by addition of a prepotent attribute-- say male, for example. In the roster as a whole, one spy in four should be relevant for a two-attribute conjunctive question. But if the spies are organized into subrosters on the basis of sex and nationality, then the female spies can be ignored and only the male subrosters need be searched. Within these subrosters one spy in two should have the required remaining attributes. Thus, adding a prepotent attribute to a one-attribute conjunctive question does not decrease the proportion of relevant spies. This holds true 107 for conjunctive questions of any number of attributes to which a prepotent attribute might be added. What happens if one of the attributes in two-attribute question is replaced by a prepotent attribute? Assuming the two original attributes were nonprepotent, one spy in four in the whole roster would have been relevant. However, substitution of a prepotent attribute confines the search to two subrosters wherein one spy in two should possess the other required attributes. The effect of the substitution is seen to be that the subset of spies which must be examined includes a higher proportion of potentially relevant spies than did the roster as a whole. Thus, substitution of a prepotent attribute should reduce the time required to locate a relevant spy. If it is the case that as spies are retrieved from the roster, they must be checked for conformity to each attri- bute specified in the question, then substitution of a prepotent attribute offers a second potential reduction in time to find a relevant spy. Because the prepotent attri- bute confines the search to a pair of subrosters wherein all spies possess that attribute, each retrieved spy need be checked for conformity only on the other attributes, i.e., on one less attribute than was the case before substitution. The preceding discussion has two implications. First, for conjunctive questions containing a fixed number of prepotent attributes, the response latencies should be an increasing function of the number of attributes in the 108 question. Second, the response latencies should be a decreasing function of the number of prepotent attributes included in questions of a fixed number of attributes. Figures 1 and 2 display the conjunctive response latencies of the five subjects who subrostered on sex and nationality and for whom latencies were recorded. Figure 1 includes data for the two whole-answer subjects for whom latencies were available, whereas Figure 2 displays the latency data for the three two-answer subjects. Each figure displays data points for 1, 2, and 3 attribute questions containing either 0 or l prepotent attributes: each data point is the mean latency for the first response to eight conjunctive questions. It is clear that every subject conforms exactly to the expectations stated above: both the O-prepotent and the l-prepotent attribute questions took longer to answer as the number of attributes increased from one to three: and without exception the l-prepotent attribute questions were answered faster than the corresponding questions with 0 prepotent attributes. It also appears from the figures that generally speaking, the two-answer subjects were quicker with their first response than the whole- answer subjects. An analysis of variance was performed on the data shown in Figures 1 and 2. Three sources of variation were examined: variation due to differences in the tasks which is a between subjects variable: variation due to different numbers of attributes in the questions (1, 2, or 3) and 109 CD 4 _1_ O prepotent 7‘ "E 1 .I (SM) I I I, l prepotent : \ 1 I Latency (seconds) to kc f- \h H I I O I l 2 No. attributes 1 1 1 \o‘r H O I I O prepotent \O l I (JL) m l ' ‘, 1 prepotent / O\ \3 l l l I {-7 Kn 1 I Latency (seconds) K10 _L I \ J I 1 2 No. attributes Figure 1. First response latencies for whole answer subjects 110 8 1— O prepotent 7 u. U) :96 .2 (CM) 8 m5 "' (D can ‘- o /, l prepotent “'" / 23 '" / a: / 4’ tz-— ,,,» +4 .2,. 1 .— o : t : ~— 1 2 3 No. attributes '§5 1” O prepotent 3:. .. (NA) 333 ..u [)1 prepotent U) .32 4- / / 81 -- " .p S _L 1 l O 1 , 4 1 2 3 No. attributes .9 0 prepotent m3 d” /1 prePOtent (JK) ”03.2 J— /’/*——- «’ +>1 ‘7 S o 1 t : l 2 3 No. attributes Figure 2. First response latencies for two-answer subjects lll variation due to differing numbers of prepotent attributes (0 or 1). Both the latter variables are within-subjects variables. An unweighted means analysis was performed since there were differing numbers of subjects within the levels of the task variable (Winer, 1962). As the summary table (Table 26) indicates, the F ratio for the attributes effect was significant at the 0.01 level and the F ratio for prepotency was significant at the 0.05 level, indicating reliable effects of both number of attributes and number of prepotent attributes. It is exceedingly improbable that all five subjects could have produced, by chance, a pattern of response latencies in conformity with expectations based on the sub— roster model. Figures 1 and 2 included six mean latencies for each subject. Suppose the six latencies were numbered from 1 to 6 corresponding to the smallest to the largest latencies, respectively. The six ordinal numbers thus obtained could then be arranged in 720 different ways in the following tabular format: 1 attr. 2 attr. 3 attr. l prepotent attribute 0 prepotent attribute 0f the 720 arrangements only five result in having the six numbers in increasing order across the rows and down the columns in conformity with the subroster model. Thus, the probability of all five subjects independently conforming 112 Table 26 Analysis of Variance on Latency Data for Responses to Conjunctive Questions in Session III Source §§ 22 Mg 2 Between Subjects Task (2 ans. vs all ans.) 37.10 1 37.10 8.30 2s w/in Tasks 13.41 3 4.47 Within Subjects Attributes (l, 2, or 3) 67.15 2 33.58 25.39** Attr. x Task 6.88 2 3.44 2.60 Attr. x 2s w/in Tasks 7.93 6 1.32 Prepotents (O or 1) 25.09 1 25.09 10.66* Prepotents x Task 0.65 1 0.65 - Prepotents x 2s w/in Tasks 7.06 3 2.35 Attributes x Prepotents 2.60 2 1.30 2.07 Attributes x Prepotents x Task 0.71 2 0.36 - Attr. x Prep. x 2s w/in Tasks 3.76 6 0.63 ’U'U 0\ LA 0 o 'o H :0: 113 to the model by chance is (5/720)5 which is considerably less than 0.0000000001. An alternative explanation. One of the expectations based on the subroster model was that response latencies would increase as a function of the number of attributes in conjunctive questions. But there is another explanation for the preceding data. The data displayed in Figures 1 and 2 which seem to support the subroster model are mean latencies for 22292 responses to conjunctive questions. These latencies represent the time interval between presentation of a question and the first response to the question. This time interval must include the time required for a subject to comprehend and prepare to answer the question as well as the time to search for the first suitable spy. Thus, it is possible that the greater latencies for questions with more attributes reflect increased time to prepare to answer the questions rather than representing increased search time. Similarly, the reduced latencies associated with increasing numbers of prepotent attributes may be due to simpler preparation for answering questions whose answers are confined to specific subrosters. If the latter explanation is correct, then the relationship between latencies for the second responses to questions need not duplicate the relationships for the first responses. 0n the other hand, the expectations based on the subroster model should apply to every response, hence, to every response latency. Thus, under the subroster model, 114 one expects the latencies for second responses to mirror the relationships between the latencies for the first responses. Figures 3 and 4 display second response latencies for the five subjects who subrostered and for whom latencies were recorded. The second response latencies duplicate the relationships between first response latencies for every subject except NA, and even in her data only one pair of points is reversed. It is clear from the latency data that conjunctive questions with prepotent attributes are answered faster than similar conjunctive questions (similar in having the same number of attributes) with fewer prepotent attributes. As noted previously, prepotent attributes can reduce search time in two ways: (1) by increasing the proportion of relevant spies within the set of potentially relevant spies (e.g. there are 5 veteran assassins and 5 veteran males, but the former are distributed across the roster of 20 spies whereas the latter are distributed among the 10 male spies found in the German male and English male subrosters), and (2) by reducing the number of attributes per spy which must be checked for relevance. Do these effects operate jointly or is one of them of primary importance? If spies are retrieved slowly and evaluated quickly, then the proportion of relevant to potentially relevant spies will be the major determinant of search time. On the other hand, if retrieval is fast and evaluation is slow, then the number of attributes examined per spy will be the primary determinant of search 115 8-- 7.... ’26-— m 235 o -. 8 (SM) m “C- ‘5’ 3.... O prepotent g 0 1'5 2-- .4 ...—no 1 prepotent 1-.. 2,4— o~”" 0 t : : 1 2 3 No. attributes 10.. 9" O prepotent 8 .. A7 v 86 L- (JL) 0 . 8 wES‘t E’“ " 9 ,91 prepotent ‘833" / FA // 2 .. /’ / // 1 -- '2”,/ 0 t 4. : 1 2 3 No. attributes Figure 3. Second response latencies for whole answer subjects 116 L: A (GM) to 8 3 0 prepotent :0 2,31 prepotent V2 / a O 5 1 +2 / 3° I .= 1 No. attributes 71- 6-. l prepotent ,\ f (NA) g-5'“ 0 prepotent 81.-. d) U) v3__ ’8 :25- 0.) *5 1 ’4 any-- 0 No. attributes 734 83 (JK) U) c, 0 prepotent 2 8 51 .—o 1 prepotent g .__.2"_._—.— ’40 7 I l 2 No. attributes Figure 4. Second response latencies for two-answer subjects 117 time. Is the number of criterial attributes equal to the number of attributes in the question or is it equal to the number of nonprepotent attributes? Does the information processor check 222 criterial attributes for each spy, or does processing of a given spy cease as soon as an attribute is discovered which does not correspond to the conjunctive question? These alternatives are implemented below in five models of the search procedure. The models relate the time interval between the first and second responses for conjunctive questions to the number of spies accessed and/or the number of attributes examined. The choice of this particular datum was governed by the following considerations. The latency of the first response confounds processing time required to discover the first relevant spy with set-up time required to code the question and select a processing strategy. However, the time interval between the onset of one name and the onset of the next should be free of set-up time. (There could be time required to re-initiate searching after naming a relevant spy, but this will be assumed to be negligible.) Since the two-answer subjects made only one response after the first, attention was given to modelling the interval between onset of the first response and onset of the second response. The data will be taken only from the five subjects who sub- rostered on sex and nationality. The five-attribute questions are precluded because each of them had only one answer . 118 The following notation will be used in the models. The total number of attributes (prepotent plus nonprepotent) will be indicated by 2, and the number of prepotent attri- butes by 2. The time in seconds between the onset of the first and the onset of the second response will be denoted by Eik for a question of k attributes of which i are prepotent. Eik indicates the expected number of spies examined to find the second relevant spy after the first relevant spy has been reported. For example, if the third spy examined after the first spy is reported is determined to be relevant. then Xik = 3. A portion of the interval between the first and second responses will be taken up by reporting the name of the first relevant spy: this time, in seconds, will be denoted by g. The processing time (seconds) per spy (Model I, below) or per attribute (Models II - V) will be denoted by 9. The following summary of the models may prove helpful: Model I: Spy retrieval is the dominant factor: attribute processing is not considered. Models II-V: Attribute processing is the dominant factor. These models are distinguished by their treatment of the number of criterial attributes (k versus k — l) and whether each spy is examined with respect to 222 criterial attributes or whether processing terminates when an attribute mismatch is discovered. 119 The following table describes the models II — V: Criterial Attributes Processing k k - 1 Complete Model II Model III Terminating Model IV Model V The models are as follows: Model I tik = a + b[Xik] (1) Model II tik = a + b[k Xik] (2) Model III tik = a + b[(k - i) xik] (3) Model IV tij = a + b [nk (xik - 1) + k] * (4) Model V tik = a + b [nk-i (Xik - l) + (k - i)]*(5) *where pm is the expected number of attributes examined per irrelevant spy where there are p criterial attributes. Models IV and V may need some comment. These are the models which assume termination of processing on irrelevant spies. If after reporting the first spy, Xik spies are examined in order to find the next relevant spy, then (Xik - l) of those spies were irrelevant. Each of the (Xik - l) spies is assumed to be processed with respect to nm attributes, the expected number of attributes processed given that a spy is irrelevant. However, the relevant spy must be processed with respect to all criterial attributes. Hence, for m criterial attributes one can expect to examine nIn (Xik - l) attributes while processing irrelevant spies 120 plus m attributes for the relevant spy. Thus, the total number of attributes examined is nm (Xik - 1) + m. Atjp seconds per attribute plus 9 seconds to complete the first response, one has tik=a+btnm(xik‘1)+"i where m = k for Model IV, m = k - i for Model V. Given values for Xik and nm, one can obtain least squares estimates of a and b for the linear regression of the observed time intervals onto the values in brackets in Equations 1 - 5. The values for Xik and nm are determined as follows. First, consider Xik' Let p be the number of potentially relevant spies of which 2 are actually relevant. Consider a set of spies ordered from 1 to n of which.p randomly chosen spies are designated as relevant. All told there are (2) different sets of r spies which could be chosen from a set of n spies. What proportion of those (3) sets have the j-th spy (léjgn - r + 1) as the first relevant spy? Following the j-th spy are n - j additional spies of which r - 1 are relevant. There are (E : {) sets of r - l spies which may be chosen from n - j spies. Thus, if Pj denotes the probability that the j-th spy is the first relevant spy as the spies are processed from 1 to n, then Pj= (339/6). 121 If L1(r,n) denotes the expected location of the first of r relevant spies randomly distributed in a set of n spies, then n-r+1 L (r n) = j- P 1 ’ J 2 l j n-r+1 =2: J:(?Ii)/(?) J=1 - +1 . r' (n - r)’ n r j n - j n! j = 1 (1) (r 1) Applying formula (12.16) from.page 62 of Feller (1957) to the preceding summation yields _ r! (n - r)! n + l _ n + l - r + l . Now, if £1 is the value of the expected location of the first relevant spy, the expected location of the second relevant spy will be .2 ‘£1 + (n "fl1) + 1 ( 2 r - l ) + 1 Hence ’ [2’I1zgii In the present study, subjects who subrostered on sex and nationality have been shown to process the spies a subroster at a time. Thus n = 5 and the value of r for each subroster 122 varies from question to question. Therefore, to establish values of Xik it was necessary to sort the conjunctive ques- tions into classes determined by the total number of attributes and the number of prepotent attributes, and then to examine the individual questions within each class to identify the relevant responses within each of the sub- rosters. Then, taking into account the orders in which the subrosters can be accessed, the value of Xik was determined for each question. Finally, a value of Xik was determined for each class of questions by averaging the values of Xik for the questions within the class. The resultant values of Elk are displayed in Table 27. The values of?ik increase as k increases when i is fixed, and the Elk values decrease as 1 increases for any fixed value of k. Next, consider how the value of nm is established. The following notation will be used: pr(rvR.Aj) = probability that a given spy is not rele- vant and that the j-th attribute examined is the first nonrelevant attribute encountered pr(jlan) = probability that the j-th attribute examined is the first nonrelevant attri- bute encountered, given that the spy is not relevant pr(rvR) = probability that the spy is not relevant 123 Table 27 Expected Number of Spies Examined to Find 2nd Relevant Spy after lst Relevant Spy is Encountered Number of prepotent Number attributes in question attributes 2 2 3 _ 0 1075 2096 5'50 - 1 1.00 1.75 2.13 3.00 2 - - 1.81 1.93 124 Then m mm = E: j- pr j=1 m . z - pr(~R’\,1) SE1 3 ° pr(~R) m .. (112)3“1- 1Q #21 J 1 - (1/2)m = 1” 'm '«(1/2)j'l 1 - (1/2)m EL J = 2m 2m+l _ 2 _ m 2m- 2 = 2- m 2m-1 Given the values of Xik and nm, the quantities in brackets in Equations 1 - 5 were calculated and then the parameters a and b were estimated for each model. The predictions of the five models are presented in the upper portion of Table 28. The lower portion of the table reports the estimated values of a and b for each model, along with the correlation between the observed values of tik and the values specified by each of the models. It is clear that the best description of the inter- response intervals is given by Model III: tik = 0.62 + 0.66 (k - i) Xik . This model assumes that only nonprepotent attributes are criterial, and that all criterial attributes are processed for each spy. Thus, the number of attributes is determined 125 Table 28 Response Intervals Between First and Second Responses: Observed Intervals and Intervals Predicted by Models I - V. Slope, Intercept, and Observed/Predicted Correlation Number prepotent Number attributes in question attributes ‘1 2 2’ fl 0 Observed 1.20 2.78 b.70 - Model I 2.20 3.39 5.81 - Model II 1.29 2. 3 6.05 - Model III 1.77 2.57 0.25 - Model IV 1.19 2.83 5.92 - Model V 1.97 3.35 5.98 - 1 Observed .u9 1.52 3.05 6.51 Model I 1.53 2.2% 2.60 3.43 Model II 1.04 1.85 2.78 “.60 Model III .62 1.77 3.43 6.56 Model IV .77 1.90 2.91 4.4” Model V 1.12 1.97 2.81 4.08 2 Observed - - 2.93 2.7a Model I - - 2.30 2.#1 Model II - - 2.h7 3.21 Model III - - 1.81 3.17 Model IV - - 2.62 3.39 Model V - - 1.99 2.68 Slope (b) Intercept (a) 3 Model I .951 .578 .68 Model II .323 .720 .87 Model III .660 .618 .96 Model IV .568 .199 .87 Model V .h82 1.122 .79 for Each Model. 126 by multiplying the number of criterial attributes per spy by the expected number of spies examined to obtain the number of attributes examined, multiplying by 0.66 seconds processing time per attribute, and adding .62 seconds for the time required to complete the first response. It is perhaps worth noting that the latter estimate can be checked against the data. For questions consisting of a single attribute which is prepotent (i.e., questions calling for male spies. female spies, German spies, or English spies), k = 1 and i = 1 so that tl,l = a + b (l - l) Xik = a In other words, for these questions the interval between responses should be equal to a, which was estimated to be .62 seconds. Each set of marker questions included all four single prepotent attribute questions. and the marker ques- tion data was not included in the estimation of a and b. The mean intername interval across the five subjects' responses to the last three repetitions of the marker ques- tions in Session III is 0.6“4 seconds. This corresponds quite closely to the estimate of .62 seconds. Results: The Subjects Who Did Not Subroster on Sex and Nationality Subject SP (A Whole Answer Subjectl Subject's self-report. Subject SP stated that she learned the spies as individual entities without any effort to group them or otherwise organize them. During Session II she attempted, in the course of answering the questions, to form groups on the basis of sex and nationality, but found that somewhat difficult. Between Session II and Session III she hit on the idea of processing the spies in alphabetical order and found that strategy fairly workable. The Session III data will be examined in the pages that follow. Analysis of SP's data. In Session III, SP said she searched the roster alphabetically, beginning with Anne and concluding with Wilfred and Werner. If this is in fact what she did, then the names in her responses should occur in alphabetical order. This should be particularly true of responses to the disjunctive questions, since these questions provide maximal opportunity for the subject to impose her own organization on the response. SP's tendency to respond with names in alphabetical order was assessed as follows: The n spies named in response to a given disjunctive question were arranged in alphabetical order and assigned numbers ranging from 1 to n (with the 127 128 exception that the two names beginning with Mar--Marta and Marlene-~were numbered in the reverse order because when they appeared contiguously in responses, SP always had them reversed. The rank order correlation between the n numbers arranged in ascending order and the n numbers arranged in the order in which the corresponding spies were named in the response indicates the extent to which the spies were named in alphabetical order. If the spies were named alphabetical- ly. both sets of numbers would be ordered from 1 to n and the rank order correlation for the given disjunctive question would be unity: to the extent the spies were not named alphabetically, the correlation would be less than unity. Figure 5 indicates the mean rank order correlation for the six disjunctive questions in each of the four question blocks in Session II (where SP said she tried to organize by subrosters) and in Session III (where she said she organized alphabetically). The mean rank order correlations for Session II range from -.03 to .32, but in Session III they range from .85 to .97. It is clear that SP switched to an alphabetizing strategy in Session III. The latency data for SP should provide further support for sequential processing of the roster in alphabetical order. The time interval between names in SP's responses should be related to the number of intervening spies in the roster who are covertly processed and rejected. Assuming SP processed the roster in alphabetical order, one can determine how many irrelevant spies occur in the roster between any 129 /\/S€SSion III 09 “'- Rank order correlation Kn I I .4 ->- .3 -- Session II .2 .1... .1 4. 0 i '1' ‘ '1 1 2 3" h —.1 1L. Blocks of 6 disjunctive questions Figure 5. Mean alphabetical order indexes for SP's responses to disjunctive questions in Session II and Session III 130 pair of relevant spies SP has named in one of her responses. Then the time intervals between the names in the responses can be plotted against the number of intervening spies. Finding that the former are an increasing function of the latter would support sequential alphabetical processing. In the case of one attribute questions, it is plausible that the time interval between names will be linearly related to the number of intervening spies. When the ques- tion specifies one attribute, each irrelevant spy is rejected on the basis of precisely one comparison. Thus, it is likely that the time to covertly examine and reject an irrelevant spy is nearly the same for every irrelevant spy. If the latter is correct, then intername intervals will be a linear function of the number of intervening spies. Figure 6 displays the observed relationship between separation of a pair of spies in the alphabetically ordered roster and the time between the pair of names in a response. The data were taken from Session III questions calling for one attribute, using only that portion of the responses up to, but not including, the first name out of alphabetical order. The latter restriction was imposed because a name out of order indicates an error in what SP described as her normal sequential processing. The small range of numbers of intervening spies (0 to 3) is due to the fact that the alphabetically ordered roster has only one instance of two spies with a given characteristic being separated by more than three intervening spies. Each point in the figure is 131 Intername interval = 2.51 + 1.00 NI Intername interval (seconds) 1.5-— 1.0:“ .5-- 0 t i 1 o 1 2 3 Number of intervening spies (NI) Figure 6. The relationship between intername intervals in seconds and number of intervening spies in the alphabetical roster. Numbers in parentheses indicate n for each point. 132 a mean time: the number of data points contributing to each is shown in parentheses. It is clear from Figure 6 that the time intervals between names are a linear function of the numbers of inter- vening spies. The slope is unity. indicating that it took SP one second to process and reject an irrelevant spy. The intercept of the linear function, which is the time interval between names of spies who are adjacent in the roster, is 2.51 seconds, indicating that SP required that much time to identify and report a relevant spy. Subject SP described herself as sequentially processing the spies in alphabetical order in Session III. The ordinal characteristics of her responses to the disjunctive ques- tions and the latency data for the one attribute conjunctive questions are consistent with that search procedure. AH: A Two-Answer Subject Who Subrostered on Experience Subject's self report. AH said that her technique for learning the roster was to list the spies and to make note of special characteristics that she could associate with TV characters or someone she knew. When asked if she had grouped the spies that were alike, she replied that she grouped them in veterans and trainees. She was asked if there were any other grouping and she said not. AH said that when she was asked a question, "I had to think back. I usually had to think either to the top half of the veterans and the trainees." From her self-report, AH sounds like yet another 133 subject who subrostered, although not along the expected dimensions of sex and nationality. If this is what she did, then her replies to the "all spies" question and her response latencies to conjunctive questions should have the structure which characterized the other subjects who sub- rostered. However, since AH is a two-answer subject. she may have developed an idiosyncratic procedure for answering the disjunctive questions. The "all spies" question. Table 29 displays the distributions of attribute strings in AH's eight responses to the "all spies" question. AH was not as consistent as some of the other subjects in her responses to this question, but the tendency to group veterans and trainees is clear. Much of the instability evident in the table occurred during the first four responses in Session II. By Session III. three of the four responses contained a string of veterans followed by a string of trainees. The fourth question was marred only by the intrusion of Faith. a veteran, in the middle of the string of trainees. Thus. the organization of the responses to the "all spies" request is consistent with AH's self-report. The latency data for the conjunctive responses. If AH capitalized on her subrosters in the same way as the other subjects who subrostered, then conjunctive questions calling for veteran or trainees should have smaller latencies than the remaining conjunctive questions. Table 30 displays mean time intervals (in seconds) between the first and 134 Table 29 String Distributions in AH’s 8 Responses Attribute Trainee Veteran German English Male Female Political Technical Assassin Burglar Disguisable Not-disg'ble to the "All Spies" Question String Lengths 267 .1. E .3. _ .. _ ._ .2 1 3 2 1 1 1 O O 6 4 O O 1 O 1 3 O 25 7 6 2 1 0 O O 0 22 6 3 3 2 O 1 O O 8 8 6 1 4 1 O O O 8 12 8 1 2 O O O O 17 11 5 2 0 2 O O O 12 12 3 2 3 O O O O 23 6 7 3 O O O O O 18 12 5 3 l O O O O 27 8 4 3 l O O O O 17 9 10 O 1 O O O O H O O O O O O O O O O NW'O 135 Table 30 Mean Intervals in Seconds Between AH's First and Second Responses to Session III Conjunctive Questions No. Prepptent No. of Attributes 1 2 3 4 .70 3.36 7.22 13.54 .36 .86 4.46 5.76 136 second responses AH made to Session III conjunctive questions of one to four attributes (no five-attribute question had two answers). The mean interval for questions with one prepotent attribute is, without exception, smaller than for similar conjunctive questions without a prepotent attribute. The latency data for AH clearly indicates that she capitalized on the subrosters to answer the conjunctive questions. Analysis of responses to disjunctive questions. The probability that AH would by chance name spies possessing a given attribute was estimated by the relative frequency with which such spies were named in response to Session III disjunctive questions specifying neither the attribute nor its complement. Let p represent this probability. Then if AH did not attend to the given attribute in naming g spies while answering the questions in which it was speci- fied, the frequency with which she named spies possessing the attribute is binomially distributed with a mean of np and standard deviation equal to the square root of np(l - p). Suppose AH named m spies who possess a given attribute specified in disjunctive questions. The quantity m can be converted to a standard score as follows: 2 - m ' np m - \/np(l - p) The greater the value of the standard score, the greater the extent to which the number of spies possessing the specified attribute exceeds the number of such spies 137 expected to occur by chance. The z scores for AH's responses to Session III disjunctive questions are displayed for all twelve attributes in Table 31. The table indicates that AH had a marked tendency to reSpond with spies matching every specified attribute except for three--political. technical. and not-disguisable. This would suggest that. except for those three attributes, AH had a conjunctive criterion for her responses to disjunctive questions, just as subject JK did. Table 32 indicates the frequency with which AH's first and second responses to Session III disjunctive questions matched one or more of the specified attributes other than the attributes political, technical. and not-disguisable. The probability that a spy matches k specified attributes by chance, given that the spy is relevant for a disjunctive question specifying n attributes is k 5161/2)“ k < n' This is clearly a decreasing function of k so that the least probable number of matches is n. Yet, it is clear for AH's first responses that she was most likely to match on all n attributes. Thus, it is unlikely that the pattern of matches indicated in Table 32 for first responses occurred by chance. Furthermore, if AH were attempting conjunctive matches on some fixed number of attributes for all disjunctive questions, the probability of matching on all specified attributes would never be larger than the 138 Table 31 The z Scores for AH's Responses to Session III Disjunctive Questions Attribute z-score Male 5.12 Female 4.88 English 1.28 German 4.48 Assassin 2.34 Burglar 6.22 Disguisable 2.95 Not-disguisable 0.06 Technical 0.18 Veteran 4.08 Political 0.18 Trainee 1.95 139 Table 32 Number of Question/Answer Attribute Matches in AH's Responses to Session III Disjunctive Questions Response I Number attributes Number of matches in question 9 1’ g 3 4 ‘5 l 0 10 - - - - 2 0 6 16 - - - 3 0 2 5 10 - _ 4 0 0 2 2 7 - 5 O O O O 2 0 Response II Number attributes Number of matches in question 9 1 2 3 4 5 1 l 9 - - - - 2 0 9 13 - - _ 3 0 1 10 - - 4 o 1 4 6 o - 5 o o o 2 o 0 Note. The attributes political, technical, and not- disguisable are ignored. 140 probability of matching on all attributes but one. But this is not consistent with AH's responses either. Rather, it appears that AH was seeking conjunctive matches and succeed- ing only imperfectly. The data for the second response is less clearcut because the frequencies do not differ much from what would be expected if AH were seeking spies who met a conjunctive criterion on all but one attribute. However, it is not implausible that AH relaxed her criterion somewhat for the second spy. 1 Did AH use her subrosters in selecting her responses to the disjunctive questions? If she did, then one would expect a higher frequency of intrasubroster response pairs than would occur by chance. If AH were not using her sub- rosters so that veterans and trainees were named by chance. one fourth of the spy pairs should be veterans, one fourth trainees. and half the pairs should be mixed-~one veteran and one trainee. Table 33 displays the observed frequency of each pair type along with the expected frequency based on chance selection of veterans and trainees. The data included in the table are the 32 responses to those Session III disjunctive questions calling neither for veterans nor trainees. The remaining questions were not included because, as Table 31 shows, AH sought veterans or trainees when either attribute was specified. However. the tendency to name veteran or trainee pairs when neither attribute is 141 Table 33 Observed and Expected Frequency of Veteran Pairs, Mixed Pairs. and Trainee Pairs in AH's Responses to Session III Disjunctive Questions Veteran Mixed Trainee pairs pairs pairs Observed 8 13 11 Expected 8 16 8 142 specified would indicate intrasubroster searching. Table 33 indicates that AH did not choose intrasubroster pairs more often than would be expected by chance (“(2)2 = 1.69, p>.25). Summary_for AH. AH said she divided the spies into a subroster of ten veterans and a subroster of ten trainees. The latency data for responses to conjunctive questions were consistent with her report in that she tended to answer conjunctive questions calling for veterans or trainees faster than questions which did not. Thus, for the conjunc- tive questions, AH resembled other subjects who subrostered-- response latencies increased with increasing numbers of attributes specified, but with the number of attributes fixed, questions specifying a prepotent attribute were answered faster than those which did not. In her responses to disjunctive questions, AH resembled JK who set a conjunctive criterion for her responses. However, unlike JK, there was no evidence that AH used her subrosters in answering disjunctive questions--even though AH did use them to answer conjunctive questions. DK: The Two-Answer Subject Who Subrostered on Nationality and Background Subject's self-report. Subject DK's report of her organization of the roster during acquisition was by far the most interesting of all the reports. The description was rather involved, so DK was asked to write down the structure she was describing. What she wrote is reproduced here. English Political (assassins) Liz Henry trainees Brian Faith' Victoria - burglar German Political (burglars) Marta Elsa veterans Klaus Hans Werner - assassin English Technical (burglars) Sarah Percy trainees Anne Wilfred Colin - assassin German Technical (assassins) Erich Otto veterans Gerda Marlene Heidi - burglar DK recognized something no other subject--nor the experimenter, for that matter-~had noticed: that background and specialty are correlated within nationalities. Thus, the English tend to be political assassins or technical burglars whereas the Germans tend to be political burglars or technical assassins. Hence, DK's subrostering was tantamount to indexing the spy roster on three dimensions rather than two. It is unfortunate that DK's data were collected before the "all spies" question was added, because it would have been interesting to check the stability of 144 this classification scheme. Surprisingly, DK was no more able to describe her search strategy than were the other two-answer subjects. Like the others, she said that names just "came to mind", and that she could easily verify whether the spies were relevant. The responses to the disjunctive questions. DK's responses to the 64 Session III disjunctive questions are listed in Table 34. She used 20 different pairs of spies: the four pairs named most often account for 36 of the 64 responses. Forty-four of the 64 response pairs were intra- subroster pairs. What was DK's search criterion for the disjunctive questions: i.e., how did she select the spies? Table 35 indicates the frequency with which BK named spies possessing certain attributes specified in the questions. The figures in the table indicate that 66 of 72 spies matched the specified nationality in DK's responses to the 36 questions in which nationality was specified. Further inspection of the table shows that when nationality was specified, DK was not consistent in matching on any other attribute. Thus, it appears that part of DK's search criterion called for selection of spies of the specified nationality, and that other attributes were ignored when nationality was specified. What did DK do if nationality was not specified? According to the data in Table 36, she selected spies of the specified specialty or sex--and if both were specified she 145 Table 34 Subject DK's Responses to 64 Session III Disjunctive Questions Within Subroster Pairs German political English technical English political Marta Elsa (3) Sarah Percy (9) Liz/Henry (ll) Hans Klaus (1) Anne Wilfred (l) Liz/Victoria (3) Hans/Werner (l) Anne/Sarah (l) Brian/Henry (3) Victoria/Faith (3) Total 5 Total 11 Liz/Faith (2) Total 22 German technical Otto/Erich (6) Cross-Subroster Pairs Within Nationality Within Background Other Hans/Erich (10) Colin/Erich (2) Erich/Brian (l) Marta/Gerda (2) Colin/Otto (2) Wilfred/Brian (1) Total 1 Liz/Sarah (1) Total 4 Otto/Hans (1) Total 15 Note. Frequencies are given in parentheses. Proportions of Spies Which Match Attributes Questions Specifying: Nat. 1. Nationality but not specialty 32/32 (n = 16) 2. Specialty but not nationality - (n = 16) 3. Specialty and nationality 34/40 (n = 20) 4. Neither specialty nor nationality - (n = 12) 146 Table 35 in DK's Responses in Session III Specified in Disjunctive Questions Spec. 31/32 24/40 Back. 9/16 10/16 2/8 16/24 Sex 16/24 7/8 11/24 15/16 Stat. Disg. 11/16 3/8 13/16 11/24 18/32 11/24 - 12/16 147 Table 36 Frequency with Which Spies Named in Response to Session III Disjunctive Questions Match One or More Attributes Specified in the Questions Number attributes specified 1 g 2 3 2 2 10 (16) 22 (16) - - - 3 10 (12) 23 (24) 15 (12) - - 4 5 (3) 9 (ll) 10 (13) 8 (5) - 5 2 (l) 4 (4) 6 (6) 2 (4) 2 (1) Note. The numbers in parentheses are the expected frequen- cies. This assumes DK seeks a match on nationality only if nationality is specified, otherwise she attempts a match on sex, specialty, or both. Matches on any other attributes occur by chance. 148 appeared to select spies matching both. She selected 24 out of 24 spies of the designated specialty when sex was not specified: she named 15 out of 16 spies of the designated sex when specialty was not specified: and 6 of 8 spies were of the indicated sex apd specialty when both were specified. All together, 60 of the 64 disjunctive questions specified nationality, specialty, or sex either singly or in combination. The remaining four questions specified disguisability, and 7 of the 8 spies DK named were of the specified disguisability. On the basis of the preceding analysis, it appears that DK was selecting spies for her responses to disjunctive questions on the basis of only one attribute--except for those questions specifying sex and specialty. In the latter case, she appeared to select spies matching both attributes. Thus, matches on any additional attributes should occur only by chance. Table 36 displays the frequency with which DK selected spies matching one or more of the attributes specified in disjunctive questions. The numbers in parentheses are the frequencies expected by chance under the conditions stated above. As the reader can see, the fit for the two-attribute questions is poor («(1)2 = 4.5. P<.03), but for the 3-. 4-, and 5-attribute questions the fit is not inconsistent with the above analysis (the largest chi-square occurs with p ).30). Thus, it seems likely that DK's processing of disjunctive questions involved retrieval of spies of the specified nationality, or of the indicated sex 149 and/or specialty when nationality was not specified. The remaining questions appear to have been answered on the basis of disguisability. Did DK use her subrosters in answering disjunctive questions? There were 44 intrasubroster responses and 20 cross-subroster responses. This many intrasubroster responses would seem to imply use of the subrosters. However, 26 of the 44 intrasubroster responses consist of repetitions of just three pairs of spies: these three pairs plus one cross-subroster pair account for 36--more than half--DK's answers to disjunctive questions. This might suggest that DK relied upon a few favorite pairs rather than using her subrosters. The use of the high-frequency pairs is detailed in Table 37. Two features of the information displayed there should be noted. First, the three intrasubroster pairs were used primarily in response to questions specifying attri- butes which would direct DK to the subroster containing the pair. Second, with the exception of the use of Sarah/Percy for 5 out of 6 questions calling for burglars, no pair accounts for a majority of responses to questions calling for one of DK's criterial attributes. And even though Sarah/Percy was the preferred response to questions calling for burglars, the pair was used nearly as often to answer questions not calling for burglars as it was to answer questions which did call for burglars. Thus, Table 37 offers no evidence that any high-frequency pair served exclusively 150 Table 37 Disjunctive Questions DK Answered with High Frequency Pairs in Session III Liz/Henry Hans/Erich Sarah/Percy Otto/Erich (n = 11) (n = 10) (n = 9) (n = 6) (English (cross- (English (German political) subroster) technical) technical) TD TMD TBd TGF VEBD TGFBD TFd VGM VEF VGAd VBD VGMA PEF PMBd VEB VGMAd PFAD PGMd PEFAD VA PD PGMBd PBd PAD tGB PGM tGFB tE PMd tB tA tG tBD tEMAd tGBD tAd N233. T = technical M = male A = assassin V = veteran F = female B = burglar P = political G = German D = disguisable t = trainee E = English d = not disguisable 151 as the primary response for questions specifying some given attribute. However, the table does indicate that the intra- subroster pairs tended to occur in response to questions which might direct DK to their respective subrosters. Furthermore, the members of each intrasubroster pair were the first two spies DK listed in their respective sub- rosters when she described her organization of the spy roster. Thus, it may be that one reason for the frequent use of the three intrasubroster pairs listed in Table 37 is that they were the first two spies in DK's arrangement of their respective subrosters. Whenever DK went to their subrosters she knew any two spies from the subroster would be relevant, and she simply named the first two spies she retrieved. How does one explain 10 occurrences of the cross- subroster pair Hans/Erich? The pair was used for seven questions calling for German, two questions calling for males, and for one question calling for male burglars. It would be appropriate for DK to go to the German political subroster for any question calling for Germans or burglars: presumably she would have no basis for choosing one sub- roster over another for questions calling for male spies. Hence, it may be that the pair Hans/Erich resulted from DK going to the German political subroster, retrieving Hans, and upon failure to retrieve a second spy, switching to the other German subroster and retrieving the top spy--Erich. 152 Responses to conjunctive questions. Response latencies for DK's first and second responses to Session III conjunc- tive questions are displayed in Table 38. Since DK noted that specialty and background are correlated within nationality. specialty is treated as a prepotent dimension in addition to nationality and background. The regularity that marked the other subjects' response latencies is not to be found in DK's data. The most striking feature is speed with which she named her second spy. Overall, the mean interval between the first and second spy she named was .58 second. This is faster than the estimated time required for the five subjects who subrostered on sex and nationality to simply state the name of the first relevant spy. Of course, given the degree of organization of the roster that was achieved by DK, it should not be surprising that her responses were fast. Indeed, this may partially account for the irregularity of her latency data--her searching was done so fast that a much greater proportion of her response time represents uncontrolled factors than was the case for other subjects. Summary for Subject DK. DK appeared to treat three attributes--nationality, sex, and specialty--as criterial for the disjunctive questions. She responded primarily to nationality. turning to the other attributes only if nationality was not specified. It appeared that she searched within subrosters for responses to the disjunctive questions. 153 Table 38 Latency Data for DK's Responses to Session III Conjunctive Questions Mean Response-One Latencies Number of attributes l .2. 2 i 2 0 prepotent 1.65 1.47 1.68 - - l prepotent 1.42 - 2.16 2.20 - 2 prepotent - 1.79 2.08 - - 3 prepotent - - 1.76 2.31 1.87 Mean Interval Between 1st and 2nd Response Number of attributes 1 2. 2 i 2 O prepotent .33 1.11 .46 - - 1 prepotent .34 - .59 .85 - 2 prepotent - .41 1.04 — - 3 prepotent - - .40 .78 - figfig. The prepotent dimensions are nationality, specialty, and background. 154 In spite of her division of the roster on the basis of nationality and background, and her observation that specialty and background are correlated within nationalities, DK's response latencies for the conjunctive questions did not conform to the pattern of other subjects who subrostered. However, it was noted that DK did process all the conjunctive questions very rapidly. CP: The Whole Answer Subject with Random Subrosters Subject's self-report. CP stated that she separated the stack of dossiers into four sets of five dossiers each. She then went about learning the spies five at a time. When she was asked how the sets of five dossiers were selected, CP stated that she had simply taken the top five dossiers as the first set, the next five as the second set, etc. In answer to a direct question, CP said that she had not based division of the roster on any of the spies' attributes, and that she was not aware of any characteristic shared in common by all the spies in any of the subrosters. CP said that she answered the questions by trying first to think of any spies who possessed at least one attribute specified in the question, and then examining each such spy in detail to make the final determination of his relevance. The responses to the disjunctivequestions. Table 39 presents the distributions of attribute strings in CP's responses to Session III disjunctive questions. Two features of the distributions are worth noting: first, the distributions for the several attributes are very similar: 155 Table 39 Distributions of Attribute Strings in CP's Responses to Session III Disjunctive Questions String Lengths Attribute l, (g ‘3 ‘4 ‘5 ‘6 ‘Z _ __ 19 German 120 62 33 18 10 O 1 O O 0 English 172 6O 19 7 4 l 0 0 0 0 Male 100 33 23 3O 11 5 2 2 1 0 Female 100 34 17 21 11 4 2 O 0 0 Political 120 86 26 7 2 1 O 0 O 0 Technical 90 66 35 22 8 1 0 0 O O Assassin 108 49 23 12 13 4 4 1 0 0 Burglar 82 44 19 20 15 4 3 O 0 O Disguisable 92 56 22 14 7 4 1 O 1 0 Not-disg'ble 69 75 42 22 3 2 2 O O 0 Veteran 152 62 26 11 0 0 0 O 0 0 Trainee 146 75 27 15 4 2 O 1 O 0 156 second, well over half the strings for each attribute are of length one or two. Thus, there is no indication that CP organized the roster of spies with respect to any given attribute--as one would expect on the basis of her descrip- tion of learning the spies. However, it is possible that some organization of the spies did take place within CP's arbitrarily formed groupings of five spies. Any such organization would be manifest in CP's responses to the disjunctive questions as recurring sets of spies who were named contiguously. Table 40 iden- tifies some pairs and triplets of spies whose members tended to be named contiguously whenever two or more occurred in the same response. The column on the right of the table expresses the number of times the members of the pairs or triplets were named contiguously as a proportion of the number of times all the members occurred in the same response. For example, the 7th pair, Brian/Marta, occurred in 16 responses, and the two spies were named contiguously in 13 of the 16 responses. Thus, there is evidence that CP developed some organization of the roster, but not necessarily along the attributes defined by the experimenter. For example, Wilfred and Werner may well have been grouped because their names start with "W": they have in common only the characteristic of being male and disguisable. It is perhaps worth noting that five of the seven groupings in Table 40 consist of spies who are either all assassins or else all burglars. It is also worth noting that there are 157 Table 40 Pairs and Triplets of Spies Occurring in CP's Responses to Session III Disjunctive Questions Proportion of occurrences with Attributes spies have members named Pair or triplet in common contiguously 1. Anne English, female, 11/17 Sarah burglar Victoria 2. Colin Assassin, disguisable 12/14 Marlene Liz 3. Werner Male, disguisable 7/8 Wilfred 4. Hans German, male, 7/8 Klaus political, burglar 5. Henry Male, assassins, 6/9 Erich disguisable 6. Otto German, technical, 16/17 Gerda veteran, assassin, not-disguisable 7. Brian Political, 13/16 Marta not-disguisable 158 two pairs of male spies, one pair of German spies, one German male pair, and a triplet of English females among the seven groups of spies. Responses to the conjunctive questions. The latencies for CP's first responses and the interval between onset of the first and onset of the second response are presented in Table 41. The latencies are classified by the number of attributes in the question and by the number of a priori prepotent attributes (i.e.. attributes specifying sex and nationality). The latencies tend to increase as the number of attributes in the questions increased, but the interesting feature of the table is that the latencies decrease with increasing numbers of prepotent attributes--as was true of the subjects who used subrosters defined by sex and nationality. Why would a subject who did not have sex/nationality subrosters produce a pattern of response latencies similar to subjects who did have such subrosters? The answer may be that CP was able to use her short lists of spies in the same way other subjects used their subrosters. The reader may have noticed in Table 40 that of the seven short lists indicated there, three were comprised of spies sharing one a priori prepotent attribute, and two consisted of spies having two of the prepotent attributes in common. CP may have turned first to these lists when questions specified prepotent attributes. Since the lists are so short, this may not always have resulted in finding a relevant spy 159 Table 41 Latency Data for Subject CP's Responses to Session III Conjunctive Questions Mean Response-One Latencies (in seconds) No. attributes in question .1 E 2, i 5 0 prepotent 3.65 6.16 6.81 - - 1 prepotent 1.95 3.56 5.55 4.91 _ 2 prepotent - - 4.85 4.37 5.66 Mean Interval Between Response 1 and Response 2 No. attributes in question 1 3, .2 i. 2 O prepotent 1.89 5.88 6.55 - - 1 prepotent 1.79 3.13 3.19 6.45 - 2 prepotent - - a a a a insufficient data 160 quickly: but a successful search even part of the time would reduce the mean search time for the questions specifying prepotent attributes. The short lists have their disadvantages. Because they are short, CP will frequently be forced to retrieve a second or third list in order to find a second relevant spy. Thus, in comparison with the subjects who subrostered, CP should be relatively slow with her second responses. Furthermore, because the short lists are apparently not an exhaustive classification of the spies in the roster, one might expect CP to omit more relevant spies than did the subjects who subrostered. Tables 42 and 43 present data relevant for these comparisons. Table 42 presents the time interval between the first and second responses to Session III conjunctive questions as a proportion of the latency of the first response. The latency ratios are classified by number of attributes in the questions and by the number of prepotent attributes. Within each cell the upper ratio is based on mean latencies for the five subjects who subrostered on sex and nationality and the lower ratio is CP's. Table 43 presents the number of omissions of relevant spies occurring in responses to Session III conjunctive questions. The first row presents the mean number of omis- sions for the three whole-answer sex/nationality subros- terers and the second row contains the number of omissions in CP's responses. It is clear that CP was more prone to omit spies than the other subjects. 161 Table 42 Time Interval Between First and Second Responses Expressed as a Proportion of the First Response Latency for Session III Conjunctive Questions Number of attributes in question 1 2 4 0 prepotent 5 SS .48 .55 .70 - — CP .52 .95 .96 _ _ l prepotent 5 SS .30 .49 .69 1.14 _ CP .90 .88 .57 1.31 - 162 Table 43 Number of Relevant Spies Omitted in Responses to All Session III Conjunctive Questions Number of attributes in question 1 2. 3. it 5 Whole-answer 4.33 5.33 3.67 1.67 0 Subrosterers Subject GP 44.00 34.00 23.00 10.00 2.00 163 Summary for subject CP. Unlike the other subjects, CP did not group the spies on the basis of common attributes when she learned them. She did, however, break the roster into four arbitrarily chosed groups of five. The responses to the disjunctive questions seemed to indicate that some degree of organization took place within the groups of five spies, because these responses were characterized by recurrent pairs and triplets of spies. The response latencies for the conjunctive questions furnished some evidence that CP used her short lists of spies the same way other subjects used their subrosters in answering conjunctive questions. Discussion Each of the ten subjects who participated in the spy retrieval task reported breaking the spy roster down into smaller sets in order to learn the spies' attributes. The most complex organization was reported by DK who divided the spies into four subrosters of five Spies each on the basis of nationality and background. She noted that the German political and English technical subrosters each contained four burglars and one assassin, and that the English political and German technical rosters each had four assassins and one burglar. Seven other subjects also organ- ized the spies in four subrosters of five spies each, using sex and nationality to define the subrosters. Of the remaining two subjects one divided the spies into a sub- roster of ten veterans and a subroster of ten trainees, and the other arbitrarily grouped the spies in sets of five without regard to any of the attributes characterizing the spies. Thus, although the subjects were unanimous in subdividing the roster for learning the spies, they differed in the way they subdivided it. The subjects also differed in the extent to which they used their subrosters in answering questions. Two whole- answer subjects did not use the subrosters at all. Because CP's subrosters were formed arbitrarily there were no 164 165 questions which identified spies to be found in some subset of her subrosters. Subject SP on the other hand started out with subrosters based on sex and nationality, but switched to processing the entire roster in alphabetical order to ensure remembering all the spies. A third subject, DK, who started out with the subroster based on nationality and background, did not report using her subrosters to answer questions. However, there is some evidence in her data that she may have used the subrosters. Her data also admit the possibility that she used a "favorite spies" approach to the disjunctive questions rather than using her subrosters. Her response latencies for the conjunctive questions showed little evidence of a subroster effect although all her responses were very fast--as one would expect, given her complex organization of the roster. It may be that her organization of the spies was so effective, and the two answer task so undemanding, that any subroster effects are too small to be observed. 0f the remaining seven subjects, three are whole answer subjects who subrostered on sex and nationality. Each was shown to have used her subrosters in responding to disjunctive questions. JL accessed the rosters in a fixed order and processed the spies in alphabetical order within the subrosters. CT and SM both accessed the rosters in a variable order depending on the attributes specified in the question. Within the subrosters SM grouped veterans and trainees. CT also had some intrasubroster structure, but not so clear-cut as SM's. 165 questions which identified spies to be found in some subset of her subrosters. Subject SP on the other hand started out with subrosters based on sex and nationality, but switched to processing the entire roster in alphabetical order to ensure remembering all the spies. A third subject, DK, who started out with the subroster based on nationality and background, did not report using her subrosters to answer questions. However, there is some evidence in her data that she may have used the subrosters. Her data also admit the possibility that she used a "favorite spies" approach to the disjunctive questions rather than using her subrosters. Her response latencies for the conjunctive questions showed little evidence of a subroster effect although all her responses were very fast--as one would expect, given her complex organization of the roster. It may be that her organization of the spies was so effective, and the two answer task so mwemmufing, that any subroster effects are too small to becflmerved. Of the remaining seven subjects, flute are whole answer subjects who subrostered on smcandnationality. Each was shown to have used her subrosters in responding to disjunctive questions. ‘ces msters in a fiXed order and proceslfi‘it ‘ )al order Within ‘1 I e rosters in a Specified in the veterans and structure, but L 166 The four remaining subjects are two-answer subjects of whom AH subrostered on experience while the other three subrostered on sex and nationality. Two of these subjects, JK and AH, appeared to adopt a conjunctive criterion for responses to disjunctive questions. JK then used her subrosters to answer the transformed disjunctive questions but AH apparently did not. The other two subjects differed in their approach to disjunctive questions. NA responded by searching through a set of mostly "technical" spies for a pair who matched the first attribute specified in the question. CM on the other hand, relied heavily on a few favorite pairs of spies which she associated with certain attributes in the questions. Although the three whole— answer and four two—answer subjects discussed just above differed in their handling of disjunctive questions, it is clear that six of them used their subrosters to process the conjunctive questions. This is shown by the fact that they tended to answer fastest those questions where the attri— butes specified spies contained in a subset of the sub- rosters. (The seventh subject, CT, was a pilot subject for whom no latencies were recorded.) The Models For every subject except DK, response latencies were an increasing function of the number of attributes specified in conjunctive questions. Only two of the models implied this relationship: the sequential list model and the subroster model. 167 Eight of the subjects can be easily classified with respect to the model which best fits their data. SP who processed the roster in alphabetical order, is an exemplar of the sequential list model. Subjects CT, SM, and JL were whole answer subjects whose responses to the disjunctive question would identify them as exemplars of the subroster model. Furthermore, their response latencies to the conjunctive questions also conform to the subroster model. Subjects JK, NA, and AH may also be classified as exemplars of the subroster model on the basis of their responses to the "all spies" question and the response latencies for the conjunctive question. CM was a two-answer subject who was not given the "all spies" question, but her response latencies for the conjunctive question also correspond to the predictions of the subroster model. Thus, of the eight subjects whose data were clear-cut, one was a sequential list processor and the other seven conformed to the subroster model. The remaining subjects are CP and DK. DK described herself as a subroster processor, but her data did not provide verification because differences in her response latencies for the conjunctive questions were small and irregular, and as a two-answer subject, she did not provide sufficient data for precise analysis of responses to the disjunctive questions. Little can be said about CP because her responses were so erratic. More than anything else she looks like a sequential list processor with a number of 168 short lists, but it is not at all clear that she consistently processed the spies in any given order. There was no indication of subrostering in her response latencies for the conjunctive questions. Thus, the results which were clear-cut supported the two models which have the subjects either processing the entire roster spy by spy, or else processing subrosters spy by spy, determining whether or not a given spy is relevant before passing on to examine the next spy. This leaves the other models—-the articulated list model, the attribute- access model, or the direct activation model-~without empirical support. Individual Differences There were marked individual differences among the subjects both in learning the spies and in answering the questions. Although it is true that all ten subjects grouped the spies in order to learn their attributes, nevertheless there were four different groupings among the ten subjects. The differences among the subjects are much greater with respect to processing the questions. Consider first the five whole—answer subjects. One used no subrosters (SP), one used arbitrarily defined and unstable subrosters (CP), and three used subrosters based on sex and nationality (CT, SM, and JL). However, although the last three may super— ficially appear to have processed the questions in a similar fashion, they processed the disjunctive questions with three 169 different strategies. Furthermore, the ordinal analysis of their disjunctive responses revealed differences of structure within their subrosters. Since these structural idiosyncracies consisted of further subdivision by attri- butes, it is likely that these three subjects also differed in their processing of conjunctive questions, but the differences were not evident because the conjunctive questions yielded so few answers. Among the five two-answer subjects there were no subjects who were even superficially similar in their processing of disjunctive questions. There were at least three discernible strategies, and no two subjects who used the same strategy implemented it the same way. Thus it may be said that while there were similarities in the approaches of the subjects to the spy retrieval problem, there were many differences in detail. The point of the preceding observation is this: In aggregates of data, the differences in tactics can obscure communalities of strategy. In the present study for example, an analysis of varience was done to show that there were reliable effects of number of attributes and number of prepotent attributes on response latencies for conjunctive questions. However, the analysis was done only on data from the five subjects who were known to have subrostered on sex and nationality. Had the data also included latencies from the subjects who subrostered on nationality and background, on experience, and the subject who processed the roster 170 alphabetically, it would probably have been the case that the effect of the prepotent attributes would have been obscured. And yet, as was stated in the results section, the effect of the prepotent attributes was evident in the individual response data for all six of the subjects who said they subrostered and for whom latency data were available. It might be argued that individual analysis of the subjects' data was necessary in the present study because the subjects were given too much freedom to organize their searching as they saw fit, whereas in most such studies the experimenter has better control. However, in the interest of gaining insight into human information processing, it seems more productive to study what subjects prefer to do, given alternatives, rather than what they must do because the experimenter has left them no other course of action. Moreover, it might be that in studies thought to be better controlled, the experimenter has neither limited the number of strategies available to his subjects nor has he checked for differences in their data. Indeed, it may be the case that the extent of individual differences observed in this study is not atypical, but would be found in most such studies if the investigators were to examine the data from individual subjects. Summary and Conclusions The study sought to capture the organization of memory in a large, complex set of information. It was successful 171 in eight of the ten subjects. One subject processed the roster of twenty spies alphabetically in answering each question. Six subjects broke the roster into four subsets using sex and nationality: i.e., into a two-dimensional nonhierarchal organization. One subject divided the roster into trainees and veterans. One subject was aware of a com- plex three-dimensional structure, but responded to the task so rapidly and so adaptively that her strategy was never identified. Finally, one subject subrostered the spies on a random basis initially, but answered questions using a strategy which may have involved searching many short lists of spies. Thus, in nine of ten subjects there is clear evidence that memory was organized in a more-than- associative structure. Among the ten subjects, seven qualitatively different memory structures were employed in responding to disjunctive questions. Furthermore, five radically different strategies were used for the disjunctive questions. A discussion noted that this order of individual differences nullifies the utility of statistical techniques using aggregates such as means, correlations, or conditional probabilities. An argument was made that the extent of individual differences found in this study might be found in many studies if the experimenters would undertake the analysis of the data of individual subjects. LIST OF REFERENCES .v'“ LIST OF REFERENCES Corballis, M. C. Serial order in recognition and recall. Journal of Experimgntal ngcholggy, 1967, 24, 99-105. Corballis, M. C., Kirby, J., & Miller, A. Access to elements of a memorized list. Journal of Experimental Psychology, 1972, 24, 185-190. Feller, W. An introduction to probability theory and its applications, volume 1 (2nd ed.). New York: Wiley, 195?. Hunt, E. B. Concept Learning: An Information Processing Problem. New York: Wiley, 1962. Kirshner, K., & Craik, F. I. M. Naming and decision processes in short-term recognition memory. Journal of Experimental Psychology, 1971, @8, 149-157. Mandler. G. Organization and memory. In K. W. Spence & J. A. Spence (Eds.), The Psycholpgy of Learning and Motivation (Vol I). New York: Academic Press, 1967. March, J. G., & Simon, H. A. Organizations. New York: Wiley, 1958. Miller, G. A. The magical number seven plus or minus two: Some limits on our capacity for processing information. Psychological Review, 1956, 63, 81-97. Miller, G. A., Galanter, E., & Pribram, K. H. Plans and the Structure of Behavior. New York: Holt, 1960. Murdock, B. B., Jr. A parallel-processing model for scanning. Perception and Psychophysics, 1971, 39, 289-291. Newell, A., Shaw, J. C., & Simon, H. A. Elements of a theory of human problem solving. Psychological Review, 1958, 6 , 151-166. Okada, R., & Burrows, D. Organizational factors in high speed scanning. Journal of Experimental Psychology, 1973. 19. 77—81. 172 173 Shevell, S. K., & Atkinson, R. C. A theoretical comparison of list scanning models. Journal of Mathematical Psychology, 1974, 31, 79-106. Simon, H. A. Administrative Behavior. New York: Macmillan, 1945. Sternberg, S. High speed scanning in human memory. Science, 1966, 153, 652-654. Sternberg, S. Two operations in character recognition: Some evidence from reaction time measurements. Perception and Psychophysics, 1967, g, 45-53. Sternberg, S. Memory-scanning: Mental processes revealed by reaction-time experiments. American Scientist, 1969, 52, 421-457. Theios, J., Smith, P. G., Haviland, S. E., Traupmann, J., & May, M. 0. Memory scanning as a serial self-terminating process. Journal of Experimental Psychology, 1973,432, 323-336. Winer, B. J. Statisticaljprinciples in experimental desigp. New York: McGraw-Hill, 1962. Wood, G. Organizational processes and free recall. In E. Tulving & W. Donaldson (Eds.), Organization of Memory. New York: Academic Press, 1972. "Alfilfllfitfllliljfllfl