Intelligent Decision Aids (IDAs) are emerging technologies used in areas such as medicine, finance, and e-commerce that leverage artificial intelligence, data mining, or related computational methods to provide recommendations to decision makers. An important goal for designers should be to help users identify and accept good recommendations and ignore poor recommendations. However, considerable research has found that IDA users frequently make poor decisions about which recommendations to follow.I present findings from three studies that provide evidence of four distinct decision-making biases related to IDA-supported decision making. These biases are characterized by an increase in users' agreement with an IDA's recommendations that is unassociated with the recommendations themselves but associated with some other aspect of the design of the IDA or of the user.In an experiment that manipulated the perceived customizability of an IDA that assisted users in predicting the outcomes of baseball games, I found that users who believed they had customized the IDA were more likely to follow both good and poor recommendations than other users who received identical recommendations from the IDA but did not customize its logic. This finding is evidence of a customization bias. Importantly, this study found that customization bias is not caused by users believing they have improved the algorithm by customizing it.In a second experiment, subjects were encouraged to believe that the system had either high or low efficacy prior to seeing recommendations. This encouragement created an expectations bias in which subjects were more likely to follow both good and poor recommendations when they had higher expectations of the IDA's efficacy than other subjects who had expected the IDA's algorithm to perform poorly. In the third experiment, I assessed decision making by users of an IDA for recommending exercise activities. Subjects who used a customizable version of this IDA, where the recommendations depended on how users configured the IDA, were more likely to agree with the recommendations than users who received recommendations of similar quality but did not customize the IDA. This finding shows additional evidence of customization bias, demonstrating that it extends to IDAs where the customizability has real influence over the recommendations rather than merely perceived customization as in the first study. In this study I also found that when users believe that an IDA's internal logic is more clear and understandable, they are more likely to follow recommendations regardless of their quality. This finding suggests a transparency bias. There was a strong relationship between the quality of recommendations that subjects received and the quality of their decisions, indicating that when decision makers are supported by IDAs, the quality of recommendations is important to system success. However, subjects who performed the decision task unaided by an IDA performed as well as the IDA-supported subjects. These findings show that when decision makers are aided by an IDA, the system affects the decision making process by requiring users to evaluate recommendations. IDA users may make biased evaluations due to characteristics of the interface and interaction design of the system as well as individual characteristics of the users. In the concluding chapter I discuss the implications of these findings for the design of IDAs and related socio-technical systems, as well as for future work on computer-supported decision making.
Read
