When do consumers prefer algorithmic versus human decisionmakers?

Imagine that you’re applying for a bank loan to finance the purchase of a new car, which you need badly. After you provide your information, the bank gives you a choice: Your application can be routed to an employee in the lending department for evaluation, or it can be processed by a computer algorithm that will determine your creditworthiness. It’s your decision. Do you pick Door Number One (human employee) or Door Number Two (software algorithm)?

The conventional wisdom is that you would have to be crazy to pick Door Number Two with the algorithm behind it. Most commentators view algorithms with a combination of fear and loathing. Number-crunching code is seen as delivering inaccurate judgments; addicting us and our children to social media sites; censoring our political views; and spreading misinformation about COVID-19 vaccines and treatments. Observers have responded with a wave of proposals to limit the role of algorithms in our lives, from transparency requirements to limits on content moderation to increased legal liability for information that platforms highlight using their recommendation engines. The underlying assumption is that there is a surge of popular demand to push algorithms offstage and to have old-fashioned humans take their place when decisions are to be made.

However, critics and reformers have largely failed to ask an important question: How do people actually feel about having algorithms make decisions that affect their daily lives? In a forthcoming paper in the Arizona State Law Journal, we did just that and surveyed people about their preferences for having a human versus an algorithm decide an issue ranging from the trivial (winning a small gift certificate from a coffee shop) to the substantial (deciding whether the respondent had violated traffic laws and should pay a hefty fine). The results are surprising. They demonstrate that greater nuance is sorely needed in debates over when and how to regulate algorithms. And they show that reflexive rejection of algorithmic decisionmaking is undesirable. No one wants biased algorithms, such as ones that increase racial disparities in health care. And there are some contexts, such as criminal trials and sentencing, where having humans decide serves important values such as fairness and due process. But human decisionmakers are also frequently biased, opaque, and unfair. Creating systematic barriers to using algorithms may well make people worse off.

The results show that people opt for algorithms far more often than one would expect from scholarly and media commentary. When asked about everyday scenarios, people are mostly quite rational—they pick between the human judge and the algorithmic one based on which costs less, makes fewer mistakes, and decides faster. The surveys show that consumers are still human: We prefer having a person in the loop when the stakes increase, and we tend to stick with whichever option we’re given initially. The data analysis also suggests some useful policy interventions if policymakers do opt to regulate code, including disclosing the most salient characteristics of algorithms, establishing realistic baselines for comparison, and setting defaults carefully.

Choosing between a human and an algorithm

Our study included approximately 4,000 people who responded to an online survey. Each respondent was randomly assigned to one of four scenarios involving a decision with varying stakes: winning a coffee shop gift card; receiving a bank loan; being included in a clinical trial for a promising treatment; and facing a sizable fine in civil traffic court. The survey then randomly picked either a human or an algorithm as the initial (default) decisionmaker. Then, the survey gave the participant information about the cost, speed, accuracy, and information used (public only, or public plus private information such as credit reports) for each type of decisionmaker. Finally, the survey asked the respondent whether they wanted to stay with the decider (human or algorithm) they were assigned initially, or whether to switch. This design let us assess how each of these factors affected a participant’s choice of human versus algorithm, as well as how they interacted with one another. The results were clear and powerful—each finding was strongly statistically significant, with two exceptions described below.

In a number of common decisionmaking situations, people prefer an algorithm to a human, but whether they prefer the algorithm is shaped by how it functions. In the aggregate, survey takers chose an algorithm to decide an issue more than half the time (52.2%), suggesting that moral panics over algorithms are overstated. This preference stands in stark contrast to the algorithmic skepticism that dominates media coverage. Second, consumers’ feelings toward algorithms are strongly and significantly determined by practical factors such as cost, speed, and accuracy—but they are not meaningfully affected by privacy concerns such as access to sensitive data. Based upon this data, we argue that policy choices about the permissible role of algorithms in decisionmaking systems must carefully consider these consumer preferences. Failure to do so will undermine the legitimacy of legal reforms, and it will impede their implementation.

Privacy was not a meaningful factor in people’s choices. Participants were indifferent between humans and algorithms in scenarios where the decisionmaker would have access to private information about them. We suspect that people have heterogeneous preferences where private information is incorporated into a decision. For example, some people may prefer algorithms because code doesn’t judge—it processes information using unemotional logic, without moralizing. But that may be the same reason that others prefer revealing sensitive information to other humans, who can place disclosures in broader context and who can analyze with empathy. Overall, though, privacy concerns did not produce a preference in either direction.

Consumer preferences

What matters most in the choice between an algorithm and a human? In short: price (relative costs and benefits), accuracy, stakes, speed, and default settings. Our study design enabled us to tease apart different variables to see which ones mattered most to people. The results showed that participants, rather than being averse to algorithms, reacted to different conditions in ways that largely track classic rational models of behavior. People preferred the option that decided more cheaply, more accurately, and more quickly. As the stakes involved rose, participants tended to pick a human at a greater rate. Our survey data also showed a strong anchoring effect: Respondents tended to stay with the type of decisionmaker that they were initially assigned by the program, especially if it was an algorithm.

Price really matters to consumers. When the algorithm offered lower cost or greater benefit, 61% of respondents selected it, versus only 43% when it was equal to a human. The effects of pricing outweighed those from accuracy (error rate) and speed. For example, consider the choice between a risky algorithm (one with a high error rate) and a human with an unknown level of accuracy, where the prices of the two options are equal. Survey takers picked the erratic algorithm just 26% of the time under these circumstances. As soon as the algorithm offered a meaningful price advantage, though, its acceptance rate rose by more than half, to 42%. Participants were willing to gamble for a bigger payoff. Similarly, if both human and algorithm offered the same speed and benefits, respondents chose the code just 38% of the time. If the algorithm was faster, that share increased to 47%. But if the algorithm offered a better price in equal time, participants went for it at a rate of 57%.

Stakes also matter to consumers. The survey scenarios asked participants about decisions of greater and lesser weight: winning a coffee shop gift card with a small monetary value ($10-$20) at the low end or being fined several hundred dollars for a traffic ticket at the high end. The results tracked standard intuitions about consumer preferences—with greater stakes, respondents were more likely to want a human in the loop. Even so, this effect was bounded. For the traffic ticket scenario, 56% opted for a person; for the gift card, 56.4% picked the algorithm.[1] At both ends of the range of stakes, a significant minority still preferred the less-popular option.

Accuracy also influenced the choice between human and algorithm. When one option had a high error rate and the alternative had a low one, 74% of respondents picked the choice offering greater accuracy. When humans and algorithms had equal accuracy, participants essentially split their votes. And in the absence of any information about either option’s accuracy, people went with the human decisionmaker 65% of the time, which may reflect baseline assumptions about how risky algorithms are.

Speed also influenced the decision. A faster algorithm led 57% of survey takers to opt for it, but when the human was just as quick, that number fell to 48%. And in each of the four scenarios, when the speed of the two decisionmakers was equal, respondents’ willingness to choose the algorithm dropped.

Finally, defaults matter. People initially assigned to an algorithmic decisionmaker stayed with that option 62% of the time, as did 58% of those initially given a human. Interestingly, the difference between the algorithm and human defaults was statistically significant: Algorithms had greater staying power. The greatest reduction in anchoring occurred when there was a difference in accuracy between the two options. For example, when participants were assigned initially to a human, but the human had a high error rate and the algorithm a low one, 72% of people switched to the formula. Thus, even though the initial selection of a type of judge was random, participants were significantly influenced by it.

These results show that consumers tend to be pragmatic when assessing whether to go for the algorithm or the human. Above all else, they are price-conscious, suggesting that utilitarian considerations drive this choice in the realm of everyday decisionmaking.

Policy recommendations

These data on consumer preferences for having humans or algorithms make decisions generate important policy insights. The first is simply that preferences matter and must be taken into account, for both principled and instrumental reasons. The principled position is that democratic states operate on the assumption that people are the best judges of their own interests. That assumption can be disproven, especially when one person’s decision harms someone else, such as when entities use algorithms that discriminate against marginalized communities and individuals. Our research, though, involved tradeoffs that only affected the person making the choice between human and formula as decider. If people are willing to trade lower accuracy for lower price, it’s not clear that society should override that preference, particularly for prosaic decisions. The instrumental reason for considering extant preferences is that reform is harder if proposed changes run contrary to popular attitudes. Even if people’s views are misguided, they are an important guide to which policy proposals are attainable and at what cost.

Second, the survey results are enormously useful for efforts to improve algorithmic transparency. Participants’ choices tell us what information is most salient for consumers’ decisions. Reforms might concentrate on having entities such as banks disclose what benefits algorithms offer, and at what costs, rather than information about how the formulas operate, which may be inscrutable for the average person in any case. Perhaps most important, transparency is relative. Human judges can also be opaque, biased, and arbitrary. The baseline for algorithms should be the real-world alternative that humans offer, not an ideal of the neutral, objective flesh-and-blood arbiter.

Third, defaults matter, but they are not absolute. Our study shows that the anchoring effects of initial assignments are real and significant. Path dependency may be consequential for algorithmic governance, making it important to set policy carefully in the first instance. The results also show how to break through the inertia of defaults: offer people relevant, accurate information in the context of meaningful choice. This could be combined with system design that minimizes the influence of defaults, such as forced choice, where there is neither the reality nor the appearance of an initial setting.

Fourth, survey participants instinctively hit upon a key point in the debate over algorithms: stakes and context matter. This study tested relatively mundane decisions. Even within its limited scenarios, though, concerns about bias and other flaws are more pressing in the traffic fine context, where several hundred dollars was at stake, than when contending for a coffee shop gift card. This has implications for both sides of the debate. Advocates for and designers of algorithms ought to concentrate on circumstances where more is at stake, and within that framework, they should be particularly mindful of the results of consumer preferences, such as the power of default settings. And critics will make a more effective case if they focus upon more consequential situations for algorithms, such as with criminal probation, bail, and sentencing. There are, in short, multiple conversations to be had about algorithms, and that discourse will be improved by attention to what’s at risk in the underlying decision.

Lastly, consumers are sensitive to certain forms of risk when choosing between a human and algorithm—in particular, they are concerned with the risk that the decisionmaker will make a mistake (error rate) or the risk that they will suffer more serious consequences (stakes based on the scenario). These empirical findings lend support to the risk-based approaches for algorithmic governance under consideration by both the European Union and some members of Congress and the Biden administration, especially if the risk analysis includes consideration of offsetting benefits.

To come full circle: for most people, the choice between algorithm and human to decide an issue is not the lady, or the tiger. It instead involves pragmatic, utilitarian questions that may be importantly affected by system design, such as through default settings. Paying attention to what consumers want, and why, when algorithms make decisions will generate both better code and better governance.

[1] This was statistically indistinguishable from the 58.4% who chose the algorithm in the bank loan scenario.

Derek E. Bambauer is a professor of law at the University of Arizona, where he teaches Internet law and intellectual property.
Michael Risch is the vice dean and a professor of law at Villanova University Charles Widger School of Law, where he is a member of the university IP Policy Board.