Papers
arxiv:2406.17055

Large Language Models Assume People are More Rational than We Really are

Published on Jun 24
· Submitted by theryanliu on Jun 26
Authors:
,
,
,

Abstract

In order for AI systems to communicate effectively with people, they must understand how we make decisions. However, people's decisions are not always rational, so the implicit internal models of human decision-making in Large Language Models (LLMs) must account for this. Previous empirical evidence seems to suggest that these implicit models are accurate -- LLMs offer believable proxies of human behavior, acting how we expect humans would in everyday interactions. However, by comparing LLM behavior and predictions to a large dataset of human decisions, we find that this is actually not the case: when both simulating and predicting people's choices, a suite of cutting-edge LLMs (GPT-4o & 4-Turbo, Llama-3-8B & 70B, Claude 3 Opus) assume that people are more rational than we really are. Specifically, these models deviate from human behavior and align more closely with a classic model of rational choice -- expected value theory. Interestingly, people also tend to assume that other people are rational when interpreting their behavior. As a consequence, when we compare the inferences that LLMs and people draw from the decisions of others using another psychological dataset, we find that these inferences are highly correlated. Thus, the implicit decision-making models of LLMs appear to be aligned with the human expectation that other people will act rationally, rather than with how people actually act.

Community

Paper submitter
edited Jun 26

LLMs assume people are more rational than we really are:
Applies to simulating decisions, predicting people’s actions, and making inferences from others’ decisions.

·

On a massive psychological dataset of 13,000+ risky choices:
We find that {GPT-4o, GPT-4-Turbo, Claude Opus, Llama3} predictions and simulations correlate highly with an existing model of rational choice (expected value theory), but only moderately correlate with humans.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2406.17055 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2406.17055 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2406.17055 in a Space README.md to link it from this page.

Collections including this paper 1