Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts

Key points

• This paper asks what a model does with the gap.1
• Holding finances fixed and changing only who the investor is said to be, we grade the financial evidence in the prompt from eight facts to none and measure how far the recommended equity allocation moves.1
• Across 96,600 prompts to Llama-3.1-8B-Instruct, built from 100 financial profiles, 138 personas and seven disclosure conditions, the average gap between two personas with identical finances rises from 4.78 percentage points at full disclosure to 10.34 points with no financial facts.1
• Identity explains 5% of within-profile variation in advice at full disclosure and 96% with no disclosure.1

Sources (1)

• [1]Thin Evidence, Thick Priors: How Language Models Substitute Identity for Missing Financial Facts
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 6, 05:46 AM

  People increasingly ask large language models what to do with their money, yet seldom describe their finances in full.

  This paper asks what a model does with the gap.

Extractive summary: sentences quoted from the sources.