Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?

Divij Chawla, Ashita Bhutada, Do Duc Anh, Abhinav Raghunathan, Vinod SP +6 more
5/25/2025
cs.CL

No confident code implementation found for this paper.

Abstract

We assess whether AI systems can credibly evaluate investment risk appetite-a task that must be thoroughly validated before automation. Our analysis was conducted on proprietary systems (GPT, Claude, Gemini) and open-weight models (LLaMA, DeepSeek, Mistral), using carefully curated user profiles that reflect real users with varying attributes such as country and gender. As a result, the models exhibit significant variance in score distributions when user attributes-such as country or gender-that should not influence risk computation are changed. For example, GPT-4o assigns higher risk scores to Nigerian and Indonesian profiles. While some models align closely with expected scores in the Low- and Mid-risk ranges, none maintain consistent scores across regions and demographics, thereby violating AI and finance regulations.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Model Risk Management for Generative AI In Financial Institutions

83% match2025

Application of AI in Credit Risk Scoring for Small Business Loans: A case study on how AI-based random forest model improves a Delphi model outcome in the case of Azerbaijani SMEs

82% match2024

An Artificial Intelligence Value at Risk Approach: Metrics and Models

81% match2025

Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selection

81% match2024

FinRobot: AI Agent for Equity Research and Valuation with Large Language Models

81% match1 repo8.1k2024

Standard Benchmarks Fail -- Auditing LLM Agents in Finance Must Prioritize Risk

80% match2025

Cite this paper

@article{chawla2025evaluating,
  title  = {Evaluating AI for Finance: Is AI Credible at Assessing Investment Risk?},
  author = {Divij Chawla and Ashita Bhutada and Do Duc Anh and Abhinav Raghunathan and Vinod SP and Cathy Guo and Dar Win Liew and Prannaya Gupta and Rishabh Bhardwaj and Rajat Bhardwaj and Soujanya Poria},
  year   = {2025},
  eprint = {2505.18953},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2505.18953v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2505.18953.svg)](https://codeofpaper.com/paper/2505.18953)

Discussion