AI-Trader: Benchmarking Autonomous Agents in Real-Time Financial Markets
Paper · https://arxiv.org/abs/2512.10971
Large Language Models (LLMs) have demonstrated remarkable potential as autonomous agents, approaching human-expert performance through advanced reasoning and tool orchestration. However, decision-making in fully dynamic and live environments remains highly challenging, requiring real-time information integration and adaptive responses. While existing efforts have explored live evaluation mechanisms in structured tasks, a critical gap remains in systematic benchmarking for real-world applications, particularly in finance where stringent requirements exist for live strategic responsiveness. To address this gap, we introduce AI-Trader, the first fully-automated, live, and data-uncontaminated evaluation benchmark for LLM agents in financial decision-making. AI-Trader spans three major financial markets: U.S. stocks, A-shares, and cryptocurrencies, with multiple trading granularities to simulate live financial environments. Our benchmark implements a revolutionary fully autonomous minimal information paradigm where agents receive only essential context and must independently search, verify, and synthesize live market information without human intervention. We evaluate six mainstream LLMs across three markets and multiple trading frequencies. Our analysis reveals striking findings: general intelligence does not automatically translate to effective trading capability, with most agents exhibiting poor returns and weak risk management. We demonstrate that risk control capability determines cross-market robustness, and that AI trading strategies achieve excess returns more readily in highly liquid markets than policy-driven environments. These findings expose critical limitations in current autonomous agents and provide clear directions for future improvements. The code and evaluation data are open-sourced to foster community research: https://github.com/HKUDS/AI-Trader.
First seen by Oriel on .
AI summary
AI-Trader is a live, automated benchmark showing that current LLM agents generally perform poorly in financial trading despite strong general intelligence, with risk control driving robustness.
- The benchmark evaluates agents across U.S. stocks, A-shares, and cryptocurrencies at multiple trading frequencies.
- Agents receive minimal essential context and must independently search, verify, and synthesize live market information without human intervention.
- Six mainstream LLMs were evaluated across the three markets and trading frequencies.
- Most agents produced poor returns and weak risk management, while risk control determined cross-market robustness.
- AI trading strategies achieved excess returns more readily in highly liquid markets than in policy-driven environments.
#llm agents #financial markets #algorithmic trading #benchmarking
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hugging Face Papers: 13, latest observation ; 0 over observed period since
Comments
Comment count reported by the source.
- Hugging Face Papers: 1, latest observation ; 0 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Papers | 2512.10971 | https://arxiv.org/abs/2512.10971 Read discussion |