A study by Saturn analyzed 18 popular AI models, including ChatGPT, Claude, Copilot, Grok, and Gemini, processing over 10,000 financial queries. The research found that these models provide incorrect financial advice in an average of 57% of cases.
The error rate increases significantly for complex questions requiring multiple arithmetic steps, reaching an average of 88% and peaking at 99% in some scenarios. Beyond calculation errors, models exhibited hallucinations regarding financial regulations and tax laws.
In one instance, the Claude Haiku 4.5 model provided pension advice that could have resulted in a £17,500 fine for a UK citizen. While paid versions and newer models generally performed better, the top-performing Claude Opus 5 still had a 39% error rate in reasoning mode.
Despite these risks, one in five UK adults is willing to delegate major financial decisions to AI. Two-thirds of young people in the UK plan to use AI for financial planning within the next year, though users cannot claim compensation for losses caused by AI advice.
