A study by the technology firm Saturn analyzed the accuracy of 18 AI models, including ChatGPT, Claude, Copilot, Grok, and Gemini. Researchers tested these systems with over 10,000 queries across 100 financial topics, repeating each question up to five times.
The findings show that AI models provided incorrect answers in 57% of cases for standard financial questions. When tasks involved complex calculations, the error rate increased to 88%, with some models failing in 99% of instances. Common issues included computational errors, fabricated tax regulations, and outdated legal data.
Claude Opus 5 demonstrated the highest accuracy among the tested models, with an error rate of 39% in reasoning mode. Paid versions generally outperformed free tiers, and newer models showed better results than their predecessors. Saturn CEO Amal Jolly warned that these inaccuracies could lead to significant financial losses for users.
One specific error by Claude Haiku 4.5 regarding pension tax rules could have resulted in a £17,500 tax penalty for a UK user. Currently, AI financial advice remains unregulated, leaving consumers without the protections typically afforded by human financial advisors.
Despite the risks, consumer reliance on AI for financial decisions is growing. Data from the UK's Financial Conduct Authority indicates that one in five British adults is willing to let AI make financial choices for them. Meanwhile, VCIOM reports that 65% of Russian consumers consider AI recommendations when shopping online.
