Recently, our team conducted a fascinating experiment that has forced us to completely reconsider the future of AI in analysis and how we should best utilize these tools. We pitted an AI against experienced gold and forex analysts in a prediction challenge across 65 cases, and the results have given us much to ponder.
The Interesting Experiment: AI vs Professional Analysts
We organized the experiment by dividing participants into two groups: the first was our FLUX model running on a Mac Mini M4 Pro, and the second was a group of analysts with at least 5 years of real-world experience in the gold and forex markets. All participants had to answer various prepared questions without knowing who was who. The experiment was scored out of 50 points, divided into 5 main categories.
Question Categories Used in the Experiment
For this experiment, we prepared questions covering 6 main categories that Thai analysts in the industry are familiar with:
- Short-term gold price trend analysis
- Factors affecting THB/USD exchange rates
- Making forex trading decisions based on economic events
- Technical analysis using predictive indicators
- Risk management in trading
- Specialized case studies from Thai market history
Each category had questions of equal difficulty, and all were appropriately adjusted for the level of both participant groups.
The Rigorous Testing Process
To ensure transparent and neutral results, we established a very strict testing process. Both the AI and analysts received the same information and had to answer questions within a specified time. Additionally, we used MetaTrader 5 on a Windows machine to provide identical insights to all participants. The evaluation was conducted by an independent team that had never seen the work of either group before.
Notable Results
When the test concluded, the results made everyone on our team reconsider everything. The total score difference between the two groups was only 1.10 points—much less than anticipated. The professional analysts scored slightly higher, but in many categories, the AI model performed just as well. What's interesting is that both groups passed all quality checks, showing that both AI and humans can make logical, data-consistent predictions.
In-Depth Result Analysis
After the test, we conducted an in-depth analysis and found that AI has advantages in processing speed and pattern recognition, while analysts have the advantage in interpreting current events and real trading experience. However, in many cases, the AI could solve complex problems as well as analysts, using broader market history data.
Lessons from This Experiment
This experiment taught us that the future of market analysis doesn't depend on who is smarter, but rather that AI should be viewed as a tool to enhance analysts' capabilities. The combination of human intelligence with AI's speed and accuracy may be the best path forward, and we should begin to consider AI implementation in terms of augmentation rather than replacement.
Next Steps for the Industry
Based on the results of this experiment, we plan to develop the iCafeFX app to feature integrated analysis results from both AI and analyst insights, along with connections to a Synology DS725+ for important data storage. Additionally, we want to build an open community for knowledge exchange between analysts and practical AI applications to continuously improve market analysis quality.
For those who want to try real gold trading, open an XM account at: https://clicks.pipaffiliates.com/c?c=72816&l=th&p=6
