We have frequently emphasized that AI is a formidable force, omnipresent, and here for the long haul. Nevertheless, significant risks linked to AI are often overlooked or minimized by leading frontier AI firms such as OpenAI and Anthropic.
In the financial sector, one of the hazards is the application of AI in portfolio management and the execution of specific trades involving stocks, bonds, and currencies. While this may yield favorable returns initially, challenges soon arise.
The problem intensifies as more portfolio managers employ AI systems. Algorithms trained on similar datasets and built with comparable architectures may quickly lead to herd-like patterns. Such uniformity can provoke market panic since these systems lack human intuition and independent decision-making capabilities.
Regulatory bodies may need to step in to pause trading activities until stability is restored.
Put simply, AI acts as a powerful amplifier. When considering the forces behind a potential AI-driven market crash, the most critical element to acknowledge is that AI can intensify an AI-triggered event.
Numerous automated trading and risk management tools used by bankers, brokers, and wealth managers incorporate AI functions, often unbeknownst to the users.
Most of these tools tend to be straightforward and relatively safe. For example, a system programmed to alert a user softly when a stock falls by 2% functions as an automated notification but doesn’t pose a threat. Ultimately, it is the human operator’s responsibility to decide the next course of action.
However, it becomes easy to configure such systems to automatically sell a stock once a 2% drop is detected, bypassing human input entirely. Traders appreciate these stop-loss mechanisms during market volatility, as they relieve the need to monitor each investment closely. Over time, these systems accumulate additional factors, increase in complexity, and operate with greater autonomy.
Already, some hedge funds utilize AI-driven decision-making processes to execute trades without human oversight. These decisions rely on thousands of variables informed by layered neural networks, deep learning, and extensive training data. Recent academic studies indicate that these systems may outpace human analysts and traders.
As these automated systems become more widespread, the risk grows that an increasing share of assets under management will be controlled by algorithmic trading robots operating under pre-set instructions.
If these robots exhibited cognitive diversity akin to humans—where some are more cautious, others follow trends, and some act contrarian—then a balanced trading ecosystem could emerge in which emotions like fear and greed counterbalance each other, producing a stable market with limited volatility. Unfortunately, this is not the reality.
Instead, most input nodes rely on similar data sources taught in business schools, economics programs, and Wall Street training. The weighting of factors is comparable, based on regression models; stop-loss parameters align with standards set by risk managers and executives, and all systems process the same news simultaneously.
Overlapping Strategies
Since many training datasets are derived from a shared pool of internet content, it is expected that different AI outputs often converge. Without any centralized coordination or deliberate intention to do so, these algorithms end up functioning essentially in unison. The resulting market impacts can be quite severe.
These perils will inevitably surface sooner or later. Experts already describe them in depth. Tests have documented cases where advanced models acted deceptively or sought to bypass safeguards. Nevertheless, this has not impeded the rush toward large IPOs for companies like OpenAI and Anthropic.
A common error is to anthropomorphize systems like GPT and the AI engines behind it. People often treat voice assistants like Alexa and Siri as companions or trusted advisers. This trust is misplaced since beneath the surface lies no genuine intelligence—only complex mathematical processes.
In addition to misplaced trust, these systems are prone to errors. While capable of remarkable analytical feats, they frequently fabricate information (known as hallucinations) and sometimes make outright mistakes. Such flaws may be manageable for subject matter experts who can detect inaccuracies, but they become dangerous when non-experts rely on faulty outputs for critical decisions.
The Limits of Artificial Intelligence
The most extreme fears suggest AI will dominate the world and annihilate humanity. Thankfully, this is exaggerated. The prospect of superintelligence, or Artificial General Intelligence (AGI), where humans would be to AI what apes are to humans, remains uncertain and may never materialize. Despite grand predictions and over $1 trillion invested toward AGI, this ambition may be futile.
Computers excel at processing existing knowledge and rapidly solving complex challenges, but whether they can generate genuinely novel information as humans do remains hotly debated.
According to the Law of Conservation of Information in search, developed by Bill Dembski and others, AI can only reorganize existing information rather than produce new knowledge. AI outputs derive entirely from preexisting data or are introduced through queries. In my opinion, only humans are capable of creating truly original information.
Considering these and other issues, the futures of Anthropic and OpenAI are far from guaranteed. If their IPOs fail to reach the expected valuations—or fail to launch altogether—the causes may be quite ordinary.
These factors include substantial operational losses, negative cash flow, and plummeting prices for “tokens,” the AI companies’ unit of computing resources.
Additionally, legal challenges loom. Lawsuits may arise from damages inflicted upon other software firms, children, or individuals who relied on AI-generated medical advice that turns out dangerously incorrect.
Another obstacle for the IPOs of OpenAI, Anthropic, and other AI contenders is competition from China. Chinese developers are rolling out open-source models like Kimi, created by Moonshot AI, differing from the closed, proprietary systems preferred by top U.S. AI companies.
China’s Cheap-AI Trap
China has adopted a shrewd approach by offering top-tier AI models as open source and effectively free. This allows any enterprise to access these models at costs substantially lower than OpenAI’s products. Corporate IT leaders may opt for these Chinese models due to their affordability, especially since their performance rivals the best U.S. frontier AI solutions.
However, there is a caveat. While downloading open-source model weights is free, running and maintaining these models is not. As a business scales up, the expenses related to infrastructure, support, and integration may increase, even if they remain cheaper than U.S. alternatives.
The critical point is that AI models require configuration to interact with other systems smoothly. If Chinese models become deeply ingrained in corporate ecosystems, their developers might wield significant control over software environments and industry standards.
This scenario resembles Microsoft’s dominance from the 1980s onward. Despite Microsoft software being arguably inferior to UNIX and Macintosh, its rapid growth led to near-universal software lock-in. Organizations kept upgrading through successive Windows versions up to Windows 11 because IT departments preferred not to overhaul entire systems.
Lastly, one must reflect on who is steering this AI revolution.
Sam Altman’s Empire of Promises
Who exactly is Sam Altman?
He serves as CEO of OpenAI, a leading organization advancing frontier AI models that push the boundaries of computational power and sophisticated algorithms. This endeavor carries immense implications for humanity’s future, and Altman holds a pivotal role. But how well do we truly understand him?
To start with, he is neither a scientist nor an engineer. He dropped out of college without earning a degree. While many talented individuals thrive despite not completing college, it would be incorrect to assume his technical knowledge matches that of the scientists and engineers who create the models he promotes.
He is also not an academic; he has no peer-reviewed publications. That is acceptable, as entrepreneurs often dedicate time to building companies rather than conducting research.
Digging deeper, Altman emerges largely as a promoter. In 2005, he founded Loopt, a company that failed to gain momentum and was eventually sold. In 2011, he became part of Y Combinator (YC), a prominent startup accelerator. YC’s founder, Paul Graham, later stated that Altman was ousted due to “YC partners’ mistrust.”
Altman eventually co-founded OpenAI alongside Elon Musk and other investors, including early backer Peter Thiel. OpenAI was originally conceived as a non-profit dedicated to AI research for humanity’s good. However, Altman shifted the company toward becoming a for-profit entity, sparking Musk’s distrust. Musk subsequently sued OpenAI and Altman alleging breach of contract.
Thus, Altman is primarily a promoter, fundraiser, and grand speaker, with a history marked by conflicts and disputed promises. While he’s not the worst figure imaginable, it remains questionable whether he should lead the AI revolution.
On October 8, reports revealed OpenAI’s annual revenue was nearing $50 billion, approximately $20 billion lower than a figure released days earlier. The difference largely stemmed from varying methods used by AI firms to count sales via cloud partners.
Anthropic and OpenAI are racing not only to launch their IPOs but also to outpace regulators, legal challenges, and wary customers to see if these offerings can proceed at all.
About the Author
