AI in Hedge Funds – Can They Beat the Markets?
May 30, 2024
Artificial intelligence (AI) and generative AI (Gen AI) have grabbed global mindshare, headlining innovations across the entire field of data capture from multiple formats and their structuring – making them ready-to-use tools that can drive real-time analytics and actionable insights. That AI has been used by hedge funds for several years too is a known fact. Past data from specific hedge funds reveal that AI-led hedge funds bested overall hedge funds half the time on annual returns in seven of the last 14 years from 2010 through 2023. The industry may have to wait some more time before Gen AI-led data capture delivers better and more consistent returns. However, recent examples1 of how these funds are using AI and satellite feeds to predict the output from a US gas plant might have lessons for the future.
What are the Use Cases for AI in Hedge Funds?
Before delving further into the future, let us pause to understand the specific areas where AI is making an impact and can do so in the future. Before getting into specifics, it is worth noting that each sector within the financial industry uses AI for specific needs, such as the insurance industry uses it to detect fraud. In wealth management, AI has found use in ensuring regulatory requirements, client segmentation, and chatbots to deal with customer queries. Hedge funds have used modeling in the investment processes with machine learning algorithms to improve decision-making. The value chain gets divided, allowing for a systemic application of models to achieve a specific outcome. The steps are detailed in this article2.
Some of the areas where AI can play a definite role and others where it has played a supporting one are listed below:
- Big data analytics – Hedge funds can now crunch vast volumes of financial data based on real-time information. This results in better-informed decisions as AI-based trading algorithms can identify signals and patterns faster from earnings reports, financial statements, news, social media, etc.
- Quantitative trading – These algorithms help hedge funds quickly test, refine, and implement trading strategies and do so at speed and high precision. They can process massive datasets within seconds and execute high-frequency trades efficiently.
- Risk management – These AI models provide continuous portfolio monitoring to the hedge funds by assessing positions and adjusting them in real time for loss mitigation. Also, machine learning and deep learning help them predict asset prices, market moves, and volatility trends more accurately.
- Alternative data analysis – Hedge funds now use alternative data such as satellite imagery, credit card transactions, web traffic and searches, mobile apps, etc. to gather insights that help them gain a competitive advantage.
- High-frequency trading – Machine learning algorithms analyze data, adapt to new information, and exploit market inefficiencies faster than humans. Companies have built tools that do all this in seconds and help the strategist make instant decisions in a fast-moving market environment.
- Portfolio optimization – AI has emerged as a key for hedge funds to optimize their portfolios across asset classes, positions, and markets while evaluating risks, returns, and existing market regimes across multiple geographies in an instant. The ability to analyze millions of combinations helps optimize asset allocation and portfolio mix.
Not About Big Data, but Quality Data
The key to delivering these outcomes rests squarely on capturing large amounts of data, from multiple sources and structuring them into a single schema. Gen AI tools, developed over the past 12 months, have made this possible and ensured close to zero processing errors. When complex algorithms parse through these datasets, they support the execution of high-frequency trades with a much higher level of efficiency compared to humans. However, there’s more to be done in this field as Michael Kharitonov, a member of the CERN nuclear research lab in Geneva says. “Finance has its own unique challenges, but over time they can be overcome,” he adds. Machines are confused by the noisy markets and often caught off guard by fickle trends. A Bloomberg report3 notes that a Eurekahedge index of 12 funds using AI actually trailed its broader hedge fund index by about 14 percentage points over the past five years.
Source: Bloomberg Law3
AI is a Must-Have – Now How Far Can It Go?
At first glance, these numbers may appear disappointing to anyone watching Gen AI making a bigger impact in healthcare, pharma, insurance, etc. Analysts note that AI and machine learning are already competing with traditional fund managers and advocates for using Gen AI say that their effort is not to seek market-bashing returns but to have a slight edge at best. Which is why some are betting big on the next wave of machine learning where massive datasets are being used to train systems to perform specific tasks. Collecting data from all possible sources, checking their quality, and structuring them for the future is getting increasingly automated. And once, these data sets are available, generating real-time analytics and actionable insights are just a step away. This means that future portfolio managers could be assisted by algorithms but would still have to use their native intelligence to make the right calls at the right time.
The Alternative Investment Management Association (AIMA) of the United States recently published a report4 on how Gen AI was impacting hedge funds based on surveying 157 hedge fund managers globally, It notes that 86% of the respondents said their teams have access to Gen AI tools at work, carrying out research tasks and supporting coding work. However, fund managers with over $1 billion AUM appear more bullish around Gen AI use cases in the portfolio management potential of the technology. They were also ready to invest in developing in-house Gen AI tools and training their staff to use them. However, there was also a consensus around adoption challenges with data security being a major concern.
Can AI-Led Funds Really Beat the Market?
While the adoption of AI in the hedge fund industry is moving ahead robustly, the question remains whether AI-led hedge funds could beat the markets any time soon. And the reasons are obvious as AI is a data-driven scientific approach that seeks a structure to build on. In the absence of one, it could become a “garbage-in, garbage-out” story as AI models often tend to overfit when not applied carefully. This is where quality of data comes into play and traders would need to ensure large and meaningful datasets to reap the best outcomes from the predictive power of artificial intelligence. More and more hedge funds now employ data scientists who are experts at quantitative methods and are exposed to actual trading. Together, the funds can create AI models based on the right datasets to increase predictive power, which when combined with automation and quick execution can bring higher returns for investors in the future.
The fact remains that even hedge funds have seen a shift towards maximizing returns from the times in the past when they were meant to “hedge” the risks by alternating between short and long-term strategies so that investors do not lose money. When investors are ready to take higher risks, the funds need to find smarter ways to mitigate them in their strategies through technology that can notice and interpret even the smallest of risk signals. So, the future of AI would focus mainly on two areas – solving the black box challenge and then training humans to interact better with artificial intelligence.