Unlock Editor’s Digest for free
FT editor Roula Khalaf has chosen her favorite stories in this weekly newsletter.
The author is a former head of global research at Morgan Stanley and a former head of research, data and analytics at UBS.
The late Byron Wien, a prominent market strategist in the 1990s, defined the best research as recommendations that are not based on consensus that turn out to be correct. Can AI pass the Vienna test for worthwhile research and eliminate the need for analysts, or at least increase the probability that recommendations are correct by 50% or more?
It is important to understand that most analyst reports are specialized in interpreting financial statements and news. This is about facilitating the work of investors. Here, modern large-scale language models simplify or replace this analysis functionality.
Second, considerable effort is spent predicting revenue. Given that there are good years after good years and vice versa, and profits tend to follow a pattern most of the time, it’s logical that a rules-based engine would work. And because the model doesn’t have to “get heard” by standing out from the crowd with outlandish predictions, its low bias and noise can outperform most analyst estimates during periods of limited uncertainty. There is a gender. Scholars wrote about this decades ago, but the practice never caught on in mainstream research. To scale, we needed to build a sufficient amount of statistics and neural networks. It’s rare in an analyst’s skill set.
Change is afoot. Academics at the University of Chicago trained a large language model to estimate the variance of returns. These exceeded the median estimate when compared to the median analyst estimate. This result is interesting because LLM does not have the edge of so-called numerical reasoning, i.e., narrowly trained algorithms, to generate insights by understanding the narrative of earnings announcements. And when they are told to mirror the steps of senior analysts, their forecasts improve. If possible, like a good junior.
But analysts have struggled to quantify the risks. Part of the problem is that investors are so obsessed with ensuring a win that they force analysts to express confidence even when there is no chance of winning. A shortcut is to adjust the estimate or multiple slightly up or down. At best, an LLM can help by taking into account a set of similar situations.
By manipulating the “temperature” of the model, which is a proxy for the randomness of the results, it is possible to statistically approximate the risk and return bands. Additionally, you can request that the model provide an estimate of its confidence in its predictions. Perhaps counterintuitive, but this is the wrong question for most people to ask. We tend to be overconfident in our ability to predict the future. And it’s not uncommon for our efforts to escalate when our predictions start to go wrong. As a practical matter, if a company creates a ‘conviction list’, it may want to think twice before blindly following that advice.
But before we throw the proverbial analyst in the water, we must recognize that AI has significant limitations. The model tries to give the most plausible answer, so you shouldn’t expect it to discover the next Nvidia or predict another global financial crisis. These stocks and events go against any trend. LLM is also unable to suggest anything “worth considering” on the earnings call, as management appears to avoid discussing value-related information. Furthermore, fluctuations in the dollar value due to political disputes cannot be predicted. Markets are volatile and opinions about the market are constantly changing. We need intuition and the flexibility to incorporate new information into our views. These are the qualities of a top analyst.
Could AI enhance our intuition? Maybe. Adventurous researchers can use the much-maligned illusion of LLM to their advantage by increasing the randomness of the model’s responses. This will give you lots of ideas to check out. Or construct geopolitical “what if” scenarios, drawing alternative lessons from more history than expert groups can provide.
Early research suggests the potential of both approaches. This is a good thing, because anyone who has sat on an investment committee understands how difficult it is to bring alternative perspectives to the table. However, please be careful. It is unlikely that any “sparkle of genius” will emerge, and there will be a lot of nonsense to eliminate.
Does it make sense to have a proper research department or follow a star analyst? That is. But we must assume that some processes can be automated, some can be enhanced, and that strategic intuition is like a needle in a haystack. It is difficult to find a non-consensus recommendation that turns out to be correct. And there are serendipities in searches.
