Large language models for quants

The story of large language models started with forecasts but what else can they do for quants? Saeed Amen shares the innovative projects where LLMs make an impact and explores their future in quant finance.

Historically, you would need task specific natural language models. For example, you might have a model for sentiment analytics, another for detecting entities. Large language models (LLMs) promise to be much more general purpose, and theoretically they can solve a much wider array of problems. We can use them to do traditional NLP (natural language processing) tasks like sentiment analysis (see Can ChatGPT Decipher FedSpeak – Sep 2023). However, we can also query them and get them to do literally anything. Ask it to write poetry and it does that. For many tasks, which seem overly creative, it appears that LLMs will give us an answer.

What about more specific problems related to finance, what can LLMs do there? Ask it to forecast inflation and it will give you some answers. On the last question, researchers from the St Louis Fed wrote a paper on it (see Artificial Intelligence and Inflation Forecasts – Jul 2023). The researchers find such an approach works well in a backtest and appears to do better than a benchmark of the Philly Fed Survey. However, I’d argue we need to be careful when asking an LLM to do such a forecast task. One of the most difficult things when doing time series forecasting, is to ensure that we have no look ahead bias, when we backtest it. In other words, the model shouldn’t know the future. In a real trading environment, unfortunately, we don’t have data from the future.

The difficulty with LLMs is that it is very difficult to know if data from the future has filtered into the model, in particular, because it isn’t clear how to train them on a rolling basis. Indeed, this can also be a problem historically with NLP models, if we are trying to use them to solve problems in a point-in-time way (even with our example with sentiment analysis).

Alexander Sokol (CompatibL) also drew the parallel between LLMs and Monte Carlo. LLMs were numerical algorithms for simulated token probabilities. Just as with Monte Carlo, where the output can change between runs, this was also something that LLMs did. Ultimately LLMs compressed a large amount of data, and “interpolated” between.

Just because LLMs might not necessarily be the best choice for this particular task of forecasting, given the problem with look ahead bias, there are many other potential use cases for quants. At the last QuantMinds International conference, there were many talks which used LLMs. One particular use case which stuck in my memory was the presentation from Vivek Anand and Ganchi Zhang (Deutsche Bank). They explained how LLMs could be useful for identifying relevant keywords for particular topics, and furthermore LLMs could explain why particular keywords could be picked.

More broadly, building on the keyword example, quants can use LLMs in the development process of their models. One obvious area that has been highlighted is in helping to write code. The traditional workflow of a quant might involve writing code, and then going online to Google searches which invariably result in falling upon a page from a website like StackOverflow. Models such as GitHub Copilot have been trained on large amounts of codes and they can be used to answer coding questions, which traditionally might have been answered in a less direct way by posts on StackOverflow. Of course, the code generated by LLMs aren’t always perfect, but they can nevertheless be a starting point. LLMs can also help with the tedium of writing documentation and generating boilerplate code which is time consuming. You could argue though that for more difficult coding, where the solutions are unlikely to have been open sourced, then LLMs might face more challenges.

Are these the only use cases for quant? I doubt it. Indeed, I’m sure in the years ahead, quants will come up with more novel ways to take advantage of the knowledge that LLMs possess. The key will be to ask the right questions!

See how quants use LLMs today at the LLMs & Advanced ML Summit, part of QuantMinds International.