Forecasting stock market volatility - HAR, GARCH, machine learning and the VIX compared

Quantooo Lab · 2026-10-22

Predicting whether the market goes up next month is close to impossible. Predicting how bumpy it will be is a different matter, because calm and stormy periods tend to last. This is where machine learning has a real, if modest, role.

Why is volatility easier to forecast than returns?

Because volatility clusters. A volatile week is usually followed by another volatile week, and a calm month by another calm month. That persistence gives a forecasting model something stable to work with. Returns have no comparable memory: if they did, traders would quickly use it up.

Which models did we compare?

We forecast the S&P 500's volatility over the next 20 trading days with:

• HAR: the standard benchmark, a simple regression on volatility over the last day, week and month.
• E1: a LightGBM machine learning model using only the index's own price history, from the research paper this site is based on.
• GARCH(1,1): a classic 1986 model of volatility clustering.
• Chronos-Bolt: a pretrained time-series foundation model, used zero-shot with no extra training.
• VIX-implied: the market's own forecast, derived from option prices.

We score forecasts with QLIKE, a loss function designed for volatility. Lower is better.

What were the results?

On the holdout, E1 beat HAR (p = 0.046), which was the result the paper had registered in advance. But E1 was worse than HAR in development, and the VIX and GARCH did better than both. GARCH and Chronos were exploratory: we added them after the main test, so their results are leads to confirm live, not findings.

What does this mean for machine learning in finance?

It means the useful question is not "can ML predict markets" but "where does it add something over simple, well-understood models". Here the summary is: a little, sometimes, against one benchmark. The market's own forecast in option prices had the lowest loss in development, which is what you would expect if option traders already use most of the available information, and on the holdout only the exploratory GARCH model beat it.

How is volatility used on this site?

The stability part of each stock's score is built from a volatility forecast: how calm a stock is expected to be next month compared with its own normal. A risk-managed S&P 500 portfolio also sizes its position by the E1 forecast, aiming for steady volatility. You can follow both on the research page, where E1, HAR, GARCH and Chronos are re-scored every night as new data arrives.

Can you forecast volatility for a single stock the same way?

Yes, the same ideas apply, though single stocks have more jumps from earnings and news, which makes forecasts noisier. Earnings dates in particular create predictable spikes that a model should know about.

Open questions for students and researchers

Questions without a single right answer. Pick one and share your reasoning in the discussion below.

1. The VIX-implied forecast beat every model in development, and only GARCH beat it on the holdout. Is that because option prices contain information the models lack, or because QLIKE favours forecasts with a particular bias? How would you check?
2. E1 beat HAR on the holdout but lost in development. How should you weigh conflicting results across periods?
3. Volatility forecasts feed position sizing. Design an evaluation that measures economic value, not just forecast loss.

Debate: three perspectives

An illustrative debate written by the author to lay out the main viewpoints. The three voices are roles, not real people. Your view is welcome in the discussion below.

The Sceptic: Volatility forecasting is a solved problem: HAR, GARCH or the VIX are enough, and machine learning adds complexity for little gain.

The Practitioner: Small improvements in volatility forecasts compound when they size every position, every day. A few percent better can matter more than a big improvement in a rarely used signal.

The Academic: The comparison should be judged by the right loss function and by economic value. Forecast evaluation is a field of its own, and the choice of loss can decide the winner.

Which view is closest to yours, and what evidence would change your mind?