The Conventional Choice: EMA
If you've ever built a quantitative signal, you probably reached for an exponentially-weighted moving average (EMA) at some point. EMAs are everywhere — moving average convergence divergence (MACD), RSI smoothing, volatility forecasting, trend following. The defaults in most charting tools assume EMA is the obvious choice.
For Dipsern v1, we did the same thing. The engine computed forward 90-day returns conditional on drawdown bucket, then smoothed those returns with an EMA (decay = 0.95) so the signal would be regime-aware. Intuitive, defensible, what everyone does.
Then we measured prediction error and the choice didn't hold up.
The Experiment
We took the S&P 500 (^GSPC) from 1975-01-01 to today — almost exactly 50 years of daily data, including the 1987 crash, 2000 dot-com bust, 2008 GFC, 2020 COVID flash crash, and 2022 bear market. Enough regimes to stress-test any forecasting method.
For every trading day in the series, we computed two forecasts of the forward 90-day return based on the current drawdown bucket:
- EMA forecast: rolling exponential moving average of forward returns in the current bucket, decay = 0.95
- Median forecast: rolling median of forward returns in the current bucket
Then 90 days later, we measured the actual realized return and computed the absolute prediction error for each method.
Aggregating across all ~12,500 trading days:
| Metric | EMA Forecast | Median Forecast | |-------------------------|--------------|-----------------| | Mean absolute error | 4.82% | 3.91% | | Median absolute error | 3.27% | 2.84% | | 95th percentile error | 14.6% | 11.3% | | Days where method won | 41% | 59% |
The median forecast had lower error 59% of the time. Aggregate error reduction was roughly 19%.
Why Median Wins on Tail-Heavy Data
The intuition is that returns are not symmetric. The S&P's distribution of forward 90-day returns has positive skew at moderate drawdowns (mean-reversion kicks in) and fat left tails during deep drawdowns (continued decline is possible). The EMA, which is just a weighted mean, gets pulled around by the tails.
The 2008 GFC is a clean illustration. At a 35% drawdown in late October 2008, the S&P had its single worst forward 90-day return in modern history (continued decline through early 2009). The EMA for that bucket immediately incorporated that observation and dragged the forecast deeply negative — just as the market was starting to bottom.
The median, by contrast, treated the 2008 extreme as one observation among many. It didn't ignore it (the 2008 episode is still in the dataset), but it didn't let one tail event dominate the central tendency.
A year later, in 2010, the median forecast at -35% drawdowns was still positive and approximately calibrated. The EMA forecast was still depressed by the 2008 tail and underpredicted forward returns for years.
The 2020 COVID Case Study
March 2020 gave us another natural experiment. The S&P dropped 34% from its mid-February peak in 22 trading days. The drawdown buckets that had been thinly populated (-30% to -35%) suddenly got 20+ new observations in rapid succession.
The EMA forecast for that bucket whipsawed: in late March it was sharply negative (incorporating the panic-day forward returns), and by July it was sharply positive (incorporating the V-shape recovery). The EMA never quite settled on a stable central estimate.
The median forecast for the same bucket was more stable. New observations entered the median list and influenced the central value, but no single observation could pull the median far. The signal was usable through the entire 2020 episode.
When EMA Wins
In fairness, EMA does win in some regimes. Specifically: when there's a structural regime change such that the long-run base rate is no longer relevant, and only recent observations matter. Small thematic ETFs that have been alive for 5-10 years and undergone a single regime shift (cannabis ETFs after legalization waves, ARK funds after the 2021 peak) are examples — the median ends up averaging across two distinct populations.
For these assets, the EMA's recency bias is a feature. For broad-market indices and assets with stable long-run dynamics, it's a bug.
What We Did
Dipsern's primary signal is now the rolling median. The EMA is still computed and shown in the segments table as a secondary reference, but the headline forecast in the app and the API is the median.
Empirically, this change reduced prediction error across 14 of 17 asset categories. The places where EMA still wins are thematic ETFs and recently-listed assets with structural regime breaks — for those we surface the EMA prominently in the comparison table.
Implications for Your Own Models
If you build any kind of forecasting model on financial returns, the median-vs-mean question is worth running explicitly on your data. The defaults in most quantitative libraries assume Gaussian-ish behavior, but returns rarely are. Robust statistics (median, trimmed mean, MAD) often outperform their non-robust counterparts when the underlying distribution has fat tails — which, for financial returns, it almost always does.
See It Live
The S&P 500 page at /signals/^GSPC shows the current drawdown bucket and historical base rates. The full app shows both the median forecast and the EMA forecast side-by-side, so you can see when they diverge and why.
For the engine source, see api/arad_engine.py in the repo — both methods are computed in a single O(n) pass through the data, with no look-ahead bias.