This article is from:Kalshi Research
Compiled by | Odaily Planet Daily (@OdailyChina); Translator | Azuma (@azuma_eth)
Editor's Note: Leading prediction market platform Kalshi announced the launch of a new research report series, Kalshi Research, yesterday. It aims to provide Kalshi's internal data to scholars and researchers interested in topics related to prediction markets. The inaugural report has been published, originally titled "Kalshi Outperforms Wall Street in Predicting Inflation" (Beyond Consensus: Prediction Markets and the Forecasting of Inflation Shocks).
Below is the content of the original report, compiled by Odaily Planet Daily.
Overview
Typically, in the week leading up to the release of important economic statistics, analysts and senior economists from large financial institutions provide their estimates of the expected figures. These forecasts are aggregated to form what is known as the "consensus expectation," which is widely regarded as a crucial reference for gaining insights into market changes and adjusting portfolio positioning.
In this research report, we compare the performance of the consensus expectation with the implied pricing from Kalshi's prediction markets (sometimes referred to as "market forecast" below) in predicting the actual value of the same core macroeconomic indicator—the year-over-year headline inflation rate (YOY CPI).
Key Highlights
- Overall Superior Accuracy: Across all market environments (including normal and shock periods), Kalshi's predictions had a Mean Absolute Error (MAE) that was 40.1% lower than the consensus expectation.
- "Shock Alpha": During significant shocks (greater than 0.2 percentage points), Kalshi's predictions had an MAE 50% lower than the consensus expectation in the one-week-ahead forecast window; this advantage expanded to 60% on the day before the data release. For moderate shocks (between 0.1 and 0.2 percentage points), Kalshi's predictions also had an MAE 50% lower than the consensus expectation one week ahead, which widened to 56.2% on the day before the release.
- Predictive Signal: When the deviation between the market forecast and the consensus expectation exceeded 0.1 percentage points, the probability of a shock occurring was approximately 81.2%, rising to about 82.4% on the day before the data release. In cases where the market forecast differed from the consensus, the market forecast was more accurate in 75% of instances.
Background
Macroeconomic forecasters face an inherent challenge: the most critical times for forecasting—periods of market disorder, policy shifts, and structural breaks—are precisely when historical models are most likely to fail. Financial market participants typically release consensus forecasts for key economic data days in advance, aggregating expert opinions into market expectations. However, while valuable, these consensus views often share similar methodological approaches and information sources.
For institutional investors, risk managers, and policymakers, the stakes of forecasting accuracy are asymmetric. During uncontroversial periods, slightly better predictions offer limited value; but during periods of market turmoil—when volatility spikes, correlations break down, or historical relationships fail—superior accuracy can yield significant alpha returns and limit drawdowns.
Therefore, understanding how parameters behave during volatile market periods is crucial. We focus on a key macroeconomic indicator—the year-over-year headline inflation rate (YOY CPI)—a core reference for future interest rate decisions and an important signal of economic health.
We compared and evaluated forecasting accuracy across multiple time windows before the official data release. Our core finding is that "shock alpha" indeed exists—during tail events, market-based predictions can achieve additional forecasting precision compared to the consensus benchmark. This outperformance is not merely of academic interest; it significantly enhances signal quality precisely when forecasting errors carry the highest economic costs. In this context, the crucial question is not whether prediction markets are "always right," but whether they provide a differentiated signal worthy of inclusion in traditional decision-making frameworks.
Methodology
Data
We analyzed the daily implied predictions from traders on the Kalshi platform at three time points: one week before the data release (matching the timing of consensus expectation releases), one day before, and the morning of the release. Each market used was (or had been) a real, tradable, active market, reflecting real-money positions at varying liquidity levels. For the consensus, we collected institution-level YoY CPI consensus forecasts, typically published about a week before the official U.S. Bureau of Labor Statistics data release.
The sample period spans from February 2023 to mid-2025, covering over 25 monthly CPI release cycles across various macroeconomic environments.
Shock Classification
We categorized events into three types based on the "surprise magnitude" relative to historical levels. A "shock" was defined as the absolute difference between the consensus expectation and the actual published data:
- Normal Events: YOY CPI forecast error below 0.1 percentage points;
- Moderate Shocks: YOY CPI forecast error between 0.1 and 0.2 percentage points;
- Major Shocks: YOY CPI forecast error exceeding 0.2 percentage points.
This classification allows us to examine whether predictive advantages vary systematically with the difficulty of the forecast.
Performance Metrics
To assess forecasting performance, we employed the following metrics:
- Mean Absolute Error (MAE): The primary accuracy metric, calculated as the average of the absolute differences between predicted and actual values.
- Win Rate: When the difference between the consensus expectation and the market forecast reached or exceeded 0.1 percentage points (rounded to one decimal place), we recorded which prediction was closer to the final actual result.
- Forecast Horizon Analysis: We tracked how the accuracy of market valuations evolved from one week before the release to the release day, revealing the value of continuously incorporating information.
Results: CPI Forecasting Performance
Overall Superior Accuracy
Across all market environments, the market-based CPI forecasts had a Mean Absolute Error (MAE) that was 40.1% lower than the consensus forecasts. Across all time horizons, the MAE for market-based CPI forecasts was lower than the consensus by 40.1% (one week ahead) to 42.3% (one day ahead).
Furthermore, in cases where the consensus expectation and the market-implied value diverged, Kalshi's market-based forecasts demonstrated a statistically significant win rate, ranging from 75.0% one week ahead to 81.2% on release day. Including ties with the consensus (to one decimal place), the market-based forecast performed equally well or better than the consensus in approximately 85% of cases one week ahead.
Such a high directional accuracy rate indicates that when market forecasts diverge from the consensus expectation, the divergence itself carries significant informational value regarding the likelihood of a shock event.
"Shock Alpha" Exists
The difference in forecasting accuracy was particularly pronounced during shock events. During moderate shock events, the MAE of market forecasts was 50% lower than the consensus expectation at the same release time; this advantage expanded to 56.2% or more one day before the data release. During major shock events, the MAE of market forecasts was also 50% lower than the consensus at the same time, reaching 60% or more one day before release. In contrast, during normal, non-shock environments, market forecasts and consensus expectations performed roughly equally well.
Although the sample size for shock events is small (reasonable in a world where shocks are inherently highly unpredictable), the overall pattern is clear: the information aggregation advantage of markets is most valuable precisely when the forecasting environment is most challenging.
However, more importantly, it's not just that Kalshi's predictions perform better during shock periods, but also that the divergence between the market forecast and the consensus expectation can itself be a signal of an impending shock. In cases of divergence, the market forecast's win rate against the consensus expectation was 75% (within comparable time windows). Furthermore, threshold analysis indicated that when the market deviation from the consensus exceeded 0.1 percentage points, the probability of predicting a shock was approximately 81.2%, rising to about 84.2% on the day before the data release.
This practically significant difference suggests that prediction markets can serve not only as a competitive forecasting tool alongside consensus expectations but also as a "meta-signal" regarding forecast uncertainty, transforming market-consensus divergence into a quantifiable early warning indicator for potential unexpected outcomes.
Further Discussion
An obvious question follows: Why do market forecasts outperform consensus forecasts during shocks? We propose three complementary mechanisms to explain this phenomenon.
Market Participant Heterogeneity and "Wisdom of the Crowd"
Traditional consensus expectations, while aggregating views from multiple institutions, often share similar methodological assumptions and information sources. Econometric models, Wall Street research reports, and government data releases form a highly overlapping common knowledge base.
In contrast, prediction markets aggregate positions held by participants with diverse information bases: including proprietary models, industry-level insights, alternative data sources, and experience-based intuition. This participant diversity has a solid theoretical foundation in the "wisdom of crowds" theory. This theory suggests that when participants possess relevant information and their prediction errors are not perfectly correlated, aggregating independent predictions from diverse sources often yields a superior estimate.
The value of this informational diversity is particularly pronounced during "state shifts" in the macro environment—individuals with scattered, local information interact in the market, combining their informational fragments to form a collective signal.
Differences in Participant Incentive Structures
Institution-level consensus forecasters often operate within complex organizational and reputational systems that systematically deviate from the goal of "purely pursuing predictive accuracy." The career risks faced by professional forecasters create an asymmetric payoff structure—large forecasting errors incur significant reputational costs, while even extreme accuracy, especially achieved by deviating substantially from peer consensus, may not yield proportional career rewards.
This asymmetry induces "herding behavior," where forecasters tend to cluster their predictions near the consensus value, even if their private information or model outputs suggest different results. The reason is that in a professional context, the cost of "being wrong alone" often outweighs the benefits of "being right alone."
In stark contrast, the incentive structure faced by prediction market participants directly aligns forecasting accuracy with economic outcomes—accurate predictions mean profits, incorrect predictions mean losses. In this system, reputational factors are almost non-existent; the only cost of deviating from the market consensus is economic loss, solely dependent on the prediction's correctness. This structure imposes stronger selection pressure for predictive accuracy—participants who can identify consensus forecast errors systematically accumulate capital and increase their influence in the market through larger position sizes;而那些 mechanically following the consensus suffer continuous losses when the consensus proves wrong.
During periods of significantly heightened uncertainty, when the career cost for institutional forecasters to deviate from the expert consensus is at its peak, this divergence in incentive structures is often most pronounced and economically most important.
Information Aggregation Efficiency
A noteworthy empirical fact is: even one week before the data release—a time point matching the typical release window of consensus expectations—market forecasts still exhibit significant accuracy advantages. This suggests that the market advantage does not stem solely from the often-cited "information speed advantage" of prediction market participants.
Instead, market forecasts may more efficiently aggregate informational fragments that are too dispersed, too industry-specific, or too nebulous to be formally incorporated into traditional econometric forecasting frameworks. The relative advantage of prediction markets may lie not in earlier access to public information, but in their ability to synthesize heterogeneous information more effectively within the same time frame—a task that survey-based consensus mechanisms, even with the same time window, often struggle to perform efficiently.
Limitations and Caveats
Our findings require an important qualification. Since the overall sample covers only about 30 months, and major shock events are by definition rare, statistical power for larger tail events remains limited. A longer time series would enhance future inferential capabilities, although the current results strongly suggest the superiority and differentiated signal of market forecasts.
Conclusion
We document systematic and economically significant outperformance of prediction markets relative to expert consensus expectations, particularly during shock periods where forecasting accuracy is most critical. Market-based CPI forecasts exhibited approximately 40% lower error overall, with error reduction reaching around 60% during periods of major structural change.
Based on these findings, several future research directions become particularly important: First, investigating whether "shock alpha" events can be predicted themselves using volatility and forecast divergence indicators, across a larger sample size and multiple macroeconomic indicators; Second, determining the liquidity threshold above which prediction markets can stably outperform traditional forecasting methods; Third, exploring the relationship between prediction market forecast values and those implied by high-frequency trading financial instruments.
In an environment where consensus forecasts heavily rely on correlated model assumptions and shared information sets, prediction markets offer an alternative information aggregation mechanism, capable of capturing state switches earlier and processing heterogeneous information more efficiently. For entities needing to make decisions in an economic environment characterized by rising structural uncertainty and tail event frequency, "shock alpha" may represent not just an incremental improvement in forecasting ability, but a fundamental component of a robust risk management infrastructure.
