← zurück

Preis

—

Anreise

—

Zugang

—

Sozial

—

Speaker: Eugene Berta, INRIA Paris Abstract: Proper scoring rules are fundamental for evaluating the quality of probabilistic forecasts. A canonical property of these scores is their decomposition into calibration and refinement errors. However, reliably estimating these two components is notoriously difficult in practice. Standard approaches are often tied to the specific setting of binary classification, and suffer from well-documented finite-sample pathologies that can lead to systematically over- or under-estimating calibration error. In this talk, I will present a variational formulation of the calibration-refinement decomposition that holds for any proper scoring rule and any forecasting space. This formulation gives a natural interpretation: refinement error is the best achievable risk obtained by optimally post-processing the forecaster's predictions, while calibration error measures how much risk is reduced by this optimal post-processing. Beyond the theory, this framework motivates a robust, cross-validation-based estimation strategy. This recipe applies across proper scores and forecast spaces, naturally avoids the estimation pathologies of prior methods, and yields lower bounds on calibration error.

• Wissen