|
Calibrating Probability Density Forecasts with Multi-objective Search |
Carney, Michael; Cunningham, Padraig
|
|
|
|
TCD-CS-2006-07 In this paper, we show that the optimization of density forecasting models for regression in machine learning can be formulated as a
multi-objective problem.We describe the two objectives of sharpness and
calibration and suggest suitable scoring
metrics for both.We use the popular negative log-likelihood as a measure of sharpness and the probability
integral transform as a measure of calibration. We show how optimization on negative log-likelihood alone often results in sub-optimal models.
To solve this problem we introduce a multi-objective evolutionary optimization framework that can produce better density forecasts from a
prediction users perspective. Our experiments show improvements over
state-of-the-art approaches.
|
|
Keyword(s):
|
Computer Science |
Publication Date:
|
2006 |
|
Type:
|
Report |
|
Peer-Reviewed:
|
Unknown |
|
Language(s):
|
English |
|
Institution:
|
Trinity College Dublin |
|
Citation(s):
|
Carney, Michael; Cunningham, Padraig. 'Calibrating Probability Density Forecasts with Multi-objective Search'. - Dublin, Trinity College Dublin, Department of Computer Science, TCD-CS-2006-07, 2006, pp12 |
|
Publisher(s):
|
Trinity College Dublin, Department of Computer Science |
|
File Format(s):
|
application/pdf |
|
First Indexed:
2014-05-13 05:30:51 Last Updated:
2017-04-26 12:24:22 |
