Get stats on number of improved forecasts

Get stats on number of improved forecasts

Usage

get_stats_improved_forecasts(
  scores,
  threshold,
  target_model,
  baseline_model,
  metric_to_compare
)

Arguments

scores

tibble of scores for every location, forecast date, and horizon

threshold

numeric indicating fold change for considering a forecast improved or worse relative to baseline, e.g. 1.1

target_model

Name of the target model

baseline_model

Name of the baseline model

metric_to_compare

Metric for which to compute relative scores. One of "wis" or "crps". Not case-sensitive. Passed as the metric_to_compare argument to scoringutils::get_pairwise_comparisons() via forecasttools::summarise_scores_with_baseline().

Value

table of the number of states with improvements, number of overall forecasts with improvements, number that got worse, etc.