Get stats on number of improved forecasts

Usage

get_stats_improved_forecasts(
  scores,
  threshold,
  target_model,
  baseline_model,
  metric_to_compare
)

get_stats_improved_forecasts(
  scores,
  threshold,
  target_model,
  baseline_model,
  metric_to_compare
)

Arguments

scores: tibble of scores for every location, forecast date, and horizon
threshold: numeric indicating fold change for considering a forecast improved or worse relative to baseline, e.g. 1.1
target_model: Name of the target model
baseline_model: Name of the baseline model
metric_to_compare: Metric for which to compute relative scores. One of "wis" or "crps". Not case-sensitive. Passed as the metric_to_compare argument to scoringutils::get_pairwise_comparisons() via forecasttools::summarise_scores_with_baseline().

Value

table of the number of states with improvements, number of overall forecasts with improvements, number that got worse, etc.