Model selection case studies

Model selection case studies by Aki Vehtari.

Cross-validation case studies

• When cross-validation is not needed
  • Simple model we trust - betablockers
• When cross-validation is useful
  • We don’t trust the model - roaches
  • Complex model with posterior dependencies - collinear
  • Large number of models - diabetes
• loo package vignettes
• Bayesian Stacking and Pseudo-BMA weights using the loo package
• Leave-one-out cross-validation for non-factorizable models
• Approximate leave-future-out cross-validation for time series models
• Using Leave-one-out cross-validation for large data
• Avoiding model refits in leave-one-out cross-validation with moment matching
• rats K-fold-CV
• LOO-R^2

Projection predictive model selection (projpred) case studies

• mesquite collinearity
• bodyfat regression
• diabetes logistic regression
• candy random data vs original data
• winequality-red
• See also projpred quick start vignette