Bayesian Data Analysis Global South (GSU) 2021
Lecturer: Aki Vehtari
All the course material is available in a git repo and via these pages are for easier navigation. All the material can be used in other courses. Text and videos licensed under CC-BY-NC 4.0. Code licensed under BSD-3.
There has been some dropouts and the registration has been re-opened here.
We have some volunteer TAs already, but a a few more would be great. All TAs will get a personal certificate from the lecturer Aki Vehtari if they actively participate helping students, answering questions, and possibly organize some TA sessions. We assume to have enough TAs that no-one needs to take part every week of the course and you can drop out if other obligations require so. The lecturer will support TAs.
To register as volunteer TA fill in your information (email, country, prerequisites check, a brief comment on justification why you think you can be a TA) here.
The electronic version of the course book Bayesian Data Analysis, 3rd ed, by by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin is available for non-commercial purposes. Hard copies are available from the publisher and many book stores. See also home page for the book, errata for the book, and chapter notes.
This course has been designed so that there is strong emphasis in computational aspects of Bayesian data analysis and using the latest computational tools.
If you find BDA3 too difficult to start with, I recommend
Assignments (67%) and a project work with presentation (33%). Minimum of 50% of points must be obtained from both the assignments and project work. But as in this course there is no formal certificate, the assignment scores are just for your own self-evaluation. The biggest benefit from the course is the support and feedback from other students and volunteer TAs.
We use peergrade.io for providing peer feedback for the assignments and the project work. See more information at Assignments.
The course consists of 12 blocks from March to May 2021. The blocks don’t match exactly specific weeks. For example, it’s good start reading the material for the next block while making the assignment for one block. There are 9 assignments and a project work with presentation, and thus the assignments are not in one-to-one correspondence with the blocks. The schedule below lists the blocks and how they connect to the topics, book chapters and assignments.
Here is an overview of the schedule. Scroll down the page to see detailed instructions for each block. Remember that blocks are overlapping so that when you are working on assignment for one block, you should start watching videos and reading text for the next block.
|Block||Readings||Lectures||Assignment||Assignment due date|
|1. Introduction||BDA3 Chapter 1||Computational probabilistic modeling,
Introduction to uncertainty and modelling,
Introduction to the course contents
|2. Basics of Bayesian inference||BDA3 Chapter 1,
BDA3 Chapter 2
Extra explanations 2,
|3. Multidimensional posterior||BDA3 Chapter 3||Lecture 3||Assignment 3,
|4. Monte Carlo||BDA3 Chapter 10||Lecture 4.1,
|5. Markov chain Monte Carlo||BDA3 Chapter 11||Lecture 5.1,
|6. Stan, HMC, PPL||BDA3 Chapter 12 + extra material on Stan||Lecture 6.1,
|7. Hierarchical models and exchangeability||BDA3 Chapter 5||Lecture 7.1,
|8. Model checking & cross-validation||BDA3 Chapter 6, BDA3 Chapter 7, Visualization in Bayesian workflow, Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC||Lecture 8.1,
|Start project work|
|9. Model comparison and selection||BDA3 Chapter 7 (not 7.2 and 7.3),
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
|10. Decision analysis||BDA3 Chapter 9||Lecture 10.1||Assignment 9,
|11. Normal approximation, frequency properties||BDA3 Chapter 4||Lecture 11.1,
|12. Extended topics||Optional: BDA3 Chapter 8,
BDA3 Chapter 14-18,
BDA3 Chapter 21
|13. Project feedback||Project feedback||28.05.|
Course practicalities, material, assignments, project work, peergrading, TA sessions, prerequisites, chat, etc.
BDA3 Chapters 1+2, basics of Bayesian inference, observation model, likelihood, posterior and binomial model, predictive distribution and benefit of integration, priors and prior information, and one parameter normal model.
Multiparameter models, joint, marginal and conditional distribution, normal model, bioassay example, grid sampling and grid evaluation. BDA3 Ch 3.
Numerical issues, Monte Carlo, how many simulation draws are needed, how many digits to report, direct simulation, curse of dimensionality, rejection sampling, and importance sampling. BDA3 Ch 10.
Markov chain Monte Carlo, Gibbs sampling, Metropolis algorithm, warm-up, convergence diagnostics, R-hat, and effective sample size. BDA3 Ch 11.
HMC, NUTS, dynamic HMC and HMC specific convergence diagnostics, probabilistic programming and Stan. BDA3 Ch 12 + extra material
Hierarchical models and exchangeability. BDA3 Ch 5.
Model checking and cross-validation.
PSIS-LOO, K-fold-CV, model comparison and selection. Extra lecture on variable selection with projection predictive variable selection.
Decision analysis. BDA3 Ch 9.
Normal approximation (Laplace approximation), and large sample theory and counter examples. BDA3 Ch 4.
Frequency evaluation of Bayesian methods, hypothesis testing and variable selection. Overview of modeling data collection, BDA3 Ch 8, linear models, BDA Ch 14-18, lasso, horseshoe and Gaussian processes, BDA3 Ch 21.
We strongly recommend using R in the course as there are more packages for Stan and statistical analysis in R. If you are already fluent in Python, but not in R, then using Python may be easier, but it can still be more useful to learn also R. Unless you are already experienced and have figured out your preferred way to work with R, we recommend
See FAQ for frequently asked questions about R problems in this course. The demo codes provide useful starting points for all the assignments.