Aalto 2021 course will be completely online (pre-recorded lectures, live zoom QA sessions, course chat, online TA sessions, assignments and project submitted online, project presentation online). The registration for the course lectures will be used to estimate the need for the online resources. If you are unable to register for the course at the moment in the Sisu, there is no need to email the lecturer. You can start taking the course and register before the end of the course. Sisu shows rooms on campus for the computer exercises, but all the computer exercises and TA sessions are online. You can choose which TA session to join each week separately, without a need to register for those sessions.

All the course material is available in a git repo (and these pages are for easier navigation). All the material can be used in other courses. Text and videos licensed under CC-BY-NC 4.0. Code licensed under BSD-3.

The material will be updated during the course. Exercise instructions and slides will be updated at latest on Monday of the corresponding week. The updated material will appear on the web pages, or you can clone the repo and pull before checking new material. If you don’t want to learn git, you can download the latest zip file.

Book: BDA3

The electronic version of the course book Bayesian Data Analysis, 3rd ed, by by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin is available for non-commercial purposes. Hard copies are available from the publisher and many book stores. Aalto library has also copies. See also home page for the book, errata for the book, and chapter notes.

Prerequisites

This course has been designed so that there is strong emphasis in computational aspects of Bayesian data analysis and using the latest computational tools.

If you find BDA3 too difficult to start with, I recommend

Communication channels

  • MyCourses is used for important announcements, linking to Zulip and Peergrade, and some questionnaires.
  • The primary communication channel is the Zulip course chat (link in MyCourses)
    • Don’t ask via email or direct messages. By asking via common streams in the course chat, more eyes will see your question, it will get answered faster and it’s likely that other students benefit from the answer.
    • Login with Aalto account to the Zulip course chat
    • If you have any questions, please ask in the public streams and get answers from course staff or other students (active students helping others will get bonus points)
    • In the chat system, we will have separate streams for each assignment and the project.
    • Stream #general can be used for any kind of general discussions and questions related to the course.
    • All important announcements will be posted to #announcements (no discussion on this stream).
    • Any kind of feedback is welcome on stream #feedback.
    • We have also streams #r, #python, and #stan for questions that are not specific to assignments or the project.
    • Stream #queue is used as a queue for getting help during TA sessions.
    • The lecturer and teaching assistants have names with “(staff)” or “(TA)” in the end of their names.
  • A weekly Q&A session with the lecturer happens in Zoom webinar (see times below, link in MyCourses)
    • Q&A session assumes you have self studied at least some of the material before the session
    • Q&A session will remind about the important announcements
    • Zoom webinar polls, Q&A feature, chat, and audio talking will be used for interaction
    • Zoom webinar polls don’t show up in browser zoom client! If you want to see the polls and the poll results, install a desktop client.
    • The form of the Q&A session will develop based on the feedback from the students
    • Q&A session is not recorded, but the answers to most relevant questions will be shared or short additional videos will be recorded
  • If you need one-to-one help, please take part in the TA sessions and ask there.
  • If you find errors in material, post in #feedback stream or submit an issue in github.
  • Peergrade alerts: If you are worried that you forget the deadlines, you can set peergade to send you email when assignment opens for submission, 24 hours before assignment close for submission, assignment is open for reviewing, 24 hours before an assignment closes for reviewing if you haven’t started yet, someone likes my feedback (once a day). Click your name -> User Settings to choose which alerts you want.

Assessment

Assignments (67%) and a project work with presentation (33%). Minimum of 50% of points must be obtained from both the assignments and project work.

Schedule 2021

The course consists of 12 blocks in periods I and II. The blocks don’t match exactly specific weeks. For example, it’s good start reading the material for the next block while making the assignment for one block. There are 9 assignments and a project work with presentation, and thus the assignments are not in one-to-one correspondence with the blocks. The schedule below lists the blocks and how they connect to the topics, book chapters and assignments.

Schedule overview

Here is an overview of the schedule. Scroll down the page to see detailed instructions for each block. Remember that blocks are overlapping so that when you are working on assignment for one block, you should start watching videos and reading text for the next block.

Block Readings Lectures Assignment Q&A Date Assignment due date
1. Introduction BDA3 Chapter 1 Computational probabilistic modeling,
Introduction to uncertainty and modelling,
Introduction to the course contents		 Assignment 1,
Rubric questions		 13.9. 19.09.
2. Basics of Bayesian inference BDA3 Chapter 1,
BDA3 Chapter 2		 Lecture 2.1,
Lecture 2.2,
Optional:
Extra explanations 2,
Summary 2.1,
Summary 2.2,
Summary 2.3		 Assignment 2,
Rubric questions		 20.9. 26.9.
3. Multidimensional posterior BDA3 Chapter 3 Lecture 3 Assignment 3,
Rubric questions		 27.9. 3.10.
4. Monte Carlo BDA3 Chapter 10 Lecture 4.1,
Lecture 4.2		 Assignment 4,
Rubric questions		 4.10. 10.10.
5. Markov chain Monte Carlo BDA3 Chapter 11 Lecture 5.1,
Lecture 5.2		 Assignment 5,
Rubric questions		 11.10. 17.10.
6. Stan, HMC, PPL BDA3 Chapter 12 + extra material on Stan Lecture 6.1,
Lecture 6.2		 Assignment 6,
Rubric questions		 18.10. 31.10.
7. Hierarchical models and exchangeability BDA3 Chapter 5 Lecture 7.1,
Lecture 7.2		 Assignment 7,
Rubric questions		 1.11. 14.11.
8. Model checking & cross-validation BDA3 Chapter 6, BDA3 Chapter 7, Visualization in Bayesian workflow, Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC Lecture 8.1,
Lecture 8.2		 Start project work 8.11. N/A
9. Model comparison and selection BDA3 Chapter 7 (not 7.2 and 7.3),
Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC		 Lecture 9.1,
Optional:
Lecture 9.2,
Lecture 9.3		 Assignment 8,
Rubric questions		 15.11. 21.11.
10. Decision analysis BDA3 Chapter 9 Lecture 10.1 Assignment 9,
Rubric questions		 22.11. 28.11.
11. Normal approximation, frequency properties BDA3 Chapter 4 Lecture 11.1,
Lecture 11.2		 Project work 29.11. N/A
12. Extended topics Optional: BDA3 Chapter 8,
BDA3 Chapter 14-18,
BDA3 Chapter 21		 Optional:
Lecture 12.1,
Lecture 12.2		 Project work 7.12 6.12.
13. Project evaluation Project presentations: 13-17.12. Evaluation week

1) Course introduction, BDA 3 Ch 1, prerequisites assignment

Course practicalities, material, assignments, project work, peergrading, QA sessions, TA sessions, prerequisites, chat, etc.

2) BDA3 Ch 1+2, basics of Bayesian inference

BDA3 Chapters 1+2, basics of Bayesian inference, observation model, likelihood, posterior and binomial model, predictive distribution and benefit of integration, priors and prior information, and one parameter normal model.

3) BDA3 Ch 3, multidimensional posterior

Multiparameter models, joint, marginal and conditional distribution, normal model, bioassay example, grid sampling and grid evaluation. BDA3 Ch 3.

4) BDA3 Ch 10, Monte Carlo

Numerical issues, Monte Carlo, how many simulation draws are needed, how many digits to report, direct simulation, curse of dimensionality, rejection sampling, and importance sampling. BDA3 Ch 10.

5) BDA3 Ch 11, Markov chain Monte Carlo

Markov chain Monte Carlo, Gibbs sampling, Metropolis algorithm, warm-up, convergence diagnostics, R-hat, and effective sample size. BDA3 Ch 11.

6) BDA3 Ch 12 + Stan, HMC, PPL, Stan

HMC, NUTS, dynamic HMC and HMC specific convergence diagnostics, probabilistic programming and Stan. BDA3 Ch 12 + extra material

7) BDA3 Ch 5, hierarchical models

Hierarchical models and exchangeability. BDA3 Ch 5.

8) BDA3 Ch 6+7 + extra material, model checking, cross-validation

Model checking and cross-validation.

9) BDA3 Ch 7, extra material, model comparison and selection

PSIS-LOO, K-fold-CV, model comparison and selection. Extra lecture on variable selection with projection predictive variable selection.

10) BDA3 Ch 9, decision analysis

Decision analysis. BDA3 Ch 9.

11) BDA3 Ch 4 + extra material, normal approximation, frequency properties

Normal approximation (Laplace approximation), and large sample theory and counter examples. BDA3 Ch 4.

  • Read Chapter 4
  • Watch Lecture 11.1 on normal approximation (Laplace approximation) and Lecture 11.2 on large sample theory and counter examples. BDA3 Ch 4.
  • Q&A Monday 29.11. 14:15-16
  • No new assignment. Work on project. TAs help with projects.
    • Review Assignment 9 done by your peers before 23:59 1.12., and reflect on your feedback
  • TA sessions 1.12 14-16, 2.12 12-14, 3.12 10-12

12) extra material + overview of BDA3 Ch 8, 14-18, 21

Frequency evaluation of Bayesian methods, hypothesis testing and variable selection. Overview of modeling data collection, BDA3 Ch 8, linear models, BDA Ch 14-18, lasso, horseshoe and Gaussian processes, BDA3 Ch 21.

  • These lectures are optional, but especially the lecture on hypothesis testing and variable selection is useful for project work.
  • Watch Lecture 12.1 on frequency evaluation, hypothesis testing and variable selection and Lecture 12.2 overview of modeling data collection, BDA3 Ch 8, linear models, BDA Ch 14-18, lasso, horseshoe and Gaussian processes, BDA3 Ch 21.
  • Q&A Tuesday 7.12. 14:15-16
  • Work on project. TAs help with projects. Project deadline 6.12. 23:59
  • TA sessions 8.12 14-16, 9.12 12-14, 10.12 10-12

13) Project evaluation

  • Project report deadline 6.12. 23:59 (submit to peergrade).
    • Review project reports done by your peers before 9.12. 23:59, and reflect on your feedback
  • Project presentations 13-17.12. (evaluation week)

R and Python

We strongly recommend using R in the course as there are more packages for Stan and statistical analysis in R. If you are already fluent in Python, but not in R, then using Python may be easier, but it can still be more useful to learn also R. Unless you are already experienced and have figured out your preferred way to work with R, we recommend

See FAQ for frequently asked questions about R problems in this course. The demo codes provide useful starting points for all the assignments.

Finnish terms

Sanasta “bayesilainen” esiintyy Suomessa muutamaa erilaista kirjoitustapaa. Muoto “bayesilainen” on muodostettu yleisen vieraskielisten nimien taivutussääntöjen mukaan: “Jos nimi on kirjoitettuna takavokaalinen mutta äännettynä etuvokaalinen, kirjoitetaan päätteseen tavallisesti takavokaali etuvokaalin sijasta, esim. Birminghamissa, Thamesilla.” Terho Itkonen, Kieliopas, 6. painos, Kirjayhtymä, 1997.