Tentative curriculum (not finalized, subject to change):
- Overview and background
- Some statistical background: point estimation, limit theorems, maximum likelihood, (maybe: probabilistic inequalities, method of moments)
- Bayesian Inference, MAP, prior and posterior distributions, marginalization trick (maybe: naive Bayes classifiers, Bayes filters)
- brief intro to ML theory (VC dimension, PAC learning)
- Bias-Variance Trade-off; Model Selection, L1 and L2 regularization
- (maybe: confidence intervals; hypothesis testing; bootstrapping)
- beyond linear regression: logistic regression, nonparametric regression
- brief intro to graphical models
- multivariate distributions, dimensionality reduction: PCA/SVD, autoencoders (maybe: Graphical Lasso)
- Mixture of Gaussians, expectation maximization (maybe: Gaussian Process Regression, Gaussian Process Optimization)
- Information Theory, KL divergence, evaluation metrics
- SVM, kernel methods
- Ensemble methods: boosting, bagging, random forests
- time permitting: reinforcement learning
- time permitting: optimization methods: gradient descent, Newton's method, SGD, Lagrange multipliers
- time permitting: MCMC (Metropolis-Hastings, Gibbs Sampling)
- time permitting: Variational Inference
Grading:
- four assignments: 40% (10% each)
- class project (proposal + final report): 50% (15% for the proposal, 35% for the final report)
- attendance and participation: 10%