Advanced Models

This week moves beyond parametric curves to models that learn flexible structure directly from data. Gaussian processes provide a principled, non-parametric approach to regression that quantifies uncertainty everywhere — not just at data points. Hierarchical models let you share information across groups while respecting group-level variation, turning “not enough data” problems into tractable ones.

Gaussian Processes I

A Gaussian process places a prior over functions, not parameters — and lets the data decide which functions are plausible. We build the framework from first principles: the function-space view, the weight-space view and the kernel trick, covariance functions (squared exponential, Matern, periodic) and how to combine them, sampling from the prior, and deriving the conditioning equations that produce predictions with calibrated uncertainties.

Gaussian Processes II

Theory in hand, we turn to practice. We cover hyperparameter optimisation via the log-marginal likelihood, full Bayesian inference over hyperparameters with MCMC, composite kernel design for the atmospheric CO$_2$ record, stellar activity modelling for exoplanet detection, computational scaling ($\mathcal{O}(N^3)$ and how to beat it with celerite), and the failure modes you need to watch for.

Hierarchical Models

Every model so far treats each dataset as an island. Hierarchical models add a population level: individual objects share a common distribution whose parameters are learned from data. We cover partial pooling, shrinkage, the joint posterior, analytic marginalisation, centered vs non-centered parameterisations, and a complete worked example estimating cluster metallicities with heteroscedastic uncertainties.