Gaussian Processes from scratch

An implementation of Gaussian Process regression in raw Python + Numpy

A Gaussian Process is a very powerful and flexible ML technique that allows you to define and fit a distribution over functions. Their advantage over classical curve fitting is that one doesn’t have to define the form of the function (for instance, a polynomial of order 3) a priori. Instead, one defines a prior over possible functions, which are modelled as a flexibly parametrised multivariate Gaussian. This allows for great flexibilty and, as the GP defines a probabilistic model, gives us an uncertainty estimate for each of the model’s predictions. Cool stuff!


Unfortunately, GPs are quite difficult to understand, especially if you’re coming from a background in Neural Networks and don’t have a strong foundation in Bayesian/Probabilistic ML.

To close this educational gap, I developed an implementation of GPs from scratch, using the Python programming language together with the Numpy package for computation, the Scipy package for numerical optimisation, and the Matplotlib package for visualisation.

You can access the results of my efforts as interactive tutorial under the following link:

Timo Flesch
Timo Flesch
PhD Candidate