Gaussian process regression: a function space perspective

This page attempts an intuitive explanation of the function space perspective of Gaussian process (GP) regression. The webapp below demonstrates how a GP prior is a distribution over functions, and how observing data conditions the prior to obtain the GP posterior.

In the beginning there is just the Gaussian process (GP) prior, which is a vague specification of the function generating the data. You can see in the app below, samples from the GP prior (shown in orange) produce all sorts of random functions.

When a data-point is observed (click on the chart to add a data point), it conditions the values the GP can take at that point. Thus, observed data-points add information that constrains the distribution of functions. Vary the GP hyper-parameters to see how these affect the conditioning of the GP. When all the data has been observed, we are left with the posterior GP distribution over functions.

Prior function covariance
Characteristic length-scale
Input noise precision