Abstract
In the Bayesian approach to modelling, noise is not considered a nuisance but rather a reflection of uncertainty, expressed in terms of probability distributions. Key to this approach is Bayes’ theorem as mathematical formalism for updating probability distributions. One starts with a prior probability distribution (akin to an initial condition) that describes expert knowledge, assumptions, and constraints. This prior is combined with a likelihood function that quantifies uncertainty around an observed datapoint. Bayes’ theorem then inverts the model and updates our prior to a posterior distribution reflecting the information gained from the new datapoint. Over time, and under appropriate conditions, this posterior will concentrate on a narrow subdomain, indicating high confidence. The value of the Bayesian approach lies in the regularizing effect of quantified uncertainty, both in parameter estimation and in state prediction. Its main challenge lies with unwieldy integrals, which may be resolved by the calculus of variations. Variational Bayesian inference optimizes an approximation of the posterior distribution, allowing for a trade-off between accuracy and computational workload. Inference algorithms based on variational Bayes are light enough to be incorporated into real-time information processing systems and resource-constrained devices. In this talk, I will demonstrate how this technique may be applied in a simple system identification problem, compare it to classical techniques, and illustrate how modularity may be utilized to accelerate the model design cycle.
