Message-Passing

System identification is an inference problem: given data, what can we infer about the data-generating system? In this talk I develop a Bayesian view of identification in which that inference is carried out as message passing on Forney-style factor graphs.

We propose a formulation of expected free energy minimization based on a Bethe free energy functional, fully supporting inference by message passing. The epistemic drive is maintained by imposing an information constraint, in addition to normalisation, marginalisation and form constraints, insisting that the mutual information between future observations, states and parameters given actions must be at least as large as the entropy of the goal prior.

This talk presents a principled mathematical framework for deriving information-seeking behavior in intelligent agents, grounded in Bayesian inference and the free energy principle.

We present the design of an autoregressive active inference agent in the form of message passing on a factor graph.

We propose a variational message passing inference algorithm for online system identification using polynomial NARMAX models. We show empirically that our variational Bayesian estimator outperforms an online recursive least-squares estimator, most notably in small sample size settings and low noise regimes, and performs on par with an iterative least-squares estimator trained offline.

Message passing on factor graphs typically relies on a scheduling procedure, in which a central algorithm or compiler figures out which nodes should pass messages where at what time. This is not a biologically plausible mechanism. I explore the possibility of passing messages without a scheduler, where the nodes merely “react” to incoming messages.