In this tutorial, we consider solving a sparse linear system of equations, and briefly discuss the classical Jacobi, Gauss-Seidel and conjugate gradient methods. We then present a probabilistic numerical solver as well as a distributed variant, based on a message passing algorithm. Solving the system becomes marginal inference in a Gaussian Markov random field.