Stacking probabilistic building blocks into deeper architectures typically breaks closed-form inference. We show this is not inevitable. We identify five factor-graph primitives: a bilinear factor, an exponential link, a Gamma prior, a Gaussian likelihood, and an equality node, and prove that any model composed from them admits closed-form variational message passing. The key insight is that edge types in the factor graph act as a type system: under mean-field factorization, every message has a known parametric form determined locally by the types, and the single non-conjugate interface (the exponential link) is resolved by exploiting the moment-generating function of the Gaussian and the sufficient statistics of the Gamma family. We demonstrate composition at increasing depth, from static ensembles through input-dependent gating to split-branch routing, and show that stacking routing layers encodes arbitrary decision trees, establishing universal function approximation with closed-form inference. Applied to ensemble time-series forecasting, the framework yields a Bayesian mixture of experts in which gating functions are inferred rather than learned, providing calibrated uncertainty over expert selection across five benchmark datasets.