Skip to main content

Jan Gerken: Emergent Equivariance in Deep Ensemble

Time: Tue 2025-05-06 10.15

Location: KTH 3418, Lindstedtsvägen 25 and Zoom

Video link: Zoom meeting ID: 632 2469 3290

Respondent: Nils Quaetaert

Supervisor: Kathlén Kohn

Export to calendar

Abstract.

In this talk, I will discuss recent results on the symmetry properties of deep ensembles trained with data augmentation. Specifically, we show that the ensemble is equivariant at any training step, provided that data augmentation is used. Crucially, this equivariance also holds off-manifold and therefore goes beyond the intuition that data augmentation leads to approximately equivariant predictions. Furthermore, equivariance is emergent in the sense that predictions of individual ensemble members are not equivariant but their collective prediction is. Therefore, the deep ensemble is indistinguishable from a manifestly equivariant predictor. In the infinite width limit, this predictor is in fact a group convolutional neural network. We prove this theoretically using neural tangent kernel theory and verify our theoretical insights using detailed numerical experiments. Based on joint work with Pan Kessel and Philipp Misof: arXiv:2403.03103 and arXiv:2406.06504.