Daniel Sharp: Mean-shift interacting particle systems for sampling and quantization

Abstract: In recent years, the fields of uncertainty quantification and data science have seen an explosion in interacting particle systems. Such methods generally require an initialized set of particles and prescribed dynamics, often inspired by gradient flow methods. The particles then are simulated using the given dynamical system. Our work introduces a new interacting particle system for quantization which we call “mean-shift interacting particles” (MSIP). In the data-driven case, MSIP is comparable to clustering algorithms (e.g., Lloyd’s k-means); in the Bayesian setting, we work within the field of variational inference and quadrature (e.g., Stein variational gradient descent). Where alternative methods in each setting are challenged by highly-concentrated target distributions and high-dimensional parameter spaces, we are able to demonstrate robustness due to the interpretation of MSIP as a preconditioned gradient descent of the maximum mean discrepancy, which is historically seen as intractable to minimize in the presence of unnormalized densities. We demonstrate the efficacy of our methods on challenging low-dimensional examples, scaling it up to higher-dimensional PDE-based inference, quantizing high-dimensional image datasets, and training Bayesian neural networks.