Density Ratio Estimation with Stein Displacement Fields

Key points

• In this paper, we estimate the density ratio between a target and a base distribution by parametrizing it through a displacement field acting on the base: the log-ratio is modeled as minus the Stein operator of the base applied to the field, up to a normalizing constant.1
• This gives both statistical and dynamical descriptions of the distribution shift through a single convex optimization problem.1
• Iterating this estimate-and-move step gives two inference algorithms: push-forward moves the model and corrects a pretrained sampler without retraining it, whereas pull-back moves the data closer to the base and fits a transformation model one layer at a time.1
• Applications to distribution shift in simulation-based inference and to nonlinear independent component analysis illustrate the benefits and limitations of the approach.1

Sources (1)

• [1]Density Ratio Estimation with Stein Displacement Fields
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 8, 05:57 PM

  Density ratios quantify distribution shift from a probability-mass point of view, whereas displacement fields describe, from a dynamical point of view, how one distribution is transported onto another.

  In this paper, we estimate the density ratio between a target and a base distribution by parametrizing it through a displacement field acting on the base: the log-ratio is modeled as minus the Stein operator of the base applied to the field, up to a normalizing constant.

Extractive summary: sentences quoted from the sources.