Minibatch Optimal Transport

Minibatch Optimal Transport and Perplexity Bound Estimation in Discrete Flow Matching

arXiv:2411.00759v5 Announce Type: replace Abstract: Discrete flow matching, a recent framework for modeling categorical data, has shown competitive performance with autoregressive models. However, unlike continuous flow matching, the rectification strategy cannot be applied due to the stochasticity of discrete paths, necessitating alternative methods to minimize state transitions. We propose a dynamic-optimal-transport-like minimization objective and derive its Kantorovich formulation for...