the Kullback-Leibler Divergence in
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
A Note on the Kullback-Leibler Divergence in Discretized Empirical Distributions
new Abstract: When empirical objects are represented as discrete probability distributions, within-distribution summaries such as Shannon entropy and Hill-type diversity indices describe how probability mass is spread inside each object, while Kullback-Leibler (KL) divergence provides pairwise asymmetric information. This note focuses on the KL difference $\Delta_{\mathrm{KL}}(p,q)=D_{\mathrm{KL}}(p|q)-D_{\mathrm{KL}}(q|p)$. Although $\Delta_{\mathrm{KL}}$ can add information beyond...
LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding
Announce Type: replace Abstract: Speculative decoding accelerates autoregressive large language model (LLM) inference by using a lightweight draft model to propose candidate tokens that are then verified in parallel by the target model. The speedup is significantly determined by the acceptance rate, yet standard training minimizes Kullback-Leibler (KL) divergence as a proxy objective. While KL divergence and acceptance rate share the same global optimum, small draft models, having limited...
Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences
arXiv:2602.02250v2 Announce Type: replace-cross Abstract: Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes. Using a unified information-geometric framework, we introduce KL analogs by replacing the Fisher-Rao geometry in the dynamical formulation of the KL with transport-based geometries, and derive closed-form expressions for common distribution families. Between...
Self-Distilled Policy Gradient
arXiv:2606.04036v1 Announce Type: new Abstract: On-policy self-distillation, where a language model conditions on privileged context to supervise its own generations, is a promising source of dense supervision for sparse-reward reinforcement learning. Actually, it can be instantiated as an auxiliary full-vocabulary student-to-teacher reverse Kullback-Leibler divergence loss.
A machine-learning-assisted progressive digit-randomness screening framework for detecting non-random patterns in raw numerical research data
Announce Type: new Abstract: Raw numerical datasets remain less systematically examined in integrity screening than images, plagiarism, or summary-statistic inconsistencies. We developed the Fabrication-risk Digit Randomness Screening model (FDRS), a statistical and machine-learning framework for detecting non-random digit-pattern irregularities in numerical research data. FDRS integrates single- and joint-decimal-digit tests, Cramer's V, entropy metrics, Kullback-Leibler divergence,...
Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry
arXiv:2511.01064v3 Announce Type: replace-cross Abstract: Variational inference (VI) approximates a target density $p$ by the best match $q$ in a family of tractable distributions. The best variational approximation is found by minimizing a divergence between distributions, $D(p||q)$, and several divergences have been proposed as objective functions for VI, with different choices leading to different approximations. We show that even when these divergences have different minimizers, the...
Even More Guarantees for Variational Inference in the Presence of Symmetries
Announce Type: replace Abstract: When approximating an intractable density via variational inference (VI) the variational family is typically chosen as a simple parametric family that very likely does not contain the target. This raises the question: Under which conditions can we recover characteristics of the target despite misspecification? In this work, we extend previous theoretical results on robust VI with location-scale families under target symmetries in two substantial ways: (1) We...
Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors
arXiv:2606.06179v1 Announce Type: cross Abstract: Score-based diffusion models are typically trained by minimizing the $L^2$ score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target distributions. We show the $L^2$ score error is not the right intrinsic measure of marginal distributional quality: a learned diffusion model can incur arbitrarily large $L^2$ score error while perfectly matching the target...
Magnetometry with Broadband Microwave Fields in Nitrogen-Vacancy Centers in Diamond
arXiv:2510.11720v2 Announce Type: replace-cross Abstract: Nitrogen-vacancy (NV) centers in diamond are optically addressable and versatile light-matter interfaces with practical application in magnetic field sensing, offering the ability to operate at room temperature and reach sensitivities below pT/$\sqrt{\mathrm{Hz}}$. We propose an approach to simultaneously probe all of the magnetically sensitive states using a broadband microwave field and demonstrate that it can be used to measure the...
Quantifying Side-Channel Leakage in Public Metrology Releases
Announce Type: new Abstract: Public scientific and metrology releases can leak the hidden settings that produced them. We formalize and quantify this risk as a profiled statistical side-channel audit: a release map exposes finite-band statistics of a power spectral density (PSD), a profiled observer trains labeled template spectra under an explicit budget, and a challenge release is drawn from one of two utility-equivalent recipes separated by a protected coordinate. Averaged PSD bins follow...