Entropy
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines
arXiv:2606.03739v1 Announce Type: new Abstract: LLM pipelines waste substantial token budgets on low-information content: repeated context, verbose responses, and redundant boilerplate. We introduce Entropy Gate, a token compression framework applying entropy quenching $-$ a thermodynamic process that progressively freezes out low-energy tokens while preserving semantic fidelity. Each token receives a multi-factor information energy $E(t)$ combining statistical, structural, and positional...
Quantum states of macrosystems and entropy
Announce Type: replace Abstract: The paper examines and critiques the expression of entropy as the logarithm of the number of quantum states of a physical system. Boltzmann method of expressing entropy as the logarithm of the number of states of a gas with a given total energy is analyzed. We demonstrate that entropy is the product of subquantum processes and show that entropy is expressed as the ratio of the logarithm of the maximum number of realizations, over the observation period, of a...
Constitutive Settings with regard to Energy- and Entropy-Balances in Non-Equilibrium Thermodynamics: the Thermodynamical Verification
Announce Type: new Abstract: Constitutive equations have to be in agreement with the energy- and entropy- balances. For achieving that, the procedure of thermodynamical verification is introduced: Because heat flux and entropy flux as well as the time differentials of internal energy and entropy are not independent of each other, energy- and entropy-balances are connected with each other by so-called internal settings laying down the theoretical frame of the applied material description...
PAEC: Position-Aware Entropy Calibration for LLM Reasoning in RLVR
arXiv:2606.08543v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning but often suffers from rapid policy-entropy collapse, where the policy prematurely concentrates on narrow high-probability reasoning paths. While global entropy regularization can encourage exploration, uniformly increasing entropy across all token positions is inefficient for long reasoning trajectories, where many tokens are not decision-relevant. We...
Policy Split: Incentivizing Dual-Mode Exploration in LLM Reinforcement with Dual-Mode Entropy Regularization
arXiv:2604.11510v2 Announce Type: replace Abstract: To encourage diverse exploration in reinforcement learning (RL) for large language models (LLMs) without compromising accuracy, we propose Policy Split, a novel paradigm that bifurcates the policy into normal and high-entropy modes with a high-entropy prompt. While sharing model parameters, the two modes undergo collaborative dual-mode entropy regularization tailored to distinct objectives. Specifically, the normal mode optimizes for task...
Entropy as a Structural Prior: How a Log-Barrier on DiT Belief Space Drives Musical Diversity and Development
arXiv:2606.07207v1 Announce Type: new Abstract: Confidence-based loss weighting is usually avoided in generative models because it accelerates errors when the model is confidently wrong, but this intuition breaks down in supervised diffusion training. We introduce the Eisbach log-barrier, a parameter-free weight derived from the entropy of the DiT output's spatial energy distribution: high entropy damps the gradient, while low entropy preserves it. Applied to LoRA fine-tuning of Stable Audio...
From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs
arXiv:2606.09508v1 Announce Type: new Abstract: Existing sparse attention and KV cache compression methods for long-context LLM inference typically apply fixed sparsity patterns or uniform budgets across all attention heads, overlooking the substantial variation in attention behavior among heads and contexts. We observe two distinct entropy patterns among attention heads: Rigid Heads, whose entropy stays near zero across input segments, and Dynamic Heads, whose entropy fluctuates...
Tensor gradient flow with quasi-entropy for smectic liquid crystals and discretizations keeping coupled physical constraints
arXiv:2606.00659v1 Announce Type: cross Abstract: A gradient flow for the concentration and a $2\times 2$ tensor is constructed to describe smectic liquid crystals. The free energy consists of the entropy term and interaction term involving squared second order spatial derivatives. The entropy term incorporates the concentration in the quasi-entropy originally proposed for the tensor only, which is a strictly convex and lower semicontinuous function imposing coupled constraints between the...
Entropy Functions on Two-Dimensional Faces of Polymatroid Region Spanned by a Matroid and a Rank-One Matroid
Announce Type: replace Abstract: Characterization of entropy functions is of fundamental importance in information theory. By imposing constraints on their Shannon outer bound, i.e., the polymatroidal region, one obtains the faces of the region and entropy functions on them with special structures. In this paper, we characterize entropy functions on 2-dimensional faces of polymatroidal region of degree n spanned by a matroid and a rank-1 matroid.
Boosting RL-Based Visual Reasoning with Selective Adversarial Entropy Intervention
arXiv:2512.10414v2 Announce Type: replace Abstract: Recently, reinforcement learning (RL) has become a common choice in enhancing the reasoning capabilities of vision-language models (VLMs). Considering existing RL-based finetuning methods, entropy intervention turns out to be an effective way to benefit exploratory ability, thereby improving policy performance. Notably, most existing studies intervene in entropy by simply controlling the update of specific tokens during policy optimization...