the Mutual Information
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
SMI: Efficient Self-Supervised Learning via Mutual-Information-Inspired Dependency Optimization
Announce Type: new Abstract: Self-supervised learning (SSL) has achieved remarkable representation learning performance, but many existing methods rely on large batch sizes, memory banks, momentum encoders, or global synchronization mechanisms that substantially increase computational cost and training complexity. In this work, we propose Semantic Mutual Information (SMI), a lightweight self-supervised objective derived from a mutual-information-inspired dependency formulation under Gaussian...
Estimating Mutual Information between Time Series and Temporal Event Sequences Across Diverse Analysis Tasks
arXiv:2606.01602v1 Announce Type: new Abstract: Pairwise dependence measures such as correlation and causality are fundamental to temporal data mining, yet there is still no principled and robust way to quantify dependence between heterogeneous data types, especially between continuous time series and discrete temporal event sequences. Existing approaches rely on ad hoc transformations or mutual-information estimators that are highly sensitive to quantization, repeated values, and event...
MIST: Mutual Information Estimation Via Supervised Training
arXiv:2511.18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators. Since any MI estimator is a function of the observed sample from two random variables, we parameterize this function with a neural network (MIST) and train it end-to-end to predict MI values. Training is performed on a large meta-dataset of 625,000 synthetic joint distributions with known ground-truth MI.
Mutual Information Optimization via K-Recursion and Automatic Differentiation for Linear Gaussian Wireless Networks
new Abstract: We present a differentiable framework for end-to-end mutual information (MI) optimization over linear Gaussian directed acyclic graphs (DAGs). The framework targets network-wide design under global constraints, such as a total transmit power budget, and covers MIMO precoding, amplify-and-forward relays, RIS-aided channels, and branching/merging topologies within a common linear Gaussian model. Its core ingredient is a \emph{K-recursion} that analytically propagates all...
Mutual Information Minimization for Side-Channel Attack Resistance via Optimal Noise Injection
arXiv:2504.20556v5 Announce Type: replace Abstract: Side-channel attacks (SCAs) pose a serious threat to system security by extracting secret keys through physical leakages such as power consumption, timing variations, and electromagnetic emissions. Among existing countermeasures, artificial noise injection is recognized as one of the most effective techniques. However, its high power consumption poses a major challenge for resource-constrained systems such as Internet of Things (IoT)...
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
arXiv:2603.19294v4 Announce Type: replace Abstract: While post-training has successfully improved large language models (LLMs) across a variety of domains, these gains heavily rely on human-labeled data or external verifiers. Existing data has already been exploited, and new data is expensive to collect. Moreover, true intelligence goes far beyond verifiable tasks.
Learning What Matters: Probabilistic Task Selection via Mutual Information for Model Finetuning
arXiv:2507.12612v3 Announce Type: replace Abstract: Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an...
Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
arXiv:2605.31445v1 Announce Type: new Abstract: In this work we study agents in simulated bargaining scenarios, where a buyer and a seller communicate through a text channel and attempt to negotiate mutually beneficial trades, under different information regimes (complete information, information asymmetry or mutual uncertainty). We evaluate their performance w.r.t. game-theoretical solutions and further investigate their honesty (their tendency to disclose or withhold information or to...
Optimal quantum locally differentially private mechanisms in the high-privacy regime
arXiv:2605.27278v2 Announce Type: replace-cross Abstract: We optimize the trade-off between privacy and utility in the high-privacy regime. We adopt local differential privacy (LDP) and its quantum extension, quantum local differential privacy (QLDP), for privacy protection, and investigate utility functions including the Holevo information (which reduces to the mutual information in the classical case) and the error exponents in symmetric and asymmetric hypothesis testing. These utility...
Finite-Resolution Information from Collision Statistics
arXiv:2606.01218v1 Announce Type: new Abstract: Collision statistics provide a finite-resolution view of information by measuring how often a fixed number of independent samples fall on the same state. These directly countable quantities form the basis of integer-order R\'enyi entropies. Here, we use low-order R\'enyi entropies to approximate Shannon entropy and mutual information, while characterizing what is necessarily lost when only finitely many collision moments are used.