Markov
No mentions found
This entity hasn't been tracked yet, or Iris is still building its knowledge base.
Related Articles from SNS
True Self-Avoiding Walk for Accelerating Markov-Chain Monte Carlo Integration
Announce Type: cross Abstract: We study true self-avoiding walk (TSAW) as a mechanism for improving empirical integral estimation via Markov chain Monte Carlo (MCMC). We consider finite-state adaptive sampling dynamics associated with an irreducible Markov kernel $P$ on a finite set, with stationary distribution $\pi$, in which the transition probabilities are penalized according to empirical overuse. Our main result is that the empirical occupation counts $L_t(i)$ and transition counts...
Identifiable Markov Switching Models with Instantaneous Effects and Exponential Families
arXiv:2606.02231v1 Announce Type: cross Abstract: Temporal systems often exhibit non-stationary behaviour, such as seasonal climate variation or glucose fluctuations in patients with type-1 diabetes. One way to model non-stationarity is through discrete latent regimes, i.e., stationary segments of time. Such systems induce a Markov Switching Model (MSM), a class of Hidden Markov Models with autoregressive dependencies among latent regimes and observed variables.
Algebraic and FFT-Based Methods for Discrete-Time Matrix Convolutions with Applications to Semi-Markov Models
arXiv:2605.30379v2 Announce Type: replace Abstract: We study the convolution product of matrix-valued sequences and its role in the computation of Markov renewal equations. Explicit representations and recursive formulae for the convolutional inverse are derived and used to construct FFT-accelerated convolution and Newton-type inversion schemes, together with a Gauss--Jordan alternative in truncated power-series rings. The proposed framework is also applied to discrete approximations of...
A Doeblin-Anchored Contrastive Chart for Learning Markov Transition Kernels
arXiv:2606.02232v1 Announce Type: new Abstract: Learning a Markov transition model is not merely conditional density estimation: the learned object must be a valid transition kernel before it is iterated in downstream dynamics. This paper introduces a Doeblin-anchored contrastive chart, a statistical-to-dynamical coordinate framework for learning transition kernels from contrastive objectives. Given a restart law and an anchor strength, the chart mixes the target transition with the restart law.
Algebraic and FFT-Based Methods for Discrete-Time Matrix Convolutions with Applications to Semi-Markov Models
arXiv:2605.30379v1 Announce Type: new Abstract: We study the convolution product of matrix-valued sequences and its role in the computation of Markov renewal equations. Explicit representations and recursive formulae for the convolutional inverse are derived and used to construct FFT-accelerated convolution and Newton-type inversion schemes, together with a Gauss--Jordan alternative in truncated power-series rings. The proposed framework is also applied to discrete approximations of matrix...
Brief Announcement: Generative Markov Model for Distributed Computing Systems
Announce Type: new Abstract: Emerging distributed computing paradigms, such as the computing continuum, are inherently heterogeneous, stochastic, and complex. Efficiently and effectively utilizing all available resources across the continuum demands a unified formal model of the system. To address this gap, we propose a general framework for modeling distributed computing systems as a generative Markov model, factorized over a structured system state.
Theoretical Analysis of Engression and Reverse Markov Engression
Announce Type: cross Abstract: Engression is a recently proposed and effective framework for conditional distribution learning. Its multi-step Reverse Markov extension further improves generative flexibility by decomposing complex conditional sampling into sequential reverse transitions. Despite their strong empirical performance, rigorous finite-sample statistical guarantees for these methods remain unavailable.
Minimax-Optimal Policy Regret in Partially Observable Markov Games
arXiv:2606.02363v1 Announce Type: new Abstract: We study sequential decision-making in partially observable environments against strategic, adaptive opponents, modeled as partially observable Markov games (POMGs). The central challenge is to learn latent dynamics from partial observations while facing an adversary whose behavior depends on the learner's strategy, making standard regret notions inadequate. We prove that an epoch-based optimistic maximum-likelihood algorithm achieves...
Policy Gradient for Continuous-Time Robust Markov Decision Processes
Announce Type: new Abstract: The framework of robust Markov decision processes (RMDPs) allows the design of reinforcement learning agents that satisfy performance guarantees under worst-case transition dynamics. Traditional RMDPs consider discrete-time dynamics and recently, sample-efficient policy gradient algorithms have been considered in this context. This paper investigates policy gradient algorithms within a continuous-time RMDP framework.
Randomization for Faster Exact Optimization of Discounted Markov Decision Processes
arXiv:2606.05110v1 Announce Type: new Abstract: We provide faster deterministic and randomized algorithms for exactly solving discounted Markov Decision Processes (DMDPs). We obtain our results by efficiently reducing computing optimal values and policies in DMDPs to the easier tasks of policy evaluation and computing approximately optimal values in DMDPs. We provide both a straightforward deterministic reduction and a more efficient randomized variant that, together with advances in...