Abstract:
This paper studies three ways to construct a nonhomogeneous jump Markov process: (i) via a compensator of the random measure of a multivariate point process, (ii) as a minimal solution of the backward Kolmogorov equation, and (iii) as a minimal solution of the forward Kolmogorov equation. The main conclusion of this paper is that, for a given measurable transition intensity, commonly called a Q-function, all these constructions define the same transition function. If this transition function is regular, that is, the probability of accumulation of jumps is zero, then this transition function is the unique solution of the backward and forward Kolmogorov equations. For continuous Q-functions, Kolmogorov equations were studied in Feller's seminal paper. In particular, this paper extends Feller's results for continuous Q-functions to measurable Q-functions and provides additional results.
This research was partially supported by NSF grants CMMI-0928490 and CMMI-1335296.
Received: 23.02.2013
Bibliographic databases:
Document Type:
Article
Language: English
Linking options:
https://www.mathnet.ru/eng/jmaa4
This publication is cited in the following 21 articles:
Manuel L. Esquível, Nadezhda P. Krasii, Gracinda R. Guerreiro, “Estimation–Calibration of Continuous-Time Non-Homogeneous Markov Chains with Finite State Space”, Mathematics, 12:5 (2024), 668
Xin Guo, Yonghui Huang, “Risk-sensitive zero-sum games for continuous-time jump processes with unbounded rates and Borel spaces”, Stochastics, 2024, 1
E. A. Feinberg, A. N. Shiryaev, “On forward and backward Kolmogorov equations for purely jump Markov processes and their generalizations”, Theory Probab. Appl., 68:4 (2024), 643–656
Madalina Deaconu, Antoine Lejay, “Probabilistic representations of fragmentation equations”, Probab. Surveys, 20:none (2023)
Eugene A. Feinberg, Manasa Mandava, Albert N. Shiryaev, “Sufficiency of Markov policies for continuous-time jump Markov decision processes”, Math. Oper. Res., 47:2 (2022), 1266–1286
Bertrand Cloez, Josué Corujo, “Uniform in time propagation of chaos for a Moran model”, Stochastic Processes and their Applications, 154 (2022), 251
Eugene Feinberg, Manasa Mandava, Albert N. Shiryaev, “Kolmogorov’s equations for jump Markov processes with unbounded jump rates”, Ann. Oper. Res., 317 (2022), 587–604
E. A. Feinberg, A. N. Shiryaev, “Kolmogorov's equations for jump Markov processes and their applications to control problems”, Theory Probab. Appl., 66:4 (2022), 582–600
Xin Guo, Yonghui Huang, “Risk-sensitive average continuous-time Markov decision processes with unbounded transition and cost rates”, J. Appl. Probab., 58:2 (2021), 523
Yi Zhang, “On the Nonexplosion and Explosion for Nonhomogeneous Markov Pure Jump Processes”, J Theor Probab, 31:3 (2018), 1322
Qingda Wei, “Zero-sum games for continuous-time Markov jump processes with risk-sensitive finite-horizon cost criterion”, Operations Research Letters, 46:1 (2018), 69
Xin Guo, Alexey Piunovskiy, Yi Zhang, “Note on discounted continuous-time Markov decision processes with a lower bounding function”, J. Appl. Probab., 54:4 (2017), 1071
Xianping Guo, Yonghui Huang, Yi Zhang, “Constrained Continuous-Time Markov Decision Processes on the Finite Horizon”, Appl Math Optim, 75:2 (2017), 317
Xianping Guo, Yi Zhang, “Constrained total undiscounted continuous-time Markov decision processes”, Bernoulli, 23:3 (2017)
David Hobson, “Mimicking martingales”, Ann. Appl. Probab., 26:4 (2016)
Xianping Guo, Xiangxiang Huang, Yonghui Huang, “Finite-horizon optimality for continuous-time Markov decision processes with unbounded transition rates”, Advances in Applied Probability, 47:4 (2015), 1064
Alexander Sokol, Niels Richard Hansen, “Exponential Martingales and Changes of Measure for Counting Processes”, Stochastic Analysis and Applications, 33:5 (2015), 823
Contact us: