Article
Full entry |
PDF
(0.4 MB)
Feedback
PDF
(0.4 MB)
Feedback
Keywords:
monotone operator; fixed point; equilibrium equation; hitting time; bounded rewards; certainty equivalent
monotone operator; fixed point; equilibrium equation; hitting time; bounded rewards; certainty equivalent
Summary:
This work is concerned with discrete-time Markov stopping games with two players. At each decision time player II can stop the game paying a terminal reward to player I, or can let the system to continue its evolution. In this latter case player I applies an action affecting the transitions and entitling him to receive a running reward from player II. It is supposed that player I has a no-null and constant risk-sensitivity coefficient, and that player II tries to minimize the utility of player I. The performance of a pair of decision strategies is measured by the risk-sensitive (expected) total reward of player I and, besides mild continuity-compactness conditions, the main structural assumption on the model is the existence of an absorbing state which is accessible from any starting point. In this context, it is shown that the value function of the game is characterized by an equilibrium equation, and the existence of a Nash equilibrium is established.
References:
[1] Alanís-Durán, A., Cavazos-Cadena, R.: An optimality system for finite average Markov decision chains under risk-aversion. Kybernetika 48 (2012), 83-104. MR 2932929
[2] Altman, E., Shwartz, A.: Constrained Markov games: Nash equilibria. In: Annals of Dynamic Games (V. Gaitsgory, J. Filar, and K. Mizukami, eds.), Birkhauser, Boston 2000, pp. 213-221. MR 1764491
[3] Atar, R., Budhiraja, A.: A stochastic differential game for the inhomogeneous Laplace equation. Ann. Probab. 38 (2010), 2, 498-531. DOI | MR 2642884
[4] Balaji, S., Meyn, S. P.: Multiplicative ergodicity and large deviations for an irreducible Markov chain. Stoch. Proc. Appl. 90 (2000), 1, 123-144. DOI | MR 1787128
[5] Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer, New York 2011. MR 2808878 | Zbl 1236.90004
[6] Bäuerle, N., Rieder, U.: More risk-sensitive Markov decision processes. Math. Oper. Res. 39 (2014), 1, 105-120. DOI | MR 3173005
[7] Bäuerle, N., Rieder, U.: Zero-sum risk-sensitive stochastic games. Stoch. Proc. Appl. 127 (2017), 2, 622-642. DOI | MR 3583765
[8] Bielecki, T. R., Hernández-Hernández, D., Pliska, S. R.: Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Mathematical Methods of OR 50 (1999), 167-188. DOI | MR 1732397 | Zbl 0959.91029
[9] Borkar, V. S., Meyn, S. F.: Risk-sensitive optimal control for Markov decision process with monotone cost. Math. Oper. Res. 27 (2002), 1, 192-209. DOI | MR 1886226
[10] Cavazos-Cadena, R., Hernández-Hernández, D.: A system of Poisson equations for a non-constant {Varadhan} functional on a finite state space. Appl. Math. Optim. 53 (2006), 101-119. DOI | MR 2190228
[11] Cavazos-Cadena, R., Hernández-Hernández, D.: Nash equilibria in a class of Markov stopping games. Kybernetika 48 (2012), 5, 1027-1044. MR 3086867
[12] Cavazos-Cadena, R., Rodríguez-Gutiérrez, L., Sánchez-Guillermo, D. M.: Markov stopping games with an absorbing state and total reward criterion. Kybernetika 57 (2021), 474-492. DOI | MR 4299459
[13] Denardo, E. V., Rothblum, U. G.: A turnpike theorem for a risk-sensitive Markov decision process with stopping. SIAM J. Control Optim. 45 (2006), 2, 414-431. DOI | MR 2246083
[14] Masi, G. B. Di, Stettner, L.: Risk-sensitive control of discrete time Markov processes with infinite horizon. SIAM J. Control Optim. 38 (1999), 1, 61-78. DOI | MR 1740607
[15] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Syst. Control Lett. 40 (2000), 15-20. DOI | MR 1829070 | Zbl 0977.93083
[16] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes under minorization property. SIAM J. Control Optim. 46 (2007), 1, 231-252. DOI | MR 2299627
[17] A.Filar, J., Vrieze, O. J.: Competitive Markov Decision Processes. Springer, New York 1996. MR 1418636
[18] Hernández-Lerma, O.: Adaptive Markov Control Processes. Springer, New York 1989. MR 0995463 | Zbl 0677.93073
[19] Howard, R. A., Matheson, J. E.: Risk-sensitive Markov decision processes. Manage. Sci. 18 (1972), 7, 349-463. DOI | MR 0292497
[20] Jaśkiewicz, A.: Average optimality for risk sensitive control with general state space. Ann. Appl. Probab. 17 (2007), 2, 654-675. DOI | MR 2308338
[21] Kolokoltsov, V. N., Malafeyev, O. A.: Understanding Game Theory. World Scientific, Singapore 2010. MR 2666863 | Zbl 1189.91001
[22] Kontoyiannis, I., Meyn, S. P.: Spectral theory and limit theorems for geometrically ergodic Markov processes. Ann. Appl. Probab. 13 (2003), 1, 304-362. DOI | MR 1952001
[23] Martínez-Cortés, V. M.: Bi-personal stochastic transient Markov games with stopping times and total reward criterion. Kybernetika 57 (2021), 1, 1-14. DOI | MR 4231853
[24] Peskir, G.: On the American option problem. Math. Finance 15 (2007), 169-181. DOI | MR 2116800 | Zbl 1109.91028
[25] Peskir, G., Shiryaev, A.: Optimal Stopping and Free-Boundary Problems. Birkhauser, Boston 2006. MR 2256030 | Zbl 1115.60001
[26] Pitera, M., Stettner, L.: Long run risk sensitive portfolio with general factors. Math. Meth. Oper. Res. 82 (2016), 2, 265-293. DOI | MR 3489700
[27] Puterman, M. L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York 1994. MR 1270015 | Zbl 1184.90170
[28] Shapley, L. S.: Stochastic games. Proc. National Academy Sci. 39 (1953), 10, 1095-1100. MR 0061807 | Zbl 1180.91042
[29] Shiryaev, A.: Optimal Stopping Rules. Springer, New York 2008. MR 2374974 | Zbl 1138.60008
[30] Sladký, K.: Growth rates and average optimality in risk-sensitive Markov decision chains. Kybernetika 44 (2008), 2, 205-226. MR 2428220
[31] Sladký, K.: Risk-sensitive average optimality in Markov decision processes. Kybernetika 54 (2018), 6, 1218-1230. DOI | MR 3902630
[32] Stettner, L.: Risk sensitive portfolio optimization. Math. Meth. Oper. Res. 50 (1999), 3, 463-474. DOI | MR 1731299
[33] Zachrisson, L. E.: Markov Games. Princeton University Press 12, Princeton 1964. MR 0170729
Search
Partner of
