Article
Full entry |
PDF
(0.3 MB)
Feedback
PDF
(0.3 MB)
Feedback
Keywords:
discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric
discrete-time Markov process; risk-seeking expected total cost; optimal stopping rule; stability index; total variation metric
Summary:
We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chain on a Borel space $X$. It is supposed that the transition probability $p(\cdot |x)$, $x\in X$ is approximated by the transition probability $\widetilde{p}(\cdot |x)$, $x\in X$, and that the stopping rule $\widetilde{f}_*$ , which is optimal for the process with the transition probability $\widetilde{p}$ is applied to the process with the transition probability $p$. We give an upper bound (expressed in term of the total variation distance: $\sup_{x\in X}\|p(\cdot |x)-\widetilde{p}(\cdot |x)\|)$ for an additional cost paid for using the rule $\widetilde{f}_*$ instead of the (unknown) stopping rule $f_*$ optimal for $p$.
References:
[1] Avila-Godoy, G., Fernández-Gaucherand, E.: Controlled Markov chains with exponential risk-sensitive criteria: modularity, structured policies and applications. In: Decision and Control 1998. Proc. 37th IEEE Conference. Vol. 1, IEEE, pp. 778-783.
[2] Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer-Verlag, Berlin 2011. MR 2808878 | Zbl 1236.90004
[3] Borkar, V. S., Meyn, S. P.: Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27 (2002), 192-209. DOI 10.1287/moor.27.1.192.334 | MR 1886226 | Zbl 1082.90577
[4] Cavazos-Cadena, R.: Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains. Math. Methods Oper. Res. 71 (2010), 47-84. DOI 10.1007/s00186-009-0285-6 | MR 2595908 | Zbl 1189.93144
[5] Cavazos-Cadena, R., Fernández-Gaucherand, E.: Controlled Markov chains with risk-sensitive criteria: Average costs, optimality equations, and optimal solutions. Math. Methods Oper. Res. 49 (1999), 299-324. MR 1687362
[6] Cavazos-Cadena, R., Montes-de-Oca, R.: Optimal stationary policies in risk-sensitive dynamic programs with finite state space and nonegative rewards. Appl. Math. 27 (2000), 167-185. MR 1768711
[7] Dijk, N. M. Van, Sladký, K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449-474. DOI 10.1023/A:1021749829267 | MR 1684679
[8] Devroye, L., Györfy, L.: Nonparametric Density Estimation: The $L_1$ View. John Wiley, New York 1986.
[9] Dynkin, E. B., Yushkevich, A. A.: Controlled Markov Processes. Springer Verlag, New York 1979. MR 0554083
[10] Gordienko, E. I., Yushkevich, A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345-365. MR 1990916 | Zbl 1116.90401
[11] Gordienko, E. I., Lemus-Rodríguez, E., Montes-de-Oca, R.: Average cost Markov control processes: stability with respect to the Kantorovich metric. Math. Methods Oper. Res. 70 (2009), 13-33. DOI 10.1007/s00186-008-0229-6 | MR 2529423 | Zbl 1176.60062
[12] Gordienko, E. I., Salem, F.: Robustness inequalities for Markov control processes with unbounded costs. Syst. Control Lett. 33 (1998), 125-130. DOI 10.1016/S0167-6911(97)00077-7 | MR 1607814
[13] Hernández-Lerma, O., Lasserre, J. B.: Further Topics on Discrete-time Markov Control Processes. Springer-Verlag, New York 1999. MR 1697198 | Zbl 0928.93002
[14] Jaśkiewicz, A.: Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17 (2007), 654-675. DOI 10.1214/105051606000000790 | MR 2308338 | Zbl 1128.93056
[15] Kartashov, N. V.: Strong Stable Markov Chains. VSP, Utrecht 1996. MR 1451375 | Zbl 0874.60082
[16] Marcus, S. I., Fernández-Gaucherand, E., Hernández-Hernández, D. E., Coraluppi, S., Fard, P.: Risk sensitive Markov decision processes. Progress in System and Control Theory 22 (1997), 263-280. MR 1427787
[17] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 (2000), 15-20. DOI 10.1016/S0167-6911(99)00118-8 | MR 1829070 | Zbl 0977.93083
[18] Meyn, S. P., Tweedie, R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993. MR 1287609 | Zbl 1165.60001
[19] Montes-de-Oca, R., Salem-Silva, F.: Estimates for perturbations of an average Markov decision processes with a minimal state and upper bounded stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772. MR 2193864
[20] Muciek, B. K.: Optimal stopping of risk processes: model with interest rates. J. Appl. Probab. 39 (2002), 261-270. DOI 10.1239/jap/1025131424 | MR 1908943
[21] Shiryaev, A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978. MR 2374974 | Zbl 1138.60008
[22] Shiryaev, A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N. J. 1999. MR 1695318
[23] Sladký, K.: Bounds on discrete dynamic programming recursions I. Kybernetika 16 (1980), 526-547. MR 0607292 | Zbl 0454.90085
[24] Zaitseva, E.: Stability estimating in optimal stopping problem. Kybernetika 44 (2008), 400-415. MR 2436040 | Zbl 1154.60326
Search
Partner of
