Article
Full entry |
PDF
(0.4 MB)
Feedback
PDF
(0.4 MB)
Feedback
Keywords:
Bregman divergence; relative entropy; exponential family; optimization
Bregman divergence; relative entropy; exponential family; optimization
Summary:
The problem to maximize the information divergence from an exponential family is generalized to the setting of Bregman divergences and suitably defined Bregman families.
References:
[1] Ay, N.: An information-geometric approach to a theory of pragmatic structuring. Ann. Probab. 30 (2002), 416-436. DOI 10.1214/aop/1020107773 | MR 1894113 | Zbl 1010.62007
[2] Ay, N., Knauf, A.: Maximizing multi-information. Kybernetika 42 (2006), 517-538. MR 2283503 | Zbl 1249.82011
[3] Barndorff-Nielsen, O.: Information and Exponential Families in Statistical Theory. Wiley, 1978. MR 0489333 | Zbl 1288.62007
[4] Csiszár, I., Matúš, F.: Closures of exponential families. Ann. Probab. 33 (2005), 582-600. DOI 10.1214/009117904000000766 | MR 2123202 | Zbl 1068.60008
[5] Csiszár, I., Matúš, F.: Generalized maximum likelihood extimates for exponential families. Probab. Theory Related Fields 141 (2008), 213-246. DOI 10.1007/s00440-007-0084-z | MR 2372970
[6] Geiger, D., Meek, C., Sturmfels, B.: On the toric algebra of graphical models. Ann. Statist. 34 (2006), 1463-1492. DOI 10.1214/009053606000000263 | MR 2278364 | Zbl 1104.60007
[7] Matúš, F.: Maximization of information divergences from binary i.i.d. sequences. In: Proc. IPMU 2 (2004), 1303-1306.
[8] Matúš, F.: Optimality conditions for maximizers of the information divergence from an exponential family. Kybernetika 43 (2007), 731-746. MR 2376334
[9] Matúš, F.: Divergence from factorizable distributions and matroid representations by partitions. IEEE Trans. Inform. Theory 55 (2009), 5375-5381. DOI 10.1109/tit.2009.2032806 | MR 2597169
[10] Matúš, F., Ay, N.: On maximization of the information divergence from an exponential family. In: Proc. WUPES 2003, University of Economics, Prague 2003, pp. 199-204.
[11] Matúš, F., Csiszár, I.: Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities. Kybernetika 48 (2012), 637-689. MR 3013394
[12] Matúš, F., Rauh, J.: Maximization of the information divergence from an exponential family and criticality. In: Proc. IEEE International Symposium on Information Theory (ISIT2011), 2011. DOI 10.1109/isit.2011.6034269 | MR 2817016
[13] Montúfar, G., Rauh, J., Ay, N.: Expressive power and approximation errors of Restricted Boltzmann Machines. In: Proc. NIPS 2011.
[14] Montúfar, G., Rauh, J., Ay, N.: Maximal information divergence from statistical models defined by neural networks. In: Proc. GSI, 2013, pp. 759-766. DOI 10.1007/978-3-642-40020-9\_85
[15] Rauh, J.: Finding the maximizers of the information divergence from an exponential family. IEEE Trans. Inform. Theory 57 (2011), 3236-3247. DOI 10.1109/tit.2011.2136230 | MR 2817016
[16] Rauh, J.: Finding the Maximizers of the Information Divergence from an Exponential Family. Ph.D. Dissertation, Universität Leipzig, 2011. MR 2817016
[17] Rockafellar, R. T.: Convex Analysis. Princeton University Press, 1970. DOI 10.1017/s0013091500010142 | MR 0274683 | Zbl 1011.49013
[18] Wang, N., Rauh, J., Massam, H.: Approximating faces of marginal polytopes in discrete hierarchical models. Ann. Statist. 47 (2019), 1203-1233. DOI 10.1214/18-aos1710 | MR 3911110
Search
Partner of
