2020 IEEE International Symposium
on Information Theory
21-26 June 2020 • Los Angeles, California, USA
| Paper ID | S.10.1 | ||
| Paper Title | Continuity of Generalized Entropy | ||
| Authors | Aolin Xu, ., United States | ||
| Session | S.10: Information Measures II | ||
| Presentation | Lecture | ||
| Track | Shannon Theory | ||
| Manuscript | Click here to download the manuscript | ||
| Virtual Presentation | Click here to watch in the Virtual Symposium | ||
| Abstract | We study the continuity property of the generalized entropy as a functional of probability distribution, defined with an action space and a loss function. Upper and lower bounds for the entropy difference of two distributions are derived in terms of several commonly used $f$-divergences, the Wasserstein distance, and a distance that depends on the action space and the loss function. Among these bounds, we notice a connection between the continuity of Shannon/differential entropy in the distribution in KL divergence and the continuity of the R\'enyi entropy in the entropy order. For information-theoretic applications, we derive several new mutual information upper bounds based on the entropy difference bounds. The general results may find broader applications in estimation theory, statistical learning theory, and control theory. | ||
Plan Ahead
2021 IEEE International Symposium on Information Theory
11-16 July 2021 | Melbourne, Victoria, Australia
