Technical Program

Paper Detail

Paper IDS.10.1
Paper Title Continuity of Generalized Entropy
Authors Aolin Xu, ., United States
Session S.10: Information Measures II
Presentation Lecture
Track Shannon Theory
Manuscript  Click here to download the manuscript
Virtual Presentation  Click here to watch in the Virtual Symposium
Abstract We study the continuity property of the generalized entropy as a functional of probability distribution, defined with an action space and a loss function. Upper and lower bounds for the entropy difference of two distributions are derived in terms of several commonly used $f$-divergences, the Wasserstein distance, and a distance that depends on the action space and the loss function. Among these bounds, we notice a connection between the continuity of Shannon/differential entropy in the distribution in KL divergence and the continuity of the R\'enyi entropy in the entropy order. For information-theoretic applications, we derive several new mutual information upper bounds based on the entropy difference bounds. The general results may find broader applications in estimation theory, statistical learning theory, and control theory.

Plan Ahead


2021 IEEE International Symposium on Information Theory

11-16 July 2021 | Melbourne, Victoria, Australia

Visit Website!