on Information Theory

Paper ID | G.5.3 | ||

Paper Title | Structure of Optimal Quantizer for Binary-Input Continuous-Output Channels with Output Constraints |
||

Authors | Thuan Nguyen, Thinh Nguyen, Oregon State University, United States | ||

Session | G.5: Signal Processing | ||

Presentation | Lecture | ||

Track | Graphs, Games, Sparsity, and Signal Processing |
||

Manuscript | Click here to download the manuscript | ||

Virtual Presentation | Click here to watch in the Virtual Symposium | ||

Abstract | In this paper, we consider a channel whose the input is a binary random source $X \in \{x_1,x_2\}$ with the probability mass function (pmf) $\textbf{p}_X = [p_{x_1}, p_{x_2}]$ and the output is a continuous random variable $Y \in \mathbb{R}$ as a result of a continuous noise, characterized by the channel conditional densities $p_{y|x_1}=\phi_1(y)$ and $p_{y|x_2}=\phi_2(y)$. A quantizer $Q$ is used to map $Y$ back to a discrete set $Z \in \{z_1, z_2, \dots, z_N\}$. To retain most amount of information about $X$, an optimal $Q$ is one that maximizes $I(X;Z)$. On the other hand, our goal is not only to recover $X$ but also ensure that $\textbf{p}_Z = [p_{z_1}, p_{z_2}, \dots, p_{z_N}]$ satisfies a certain constraint. In particular, we are interested in designing a quantizer that maximizes $\beta I(X;Z) - C(\textbf{p}_Z)$ where $\beta$ is a trade-off parameter and $C(\textbf{p}_Z)$ is an arbitrary cost function of $\textbf{p}_Z$. Let the posterior probability $p_{x_1|y} = r_y=\dfrac{p_{x_1} \phi_1(y)}{p_{x_1} \phi_1(y) + p_{x_2} \phi_2(y)}$, our result shows that the structure of the optimal quantizer separates $r_y$ into convex cells. In other words, the optimal quantizer has the form: $Q^*(r_y)= z_i, \text{ if } a^*_{i-1} \leq r_y < a^*_i,$ for some optimal thresholds $a^*_0=0 < a^*_1 < a^*_2< \dots < a^*_{N-1} < a^*_N = 1.$ Based on this optimal structure, we describe some fast algorithms for determining the optimal quantizers. |

Plan Ahead

2021 IEEE International Symposium on Information Theory

**11-16 July 2021 ** | **Melbourne, Victoria, Australia**