Example 1

The forward transition matrix looks like the following:

$$\mathbf{Q} = \left[ \begin{array}{cccc} 1 & 0 & 0 & 0 \\ 0 & q & 1-q & 0 \\ 0 & 0 & 0 & 1 \end{array}\right]$$

As can be observed, $H(X\vert Y)=0$ , i.e., when output is observed, the input can be uniquely determined. This is because that channel can be viewed as three parallel reliable subchannels. As can be expected, changing on parameter $q$ has no effect on the overall channel capacity.

The unconstrained capacity can be calculated as the following:

$$C=\max_{\mathbf{p}} {I(X;Y)}=\max_{\mathbf{p}} {H(X)-H(X\vert Y)},$$ (6)

since $H(X\vert Y)=0$ , eq. 6 can be further written as:

$$C=\max_{\mathbf{p}}{H(X)},$$ (7)

$H(X)$ is maximized when $\mathbf{p}=[1/3\ 1/3\ 1/3]^T$ . And $C=\log_2{3}\approx1.59\ bits/channel\ use$ .

When $e=[0,1,0]^T$ is assigned as expense schedule, $E_{min}$ is achived when $p=[\alpha/2\ 0\ \alpha/2]^T$ , because of channel symetry. $C_{min}=log_2{3}\ bits/channel\ use$ as the channel reduces to BSC in this case.

Naturally, $C_{max}=C_{unconstrained}$ is achieved as $\mathbf{p}=[1/3\ 1/3\ 1/3]^T$ . Therefore $E_{max}=1/3$ in this case.

Figure 4 shows the Blahut algorithm estimation on this channel for expense schedule $e=[0,1,0]^T$ .

Figure 4: Capacity Expense Curve of Channel Described in Example 1. $e=[0,1,0]^T$ .

As shown in the figure, the estimation agrees with theoretical analyse perfectly.