Example 2

The forward transition matrix looks like:

$$\mathbf{Q}=\left[ \begin{array}[c]{ccc} 1-p & p & 0 \\ 0 & 1 & 0 \\ 0 & p & 1-p \end{array}\right],$$

with expense schedule $e=[1,0,1]^T$ .

It's easy to show from Kuhn-Tuckor condition that the unconstrained capacity can be achieved when $p^*=[\frac{1-\alpha}{2}\ \alpha \frac{1-\alpha}{2}]^T$ . The unconstrained capacity is

$$C=log{\frac{1/p}{1+\alpha^*\left(\frac{1-p}{p}\right)}}$$ (8)

For example, $p=1/3$ , we solve for

$$\frac{1-\alpha}{1+2\alpha}=2\left(\frac{1}{3}\right)^{3/2}$$ (9)

in which case

$$\alpha^*=\frac{3\sqrt{3}-2}{3\sqrt{3}+4}\approx0.348$$

, and

$$C=log{\frac{3}{1+2\alpha^*}}\approx0.824\ bits/channel\ use$$

.

Now let's consider capacity expense function C(E). Clearly we have $E_{min}=0$ which is achieved when $p_{E_{min}}^*=[0,1,0]^T$ . The corresponding capacity is $C(E_{min})=0$ .

For $E\geq0$ , $p_E^*=[\frac{1-\alpha_E^*}{2},\alpha_E^*,\frac{1-\alpha_E^*}{2}]^T$ , due to channel symmetry. Since $1-\alpha_E^*=E$ , we have

$$p_E^*=[E/2,1-E,E/2]^T$$ (10)

Finally we have

$$C(E)=I(p_E^*;Q)=H(Y)-H(Y\vert X)=E(1-p)+\mathcal{H}(E(1-p))-E\mathcal{H}(p)$$ (11)

For different value of $p$ , we have different $C(E_{max})$ and the corresponding $E_{max}$ . For the example that $p=1/3$ , we have $C(E_{max})\approx0.824$ , when $E_{max}=1-\alpha^*\approx0.652$ .

The case when $p=1/3$ is estimated using Blahut algorithm and is shown in figure 5. Also, cases with different $p$ values are plotted together in figure 6.

Figure 5: Capacity Expense Curve of Example 2 with p=1/3

Figure 6: Capacity Expense Curve of Example 2 with different values of p