Accuracy vs. Correlated Noise

By Peyman Khorsand

Taken from: “The Effect of Correlated Variability on the Accuracy of a Population Code” by L.F. Abbott and P. Dayan

They answer to the following questions for three different class of models: “(1) Does correlation increase or decrease the accuracy with which the value of an encoded quantity can be extracted from a population of $N$ neurons? (2) Does this accuracy approach a fixed limit as $N$ increases?”

The average firing rate is noted as $f_i(s)=\langle r_i(s) \rangle$ . The correlation matrix $Q_{ij}(s)= \langle (r_i (s)- \langle r_i (s)\rangle)(r_j(s) - \langle r_j (s)\rangle ) \rangle$ is used to calculate Fisher information $I_{\mathrm F}$ . The Fisher information can be calculated assuming Gaussian character of correlation

$P[\mathbf {r}|s] = {\cal N} \exp \left[-{1\over 2}\sum_{i,j}(r_i - f_i)Q_{ij}^{-1} (r_j - f_j) \right]$

$I_{\mathrm F}(s)=\sum_{i,j} {d f_{i}\over ds} Q^{-1}_{ij} {d f_{j}\over ds} + {1\over 2}\sum_{i,j,k,l} {d Q_{ij} \over ds} Q_{jk}^{-1} {dQ_{kl} \over ds } Q_{li}^{-1}$
$I_{\mathrm F}(s)= {d \mathbf{f}^{\mathrm T} \over ds} \mathbf{Q}^{-1} {d\mathbf{f} \over ds}+ {1\over 2} \mathbf {Tr} \left[{d\mathbf{Q} \over ds} \mathbf{Q}^{-1}{d \mathbf{Q}\over ds}\mathbf{Q}^{-1} \right]$

The models being studied are:

Additive Noise Model:

$Q_{ij}= \sigma^2 \left[ \delta_{ij} + c(1 - \delta_{ij})\right]$

A nice example of collective quantities $R= {1\over N} \sum r_i$ and $\tilde R={1\over N} \sum (-1)^i r_i$ and the $N$ dependence of their respective variance, $\sigma^2_R$ and $\sigma^2_{\tilde R}$ , is presented.

$\lim _{N\rightarrow \infty} I_{\mathrm F}= {N [F_1(s)- F_{2}(s)]\over \sigma^2 (1-c)}$

where $F_{1}(s)= {1\over N} \sum_i \left(d f_i(s) \over ds\right)^2$ and $F_{2}(s)=\left( {1\over N} \sum_i {d f_i(s) \over ds} \right)^2$ . It worths mentioning that when $F_1=F_2$ the Fisher information fails to grow linearly. This will put a constraint on the individual neurons tuning curves among the population

$F_1=F_2 \Rightarrow f_i(s)=p(s)+q_i$

Multiplicative Noise Model:

$Q_{ij}(s)= \sigma^2 \left[ \delta_{ij} + c(1-\delta_{ij})\right] f_i (s) f_{j}(s)$

$\lim_{N \rightarrow \infty}I_{\mathrm F}= {N [G_1(s)- G_{2}(s)] \over \sigma^2 (1-c)} +{N [(2-c)G_1(s)-c G_{2}(s)]\over (1-c)}$

Limited-Range Correlation Model:

$Q_{ij}=\sigma^2 \rho^{|i-j|}$

$I_{\mathrm F}={N(1-\rho)F_{1}(s)\over\sigma^2 (1+\rho)} +O(N^{1-2/D})$

$D$ is the number of encoded variables. “The main conclusion of this paper is that neurons should have different selectivity to the quantities they are encoding. In particular their tuning curves should not be additively or multiplicatively separable”.

This entry was posted on October 16, 2008 at 7:33 pm and is filed under Information Theory. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

Randomness

Accuracy vs. Correlated Noise

Leave a Reply