Effect of Measurement Noise

In practice, measurements are never perfect. Let $\hat{\underline{y}}=\underline{y}+\underline{e}$ denote the measured output signal, where $\underline{e}$ is a vector of ``measurement noise'' samples. Then we have

$$\hat{\underline{y}}= \underline{y}+ \underline{e}= \mathbf{x}\underline{h}+ \underline{e}.$$

By the orthogonality principle [38], the least-squares estimate of $\underline{h}$ is obtained by orthogonally projecting $\hat{\underline{y}}$ onto the space spanned by the columns of $\mathbf{x}$. Geometrically speaking, choosing $\underline{h}$ to minimize the Euclidean distance between $\hat{\underline{y}}$ and $\mathbf{x}\underline{h}$ is the same thing as choosing it to minimize the sum of squared estimated measurement errors $\vert\vert\,\underline{e}\,\vert\vert ^2$. The distance from $\mathbf{x}\underline{h}$ to $\hat{\underline{y}}$ is minimized when the projection error $\underline{e}=\hat{\underline{y}}-\mathbf{x}\underline{h}$ is orthogonal to every column of $\mathbf{x}$, which is true if and only if $\mathbf{x}^T\underline{e}=0$ [83]. Thus, we have, applying the orthogonality principle,

$$0 = \mathbf{x}^T\underline{e}= \mathbf{x}^T(\underline{y}- \mathb... ...nderline{h}) = \mathbf{x}^T\underline{y}- \mathbf{x}^T\mathbf{x}\underline{h}.$$

Solving for $\underline{h}$ yields Eq. (5.16) as before, but this time we have derived it as the least squares estimate of $\underline{h}$ in the presence of output measurement error.

It is also straightforward to introduce a weighting function in the least-squares estimate for $\underline{h}$ by replacing $\mathbf{x}^T$ in the derivations above by $\mathbf{x}^TR$, where $R$ is any positive definite matrix (often taking to be diagonal and positive). In the present context, (time-domain formulations), it is difficult to choose a weighting function that corresponds well to audio perception. Therefore, in audio applications, frequency-domain formulations are generally more powerful for linear-time-invariant system identification. A practical example is the frequency-domain equation-error method described in §G.4.3 [75].