Estimating the asymptotic variance of the OLS estimator

In this post, we show that the sandwich estimator of the asymptotic variance is consistent; i.e. $\hat{Avar} [\hat{β}] \overset{p}{\to} Avar$ , where

\hat{Avar} [\hat{β}] ≜ {\hat{Σ}}_{x}^{- 1} {\hat{Σ}}_{g} {\hat{Σ}}_{x}^{- 1}, \begin{array}{r} {\hat{Σ}}_{x} ≜ \frac{1}{n} \sum_{i = 1}^{n} x_{i} x_{i}^{T}, \\ {\hat{Σ}}_{g} ≜ \frac{1}{n} \sum_{i = 1}^{n} {\hat{ϵ}}_{i}^{2} x_{i} x_{i}^{T} . \end{array}

We shall show that the sandwich estimator is consistent in two steps

show that ${\hat{Σ}}_{x}$ and ${\hat{Σ}}_{g}$ are consistent estimators of $Σ_{x}$ and $Σ_{g}$ respectively
use the continuous mapping theorem (CMT) to conclude the sandwich estimator is consistent.

The consistency of ${\hat{Σ}}_{x}$ is a straightforward consequence of the law of large numbers:

{\hat{Σ}}_{x} = \frac{1}{n} \sum_{i = 1}^{n} x_{i} x_{i}^{T} \overset{p}{\to} E [x_{1} x_{1}^{T}] = Σ_{x} .

The consistency of ${\hat{Σ}}_{g}$ is trickier. Recall ${\hat{ϵ}}_{i} ≜ y_{i} - x_{i}^{T} \hat{β} = ϵ_{i} - x_{i}^{T} (\hat{β} - β_{*})$ . This implies

\begin{aligned} {\hat{Σ}}_{g} & = \frac{1}{n} \sum_{i = 1}^{n} {\hat{ϵ}}_{i}^{2} x_{i} x_{i}^{T} \\ = \underset{I}{\underset{⏟}{\frac{1}{n} \sum_{i = 1}^{n} ϵ_{i}^{2} x_{i} x_{i}^{T}}} - \underset{I I}{\underset{⏟}{\frac{2}{n} \sum_{i = 1}^{n} x_{i} ϵ_{i} x_{i}^{T} (\hat{β} - β_{*}) x_{i}^{T}}} + \underset{I I I}{\underset{⏟}{\frac{1}{n} \sum_{i = 1}^{n} x_{i} (\hat{β} - β_{*})^{T} x_{i} x_{i}^{T} (\hat{β} - β_{*}) x_{i}^{T}}} . \end{aligned}

The first term $I$ converges in probability to $Σ_{g}$ . This is a consequence of the law of large numbers. All the entries of the second term $I I$ converges in probability to zero: the (probability) limit of its $j, k$ -th entry is

\begin{aligned} \frac{2}{n} \sum_{i = 1}^{n} x_{i, j} ϵ_{i} x_{i}^{T} (\hat{β} - β_{*}) x_{i, k} & = \frac{2}{n} \sum_{i = 1}^{n} x_{i, j} ϵ_{i} x_{i, k} x_{i}^{T} (\hat{β} - β_{*})^{T} \\ = (\frac{2}{n} \sum_{i = 1}^{n} x_{i, j} ϵ_{i} x_{i, k} x_{i}^{T}) (\hat{β} - β_{*}) \\ \overset{p}{\to} 2 E [x_{1, j} ϵ_{1} x_{1, k} x_{1}^{T}] \cdot 0 \end{aligned}

Similarly, all the entries of $I I I$ converge to zero: the (probability) limit of its $j, k$ -th entry is

\begin{aligned} \frac{1}{n} \sum_{i = 1}^{n} x_{i, j} (\hat{β} - β_{*})^{T} x_{i} x_{i}^{T} (\hat{β} - β_{*}) x_{i, k} & = \frac{1}{n} \sum_{i = 1}^{n} (\hat{β} - β_{*})^{T} x_{i} x_{i, j} x_{i, k} x_{i}^{T} (\hat{β} - β_{*}) \\ = (\hat{β} - β_{*})^{T} (\frac{1}{n} \sum_{i = 1}^{n} x_{i} x_{i, j} x_{i, k} x_{i}^{T}) (\hat{β} - β_{*}) \\ \overset{p}{\to} E [x_{1} x_{1, j} x_{1, k} x_{1}^{T}] \cdot 0 \end{aligned}

We deduce ${\hat{Σ}}_{g} \overset{p}{\to} Σ_{g}$ . Finally, we use the CMT to conclude the sandwich estimator is consistent: ${\hat{Σ}}_{x}^{- 1} {\hat{Σ}}_{g} {\hat{Σ}}_{x}^{- 1} \overset{p}{\to} Σ_{x}^{- 1} Σ_{x} Σ_{x}^{- 1}$ .

Posted on November 08, 2021 from Ann Arbor, MI