Adjusting standard errors for clustering can be a very important part of any statistical analysis. For example, duplicating a data set will reduce the standard errors dramatically despite there being no new information.
I have previously dealt with this topic with reference to the linear regression model. However, in many cases one would like to obtain cluster-robust standard errors for more elaborate statistical analyses than a simple linear regression model. Typically, increases in model complexity are met with similar increases in the calculations required to estimate the model’s uncertainty via analytical means. Adding the required adjustment for group clustering furthers this often burdensome process.
Bootstrapping procedures offer a simple solution to this issue. Essentially, a nonparametric bootstrap procedure estimates a model for a specified number of repetitions using samples of the data frame. The simplest bootstrap draws a random sample from the data frame such that the sample data frame is the same dimension as the main data frame, although the observations are not the same because it allows for random sampling with replacement. For each repetition the main analysis is repeated on the sample data, and the estimate stored (so for a linear regression model this would be the model’s coefficients). Once all repetitions have been computed the standard errors can be calculated by taking the standard deviation of the stored model estimates.
Adapting this technique to tackle clustering is relatively straightforward. Instead of drawing the observation units with replacement, one draws the cluster units with replacement. For example, imagine we have a data frame consisting of 1,000 students from 100 schools. The non-cluster bootstrap draws 1,000 observations from the 1,000 students with replacement for each repetition. The cluster bootstrap will instead draw 100 schools with replacement.
In the below, I show how to formulate a simple cluster bootstrap procedure for a linear regression in R. In this analysis, I simulate some data and then falsely replicate the data frame three times which causes the standard errors to drop. I then show how a simple cluster-robust command solves this issue (i.e. gives standard errors like the original data frame that had not been replicated). Finally, I show how these cluster-robust standard errors can be obtained via bootstrapping. The code is anything but succinct (and therefore probably very inefficient), so I would be happy to take suggestions on how one could improve my clusbootreg() function. Nevertheless, it is clear from the output that this function is giving the right answers.
# cluster-boot
library(lmtest) ; library(sandwich)
rm(list=ls())
# data sim
x <- rnorm(1000)
y <- 5 + 2*x + rnorm(1000)
# regression
summary(lm(y~x))
# triple data
dat <- data.frame(x=c(x,x,x),y=c(y,y,y),g=c(1:1000,1:1000,1:1000))
# regressions
m1 <- lm(y~x, dat) # smaller StErrs
summary(m1)
# cluster robust standard error function
robust.se <- function(model, cluster){
require(sandwich)
require(lmtest)
M <- length(unique(cluster))
N <- length(cluster)
K <- model$rank
dfc <- (M/(M - 1)) * ((N - 1)/(N - K))
uj <- apply(estfun(model), 2, function(x) tapply(x, cluster, sum));
rcse.cov <- dfc * sandwich(model, meat = crossprod(uj)/N)
rcse.se <- coeftest(model, rcse.cov)
return(list(rcse.cov, rcse.se))
}
robust.se(m1,dat$g)[[2]] # StErrs now back to what they are
# cluster bootstrap function
clusbootreg <- function(formula, data, cluster, reps=1000){
reg1 <- lm(formula, data)
clusters <- names(table(cluster))
sterrs <- matrix(NA, nrow=reps, ncol=length(coef(reg1)))
for(i in 1:reps){
index <- sample(1:length(clusters), length(clusters), replace=TRUE)
aa <- clusters[index]
bb <- table(aa)
bootdat <- NULL
for(j in 1:max(bb)){
cc <- data[cluster %in% names(bb[bb %in% j]),]
for(k in 1:j){
bootdat <- rbind(bootdat, cc)
}
}
sterrs[i,] <- coef(lm(formula, bootdat))
}
val <- cbind(coef(reg1),apply(sterrs,2,sd))
colnames(val) <- c("Estimate","Std. Error")
return(val)
}
> # OUTPUT > summary(lm(y~x)) #correct StErrs Call: lm(formula = y ~ x) Residuals: Min 1Q Median 3Q Max -3.10461 -0.63043 0.02924 0.70622 2.78926 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.96345 0.03084 160.97 <2e-16 *** x 2.02747 0.03048 66.51 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.9748 on 998 degrees of freedom Multiple R-squared: 0.8159, Adjusted R-squared: 0.8157 F-statistic: 4424 on 1 and 998 DF, p-value: < 2.2e-16 > summary(lm(y~x, dat)) #smaller StErrs Call: lm(formula = y ~ x, data = dat) Residuals: Min 1Q Median 3Q Max -3.10461 -0.63043 0.02924 0.70622 2.78926 Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.96345 0.01779 279.0 <2e-16 *** x 2.02747 0.01759 115.3 <2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Residual standard error: 0.9742 on 2998 degrees of freedom Multiple R-squared: 0.8159, Adjusted R-squared: 0.8159 F-statistic: 1.329e+04 on 1 and 2998 DF, p-value: < 2.2e-16 > robust.se(m1,dat$g)[[2]] #correct StErrs t test of coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) 4.963447 0.030810 161.099 < 2.2e-16 *** x 2.027469 0.029347 69.086 < 2.2e-16 *** --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 > clusbootreg(formula=y~x, data=dat, cluster=dat$g, reps=1000) #correct StErrs Estimate Std. Error (Intercept) 4.963447 0.03197737 x 2.027469 0.02920071
, using the 
using the all too familiar: 
least squares estimator.![E[\textbf{uu}'|\textbf{X}] = \mathbf{\Sigma_{u}}](https://s0.wp.com/latex.php?latex=E%5B%5Ctextbf%7Buu%7D%27%7C%5Ctextbf%7BX%7D%5D+%3D+%5Cmathbf%7B%5CSigma_%7Bu%7D%7D&bg=ffffff&fg=666666&s=0&c=20201002)
![\textrm{Var}[\hat{\mathbf{\beta}}|\textbf{X}] = (\textbf{X}'\textbf{X})^{-1} (\textbf{X}' \mathbf{\Sigma_{u}} \textbf{X}) (\textbf{X}'\textbf{X})^{-1}](https://s0.wp.com/latex.php?latex=%5Ctextrm%7BVar%7D%5B%5Chat%7B%5Cmathbf%7B%5Cbeta%7D%7D%7C%5Ctextbf%7BX%7D%5D+%3D+%28%5Ctextbf%7BX%7D%27%5Ctextbf%7BX%7D%29%5E%7B-1%7D+%28%5Ctextbf%7BX%7D%27+%5Cmathbf%7B%5CSigma_%7Bu%7D%7D+%5Ctextbf%7BX%7D%29+%28%5Ctextbf%7BX%7D%27%5Ctextbf%7BX%7D%29%5E%7B-1%7D&bg=ffffff&fg=666666&s=0&c=20201002)
![\textrm{Var}[\hat{\mathbf{\beta}}|\textbf{X}] = \sigma_{u}^{2}(\textbf{X}'\textbf{X})^{-1}](https://s0.wp.com/latex.php?latex=%5Ctextrm%7BVar%7D%5B%5Chat%7B%5Cmathbf%7B%5Cbeta%7D%7D%7C%5Ctextbf%7BX%7D%5D+%3D+%5Csigma_%7Bu%7D%5E%7B2%7D%28%5Ctextbf%7BX%7D%27%5Ctextbf%7BX%7D%29%5E%7B-1%7D&bg=ffffff&fg=666666&s=0&c=20201002)
, with the appropriate degrees of freedom adjustment. Code is below. As you can see, these standard errors correspond exactly to those reported using the lm function.
will, for the most part, be different , so the j-th row-column element will be 
. Once again, in R this is trivially implemented.
or ‘meat’ part, that needs to be adjusted for clustering. In practice, this involves multiplying the residuals by the predictors for each cluster separately, and obtaining 
, an m by k matrix (where k is the number of predictors). ‘Squaring’ 
results in a k by k matrix (the meat part). To get the standard errors, one performs the same steps as before, after adjusting the degrees of freedom for clusters.
DiffusePrioR 