Robust Estimation of Standard Errors, Confidence Intervals and p-values

The tab_model() function also allows the computation of standard errors, confidence intervals and p-values based on robust covariance matrix estimation from model parameters. Robust estimation is based on the packages sandwich and clubSandwich, so all models supported by either of these packages work with tab_model().

Classical Regression Models

Robust Covariance Matrix Estimation from Model Parameters

There are three arguments that allow for choosing different methods and options of robust estimation: vcov.fun, vcov.type and vcov.args. Let us start with a simple example, which uses a heteroskedasticity-consistent covariance matrix estimation with estimation-type “HC3” (i.e. sandwich::vcovHC(type = "HC3") is called):

  Petal Length
Predictors Estimates std. Error CI p
(Intercept) 0.87 0.45 -0.03 – 1.76 0.059
Sepal.Length 0.04 0.12 -0.19 – 0.28 0.711
Species [versicolor] -0.78 0.69 -2.15 – 0.59 0.265
Species [virginica] -0.41 0.63 -1.66 – 0.83 0.513
Sepal.Width 0.11 0.08 -0.05 – 0.27 0.190
Sepal.Length * Species
[versicolor]
0.61 0.13 0.35 – 0.87 <0.001
Sepal.Length * Species
[virginica]
0.68 0.12 0.45 – 0.91 <0.001
Observations 150
R2 / R2 adjusted 0.979 / 0.978

Cluster-Robust Covariance Matrix Estimation (sandwich)

If another covariance matrix estimation is required, use the vcov.fun-argument. This argument needs the suffix for the related vcov*()-functions as value, i.e. vcov.fun = "CL" would call sandwich::vcovCL(), or vcov.fun = "HAC" would call sandwich::vcovHAC().

The specific estimation type can be changed with vcov.type. E.g., sandwich::vcovCL() accepts estimation types HC0 to HC3. In the next example, we use a clustered covariance matrix estimation with HC1-estimation type.

  Petal Length
Predictors Estimates std. Error CI p
(Intercept) 0.87 0.42 0.03 – 1.70 0.042
Sepal.Length 0.04 0.11 -0.18 – 0.26 0.692
Species [versicolor] -0.78 0.65 -2.07 – 0.51 0.237
Species [virginica] -0.41 0.59 -1.57 – 0.75 0.483
Sepal.Width 0.11 0.08 -0.05 – 0.27 0.170
Sepal.Length * Species
[versicolor]
0.61 0.12 0.37 – 0.85 <0.001
Sepal.Length * Species
[virginica]
0.68 0.11 0.46 – 0.90 <0.001
Observations 150
R2 / R2 adjusted 0.979 / 0.978

Usually, clustered covariance matrix estimation is used when there is a cluster-structure in the data. The variable indicating the cluster-structure can be defined in sandwich::vcovCL() with the cluster-argument. In tab_model(), additional arguments that should be passed down to functions from the sandwich package can be specified in vcov.args:

  Petal Length
Predictors Estimates std. Error CI p
(Intercept) 0.87 0.34 0.20 – 1.53 0.011
Sepal.Length 0.04 0.07 -0.10 – 0.19 0.540
Species [versicolor] -0.78 0.52 -1.80 – 0.25 0.137
Species [virginica] -0.41 0.26 -0.94 – 0.11 0.120
Sepal.Width 0.11 0.07 -0.03 – 0.25 0.131
Sepal.Length * Species
[versicolor]
0.61 0.10 0.42 – 0.80 <0.001
Sepal.Length * Species
[virginica]
0.68 0.05 0.58 – 0.78 <0.001
Observations 150
R2 / R2 adjusted 0.979 / 0.978

Cluster-Robust Covariance Matrix Estimation (clubSandwich)

Cluster-robust estimation of the variance-covariance matrix can also be achieved using clubSandwich::vcovCR(). Thus, when vcov.fun = "CR", the related function from the clubSandwich package is called. Note that this function requires the specification of the cluster-argument.

  Petal Length
Predictors Estimates std. Error CI p
(Intercept) 0.87 0.33 0.21 – 1.52 0.010
Sepal.Length 0.04 0.07 -0.10 – 0.18 0.531
Species [versicolor] -0.78 0.51 -1.78 – 0.23 0.129
Species [virginica] -0.41 0.26 -0.92 – 0.10 0.112
Sepal.Width 0.11 0.07 -0.03 – 0.25 0.123
Sepal.Length * Species
[versicolor]
0.61 0.09 0.42 – 0.79 <0.001
Sepal.Length * Species
[virginica]
0.68 0.05 0.58 – 0.78 <0.001
Observations 150
R2 / R2 adjusted 0.979 / 0.978

Robust Covariance Matrix Estimation on Standardized Model Parameters

Finally, robust estimation can be combined with standardization. However, robust covariance matrix estimation only works for show.std = "std".

  Petal Length
Predictors Estimates std. Beta CI standardized CI p std. p
(Intercept) 0.87 -1.30 -0.03 – 1.76 -1.44 – -1.16 0.059 <0.001
Sepal.Length 0.04 0.02 -0.19 – 0.28 -0.09 – 0.13 0.711 0.711
Species [versicolor] -0.78 1.57 -2.15 – 0.59 1.40 – 1.74 0.265 <0.001
Species [virginica] -0.41 2.02 -1.66 – 0.83 1.84 – 2.20 0.513 <0.001
Sepal.Width 0.11 0.03 -0.05 – 0.27 -0.01 – 0.07 0.190 0.190
Sepal.Length * Species
[versicolor]
0.61 0.28 0.35 – 0.87 0.16 – 0.41 <0.001 <0.001
Sepal.Length * Species
[virginica]
0.68 0.32 0.45 – 0.91 0.21 – 0.43 <0.001 <0.001
Observations 150
R2 / R2 adjusted 0.979 / 0.978

Mixed Models

Robust Covariance Matrix Estimation for Mixed Models

For linear mixed models, that by definition have a clustered (“hierarchical” or multilevel) structure in the data, it is also possible to estimate a cluster-robust covariance matrix. This is possible due to the clubSandwich package, thus we need to define the same arguments as in the above example.

  Sepal Length
Predictors Estimates CI p
(Intercept) 1.55 0.77 – 2.34 <0.001
Species [versicolor] 0.41 -0.66 – 1.49 0.453
Species [virginica] -0.41 -1.55 – 0.73 0.482
Sepal.Width 0.66 0.44 – 0.88 <0.001
Petal.Length 0.82 0.69 – 0.95 <0.001
Species [versicolor] *
Sepal.Width
-0.48 -0.85 – -0.12 0.009
Species [virginica] *
Sepal.Width
-0.36 -0.71 – -0.01 0.046
Random Effects
σ2 0.09
τ00 grp 0.01
ICC 0.07
N grp 3
Observations 150
Marginal R2 / Conditional R2 0.860 / 0.870
  Sepal Length
Predictors Estimates CI p
(Intercept) 1.55 0.76 – 2.35 <0.001
Species [versicolor] 0.41 -1.17 – 1.99 0.608
Species [virginica] -0.41 -0.78 – -0.03 0.033
Sepal.Width 0.66 0.46 – 0.86 <0.001
Petal.Length 0.82 0.72 – 0.91 <0.001
Species [versicolor] *
Sepal.Width
-0.48 -1.18 – 0.21 0.172
Species [virginica] *
Sepal.Width
-0.36 -0.57 – -0.15 0.001
Random Effects
σ2 0.09
τ00 grp 0.01
ICC 0.07
N grp 3
Observations 150
Marginal R2 / Conditional R2 0.860 / 0.870

Robust Covariance Matrix Estimation on Standardized Mixed Model Parameters

Again, robust estimation can be combined with standardization for linear mixed models as well, which in such cases also only works for show.std = "std".

  Sepal Length
Predictors Estimates std. Beta CI standardized CI p std. p
(Intercept) 1.55 0.97 0.76 – 2.35 0.82 – 1.12 <0.001 <0.001
Species [versicolor] 0.41 -1.29 -1.17 – 1.99 -1.95 – -0.63 0.608 <0.001
Species [virginica] -0.41 -1.81 -0.78 – -0.03 -2.26 – -1.37 0.033 <0.001
Sepal.Width 0.66 0.35 0.46 – 0.86 0.24 – 0.45 <0.001 <0.001
Petal.Length 0.82 1.74 0.72 – 0.91 1.54 – 1.94 <0.001 <0.001
Species [versicolor] *
Sepal.Width
-0.48 -0.25 -1.18 – 0.21 -0.62 – 0.11 0.172 0.172
Species [virginica] *
Sepal.Width
-0.36 -0.19 -0.57 – -0.15 -0.30 – -0.08 0.001 0.001
Random Effects
σ2 0.09
τ00 grp 0.01
ICC 0.07
N grp 3
Observations 150
Marginal R2 / Conditional R2 0.860 / 0.870