Barebones Meta-Analysis of Correlations

Brenton Wiernik

2020-02-19

This vignette will walk you through estimating barebones meta-analyses of correlations between multiple constructs.

Getting Started

To begin, you will need your meta-analytic data sheet for analysis. We recommend the rio package for importing data to R. For an introduction to rio, see vignette("rio", "rio").

psychmeta assumes that your data are in “wide” format, with each row corresponding to one effect size. For example, this is the format used in this data frame:

knitr::kable(data_r_meas_multi[1:10,])
sample_id moderator x_name y_name n rxyi rxxi ryyi citekey
1 1 X Y 416 0.49 0.79 0.77 Watson2005
1 1 X Z 416 0.40 0.79 0.77 Watson2005
1 1 Y Z 416 0.36 0.77 0.77 Watson2005
2 1 X Y 241 0.54 0.82 0.84 Watson2005
2 1 X Z 241 0.56 0.82 0.89 Watson2005
2 1 Y Z 241 0.62 0.84 0.89 Watson2005
3 1 X Y 479 0.34 0.73 0.87 Zellars2006
3 1 X Z 479 0.40 0.73 0.79 Zellars2006
3 1 Y Z 479 0.53 0.87 0.79 Zellars2006
4 1 X Y 167 0.37 0.78 0.79 Bandura1980

In this table, - sample_id contains labels indicating the sample each effect size is drawn from; - moderator is a moderator variable, each row containing the effect size’s level for that moderator; - x_name and y_name are columns indicating the variables/constructs being related in the effect size; - n is the sample size for the effect size; - rxyi is the effect size (the correlation between the two constructs/variables); - rxxi and rxyi are the sample reliability values for the measures of the x_name and y_name variables, respectively; - citekey containts the citations keys for each study (used to generate bibliographies of included studies).

You can see this data set includes correlations among three variables: X, Y, and Z, and that each sample contributes several effect sizes, one each for for different pairs of variables/constructs.

If your data are in a different format, you can use the reshape_wide2long() function to reshape it. See also vignette("pivot", "tidyr").

Estimating a Barebones Meta-Analysis

Let’s assume your data sheet is called coding_sheet.

coding_sheet <- data_r_meas_multi

head(coding_sheet)
#>   sample_id moderator x_name y_name   n rxyi rxxi ryyi    citekey
#> 1         1         1      X      Y 416 0.49 0.79 0.77 Watson2005
#> 2         1         1      X      Z 416 0.40 0.79 0.77 Watson2005
#> 3         1         1      Y      Z 416 0.36 0.77 0.77 Watson2005
#> 4         2         1      X      Y 241 0.54 0.82 0.84 Watson2005
#> 5         2         1      X      Z 241 0.56 0.82 0.89 Watson2005
#> 6         2         1      Y      Z 241 0.62 0.84 0.89 Watson2005

The primary function to conduct meta-analyses of correlations is ma_r(). To conduct barebones meta-analyses, run:

ma_res <- ma_r(rxyi = rxyi, 
               n = n, 
               construct_x = x_name,
               construct_y = y_name,
               sample_id = sample_id, 
               moderators = moderator,
               data = coding_sheet
               )
#>  **** Running ma_r: Meta-analysis of correlations ****

To conduct a barebones meta-analysis, at minimum, n and rxyi are needed.

Modeling Options

The psychmeta Meta-Analysis Object

A psychmeta meta-analsyis object is a data frame, with each row being a meta-analysis or subanalysis and each column containing information about or results from that analysis. For example, the results of the analysis above look like this:

ma_res
#> Overview tibble of psychmeta meta-analysis of correlations  
#> ---------------------------------------------------------------------- 
#> # A tibble: 9 x 8
#>   analysis_id pair_id construct_x construct_y analysis_type    moderator  meta_tables      escalc          
#> *       <int>   <int> <fct>       <fct>       <fct>            <fct>      <named list>     <named list>    
#> 1           1       1 X           Y           Overall          All Levels <named list [3]> <named list [4]>
#> 2           2       1 X           Y           Simple Moderator 1          <named list [3]> <named list [4]>
#> 3           3       1 X           Y           Simple Moderator 2          <named list [3]> <named list [4]>
#> 4           4       2 X           Z           Overall          All Levels <named list [3]> <named list [4]>
#> 5           5       2 X           Z           Simple Moderator 1          <named list [3]> <named list [4]>
#> 6           6       2 X           Z           Simple Moderator 2          <named list [3]> <named list [4]>
#> 7           7       3 Y           Z           Overall          All Levels <named list [3]> <named list [4]>
#> 8           8       3 Y           Z           Simple Moderator 1          <named list [3]> <named list [4]>
#> 9           9       3 Y           Z           Simple Moderator 2          <named list [3]> <named list [4]>
#> 
#> To extract results, try summary() or the get_stuff functions (run ?get_stuff for help).

Each row corresponds to a different pair of variables/constructs (X-Y; X-Z; Y-Z) and level of the moderator variable (overall/all levels pooled together; moderator = 1; moderator = 2).

Viewing Results Summaries

To view meta-anlaysis results tables, use the summary() function:

summary(ma_res)
#> Bare-bones meta-analysis results 
#> ---------------------------------------------------------------------- 
#> # A tibble: 9 x 16
#>   analysis_id pair_id construct_x construct_y analysis_type    moderator      k     N mean_r   sd_r   se_r sd_res CI_LL_95 CI_UL_95 CR_LL_80 CR_UL_80
#>         <int>   <int> <fct>       <fct>       <fct>            <fct>      <dbl> <dbl>  <dbl>  <dbl>  <dbl>  <dbl>    <dbl>    <dbl>    <dbl>    <dbl>
#> 1           1       1 X           Y           Overall          All Levels    40 11927  0.317 0.125  0.0198 0.113     0.277    0.357    0.169    0.465
#> 2           2       1 X           Y           Simple Moderator 1             20  5623  0.397 0.0886 0.0198 0.0729    0.356    0.439    0.300    0.494
#> 3           3       1 X           Y           Simple Moderator 2             20  6304  0.245 0.109  0.0243 0.0948    0.194    0.296    0.119    0.371
#> 4           4       2 X           Z           Overall          All Levels    40 11927  0.324 0.129  0.0204 0.118     0.282    0.365    0.170    0.477
#> 5           5       2 X           Z           Simple Moderator 1             20  5623  0.422 0.0922 0.0206 0.0780    0.379    0.465    0.319    0.526
#> 6           6       2 X           Z           Simple Moderator 2             20  6304  0.236 0.0861 0.0193 0.0677    0.195    0.276    0.146    0.326
#> 7           7       3 Y           Z           Overall          All Levels    40 11927  0.311 0.137  0.0217 0.126     0.268    0.355    0.146    0.476
#> 8           8       3 Y           Z           Simple Moderator 1             20  5623  0.405 0.110  0.0246 0.0979    0.354    0.457    0.275    0.535
#> 9           9       3 Y           Z           Simple Moderator 2             20  6304  0.228 0.0997 0.0223 0.0841    0.181    0.274    0.116    0.339
#> 
#> 
#> Information available in the meta-analysis object includes:
#>  - meta_tables   [ access using get_metatab() ]
#>  - escalc        [ access using get_escalc() ]

In this table,

To view additional results, such as observed variance (var_r) or standard deviation of sampling errors (sd_e), use the get_metatab() function and select the appropriate columns:

names(get_metatab(ma_res))
#>  [1] "analysis_id"   "pair_id"       "construct_x"   "construct_y"   "analysis_type" "moderator"     "k"             "N"             "mean_r"       
#> [10] "var_r"         "var_e"         "var_res"       "sd_r"          "se_r"          "sd_e"          "sd_res"        "CI_LL_95"      "CI_UL_95"     
#> [19] "CR_LL_80"      "CR_UL_80"

get_metatab(ma_res)$var_r
#> [1] 0.015607018 0.007854615 0.011804107 0.016601343 0.008502949 0.007418860 0.018751347 0.012097853 0.009938716

To view all columns of this table, convert it to a data.frame or tibble:

dplyr::as_tibble(get_metatab(ma_res))
#> # A tibble: 9 x 20
#>   analysis_id pair_id construct_x construct_y analysis_type moderator     k     N mean_r   var_r   var_e var_res   sd_r   se_r   sd_e sd_res CI_LL_95 CI_UL_95
#>         <int>   <int> <fct>       <fct>       <fct>         <fct>     <dbl> <dbl>  <dbl>   <dbl>   <dbl>   <dbl>  <dbl>  <dbl>  <dbl>  <dbl>    <dbl>    <dbl>
#> 1           1       1 X           Y           Overall       All Leve…    40 11927  0.317 0.0156  0.00273 0.0129  0.125  0.0198 0.0523 0.113     0.277    0.357
#> 2           2       1 X           Y           Simple Moder… 1            20  5623  0.397 0.00785 0.00255 0.00531 0.0886 0.0198 0.0504 0.0729    0.356    0.439
#> 3           3       1 X           Y           Simple Moder… 2            20  6304  0.245 0.0118  0.00281 0.00899 0.109  0.0243 0.0530 0.0948    0.194    0.296
#> 4           4       2 X           Z           Overall       All Leve…    40 11927  0.324 0.0166  0.00270 0.0139  0.129  0.0204 0.0520 0.118     0.282    0.365
#> 5           5       2 X           Z           Simple Moder… 1            20  5623  0.422 0.00850 0.00242 0.00608 0.0922 0.0206 0.0492 0.0780    0.379    0.465
#> 6           6       2 X           Z           Simple Moder… 2            20  6304  0.236 0.00742 0.00284 0.00458 0.0861 0.0193 0.0533 0.0677    0.195    0.276
#> 7           7       3 Y           Z           Overall       All Leve…    40 11927  0.311 0.0188  0.00275 0.0160  0.137  0.0217 0.0525 0.126     0.268    0.355
#> 8           8       3 Y           Z           Simple Moder… 1            20  5623  0.405 0.0121  0.00251 0.00959 0.110  0.0246 0.0501 0.0979    0.354    0.457
#> 9           9       3 Y           Z           Simple Moder… 2            20  6304  0.228 0.00994 0.00286 0.00708 0.0997 0.0223 0.0535 0.0841    0.181    0.274
#> # … with 2 more variables: CR_LL_80 <dbl>, CR_UL_80 <dbl>
as.data.frame(get_metatab(ma_res))
#>   analysis_id pair_id construct_x construct_y    analysis_type  moderator  k     N    mean_r       var_r       var_e     var_res       sd_r       se_r
#> 1           1       1           X           Y          Overall All Levels 40 11927 0.3166643 0.015607018 0.002731165 0.012875853 0.12492805 0.01975286
#> 2           2       1           X           Y Simple Moderator          1 20  5623 0.3971059 0.007854615 0.002545026 0.005309589 0.08862626 0.01981744
#> 3           3       1           X           Y Simple Moderator          2 20  6304 0.2449125 0.011804107 0.002813166 0.008990941 0.10864671 0.02429414
#> 4           4       2           X           Z          Overall All Levels 40 11927 0.3235671 0.016601343 0.002704401 0.013896942 0.12884620 0.02037237
#> 5           5       2           X           Z Simple Moderator          1 20  5623 0.4220597 0.008502949 0.002422999 0.006079950 0.09221143 0.02061910
#> 6           6       2           X           Z Simple Moderator          2 20  6304 0.2357143 0.007418860 0.002839689 0.004579171 0.08613281 0.01925988
#> 7           7       3           Y           Z          Overall All Levels 40 11927 0.3113684 0.018751347 0.002751395 0.015999952 0.13693556 0.02165141
#> 8           8       3           Y           Z Simple Moderator          1 20  5623 0.4053224 0.012097853 0.002505340 0.009592513 0.10999024 0.02459457
#> 9           9       3           Y           Z Simple Moderator          2 20  6304 0.2275640 0.009938716 0.002862440 0.007076275 0.09969311 0.02229206
#>         sd_e     sd_res  CI_LL_95  CI_UL_95  CR_LL_80  CR_UL_80
#> 1 0.05226055 0.11347182 0.2767104 0.3566182 0.1687380 0.4645905
#> 2 0.05044825 0.07286692 0.3556276 0.4385843 0.3003585 0.4938534
#> 3 0.05303929 0.09482057 0.1940642 0.2957607 0.1190165 0.3708084
#> 4 0.05200385 0.11788529 0.2823600 0.3647741 0.1698872 0.4772469
#> 5 0.04922396 0.07797403 0.3789034 0.4652160 0.3185314 0.5255880
#> 6 0.05328873 0.06766957 0.1954029 0.2760257 0.1458675 0.3255611
#> 7 0.05245374 0.12649092 0.2675743 0.3551626 0.1464700 0.4762669
#> 8 0.05005337 0.09794137 0.3538453 0.4567994 0.2752828 0.5353619
#> 9 0.05350178 0.08412060 0.1809062 0.2742218 0.1158747 0.3392533

Moderator Analyses

Results for subgroup analyses for different levels of categorical moderators are shown in the rows of the meta-analysis results table. To estimate confidence intervals for differences between levels or an omnibus ANOVA statistic, use the anova() function:

anova(ma_res)
#> # A tibble: 3 x 14
#>   pair_id construct_x construct_y moderator `F value` df_num df_denom level_1 level_2 mean_1 mean_2  diff CI_LL_95 CI_UL_95
#>     <int> <fct>       <fct>       <chr>         <dbl>  <dbl>    <dbl> <chr>   <chr>    <dbl>  <dbl> <dbl>    <dbl>    <dbl>
#> 1       1 X           Y           moderator      23.6      1     36.5 1       2        0.397  0.245 0.152   0.0887    0.216
#> 2       2 X           Z           moderator      43.6      1     37.8 1       2        0.422  0.236 0.186   0.129     0.243
#> 3       3 Y           Z           moderator      28.7      1     37.6 1       2        0.405  0.228 0.178   0.111     0.245

Outputting Results

To output the main meta-analysis results table to RMarkdown, Word, HTML, PDF, or other formats, use the metabulate() function. For example, to output the above results to a Word document, run:

metabulate(ma_res, file = "meta-analysis_results.docx", output_format = "word")

Follow-Up Analyses

Plotting

You can add plots for each meta-analysis in ma_res using the plot_forest() and plot_funnel() functions:

ma_res <- plot_funnel(ma_res)
#> Funnel plots have been added to 'ma_obj' - use get_plots() to retrieve them.
ma_res <- plot_forest(ma_res)
#> Forest plots have been added to 'ma_obj' - use get_plots() to retrieve them.

You can view these plots using the get_plots() function. This will return a list of all of the plots in this results. Specify which meta-analysis you want to view plots for by passing its analysis_id to [[:

get_plots(ma_res)[["forest"]][[2]]
#> $moderated
#> NULL
#> 
#> $unmoderated
#> $unmoderated$barebones
#> 
#> $unmoderated$individual_correction
#> NULL
get_plots(ma_res)[["funnel"]][[2]]
#> $barebones
#> 
#> $individual_correction
#> list()
#> 
#> $artifact_distribution
#> list()

For forest plots, if you select an “Overall” meta-analysis, it will include plots faceted by moderator levels ("moderated") and not ("unmoderated"):

get_plots(ma_res)[["forest"]][[1]][["moderated"]][["barebones"]]
get_plots(ma_res)[["forest"]][[1]][["unmoderated"]][["barebones"]]

Heterogeneity Analyses

psychmeta reports the random-effects standard deviaton (τ or SD_res_) and credibility intervals (mean_r ± crit × SDres) in the main meta-analaysis results tables. To view confidence intervals for SD_res_ or additional heterogeneity statistics, use the heterogeneity() function:

ma_res <- heterogeneity(ma_res)
#> Heterogeneity analyses have been added to 'ma_obj' - use get_heterogeneity() to retrieve them.
get_heterogeneity(ma_res)[[1]][["barebones"]]
#> 
#> Heterogeneity results for r 
#> ---------------------------
#> 
#> Accounted for a total of 17.500% of variance
#> 
#> Correlation between r values and artifactual perturbations: 0.418
#> 
#> The reliability of observed effect sizes is: 0.825
#> 
#> 
#> Random effects variance estimates
#> ---------------------------------
#> Hunter-Schmidt method:
#>   sd_res  (tau):   0.113, SE = 0.016, 95% CI = [0.088, 0.152] 
#>   var_res (tau^2): 0.013, SE = 0.004, 95% CI = [0.008, 0.023] 
#> 
#>   Q statistic: 222.862 (df = 39, p = 0.000) 
#>   H: 2.390   H^2: 5.714   I^2: 82.500
#> 
#> DerSimonian-Laird method:
#>   sd_res  (tau):   0.114
#>   var_res (tau^2): 0.013
#> 
#>   Q statistic: 223.408
#>   H: 2.393   H^2: 5.728   I^2: 82.543
#> 
#> Outlier-robust method (absolute deviation from mean):
#>   sd_res  (tau_r):   0.121
#>   var_res (tau_r^2): 0.015
#> 
#>   Q_r statistic: 78.734
#>   H_r: 2.498   H_r^2: 6.242   I_r^2: 0.840
#> 
#> Outlier-robust method (absolute deviation from median):
#>   sd_res  (tau_m):   0.118
#>   var_res (tau_m^2): 0.014
#> 
#>   Q_m statistic: 77.813
#>   H_m: 2.438   H_m^2: 5.944   I_m^2: 0.832

Publication Bias and Sensitivity Analyses

psychmeta supports cumulative meta-analysis for publication/small-sample bias detection, leave-1-out sensitivity analyses, and bootstrap confidence interavals using the sensitivity function:

ma_res <- sensitivity(ma_res)
#>  **** Computing sensitivity analyses ****
#> Bootstrapped meta-analyses have been added to 'ma_obj' - use get_bootstrap() to retrieve them.
#> leave-1-out meta-analyses have been added to 'ma_obj' - use get_leave1out() to retrieve them.
#> Cumulative meta-analyses have been added to 'ma_obj' - use get_cumulative() to retrieve them.
get_cumulative(ma_res)[[1]][["barebones"]]
#> Cumulative meta-analysis results 
#> ---------------------------------------- 
#>    study_added  k     N mean_r    var_r   var_e   var_res   sd_r   se_r   sd_e sd_res CI_LL_95 CI_UL_95 CR_LL_80 CR_UL_80
#> 1           28  1   487  0.300       NA 0.00170        NA     NA 0.0413 0.0413     NA   0.2194    0.381       NA       NA
#> 2            3  2   966  0.320 0.000802 0.00167 -0.000869 0.0283 0.0200 0.0409 0.0000   0.0658    0.575    0.320    0.320
#> 3           31  3  1407  0.270 0.008675 0.00184  0.006838 0.0931 0.0538 0.0429 0.0827   0.0387    0.501    0.114    0.426
#> 4           33  4  1841  0.268 0.005916 0.00188  0.004039 0.0769 0.0385 0.0433 0.0636   0.1453    0.390    0.164    0.372
#> 5           14  5  2273  0.308 0.013208 0.00181  0.011402 0.1149 0.0514 0.0425 0.1068   0.1655    0.451    0.144    0.472
#> 6            1  6  2689  0.336 0.015939 0.00176  0.014180 0.1263 0.0515 0.0419 0.1191   0.2039    0.469    0.161    0.512
#> 7           17  7  3086  0.342 0.013757 0.00177  0.011985 0.1173 0.0443 0.0421 0.1095   0.2336    0.451    0.184    0.500
#> 8           38  8  3461  0.338 0.012208 0.00182  0.010389 0.1105 0.0391 0.0426 0.1019   0.2452    0.430    0.193    0.482
#> 9            6  9  3818  0.338 0.010894 0.00185  0.009040 0.1044 0.0348 0.0431 0.0951   0.2576    0.418    0.205    0.471
#> 10          32 10  4165  0.326 0.011469 0.00192  0.009547 0.1071 0.0339 0.0438 0.0977   0.2498    0.403    0.191    0.462
#> 11          27 11  4510  0.331 0.010806 0.00194  0.008868 0.1040 0.0313 0.0440 0.0942   0.2614    0.401    0.202    0.460
#> 12          37 12  4855  0.317 0.012867 0.00200  0.010862 0.1134 0.0327 0.0448 0.1042   0.2449    0.389    0.175    0.459
#> 13          21 13  5197  0.317 0.011939 0.00203  0.009909 0.1093 0.0303 0.0451 0.0995   0.2505    0.383    0.182    0.452
#> 14          20 14  5536  0.321 0.011481 0.00204  0.009442 0.1072 0.0286 0.0452 0.0972   0.2592    0.383    0.190    0.452
#> 15          24 15  5872  0.324 0.010973 0.00205  0.008923 0.1048 0.0270 0.0453 0.0945   0.2665    0.383    0.197    0.452
#> 16          30 16  6195  0.309 0.015242 0.00212  0.013122 0.1235 0.0309 0.0460 0.1146   0.2428    0.374    0.155    0.462
#> 17          36 17  6515  0.301 0.015529 0.00216  0.013366 0.1246 0.0302 0.0465 0.1156   0.2373    0.365    0.147    0.456
#> 18          23 18  6828  0.306 0.015222 0.00217  0.013051 0.1234 0.0291 0.0466 0.1142   0.2445    0.367    0.154    0.458
#> 19          22 19  7138  0.300 0.015323 0.00221  0.013113 0.1238 0.0284 0.0470 0.1145   0.2403    0.360    0.148    0.452
#> 20          18 20  7441  0.308 0.016155 0.00221  0.013947 0.1271 0.0284 0.0470 0.1181   0.2483    0.367    0.151    0.465
#> 21          11 21  7739  0.313 0.016082 0.00222  0.013867 0.1268 0.0277 0.0471 0.1178   0.2548    0.370    0.156    0.469
#> 22          19 22  8030  0.316 0.015749 0.00223  0.013522 0.1255 0.0268 0.0472 0.1163   0.2601    0.371    0.162    0.470
#> 23          12 23  8315  0.319 0.015428 0.00224  0.013188 0.1242 0.0259 0.0473 0.1148   0.2649    0.372    0.167    0.470
#> 24          40 24  8586  0.315 0.015286 0.00227  0.013012 0.1236 0.0252 0.0477 0.1141   0.2630    0.367    0.165    0.466
#> 25          34 25  8854  0.313 0.014968 0.00230  0.012664 0.1223 0.0245 0.0480 0.1125   0.2624    0.363    0.165    0.461
#> 26          16 26  9110  0.318 0.015639 0.00231  0.013328 0.1251 0.0245 0.0481 0.1154   0.2680    0.369    0.167    0.470
#> 27           5 27  9365  0.320 0.015334 0.00233  0.013006 0.1238 0.0238 0.0483 0.1140   0.2715    0.369    0.171    0.470
#> 28           9 28  9619  0.318 0.015125 0.00236  0.012766 0.1230 0.0232 0.0486 0.1130   0.2704    0.366    0.170    0.467
#> 29          35 29  9869  0.318 0.014723 0.00238  0.012342 0.1213 0.0225 0.0488 0.1111   0.2720    0.364    0.172    0.464
#> 30          39 30 10116  0.320 0.014558 0.00240  0.012163 0.1207 0.0220 0.0489 0.1103   0.2754    0.365    0.176    0.465
#> 31          10 31 10360  0.322 0.014260 0.00241  0.011847 0.1194 0.0214 0.0491 0.1088   0.2778    0.365    0.179    0.464
#> 32           2 32 10601  0.327 0.015026 0.00242  0.012610 0.1226 0.0217 0.0492 0.1123   0.2824    0.371    0.180    0.474
#> 33          13 33 10838  0.328 0.014805 0.00243  0.012373 0.1217 0.0212 0.0493 0.1112   0.2851    0.371    0.183    0.474
#> 34          25 34 11075  0.323 0.015912 0.00247  0.013440 0.1261 0.0216 0.0497 0.1159   0.2787    0.367    0.171    0.474
#> 35           8 35 11288  0.321 0.015838 0.00250  0.013334 0.1258 0.0213 0.0500 0.1155   0.2773    0.364    0.170    0.471
#> 36          29 36 11471  0.319 0.015734 0.00254  0.013193 0.1254 0.0209 0.0504 0.1149   0.2765    0.361    0.169    0.469
#> 37           4 37 11638  0.320 0.015536 0.00257  0.012964 0.1246 0.0205 0.0507 0.1139   0.2782    0.361    0.171    0.468
#> 38          15 38 11786  0.318 0.015541 0.00261  0.012928 0.1247 0.0202 0.0511 0.1137   0.2771    0.359    0.170    0.466
#> 39          26 39 11916  0.317 0.015602 0.00266  0.012942 0.1249 0.0200 0.0516 0.1138   0.2760    0.357    0.168    0.465
#> 40           7 40 11927  0.317 0.015607 0.00273  0.012876 0.1249 0.0198 0.0523 0.1135   0.2767    0.357    0.169    0.465
#> 
#> See the 'plots' list for data visualizations.
get_cumulative(ma_res)[[1]][["barebones"]][["plots"]]
#> $mean_plot
#> Warning: Removed 1 rows containing missing values (geom_pointrange).
#> Warning: Removed 1 rows containing missing values (geom_point).

#> Warning: Removed 1 rows containing missing values (geom_point).
#> 
#> $sd_plot
#> Warning: Removed 1 rows containing missing values (geom_point).
#> Warning: Removed 1 rows containing missing values (geom_path).

get_bootstrap(ma_res)[[1]][["barebones"]]
#> Bootstrapped meta-analysis results 
#> ---------------------------------------- 
#>          boot_mean boot_var CI_LL_95 CI_UL_95
#> k         4.00e+01 0.00e+00 4.00e+01 4.00e+01
#> N         1.19e+04 3.23e+05 1.08e+04 1.30e+04
#> mean_r    3.17e-01 3.95e-04 2.78e-01 3.55e-01
#> var_r     1.51e-02 9.26e-06 1.10e-02 2.38e-02
#> var_e     2.73e-03 2.52e-08 2.47e-03 3.12e-03
#> var_res   1.24e-02 9.05e-06 8.35e-03 2.13e-02
#> sd_r      1.22e-01 1.55e-04 1.05e-01 1.54e-01
#> se_r      1.93e-02 3.86e-06 1.66e-02 2.44e-02
#> sd_e      5.22e-02 2.29e-06 4.97e-02 5.58e-02
#> sd_res    1.10e-01 1.86e-04 9.13e-02 1.46e-01
#> CI_LL_95  2.78e-01 4.59e-04 2.34e-01 3.17e-01
#> CI_UL_95  3.56e-01 3.62e-04 3.20e-01 3.93e-01
#> CR_LL_80  1.73e-01 9.11e-04 1.05e-01 2.23e-01
#> CR_UL_80  4.61e-01 5.11e-04 4.25e-01 5.14e-01
#> 
#> See list item 'boot_data' for meta-analysis results from each bootstrap iteration