How to simulate experiments

2021-09-24

Simulating data is a good way to test an experimental design prior to running a costly experiment. The isotracer package provides some basic functionality to simulate data for a network model in which the true parameter values are given by the user.

In this vignette, you will learn:

By repeating those basic steps, one can test different assumptions on the real system under study and different experimental designs to decide on the most cost-effective approach for the real experiment to be run.

library(isotracer)
library(tidyverse)

Creating a network model

In this vignette, we will use the same example as in the Including Fixed Effects of Covariates tutorial. The modelled foodweb has three compartments:

The experiment is done in two aquariums, with one aquarium exposed to light while the other is kept in the dark.

Building the network model

The first step is to build the network model structure. This is done in exactly the same way as in the previous vignettes: we have to specify the network topology, some initial values, and potentially some covariates.

Let's start with the topology:

mod <- new_networkModel() %>%
    set_topo("NH4 -> algae -> daphnia -> NH4")
## Using default distribution family for proportions ("gamma_cv").
##   (eta is the coefficient of variation of gamma distributions.)
## Using default distribution family for sizes ("normal_cv").
##   (zeta is the coefficient of variation of normal distributions.)

We prepare a table of initial values which could be used in the real-life experiment we want to prepare:

inits <- tibble::tribble(
     ~comps, ~sizes, ~props, ~treatment,
      "NH4",    0.2,    0.8,    "light",
    "algae",      1,  0.004,    "light",
  "daphnia",      2,  0.004,    "light",
      "NH4",    0.5,    0.8,     "dark",
    "algae",    1.2,  0.004,     "dark",
  "daphnia",    1.3,  0.004,     "dark")

We had the initial values to the model, and we indicate that we want to group initial values by "treatment":

mod <- set_init(mod, inits, comp = "comps", size = "sizes",
                prop = "props", group_by = "treatment")
mod
## # A tibble: 2 × 5
##   topology           initial          observations parameters       group    
##   <list>             <list>           <list>       <list>           <list>   
## 1 <topology [3 × 3]> <tibble [3 × 3]> <NULL>       <tibble [8 × 2]> <chr [1]>
## 2 <topology [3 × 3]> <tibble [3 × 3]> <NULL>       <tibble [8 × 2]> <chr [1]>

Setting parameter values

We have the basic model ready to be given some "true" parameter values. What are the parameters we have to specify?

params(mod)
## [1] "eta"                      "lambda_algae"             "lambda_daphnia"          
## [4] "lambda_NH4"               "upsilon_algae_to_daphnia" "upsilon_daphnia_to_NH4"  
## [7] "upsilon_NH4_to_algae"     "zeta"

Let's say that we want to simulate an effect of "treatment" (light/dark) on the uptake of NH4 by the algae:

mod <- add_covariates(mod, upsilon_NH4_to_algae ~ treatment)

Now we have more parameters to specify:

params(mod)
## [1] "eta"                        "lambda_algae"              
## [3] "lambda_daphnia"             "lambda_NH4"                
## [5] "upsilon_algae_to_daphnia"   "upsilon_daphnia_to_NH4"    
## [7] "upsilon_NH4_to_algae|dark"  "upsilon_NH4_to_algae|light"
## [9] "zeta"

We can set the parameter values with the set_params() function:

mod <- mod %>%
  set_params(c("eta" = 0.2, "lambda_algae" = 0, "lambda_daphnia" = 0,
               "lambda_NH4" = 0, "upsilon_NH4_to_algae|light" = 0.3,
               "upsilon_NH4_to_algae|dark" = 0.1,
               "upsilon_algae_to_daphnia" = 0.13,
               "upsilon_daphnia_to_NH4" = 0.045, "zeta" = 0.1))

Once the parameter values are stored in the network model, they are visible in the parameters column:

mod$parameters
## [[1]]
## # A tibble: 8 × 3
##   in_replicate             in_model                   value
##   <chr>                    <chr>                      <dbl>
## 1 eta                      eta                        0.2  
## 2 lambda_algae             lambda_algae               0    
## 3 lambda_daphnia           lambda_daphnia             0    
## 4 lambda_NH4               lambda_NH4                 0    
## 5 upsilon_algae_to_daphnia upsilon_algae_to_daphnia   0.13 
## 6 upsilon_daphnia_to_NH4   upsilon_daphnia_to_NH4     0.045
## 7 upsilon_NH4_to_algae     upsilon_NH4_to_algae|light 0.3  
## 8 zeta                     zeta                       0.1  
## 
## [[2]]
## # A tibble: 8 × 3
##   in_replicate             in_model                  value
##   <chr>                    <chr>                     <dbl>
## 1 eta                      eta                       0.2  
## 2 lambda_algae             lambda_algae              0    
## 3 lambda_daphnia           lambda_daphnia            0    
## 4 lambda_NH4               lambda_NH4                0    
## 5 upsilon_algae_to_daphnia upsilon_algae_to_daphnia  0.13 
## 6 upsilon_daphnia_to_NH4   upsilon_daphnia_to_NH4    0.045
## 7 upsilon_NH4_to_algae     upsilon_NH4_to_algae|dark 0.1  
## 8 zeta                     zeta                      0.1

The model is now complete and can be used to generate data!

Generating simulated data

One can calculate predicted trajectories with the project() function:

proj <- mod %>% project(end = 10)
plot(proj, facet_row = "type", facet_col = "group")

plot of chunk unnamed-chunk-11

Real-life data will incorporate some variability around those trajectories. To simulate data with variability around expected compartment size (coefficient of variation "zeta") and around expected proportion of tracer (c.v. "eta"), one can use the sample_from() function:

spl <- mod %>% sample_from(at = 1:10)
spl
## # A tibble: 60 × 5
##     time comp   size   prop treatment
##    <dbl> <chr> <dbl>  <dbl> <chr>    
##  1     1 algae 0.958 0.0459 light    
##  2     2 algae 0.842 0.0895 light    
##  3     3 algae 0.930 0.0783 light    
##  4     4 algae 0.875 0.103  light    
##  5     5 algae 0.936 0.153  light    
##  6     6 algae 0.843 0.116  light    
##  7     7 algae 0.677 0.0982 light    
##  8     8 algae 0.750 0.0935 light    
##  9     9 algae 0.904 0.0775 light    
## 10    10 algae 0.888 0.0968 light    
## # … with 50 more rows

To visualize the simulated data, we can add it to the projected trajectories:

proj <- proj %>%
  set_obs(spl, comp = "comp", size = "size", prop = "prop", time = "time",
          group = "treatment")
plot(proj, facet_row = "type", facet_col = "group")

plot of chunk unnamed-chunk-13

Fitting a model on the simulated data

We can use the simulated data in spl to fit parameters using MCMC. By using different versions of the dataset, we can compare different experimental designs. Here, to test how sample size affects the uncertainty of parameter estimates, we will perform MCMC runs with either the full spl dataset (ten time points) or a reduced dataset with only three time points:

spl_reduced <- spl %>% filter(time %in% c(4, 7, 10))

plot of chunk unnamed-chunk-15

Run with ten time points

We add the simulated data to the model as we would do for real data:

mod_full <- mod %>%
  set_obs(spl, comp = "comp", size = "size", prop = "prop", time = "time",
          group = "treatment")

We run the MCMC:

run_full <- run_mcmc(mod_full, iter = 1000)
plot(run_full)
# Note: the figure below only shows a few of the traceplots for vignette concision

plot of chunk unnamed-chunk-18

and we do a posterior predictive check:

pred_full <- predict(mod_full, run_full)
plot(pred_full, facet_row = c("group", "type"),
     facet_col = "compartment",
     scale = "all")

plot of chunk unnamed-chunk-20

Run with three time points

We use the reduced dataset this time:

mod_red <- mod %>%
  set_obs(spl_reduced, comp = "comp", size = "size", prop = "prop",
          time = "time", group = "treatment")

We run the MCMC:

run_red <- run_mcmc(mod_red, iter = 1000)
plot(run_red)
# Note: the figure below only shows a few of the traceplots for vignette concision

plot of chunk unnamed-chunk-23

and we do a posterior predictive check:

pred_red <- predict(mod_red, run_red)
plot(pred_red, facet_row = c("group", "type"),
     facet_col = "compartment",
     scale = "all")

plot of chunk unnamed-chunk-25

Comparison

Does using ten time points (spl) instead of three (spl_reduced) improve a lot the parameter estimates? Let's compare the uncertainty in their posteriors:

signif(summary(run_full)$quantiles, 2)
##                                2.5%     25%    50%    75%  97.5%
## eta                        0.170000 0.19000 0.2100 0.2200 0.2500
## lambda_algae               0.000170 0.00180 0.0039 0.0068 0.0150
## lambda_daphnia             0.000078 0.00071 0.0015 0.0027 0.0058
## lambda_NH4                 0.000160 0.00150 0.0035 0.0062 0.0140
## upsilon_algae_to_daphnia   0.120000 0.12000 0.1300 0.1300 0.1400
## upsilon_daphnia_to_NH4     0.047000 0.05000 0.0510 0.0520 0.0550
## upsilon_NH4_to_algae|dark  0.087000 0.09300 0.0960 0.1000 0.1100
## upsilon_NH4_to_algae|light 0.280000 0.30000 0.3100 0.3200 0.3300
## zeta                       0.069000 0.07800 0.0830 0.0890 0.1000
signif(summary(run_red)$quantiles, 2)
##                               2.5%     25%    50%    75%  97.5%
## eta                        0.16000 0.20000 0.2300 0.2500 0.3400
## lambda_algae               0.00019 0.00270 0.0059 0.0100 0.0230
## lambda_daphnia             0.00005 0.00078 0.0020 0.0037 0.0089
## lambda_NH4                 0.00035 0.00300 0.0066 0.0120 0.0240
## upsilon_algae_to_daphnia   0.11000 0.12000 0.1300 0.1300 0.1500
## upsilon_daphnia_to_NH4     0.04300 0.04800 0.0500 0.0520 0.0580
## upsilon_NH4_to_algae|dark  0.07400 0.08700 0.0930 0.0990 0.1100
## upsilon_NH4_to_algae|light 0.23000 0.26000 0.2800 0.3000 0.3300
## zeta                       0.06500 0.08300 0.0940 0.1100 0.1500

For most parameters, it didn't change things much. Maybe in this case an experiment with three time points would be sufficient.

To go further

This example only used one simulated dataset. When preparing a real-life experiment, it would make sense to try a range of "true" values for the parameters to be estimated and different initial conditions in addition to different experimental designs (number of replicates, number of samples, timing of sampling).

A convenient way to structure many data simulations is to create a tibble containing the simulation variables (e.g. parameter values, sampling points, alternative models) and store the resulting simulated datasets and their corresponding MCMC fits in list columns to make the analysis of the simulations easier.