AAT Double Difference of Medians


This vignette describes a scoring method similar to Heuer, Rinck, and Becker (2007); double difference of median reaction times (RTs) for correct responses on Approach Avoidance Task data. It is a subtraction comparing approach bias towards test stimuli relative to approach bias towards control stimuli (avoid_test - approach_test) - (avoid_control - approach_control).


Load the included AAT dataset and inspect its documentation.

data("ds_aat", package = "splithalfr")

Relevant variables

The columns used in this example are:


Only select trials from assessment blocks, using the first response of the first attempt of each trial.

ds_aat <- subset(ds_aat, block_type %in% c("assess1", "assess2"))
ds_aat <- subset(ds_aat, attempt == 0)

Scoring the AAT

Writing a scoring method for the splithalfr requires implementing two functions; a sets function that describes which sets of data should be split into halves and a score function that calculates a score.

Defining the sets function

The sets function receives data from a single participant and returns a list of datasets for each condition. In this case, we will generate four data frames, containing the trials from: avoid test, approach test, avoid control, and approach control.

aat_fn_sets <- function (ds) {
  return (list(
    avoid_test = subset(ds, trial_type == "avoid" & cat == "test"),
    approach_test = subset(ds, trial_type == "approach" & cat == "test"),
    avoid_control = subset(ds, trial_type == "avoid" & cat == "control"),
    approach_control = subset(ds, trial_type == "approach" & cat == "control")

Defining the score function

The score function receives these four data frames from a single participant and for each:

  1. selects only correct responses
  2. calculates the median RT of remaining responses

Finally, it returns the double difference between the four median RTs.

aat_fn_score <- function (sets) {
  median_avoid_test <- median(subset(sets$avoid_test, response == 1)$rt)
  median_approach_test <- median(subset(sets$approach_test, response == 1)$rt)
  median_avoid_control <- median(subset(sets$avoid_control, response == 1)$rt)
  median_approach_control <- median(subset(sets$approach_control, response == 1)$rt)
  return ((median_avoid_test - median_approach_test) - (median_avoid_control - median_approach_control))

Calculating a score without the splithalfr

By combining the sets and score functions, a score for a single participant can be calculated. For instance, the score of UserID 1 can be calculated via the statement below.

aat_fn_score(aat_fn_sets(subset(ds_aat, UserID == 1)))

Calculating scores with the spltihalfr

To calculate scores for each participant, call sh_apply with four arguments:

  1. the dataset
  2. the column that identifies participants in the dataset
  3. the sets function
  4. the score function

The sh_apply function will return a data frame with one row per participant, and two columns: one that identifies participants (“UserID” in this example) and a column “score”, that contains the output of the score function.

aat_scores <- sh_apply(ds_aat, "UserID", aat_fn_sets, aat_fn_score)

Checking scores

It is recommended to check your scoring method by calculating the score of a representative participant via a different approach. For splithalfr tests, the author has done so via Excel.

Estimating split-half reliability

Calculating split scores

To calculate split-half scores for each participant, call sh_apply with an additional split_count argument, which specifies how many splits should be calculated. For each participant and split, the splithalfr will randomly divide the dataset of each element of sets into two halves that differ at most by one in size. When called with a split_count argument that is higher than zero, sh_apply returns a data frame with the following columns:

Note that in this example UserID 294 had a relatively low number of correct responses; it was sufficient to yield an AAT score but may yield missing data in some of the random splits.

aat_splits <- sh_apply(ds_aat, "UserID", aat_fn_sets, aat_fn_score, 1000)

Estimating reliability averaged over splits

Next, the output of sh_apply can be analyzed in order to estimate reliability. By default, functions are provided that automatically calculate mean Spearman-Brown (mean_sb_by_split) and Flanagan-Rulon (mean_fr_by_split) coefficients. If any missing values were encountered in the data provided to these functions, they give a warning, and then pairwise remove the missing data before calculating reliability.

# Spearman-Brown