Our Experiment: Each eyetrackingR vignette uses the eyetrackingR package to analyze real data from a simple 2-alternative forced choice (2AFC) word recognition task administered to 19- and 24-month-olds. On each trial, infants were shown a picture of an animate object (e.g., a horse) and an inanimate object (e.g., a spoon). After inspecting the images, they disappeared and they heard a label referring to one of them (e.g., “The horse is nearby!”). Finally, the objects re-appeared on the screen and they were prompted to look at the target (e.g., “Look at the horse!”).

Overview of this vignette

In this vignette, we want to examine our data without throwing out time as a predictor. That is, we want to ask how are differences (if any) between the Target conditions emerged over time in each trial. To do so, we will perform a growth curve analysis.

Prerequisites

Before performing this analysis, we’ll need to prepare and clean our dataset. Here we will to do this quickly and with few notes but, for more information, see the vignette on preparing your data.

library("Matrix")
library("lme4")
library("ggplot2")
library("eyetrackingR")

data("word_recognition")
data <- make_eyetrackingr_data(word_recognition, 
                       participant_column = "ParticipantName",
                       trial_column = "Trial",
                       time_column = "TimeFromTrialOnset",
                       trackloss_column = "TrackLoss",
                       aoi_columns = c('Animate','Inanimate'),
                       treat_non_aoi_looks_as_missing = TRUE
)

# subset to response window post word-onset
response_window <- subset_by_window(data, 
                                    window_start_time = 15500, 
                                    window_end_time = 21000, 
                                    rezero = FALSE)
## Avg. window length in new data will be 5500
# analyze amount of trackloss by subjects and trials
(trackloss <- trackloss_analysis(data = response_window))

# remove trials with > 25% of trackloss
response_window_clean <- clean_by_trackloss(data = response_window, trial_prop_thresh = .25)
## Performing Trackloss Analysis...
## Will exclude trials whose trackloss proportion is greater than : 0.25
##  ...removed  33  trials.
# create Target condition column
response_window_clean$Target <- as.factor( ifelse(test = grepl('(Spoon|Bottle)', response_window_clean$Trial), 
                                       yes = 'Inanimate', 
                                       no  = 'Animate') )

Visualize the timecourse

In this analysis, we are interested in the change of our data over time. Therefore, it’s very important that we visualize our data to see whether our statistical estimates make any sense at all.

To plot the time-course of our data, we first need to convert it to eyetrackingR’s time_sequence_data format, which summarizes the data into time-bins and calculates proportion-looking for each (here we use 100ms time bins). From there, we simply use the plot method to plot the time-course.

# aggregate across trials within subjects in time analysis
response_time <- make_time_sequence_data(response_window_clean, time_bin_size = 100, 
                                 predictor_columns = c("Target"),
                                 aois = "Animate"
                            )

# visualize time results
plot(response_time, predictor_column = "Target") + 
  theme_light() +
  coord_cartesian(ylim = c(0,1))

From this plot, we should expect to see some big differences between Target conditions over time. However, an ideal analysis will also be consistent with the observation that these differences emerged after ~500ms – i.e, they were not present at window onset.

Growth curve analysis

Growth curve analysis (GCA) lets us model the timecourse of attention by fitting curves to proportion-looking over the course of the trial, and statistically assessing the bends in these curves. Our implementation of a growth curve analysis modelled after Mirman et al. (2008).

eyetrackingR sets us up nicely for GCA. Above, we used make_time_sequence_data to generate the dataframe. This dataframe includes everything we need for GCA.

First, it includes the same dependent variable columns as make_time_window_data, giving us an option of analyzing raw proportions or its transformations (for more detail, see the documentation or the window analysis vignette ).

Second, time_sequence_data has a series columns corresponding to ‘orthogonal polynomial timecodes.’ You can think of these as the linear, quadratic, cubic, etc. component of our Time predictor (so Time, Time^2, Time^3, etc.). However, unlike simply taking the power, these transformations are uncorrelated with each other, and therefore more appropriate for multiple regression.

If this seems confusing, it might help to simply visualize each of these vectors:

# generate dataframe summarizing values of each vector
timecodes <- unique(response_time[, c('ot1','ot2','ot3','ot4','ot5','ot6','ot7')])
timecodes$num <- 1:nrow(timecodes)

ggplot(timecodes, aes(x=num, y=ot1)) +
               geom_line() +
               geom_line(aes(y=ot2), color='red') +    # quadratic
               geom_line(aes(y=ot3), color='blue') +   # cubic
               geom_line(aes(y=ot4), color='green') +  # quartic
               geom_line(aes(y=ot5), color='purple') + # quintic 
               geom_line(aes(y=ot6), color='yellow') + # sextic
               geom_line(aes(y=ot7), color='pink') +   # septic
               scale_x_continuous(name="") +
               scale_y_continuous(name="")

round(cor(timecodes[, c(1:7)]),5)
##     ot1 ot2 ot3 ot4 ot5 ot6 ot7
## ot1   1   0   0   0   0   0   0
## ot2   0   1   0   0   0   0   0
## ot3   0   0   1   0   0   0   0
## ot4   0   0   0   1   0   0   0
## ot5   0   0   0   0   1   0   0
## ot6   0   0   0   0   0   1   0
## ot7   0   0   0   0   0   0   1

You can see the linear time code moving from the bottom left to top right corner of that plot (in black). Each of the other coloured lines corresponds to a different growth function.

The idea behind GCA is that we can simulanteously regress differences between our conditions (here, Target) on each of these to see which (in combination, or independently) best captures the pattern of growth in our data. Importantly, because they are uncorrelated, they are sure to capture distinct variance in our data.

Let’s fit our first GCA model, including only the linear time code for now (i.e., we temporarily want to ignore non-linear change over time):

# sum-code and center our predictor:
response_time$TargetC <- ifelse(response_time$Target == 'Animate', .5, -.5)
response_time$TargetC <- as.numeric(scale(response_time$TargetC, center=TRUE, scale=FALSE))

# Construct model
model_time_sequence <- lmer(Elog ~ TargetC*(ot1) + (1 + ot1 | Trial) + (1 + ot1 | ParticipantName), 
              data = response_time, REML = FALSE)

# cleanly show important parts of model (see `summary()` for more)
broom::tidy(model_time_sequence, effects = "fixed")
##          term   estimate std.error statistic
## 1 (Intercept)  0.7198634 0.1352025  5.324334
## 2     TargetC  1.7104983 0.1281708 13.345459
## 3         ot1 -2.9856824 0.5955429 -5.013379
## 4 TargetC:ot1 -0.9965965 0.6639581 -1.500993
drop1(model_time_sequence, ~., test="Chi")
## Single term deletions
## 
## Model:
## Elog ~ TargetC * (ot1) + (1 + ot1 | Trial) + (1 + ot1 | ParticipantName)
##             Df   AIC     LRT   Pr(Chi)    
## <none>         28855                      
## TargetC      1 28873 20.0611 7.501e-06 ***
## ot1          1 28868 15.0372 0.0001054 ***
## TargetC:ot1  1 28854  1.9262 0.1651718    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

eyetrackingR includes a plot method for time_sequence_data that can easily overlay the predictions of this model on the raw data:

plot(response_time, predictor_column = "Target", dv = "Elog", model = model_time_sequence) +
  theme_light()

This model reveals two main effects of TargetC and ot1, respectively. These may seem OK at first, but they are actually not a great match to our data (as revealed by the plot of their predictions).

But what we are missing is a result that reflects that these differences between Target emerged after the start of the window.

To remedy this issue, we can allow for non-linear change overtime by including a few higher-order orthogonal polynomials:

model_time_sequence <- lmer(Elog ~ TargetC*(ot1 + ot2 + ot3 + ot4) + (1 | Trial) + (1 | ParticipantName), 
                            data = response_time, REML = FALSE)

# commented out because this many random slopes takes too long to fit for a vignette
# model_time_sequence <- lmer(Elog ~ TargetC*(ot1 + ot2 + ot3 + ot4) + 
#                               (1 + ot1 + ot2 + ot3 + ot4 | Trial) + (1 + ot1 + ot2 + ot3 + ot4 | ParticipantName), 
#                             data = response_time, REML = FALSE)

# cleanly show important parts of model (see `summary()` for more)
broom::tidy(model_time_sequence, effects = "fixed")
##           term   estimate std.error  statistic
## 1  (Intercept)  0.7203362 0.1354240   5.319117
## 2      TargetC  1.7087990 0.1285408  13.293830
## 3          ot1 -2.8035128 0.1861832 -15.057818
## 4          ot2  0.2320570 0.1861832   1.246390
## 5          ot3 -0.5809525 0.1861832  -3.120327
## 6          ot4 -0.3634757 0.1861832  -1.952248
## 7  TargetC:ot1 -0.9024639 0.3877276  -2.327572
## 8  TargetC:ot2 -1.8994471 0.3877276  -4.898922
## 9  TargetC:ot3  4.0342392 0.3877276  10.404829
## 10 TargetC:ot4 -1.8634407 0.3877276  -4.806057
drop1(model_time_sequence, ~., test="Chi")
## Single term deletions
## 
## Model:
## Elog ~ TargetC * (ot1 + ot2 + ot3 + ot4) + (1 | Trial) + (1 | 
##     ParticipantName)
##             Df   AIC     LRT   Pr(Chi)    
## <none>         28837                      
## TargetC      1 28855  19.838 8.430e-06 ***
## ot1          1 29058 222.973 < 2.2e-16 ***
## ot2          1 28837   1.553  0.212648    
## ot3          1 28845   9.729  0.001813 ** 
## ot4          1 28839   3.810  0.050942 .  
## TargetC:ot1  1 28840   5.415  0.019960 *  
## TargetC:ot2  1 28859  23.956 9.854e-07 ***
## TargetC:ot3  1 28942 107.392 < 2.2e-16 ***
## TargetC:ot4  1 28858  23.058 1.572e-06 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
plot(response_time, predictor_column = "Target", dv = "Elog", model = model_time_sequence) +
  theme_light()

This model better captures our data because it shows critical interactions between Target and higher-order polynomials. These allow the model to capture the steep drop in the Inanimate condition as well as the bend that follows it.

But when did the conditions diverge?

GCA can tell you that your conditions/predictors had some predictive value (i.e., changed differently) over time. They can also tell you the form of that change by seeing the polynomials with which they reliably interacted. However, they can’t tell you at what time these predictors had an effect.

In the case of ascertaining reaction times, one way to answer this question is by using onset-contingent analyses.

For more general approaches to estimating time windows of divergence (estimating both the onset and offset of divergences), take a look at the divergence vignette.

References

Mirman, D., Dixon, J., & Magnuson, J. S. (2008). Statistical and computational models of the visual world paradigm: Growth curves and individual differences. Journal of Memory and Language, 59, 474–494.