The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Nested data

library(tidyr)
library(dplyr)
library(purrr)

Basics

A nested data frame is a data frame where one (or more) columns is a list of data frames. You can create simple nested data frames by hand:

df1 <- tibble(
  g = c(1, 2, 3),
  data = list(
    tibble(x = 1, y = 2),
    tibble(x = 4:5, y = 6:7),
    tibble(x = 10)
  )
)

df1
#> # A tibble: 3 × 2
#>       g data            
#>   <dbl> <list>          
#> 1     1 <tibble [1 × 2]>
#> 2     2 <tibble [2 × 2]>
#> 3     3 <tibble [1 × 1]>

(It is possible to create list-columns in regular data frames, not just in tibbles, but it’s considerably more work because the default behaviour of data.frame() is to treat lists as lists of columns.)

But more commonly you’ll create them with tidyr::nest():

df2 <- tribble(
  ~g, ~x, ~y,
   1,  1,  2,
   2,  4,  6,
   2,  5,  7,
   3, 10,  NA
)
df2 %>% nest(data = c(x, y))
#> # A tibble: 3 × 2
#>       g data            
#>   <dbl> <list>          
#> 1     1 <tibble [1 × 2]>
#> 2     2 <tibble [2 × 2]>
#> 3     3 <tibble [1 × 2]>

nest() specifies which variables should be nested inside; an alternative is to use dplyr::group_by() to describe which variables should be kept outside.

df2 %>% group_by(g) %>% nest()
#> # A tibble: 3 × 2
#> # Groups:   g [3]
#>       g data            
#>   <dbl> <list>          
#> 1     1 <tibble [1 × 2]>
#> 2     2 <tibble [2 × 2]>
#> 3     3 <tibble [1 × 2]>

I think nesting is easiest to understand in connection to grouped data: each row in the output corresponds to one group in the input. We’ll see shortly this is particularly convenient when you have other per-group objects.

The opposite of nest() is unnest(). You give it the name of a list-column containing data frames, and it row-binds the data frames together, repeating the outer columns the right number of times to line up.

df1 %>% unnest(data)
#> # A tibble: 4 × 3
#>       g     x     y
#>   <dbl> <dbl> <dbl>
#> 1     1     1     2
#> 2     2     4     6
#> 3     2     5     7
#> 4     3    10    NA

Nested data and models

Nested data is a great fit for problems where you have one of something for each group. A common place this arises is when you’re fitting multiple models.

mtcars_nested <- mtcars %>% 
  group_by(cyl) %>% 
  nest()

mtcars_nested
#> # A tibble: 3 × 2
#> # Groups:   cyl [3]
#>     cyl data              
#>   <dbl> <list>            
#> 1     6 <tibble [7 × 10]> 
#> 2     4 <tibble [11 × 10]>
#> 3     8 <tibble [14 × 10]>

Once you have a list of data frames, it’s very natural to produce a list of models:

mtcars_nested <- mtcars_nested %>% 
  mutate(model = map(data, function(df) lm(mpg ~ wt, data = df)))
mtcars_nested
#> # A tibble: 3 × 3
#> # Groups:   cyl [3]
#>     cyl data               model 
#>   <dbl> <list>             <list>
#> 1     6 <tibble [7 × 10]>  <lm>  
#> 2     4 <tibble [11 × 10]> <lm>  
#> 3     8 <tibble [14 × 10]> <lm>

And then you could even produce a list of predictions:

mtcars_nested <- mtcars_nested %>% 
  mutate(model = map(model, predict))
mtcars_nested  
#> # A tibble: 3 × 3
#> # Groups:   cyl [3]
#>     cyl data               model     
#>   <dbl> <list>             <list>    
#> 1     6 <tibble [7 × 10]>  <dbl [7]> 
#> 2     4 <tibble [11 × 10]> <dbl [11]>
#> 3     8 <tibble [14 × 10]> <dbl [14]>

This workflow works particularly well in conjunction with broom, which makes it easy to turn models into tidy data frames which can then be unnest()ed to get back to flat data frames. You can see a bigger example in the broom and dplyr vignette.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.