Case study: converting a Shiny app to async

User interface

The user interface is a pretty typical shinydashboard. It’s important to note that the UI part of the app is entirely agnostic to whether the server is written in the sync or async style; when we port the app to async, we won’t touch the UI at all.

There are two major pieces of input we need from users: what date to examine (this app only lets us look at one day at a time) and how many of the most prolific downloaders to look at. We’ll put these two controls in the dashboard sidebar.

dashboardSidebar(
  dateInput("date", "Date", value = Sys.Date() - 2),
  numericInput("count", "Show top N downloaders:", 6)
)

(We set date to two days ago by default, because there’s some lag between when a day ends and when its logs are published.)

The rest of the UI code is just typical shinydashboard scaffolding, plus some shinydashboard::valueBoxOutputs and plotOutputs. These are so trivial that they’re hardly worth talking about, but I’ll include the code here for completeness. Finally, there’s detailViewUI, a Shiny module that just contains more of the same (value boxes and plots).

  dashboardBody(
    fluidRow(
      tabBox(width = 12,
        tabPanel("All traffic",
          fluidRow(
            valueBoxOutput("total_size", width = 4),
            valueBoxOutput("total_count", width = 4),
            valueBoxOutput("total_downloaders", width = 4)
          ),
          plotOutput("all_hour")
        ),
        tabPanel("Biggest whales",
          plotOutput("downloaders", height = 500)
        ),
        tabPanel("Whales by hour",
          plotOutput("downloaders_hour", height = 500)
        ),
        tabPanel("Detail view",
          detailViewUI("details")
        )
      )
    )
  )

Server logic

Based on these inputs and outputs, we’ll write a variety of reactive expressions and output renderers to download, manipulate, and visualize the relevant log data.

The reactive expressions:

• data (eventReactive): Whenever input$date changes, the data reactive downloads the full log for that day from http://cran-logs.rstudio.com, and parses it.
• whales (reactive): Reads from data(), tallies the number of downloads performed by each unique IP, and returns a data frame of the top input$count most prolific downloaders, along with their download counts.
• whale_downloads (reactive): Joins the data() and whales() data frames, to return all of the details of the cetacean downloads.

The whales reactive expression depends on data, and whale_downloads depends on data and whales.

The outputs in this app are mostly either renderPlots that we populate with ggplot2, or shinydashboard::renderValueBoxes. They all rely on one or more of the reactive expressions we just described. We won’t catalog them all here, as they’re not individually interesting, but we will look at some archetypes below.

Loading promises and future

The first thing we’ll do is load the basic libraries of async programming.

library(promises)
library(future)
plan(multisession)

I originally used multiprocess but file downloading inside a future seemed to fail on Mac. (I’ve found that it’s usually not worth spending a lot of time trying to figure out why multiprocess doesn’t work for some specific code; instead, just use multisession, since that’s probably going to be the solution anyway.)

The data reactive: future_promise() all the things

The next thing we’ll do is convert the data event reactive to use future for the expensive bits. The original code looks lke this:

# SYNCHRONOUS version

data <- eventReactive(input$date, {
  date <- input$date  # Example: 2018-05-28
  year <- lubridate::year(date)  # Example: "2018"

  url <- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
  path <- file.path("data_cache", paste0(date, ".csv.gz"))

  withProgress(value = NULL, {

    if (!file.exists(path)) {
      setProgress(message = "Downloading data...")
      download.file(url, path)
    }

    setProgress(message = "Parsing data...")
    read_csv(path, col_types = "Dti---c-ci", progress = FALSE)

  })
})

(Earlier, I said we wouldn’t take advantage of precomputation or caching. That wasn’t entirely true; in the code above, we cache the log files we download in a data_cache directory. I couldn’t bring myself to put my internet connection through that level of abuse, as I knew I’d be running this code thousands of times as I load tested it.)

For now, we’ll lose the withProgress/setProgress reporting, since doing that correctly requires some more advanced techniques that we’ll talk about later. We’ll come back and fix this code later, but for now:

# ASYNCHRONOUS version

data <- eventReactive(input$date, {
  date <- input$date
  year <- lubridate::year(date)

  url <- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
  path <- file.path("data_cache", paste0(date, ".csv.gz"))

  future_promise({
    if (!file.exists(path)) {
      download.file(url, path)
    }
    read_csv(path, col_types = "Dti---c-ci", progress = FALSE)
  })
})

Pretty straightforward. This reactive now returns a future (which counts as a promise), not a data frame.

Remember that we must read any reactive values (including input) and reactive expressions from outside the future. (You will get an error if you attempt to read one from inside the future.)

At this point, since there are no other long-running operations we want to make asynchronous, we’re actually done interacting directly with the future package. The rest of the reactive expressions will deal with the future returned by data using general async functions and operators from promises.

The whales reactive: simple pipelines are simple

The whales reactive takes the data frame from data, and uses dplyr to find the top input$count most prolific downloaders.

# SYNCHRONOUS version

whales <- reactive({
  data() %>%
    count(ip_id) %>%
    arrange(desc(n)) %>%
    head(input$count)
})

Since data() now returns a promise, the whole function needs to be modified to deal with promises.

This is basically a best-case scenario for working with promises. The whole expression consists of a single magrittr pipeline. There’s only one object (data()) that’s been converted to a promise. The promise object only appears once, at the head of the pipeline.

When the stars align like this, converting this code to async is literally as easy as replacing each %>% with %...>%:

# ASYNCHRONOUS version

whales <- reactive({
  data() %...>%
    count(ip_id) %...>%
    arrange(desc(n)) %...>%
    head(input$count)
})

The input (data()) is a promise, the resulting output object is a promise, each stage of the pipeline returns a promise; but we can read and write this code almost as easily as the synchronous version!

An example this simple may seem reductive, but this best-case scenario happens surprisingly often, if your coding style is influenced by the tidyverse. In this example app, 59% of the reactives, observers, and outputs were converted using nothing more than replacing %>% with %...>%.

One last thing before we move on. In the last section, I emphasized that reactive values cannot be read from inside a future. Here, we’re using head(input$count) inside a promise-pipeline; since data() is written using a future, doesn’t that mean… well… isn’t this wrong?

Nope—this code is just fine. The prohibition is against reading reactive values/expressions from inside a future, because code inside a future is executed in a totally different R process. The steps in a promise-pipeline aren’t futures, but promise handlers. These aren’t executed in a different process; rather, they’re executed back in the original R process after a promise is resolved. We’re allowed and expected to access reactive values and expressions from these handlers.

The whale_downloads reactive: reading from multiple promises

The whale_downloads reactive is a bit more complicated case.

# SYNCHRONOUS version

whale_downloads <- reactive({
  data() %>%
    inner_join(whales(), "ip_id") %>%
    select(-n)
})

Looks simple, but we can’t just do a simple replacement this time. Can you see why?

# BAD VERSION DOESN'T WORK

whale_downloads <- reactive({
  data() %...>%
    inner_join(whales(), "ip_id") %...>%
    select(-n)
})

Remember, both data() and whales() now return a promise object, not a data frame. None of the dplyr verbs know how to deal with promises natively (and the same is true for almost every other R function, anywhere in the R universe).

We’re able to use %...>% with promises on the left-hand side and regular dplyr calls on the right-hand side, only because the %...>% operator “unwraps” the promise object for us, yielding a regular object (data frame or whatever) to be passed to dplyr. But in this case, we’re passing whales(), which a promise object, directly to inner_join, and inner_join has no idea what to do with it.

The fundamental thing to pattern-match on here, is that we have a block of code that relies on more than one promise object, and that means %...>% won’t be enough. This is a pretty common situation as well, and occurs in 12% of reactives and outputs in this example app.

Here’s what the real solution looks like:

# ASYNCHRONOUS version

whale_downloads <- reactive({
  promise_all(data_df = data(), whales_df = whales()) %...>% with({
    data_df %>%
      inner_join(whales_df, "ip_id") %>%
      select(-n)
  })
})

> promise_all(a = future_promise("Hello"), b = future_promise("World")) %...>% print()
$a
[1] "Hello"

$b
[1] "World"

> x + y
Error: object 'x' not found

> with(list(x = 1, y = 2), {
+   x + y
+ })
[1] 3

> promise_all(x = future_promise("Hello"), y = future_promise("World")) %...>%
+   with({ paste(x, y) }) %...>%
+   print()
[1] "Hello World"

The total_downloaders value box: simple pipelines are for output, too

All of the value boxes in this app ended up looking a lot like this:

# SYNCHRONOUS version

output$total_downloaders <- renderValueBox({
  data() %>%
    pull(ip_id) %>%
    unique() %>%
    length() %>%
    format(big.mark = ",") %>%
    valueBox("unique downloaders")
})

This is structurally no different than the whales best-case scenario reactive. One thing worth pointing out is that an async renderValueBox means you return a promise that returns a valueBox; you don’t return a valueBox to whom you have passed a promise.

Meaning, you don’t do this:

# BAD VERSION DOESN'T WORK

output$total_downloaders <- renderValueBox({
  valueBox(
    data() %...>%
      pull(ip_id) %...>%
      unique() %...>%
      length() %...>%
      format(big.mark = ","),
    "unique downloaders"
  )
})

Instead, you do this:

# ASYNCHRONOUS version

output$total_downloaders <- renderValueBox({
  data() %...>%
    pull(ip_id) %...>%
    unique() %...>%
    length() %...>%
    format(big.mark = ",") %...>%
    valueBox("unique downloaders")
})

The other trick worth nothing is the pull verb, which is used to retrieve a specific column of a data frame as a vector (similar to $ or [[). In this case, pull(data, ip_id) is equivalent to data[["ip_id"]]. Note that pull is part of dplyr and isn’t specific to promises.

The biggest_whales plot: getting untidy

In a cruel twist of API design fate, one of the cornerstone packages of the tidyverse lacks a tidy API. I’m referring, of course, to ggplot2:

# SYNCHRONOUS version

output$downloaders <- renderPlot({
  whales() %>%
    ggplot(aes(ip_name, n)) +
    geom_bar(stat = "identity") +
    ylab("Downloads on this day")
})

While dplyr and other tidyverse packages are designed to link calls together with %>%, the older ggplot2 package uses the + operator. This is mostly a small aesthetic wart when synchronous code, but it’s a real problem with async, because the promises package doesn’t currently have a promise-aware replacement for + like it does for %>%.

Fortunately, there’s a pretty good escape hatch for %>%, and %...>% inherited it too. Instead of a pipeline stage being a simple function call, you can put a { and } delimited code block, and inside of that code block, you can access the “it” value using a period (.).

# ASYNCHRONOUS version

output$downloaders <- renderPlot({
  whales() %...>% {
    whale_df <- .
    ggplot(whale_df, aes(ip_name, n)) +
      geom_bar(stat = "identity") +
      ylab("Downloads on this day")
  }
})

The importance of this pattern cannot be overstated! Using %...>% and simple calls alone, you’re restricted to doing pipeline-compatible operations. But %...>% together with a curly-brace code block means your handler code can be any shape or size. Once inside that code block, you have a regular, non-promise value in . (if you even want to use it—sometimes you don’t, as we’ll see later). You can have zero, one, or more statements. You can use the . multiple times, in nested expressions, whatever.

Tip: if you have extensive or complex code to put in a code block, start the block by creating a properly named variable to store the value of .. The reason for this is that . may acquire a different meaning than you intend as you add code to the code block. For example, if a magrittr pipeline starts with ., instead of evaluating the pipeline and returning a value, it creates a function that takes a single argument. So the following code wouldn’t filter the resolved value of whales(), but instead, create an anonymous function that calls filter(n > 1000) on whatever you pass it.

whales() %...>% {
  . %>% filter(n > 1000)
}

This fixes it:

whales() %...>% {
  whales_df <- .
  whales_df %>% filter(n > 1000)
}

There are other ways to work around the above problem as well, but I like this fix because it doesn’t require any thought or care. Just give the . value a new name, and forget the . exists.

For untidy code with a single promise object, just remember: pair a single %...>% with a code block and you should be able to do almost anything.

Revisiting the data reactive: progress support

Now that we have discussed a few techniques for writing async code, let’s come back to our original data event reactive, and this time do a more faithful async conversion that preserves the progress reporting functionality of the original.

Again, here’s the original sync code:

# SYNCHRONOUS version

data <- eventReactive(input$date, {
  date <- input$date  # Example: 2018-05-28
  year <- lubridate::year(date)  # Example: "2018"

  url <- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
  path <- file.path("data_cache", paste0(date, ".csv.gz"))

  withProgress(value = NULL, {

    if (!file.exists(path)) {
      setProgress(message = "Downloading data...")
      download.file(url, path)
    }

    setProgress(message = "Parsing data...")
    read_csv(path, col_types = "Dti---c-ci", progress = FALSE)

  })
})

Progress reporting currently presents two challenges for future.

First, the withProgress({...}) function cannot be used with async. withProgress is designed to wrap a slow synchronous action, and dismisses its progress dialog when the block of code it wraps is done executing. Since the call to future_promise() will return immediately even though the actual task is far from done, using withProgress won’t work; the progress dialog would be dismissed before the download even got going.

It’s conceivable that withProgress could gain promise compatibility someday, but it’s not in Shiny v1.1.0. In the meantime, we can work around this by using the alternative, object-oriented progress API that Shiny offers. It’s a bit more verbose and fiddly than withProgress/setProgress, but it is flexible enough to work with futures/promises.

Second, progress messages can’t be sent from futures. This is simply because futures are executed in child processes, which don’t have direct access to the browser like the main Shiny process does.

It’s conceivable that future could gain the ability for child processes to communicate back to their parents, but no good solution exists at the time of this writing. In the meantime, we can work around this by taking the one future that does both downloading and parsing, and splitting it into two separate futures. After the download future has completed, we can send a progress message that parsing is beginning, and then start the parsing future.

The regrettably complicated solution is below.

# ASYNCHRONOUS version

data <- eventReactive(input$date, {
  date <- input$date
  year <- lubridate::year(date)

  url <- glue("http://cran-logs.rstudio.com/{year}/{date}.csv.gz")
  path <- file.path("data_cache", paste0(date, ".csv.gz"))

  p <- Progress$new()
  p$set(value = NULL, message = "Downloading data...")
  future_promise({
    if (!file.exists(path)) {
      download.file(url, path)
    }
  }) %...>%
    { p$set(message = "Parsing data...") } %...>%
    { future_promise(read_csv(path, col_types = "Dti---c-ci", progress = FALSE)) } %>%
    finally(~p$close())
})

The single future we wrote earlier has now become a pipeline of promises:

1. future (download)
2. send progress message
3. future (parse)
4. dismiss progress dialog

Note that neither the R6 call p$set(message = ...) nor the second future_promise() call are tidy, so they use curly-brace blocks, as discussed in the above section about biggest_whales.

The final step of dismissing the progress dialog doesn’t use %...>% at all; because we want the progress dialog to dismiss whether the download and parse operations succeed or fail, we use the regular pipe %>% and finally() function instead. See the relevant section in Working with promises in R to learn more.

With these changes in place, we’ve now covered all of the changes to the application. You can see the full changes side-by-side via this GitHub diff.

Load testing with Shiny (coming soon)

At the time of this writing, we are working on a suite of load testing tools for Shiny that is not publicly available yet, but was previewed by Sean Lopp during his epic rstudio::conf 2018 talk about running a Shiny load test with 10,000 simulated concurrent users.

You use these tools to easily record yourself using your Shiny app, which creates a test script; then play back that test script, but multiplied by dozens/hundreds/thousands of simulated concurrent users; and finally, analyze the timing data generated during the playback step to see what kind of latency the simulated users experienced.

To examine the effects of my async refactor, I recorded a simple test script by loading up the app, waiting for the first tab to appear, then clicking through each of the other tabs, pausing for several seconds each time before moving on to the next. When using the app without any other visitors, the homepage fully loads in less than a second, and the initial loading of data and rendering of the plot on the default tab takes about 7 seconds. After that, each tab takes no more than a couple of seconds to load. Overall, the entire test script, including time where the user is thinking, takes about 40 seconds under ideal settings (i.e. only a single concurrent user).

I then used this test script to generate load against the Shiny app running in my local RStudio. With the settings I chose, the playback tool introduced one new “user” session every 5 seconds, until 50 sessions total had been launched; then it waited until all the sessions were complete. I ran this test on both the sync and async versions in turn, which generated the following results.

Sync vs. async performance

In this plot, each row represents a single session, and the x dimension represents time. Each of the rectangles represents a single “step” in the test script, be it downloading the HTML for the homepage, fetching one of the two dozen JavaScript/CSS files, or waiting for the server to update outputs. So the wider a rectangle is, the longer the user had to wait. (The empty gaps between rectangles represents time the app is waiting for the user to click an input; their widths are hard-coded into the test script.)

Of particular importance are the red and pink rectangles, as these represent the initial page load. While these are taking place, the user is staring at a blank page, probably wondering if the server is down. Long waits during this stage are not only undesirable, but surprising and incomprehensible to the user; whereas the same user is probably prepared to wait a little while for a complicated visualization to be rendered in response to an input change.

And as you can see from this plot, the behavior of the async app is much improved in the critical metric of homepage/JS/CSS loading time. The sync version of the app starts displaying unacceptably long red/pink loading times as early as session 15, and by session #44 the maximum page load time has exceeded one minute. The async version at that point is showing 25 second load times, which is far from great, but still a significant step in the right direction.

Further optimizations

I was surprised that the async version’s page load times weren’t even faster, and even more surprised to see that the blue rectangles were just as wide as the sync version. Why isn’t the async version way faster? The sync version does all of its work on a single thread, and I specifically designed this app to be a nightmare for scalability by having each session kick off by parsing hundreds of megabytes of CSV, an operation that is quite expensive. The async version gets to spread these jobs across several workers. Why aren’t we seeing a greater time savings?

Mostly, it’s because calling future_promise(read_csv("big_file.csv")) is almost a worst-case scenario for future and async. read_csv is generally fast, but because the CRAN log files are so big, read_csv("big_file.csv") is slow. The value it returns is a very large data frame, that has now been loaded not into the Shiny process, but a future worker process. In order to return that data frame to the Shiny process, that data must first be serialized (I believe future essentially uses saveRDS for this), transmitted to the Shiny process, and then deserialized; to make matters worse, the transmitting and deserialization steps happen on the main R thread that we’re working so hard to try to keep idle. The larger the data we send back and forth to the future, the more performance suffers, and in this case we’re sending back quite a lot of data.

We can make our code significantly faster by doing more summarizing, aggregation, and filtering inside the future; not only does this make more of the work happen in parallel, but by returning the data in already-processed form, we can have much less data to transfer from the worker process back to the Shiny process. (For example, the data for May 31, 2018 weighs 75MB before optimization, and 8.8MB afterwards.)

Compare all three runs in the image below (the newly optimized version is labelled “async2”). The homepage load times have dropped further, and the calculation times are now dramatically faster than the sync code.

Looking at the “async2” graph, the leading (bottom-left) edge has the same shape as before, as that’s simply the rate at which the load testing tool launches new sessions. But notice how much more closely the trailing (upper-right) edge matches the leading edge! It means that even as the number of active sessions ramped up, the amount of latency didn’t get dramatically worse, unlike with the “sync” and “async” versions. And each of the individual blue rectangles in the “async2” are comparatively tiny, meaning that users never have to wait more than a dozen seconds at the most for plots to update.

This last plot shows the same data as above, but with the sessions aligned by start time. You can clearly see how the sessions are both shorter and less variable in “async2” compared to the others. I’ve added a yellow vertical line at the 10 second mark; if the page load (red/pink) has not completed at this point, it’s likely that your visitor has left in disgust. While “async” does better than “sync”, they both break through the 10 second mark early and often. In contrast, the “async2” version just barely peeks over the line three times.

To get a visceral sense for what it feels like to use the app under load, here’s a video that shows what it’s like to browse the app while the load test is running at its peak. The left side of the screen shows “sync”, the right shows “async2”. In both cases, I navigated to the app when session #40 was started.

Take a look at the code diff for async vs. async2. While the code has not changed very dramatically, it has lost a little elegance and maintainability: the code for each of the affected outputs now has one foot in the the render function and one foot in the future. If your app’s total audience is a team of a hundred analysts and execs, you may choose to forgo the extra performance and stick with the original async (or even sync) code. But if you have serious scaling needs, the refactoring is probably a small price to pay.

Let’s get real for a second, though. If this weren’t an example app written for exposition purposes, but a real production app that was intended to scale to thousands of concurrent users across dozens of R processes, we wouldn’t download and parse CSV files on the fly. Instead, we’d establish a proper ETL procedure to run every night and put the results into a properly indexed database table, or RDS files with just the data we need. As I said earlier, a little precomputation and caching can make a huge difference!

Much of the remaining latency for the async2 branch is from ggplot2 plotting. Sean’s talk alluded to some upcoming plot caching features we’re adding to Shiny, and I imagine they will have as dramatic an effect for this test as they did for Sean.