Usage
Example Images
A folder has been provided in this package called sample_images. It contains five sample of images provided by Tennis Australia, they depict scenes from the Channel 7 broadcast of the Australian Open 2016.
Creating your app
The following code creates the app that was used to tag images from the Australian Open in 2016.
First load taipan, and shiny.
# load packages
library(shiny)
library(taipan)Create questions
The survey creator provides questions using Shiny widgets, and the tutorials expand on the types of widgets you can use to collect information from users. Radio button widgets are used in the sample app, as all questions required one answer, and were limited to one answer.
Two sets of questions are required, one set that relates to the entire scene, and another set that relates to the smaller areas that can be selected in the images.
The following code shows the basic layout to include questions.
questions <- taipanQuestions(
scene = div(),
selection = div())The following code creates the taipanQuestions set for the tennis images:
# create questions to ask users
questions <- taipanQuestions(
scene = div(radioButtons("graphic", label = ("2D Graphic"),
choices = list("Live image", "2D Graphic")),
radioButtons("bg", label = ("Background"),
choices = list("Crowd",
"Court", "Logo wall", "Not applicable")),
radioButtons("person", label = ("Detectable Person"),
choices = list("Yes", "No"), selected = "Yes"),
radioButtons("shotangle", label = ("Shot angle"),
choices = list("Level with players",
"Birds eye",
"Upward angle")),
radioButtons("situation", label = ("Situation"),
choices = list("Court in play",
"Court player close-up",
"Court close-up not player",
"Crowd",
"Off court close up of player",
"Transition"))),
selection = div(radioButtons("detect", label = ("Detect Face"),
choices = list("Player" ,
"Other staff on court", "Fan", "None")),
radioButtons("obscured", label = ("Face obscured"),
choices = list("Yes", "No"), selected = "No"),
radioButtons("lighting", label = ("Lighting"),
choices = list("Direct sunlight", "Shaded", "Partially shaded")),
radioButtons("headangle", label = ("Head angle"),
choices = list("Front on", "Back of head",
"Profile", "Other")),
radioButtons("glasses", label = ("Glasses"),
choices = list("Yes", "No"), selected = "No"),
radioButtons("visorhat", label = ("Visor/hat"),
choices = list("Yes", "No")))
)Build app
To build the shiny app the following buildTaipan function is used. It requires the taipanQuestions, the location to the save the new app, and the location of images. Images can be accessed via a URL, or a file path is held locally. Note that these images will be downloaded or copied to the directory of the new app.
images <-
c("https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/blue_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/pink_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/red_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/yellow_player.png",
"https://raw.githubusercontent.com/srkob1/taipan/master/sample_images/disgruntled_player.png")
buildTaipan(
questions = questions,
images = images,
file.path(tempdir(), "taipan")
)The sample app is stored in a temporary directory. It will launch after it has been created, this default behaviour can be changed by setting launch = FALSE.
Using the app
Annotating Images
The scene questions need to be answered only once per image. These questions appear for the first image when the app begins, and will be displayed as long as there is no area selected.
Taipan application scene questions.
Selecting areas
The selection questions are answered for each highlighted area. To make a selection, click on the image and drag the mouse to create a shaded box covering the desired area. When the area is selected, the selection questions become available.
Taipan application selection questions appear when selecting an area.
When a user is finished answering questions for ALL selections in an image, click the Next Image button to continue to the next image available. A user can also choose the Export responses button at any time, downloading and saving all current information as a text file.
Saving answers
When the questions for an area have been answered, the user should click the Save Selection button, this will save the current answers as information for the currently shaded area of the image. Users can repeat the process of highlighting an area, answering and savings related questions as many times as needed.
Taipan application selection questions can be answered for multiple selections.
Exporting answers
While the answers provided are regularly saved, they are not exported unless the Export responses button is used. This will save the comma separated values in the chosen directory.
The answers are provided in a tidy format. Each row contains one observation, we consider each selected area in an image an observation. The columns contain information regarding each selected area. This includes:
- The file name of the image, labelled image_name
- The answer to each scene question, labelled as per the input id
- The bounding coordinates of the location, labelled xmin, xmax, ymin, ymax
- The answer to each selection question, labelled as per the input id
taipan_export_2018_09_04 <- read.csv("taipan-export-2018-09-04.csv")
taipan_export_2018_09_04
#> image_name graphic bg person shotangle
#> 1 blue_player.png Live image Logo wall Yes Level with players
#> 2 blue_player.png Live image Logo wall Yes Level with players
#> 3 disgruntled_player.png Live image Crowd Yes Level with players
#> 4 pink_player.png Live image Crowd Yes Level with players
#> 5 pink_player.png Live image Crowd Yes Level with players
#> situation xmin xmax ymin ymax
#> 1 Court player close-up 335.9307 440.9812 164.71154 285.8654
#> 2 Court player close-up 623.3766 711.1111 213.17308 292.7885
#> 3 Court in play 214.7186 346.3203 33.17308 184.3269
#> 4 Court in play 364.7908 468.6869 35.48077 179.7115
#> 5 Court in play 278.2107 349.7835 72.40385 154.3269
#> detect obscured lighting headangle glasses visorhat
#> 1 Player No Shaded Other No Yes
#> 2 Other staff on court No Shaded Front on Yes Yes
#> 3 Player No Shaded Other No No
#> 4 Player No Direct sunlight Front on No No
#> 5 Other staff on court No Shaded Profile No YesEditing results
It is possible to view and edit answers for a selected area by double clicking within the box shown on the image.
Taipan application selection answers can be viewed and edited by double clicking within an area.
Saving the new answers for this selection will replace the previous answers, if the answers are not saved or deleted they will remain unchanged in the data set.