dataMeta: Making and appending a data dictionary to an R dataset

Data

The data used for this vignette will be obtained from the Centers for Disease Control and Prevention (CDC) github repository for publicly avaiable Zika data: https://github.com/cdcepi/zika. This repo contains Zika data that has been scraped from the published records of the Health Department of various countries. The dataset used contains information related with Zika infection in the United States Virgin Islands (USVI) as published by their Health Department on their January 03, 2017 report and can be obtained using the code below. Please, note that the data is read with stringsAsFactors = FALSE. Below is a portion of this dataset:

data(my.data)

kable(head(my.data, 10), format = "html", caption = "Portion of dataset")

Another way to load the data is using it’s raw link from the github repo where it is stored:

path = "http://raw.githubusercontent.com/cdcepi/zika/master/"
path2 = "USVI/USVI_Zika/data/USVI_Zika-2017-01-03.csv"
url <- paste0(path, path2, collapse="")

my.data <- read.csv(url, header = TRUE, stringsAsFactors = FALSE)

Linker:

To build a data dictionary for the previous dataset, a linker data frame is constructed. This linker will serve as an intermediary to build the data dictionary. It will contain the names of the variables, a description of each variable provided by the user and a “variable type.” The variable type will give each variable a value of 1 or 0, as follows:

• 0: for those variables that have options that can be portrayed as a range of values. For example, age or dates or any categorical factors that the user does not want to list out (NA values will be maintained).
  
• 1: for those variables that have options that need to be listed and/or descrcibed later on.

The linker is built using one of the following two functions:

• prompt_linker.R will prompt the user to add the description of each variable in the console and the variable type.
  
• build_linker.R will require that the user create two vectors that will fill out the variable descriptions and variable types. This is shown below:

var_desc <- c("Date when report was published", "Regional location", 
             "Description of regional location", "Type of case",
             "A specific code for each data field", "The time period of each week",
             "The type of time period", "The number of cases per data field type",
             "The unit in which cases are reported")

var_type <- c(0, 1, 0, 1, 0, 0, 0, 0, 1)
                          
linker <- build_linker(my.data, variable_description = var_desc, variable_type = var_type)

Dictionary build:

Next, the dicitonary is built using the linker created above and the original dataset using the build_dict. Here, option_description is set to NULL because the options of each variable are self explanatory, thus the dictionary will not contain a description for each variable option. If option_descriptions will be added, then a vector of each option description is constructed. The length of this vector needs to be the same as the number of rows in the dictionary and it will depend on the the variable_type used in the linker. Another way to add option descriptions is to use the prompt_varopts function, which prompt the user for a description of each variable option, without the need to write the descriptions vector beforehand. If using this option, then prompt_varopts must be set to TRUE and option_description must be NULL. The following code builds a data dictionary for the USVI report:

dict <- build_dict(my.data = my.data, linker = linker, option_description = NULL, 
                   prompt_varopts = FALSE)

The dictionary will look as follows:

kable(dict, format = "html", caption = "Data dictionary for original dataset")

Adding the dictionary as a data attribute and incorporating other attributes:

To add the dictionary as an attribute to the original dataset, the function incorporate_attr is used. This function will require that the user add a main description of the dataset as shown below. There is also a prompt_attr function that will prompt the user to add this description through the console. Once the attributes are added to the dataset, the function save_it will save the file as an R data file .rds, which will preserve all attributes to the dataset. Accessing the attributes of the data is done using the following comand: attributes(my.new.data) and single attributes, like the main description of the dataset, can be obtained by typing: attributes(my.new.data)$main, as is shown below:

data_desc = "This data set portrays Zika infection related cases as reported by USVI."

my.new.data <- incorporate_attr(my.data = my.data, data.dictionary = dict, main_string = data_desc)

attributes(my.new.data)

$names
[1] "report_date"      "location"         "location_type"    "data_field"       "data_field_code" 
[6] "time_period"      "time_period_type" "value"            "unit"            

$class
[1] "data.frame"

$row.names
 [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

$main
[1] "This data set portrays Zika infection related cases as reported by USVI."

$dictionary
      variable_name                    variable_description                          variable_options
1        data_field                            Type of case                             zika_reported
2                                                                                   zika_lab_positive
3                                                                                            zika_not
4                                                                                        zika_pending
5                                                                               confirmed_age_under20
6                                                                                confirmed_age_20to39
7                                                                                confirmed_age_40to59
8                                                                                confirmed_age_over59
9                                                                                   confirmed_age_unk
10                                                                                     confirmed_male
11                                                                                   confirmed_female
12                                                                                    confirmed_fever
13                                                                              confirmed_acute_fever
14                                                                               confirmed_arthralgia
15                                                                                confirmed_arthritis
16                                                                                     confirmed_rash
17                                                                           confirmed_conjunctivitis
18                                                                                  confirmed_eyepain
19                                                                                 confirmed_headache
20                                                                                  confirmed_malaise
21                                                                                   zika_no_specimen
22                                                                               zika_tested_pregnant
23                                                                             zika_positive_pregnant
24                                                                             zika_negative_pregnant
25                                                                              zika_pending_pregnant
26                                                                          zika_no_specimen_pregnant
27  data_field_code     A specific code for each data field                          VI0001 to VI0026
28         location                       Regional location              United_States_Virgin_Islands
29                                                          United_States_Virgin_Islands-Saint_Thomas
30                                                           United_States_Virgin_Islands-Saint_Croix
31                                                            United_States_Virgin_Islands-Saint_John
32    location_type        Description of regional location                       county to territory
33      report_date          Date when report was published                  2017-01-03 to 2017-01-03
34      time_period            The time period of each week                                  NA to NA
35 time_period_type                 The type of time period                                  NA to NA
36             unit    The unit in which cases are reported                                     cases
37            value The number of cases per data field type                                 0 to 1930

$last_edit_date
[1] "2017-08-11 20:01:20 -04"

$author
[1] "Dania Rodriguez"

The user can also export the data dicitonary using as .csv or .xlsx by itlself and also save the data with all of its attributes, as shown below:

# Exporting dictionary only:
dict_only <- attributes(my.new.data)$dictionary
write.csv(dict_only, "dict_only.csv")

# Saving as .rds (dataset with appended dictionary)
save_it(complete_dict = my.new.data, name_of_file = "My Complete Dataset")