Introduction to aidR

Without cleaning

In the following example, we set clean = FALSE. This reads all of the files present in the export and applies minimal standardization (e.g. formatting timestamps), but performs no further cleaning or tidying. This is useful for (i) inspecting non-standard data types (e.g. alarms or pump settings rather than CGM), or (ii) simply reading the data of a single subject, where merging across platforms does not matter.

path <- system.file("extdata", "tidepool_example.xlsx", package = "aidR")
data_raw <- parse_data(id = "1", paths = path, clean = FALSE)
#> Parsing data for id 1...
str(data_raw, max.level = 1)
#> List of 11
#>  $ cgm                  : tibble [36,873 × 14] (S3: tbl_df/tbl/data.frame)
#>  $ basal                : tibble [2,392 × 22] (S3: tbl_df/tbl/data.frame)
#>  $ bolus                : tibble [595 × 19] (S3: tbl_df/tbl/data.frame)
#>  $ device_event         : tibble [1,224 × 28] (S3: tbl_df/tbl/data.frame)
#>  $ bolus_calculator     : tibble [587 × 26] (S3: tbl_df/tbl/data.frame)
#>  $ basal_schedules      : tibble [330 × 22] (S3: tbl_df/tbl/data.frame)
#>  $ bg_targets           : tibble [330 × 27] (S3: tbl_df/tbl/data.frame)
#>  $ carb_ratios          : tibble [330 × 24] (S3: tbl_df/tbl/data.frame)
#>  $ insulin_sensitivities: tibble [330 × 24] (S3: tbl_df/tbl/data.frame)
#>  $ SMBG                 : tibble [646 × 15] (S3: tbl_df/tbl/data.frame)
#>  $ upload               : tibble [30 × 20] (S3: tbl_df/tbl/data.frame)

Here, path happens to be a single Tidepool export, so data_raw contains the standard cgm, basal, bolus, and SMBG data frames alongside Tidepool-specific extras (device_event, bolus_calculator, basal_schedules, …).

If a subject’s data spans several files (i.e. paths is a vector of length > 1), parse_data() merges all of them together by data type and sorts each resulting data frame by timestamp. Duplicate rows within a data type are removed (while ignoring the format column to also filter data downloaded from two different platforms), see Merging multiple exports of one subject.

With cleaning

Let us re-run the example above, but now with clean = TRUE. This is in fact the default, so the argument can be omitted.

path <- system.file("extdata", "tidepool_example.xlsx", package = "aidR")
data_clean <- parse_data(id = "1", paths = path, clean = TRUE)
#> Parsing data for id 1...
str(data_clean, max.level = 1)
#> List of 7
#>  $ cgm        : tibble [36,873 × 7] (S3: tbl_df/tbl/data.frame)
#>  $ basal      : tibble [2,389 × 8] (S3: tbl_df/tbl/data.frame)
#>  $ SMBG       : tibble [646 × 7] (S3: tbl_df/tbl/data.frame)
#>  $ bolus      : tibble [595 × 12] (S3: tbl_df/tbl/data.frame)
#>  $ carbs      : tibble [427 × 10] (S3: tbl_df/tbl/data.frame)
#>  $ total_basal:'data.frame': 137 obs. of  5 variables:
#>  $ total_bolus:'data.frame': 137 obs. of  5 variables:

The additional platform-specific data types from before (e.g. device_event, bolus_calculator) have disappeared. Instead, the major data types (CGM, basal, bolus, carbohydrates, and SMBG) are standardized to a common column layout.

All five start with the same four columns:

• id: the subject’s id, as passed to parse_data().
• format: the export format this row was parsed from, one of glooko, carelink, mylife, tidepool, yourloops, tandem_source or dexcom_clarity.
• timestamp: the time of the data point (POSIXct), in local (wall-clock) time.
• timezone_offset: the offset of that local time from UTC, in hours (NA if the export does not report it). UTC time is therefore timestamp - lubridate::hours(timezone_offset).

Rows are sorted by timestamp, and the columns that follow are specific to the data type:

• cgm: CGM values, one row per sensor reading
  • value: the measured CGM value, always in mg/dL (readings exported in mmol/L are converted).
  • unit: the unit of value (always mg/dL).
  • sensor_name: the name of the CGM sensor, NA if not available.
• basal: delivered basal rates, one row per interval over which the rate was constant
  • duration: the duration of the basal rate, in hours, starting at timestamp.
  • rate: the delivered basal rate, in U/h. The insulin delivered over the interval is rate * duration.
  • unit: the unit of rate (always U/h).
  • pump_name: the name of the pump, NA if not available.
• bolus: bolus information, one row per bolus
  • type: the type of the bolus. Either normal (delivered in one go), square_wave (delivered gradually and evenly over a longer period), dual_wave (part delivered immediately, the rest gradually over a longer period), biphasic (YourLoops only, a bolus split into two boluses, separated by a period of time; each part is its own row), or pen (YourLoops only).
  • total: the total amount of insulin in the bolus (normal + extended), in U.
  • normal: the portion delivered immediately, in U (applicable for normal, dual_wave and biphasic; 0 for square_wave).
  • extended: the portion delivered gradually over a longer period, in U (applicable for dual_wave and square_wave, 0 otherwise).
  • unit: the unit of total, normal and extended (always U).
  • duration_extended: the duration of the extended portion, in hours (for dual_wave and square_wave, NA otherwise, and also NA where the platform does not report it).
  • pump_name: the name of the pump, NA if not available.
  • link_id: an identifier linking the bolus to the carbohydrate entries that triggered it, NA if not available (see Linking boluses and carbohydrates).
• carbs: user-reported carbohydrate consumption, one row per entry
  • value: the carbohydrate value (in g).
  • unit: the unit of value (always g).
  • label: an optional label for the entry, NA if not available. Its content is platform-specific: it may be a meal name entered by the user, or summarize additional information on the entry (e.g. that it was a fatty meal).
  • estimated_absorption_duration: the absorption duration in hours, as estimated by the user, NA if not available.
  • is_hypo_treatment: a logical flagging entries labeled as hypoglycemia treatment, NA if not available.
  • link_id: an identifier linking the carbohydrate entry to the bolus it triggered, NA if not available (see Linking boluses and carbohydrates).
• SMBG: self-monitored blood glucose, one row per measurement
  • value: the measured SMBG value, always in mg/dL (readings exported in mmol/L are converted).
  • unit: the unit of value (always mg/dL).
  • sensor_name: the name of the SMBG device, NA if not available.

This layout is the same for all platforms and can therefore be merged across subjects.

In addition, two data types hold the daily insulin totals, with one row per day rather than one row per data point:

• total_basal: id, format, date, total_basal, and source
• total_bolus: id, format, date, total_bolus, and source

total_basal and total_bolus give the insulin delivered on that day, in units. Where a platform reports a daily aggregate itself, that value is used and source is "reported"; otherwise the total is summed over the cleaned basal (as rate * duration) or bolus data and source is "summed":

While the summed total merely reflects the underlying records, the reported total can indicate whether those records are complete. This matters in particular for Omnipod users on Glooko: the exported basal field contains only rates delivered in manual mode, omitting those delivered in automated mode, while the reported daily total covers both. Comparing the two therefore provides a check on the completeness of the export.

Note that there is no total_basal for mylife, since a mylife export contains no basal data at all, and none for CareLink, since a CareLink export reports only the scheduled basal rates rather than the basal insulin effectively delivered. For the same reason, neither platform yields a cleaned basal data type. A Dexcom Clarity export carries neither total_basal nor total_bolus, as it is a CGM export and contains no insulin data at all.

Merging multiple exports of one subject

If paths holds several exports, parse_data() merges them per data type and sorts each data frame by timestamp. However, exports may overlap, for example for consecutive downloads covering the same days, or the same pump uploaded to two platforms. aidR therefore removes duplicates in three steps:

1. Exact duplicates: rows that are identical in all columns except format are kept once (the first occurrence). Rows where all measurement columns are NA are dropped.
2. Daily totals: if a day appears more than once in total_basal or total_bolus (e.g. a partial day at the edge of one export and the full day in another), the largest value is kept, as it comes from the export covering the full day.
3. Near-duplicates (cleaned data only): different platforms may round timestamps or values differently, so the same event is not always exported identically. For carbs, bolus, SMBG, and basal, two rows are considered near-duplicates if they
   • come from different formats,
   • have the same unit,
   • are at most 5 minutes apart (2 minutes for basal), and
   • differ in value by at most 1 g (carbs), 0.05 U (total of bolus), 1 mg/dL (SMBG), or 0.05 U/h (rate of basal).
   Since basal rates change frequently, neighbouring basal rows often have similar rates. Two basal rows must therefore also have the same duration (within 1 minute); rows with a missing duration are never considered near-duplicates. Of each duplicate pair, the row with a link_id is kept to maintain the link of carbohydrate and bolus - if none of the two rows contains a link_id, the row with more non-missing columns is kept. Missing columns of the kept row (e.g. a label) are filled from the removed row, in which case format combines both export platforms to indicate that information is pulled from two sources (e.g. "glooko/tidepool"). The removed rows can be inspected with removed_entries().

Near-duplicates are not detected if two exports are shifted in time, e.g. due to daylight saving time or differing time zone settings. aidR therefore warns if the entries of two formats match much better when shifted by one hour, but leaves the data unchanged.

Linking boluses and carbohydrates

Since the cleaned data is split by data type, a bolus and the carbohydrates entered for it in the bolus calculator end up in two different data frames. For exports that are informative about which carbohydrate entry triggered which bolus, aidR keeps this information in the column link_id of both bolus and carbs: rows sharing a link_id belong to the same event. This is the case for

• Glooko and Tandem Source, where the bolus files hold the bolus and the entered carbohydrates in the same row, and
• Tidepool exports with a Bolus Calculator sheet, where the Bolus column refers to the Id of the bolus in the Bolus sheet.

For all other platforms (CareLink, mylife, YourLoops), and for Tidepool exports with a Food sheet, link_id is NA. The same goes for boluses delivered without entering carbohydrates. While it would be possible to match bolus and carbohydrate entries for these platforms using timestamps, aidR currently does not implement this.

Slowly absorbed carbohydrates in Glooko

CamAPS FX lets users flag a meal as slowly absorbed (over 3 to 4 hours). Although this is a single meal, Glooko exports encode it as a series of equal-sized carbohydrate entries, one every 30 minutes, possibly with unrelated entries in between. For example:

"","Zeitstempel","Insulin.Typ","Blutzuckereingabe..mg.dl.","Kohlenhydrataufnahme..g.","Kohlenhydratverhältnis","Abgegebenes.Insulin..E.","Anfängliche.Abgabe..E.","Verzögerte.Abgabe..E.","Seriennummer"
"1",2026-05-19 05:33:00,"Normal",0,21,NA,2.4,NA,NA,"CamAPS mylife YpsoPump"
"2",2026-05-19 05:34:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"3",2026-05-19 06:04:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"4",2026-05-19 06:34:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"5",2026-05-19 07:04:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"6",2026-05-19 07:34:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"7",2026-05-19 08:04:00,"Normal",0,1.2,NA,0,NA,NA,"CamAPS mylife YpsoPump"
"8",2026-05-19 14:18:00,"Normal",0,32,NA,2.9,NA,NA,"CamAPS mylife YpsoPump"
"9",2026-05-19 17:53:00,"Normal",0,27,NA,2.5,NA,NA,"CamAPS mylife YpsoPump"

Here, the user entered 7 g of slowly absorbed carbohydrates at 05:34 with an absorption time of 3 hours. The Glooko export splits this into six entries of 7/6 \(\approx\) 1.2 g, spaced 30 minutes apart.

To stay consistent with platforms that do not split slowly absorbed carbohydrates, aidR merges these entries back into a single entry and records the absorption time in the column estimated_absorption_duration. The merged entry keeps the timestamp of the first entry, holds the summed carbohydrate amount, and gets an estimated_absorption_duration of 0.5 hours per merged entry (3 hours in this example). The remaining entries and their associated zero-unit boluses are removed; you can inspect them with removed_entries().

To turn off merging, set merge_slow_carbs = FALSE in parse_data():

data <- parse_data(id = "1", paths = path, merge_slow_carbs = FALSE)

CGM

AID data is often noisy and may require additional filtering. While we encourage users to inspect the parsed data for platform-specific issues, aidR implements a few filters that we have found particularly useful. CGM data in particular is frequently affected in two ways: individual values may be physiologically implausible (below or above the range a sensor can measure), and the same period may be covered by two overlapping data streams whose timestamps are shifted by a few seconds or minutes but that otherwise carry the same signal (often the case for DIY open-source systems).

filter_CGM() addresses both. It first truncates values to a plausible range, clamping anything below truncate_min_cgm (40 mg/dL by default) up to that value and anything above truncate_max_cgm (400 mg/dL by default) down to it. It then downsamples the CGM series to a regular time grid (5-minute bins by default): each reading is assigned to a time bin, and within each bin only the reading closest to the bin’s start is kept. Two streams that differ only by a shift of a few seconds or minutes therefore fall into the same bins and collapse to a single value, so a duplicated stream is removed rather than doubling the number of readings for that period.

nrow(data_clean$cgm)
#> [1] 36873
data_clean_filter <- filter_CGM(data_clean)
nrow(data_clean_filter$cgm)
#> [1] 36702

The two truncation bounds and the downsampling interval can each be adjusted or disabled: truncate_min_cgm (default 40 mg/dL), truncate_max_cgm (default 400 mg/dL), and min_minutes_difference_cgm (default 5 minutes); set any of them to NULL to skip that particular step. Only CGM is filtered, and only once it has been cleaned (i.e. after parse_data(clean = TRUE)); all other data types are returned unchanged.

Carbohydrates

Raw data sometimes contains a pattern like this:

"","Zeitstempel","Insulin.Typ","Blutzuckereingabe..mg.dl.","Kohlenhydrataufnahme..g.","Kohlenhydratverhältnis","Abgegebenes.Insulin..E.","Anfängliche.Abgabe..E.","Verzögerte.Abgabe..E.","Seriennummer"
"1",2026-06-21 17:42:00,"Normal",353,0,NA,2.44,NA,NA,"XXX"
"2",2026-06-21 18:33:00,"Normal",0,403,NA,0,NA,NA,"XXX"
"3",2026-06-21 18:34:00,"Normal",0,40,5,8,NA,NA,"XXX"
"4",2026-06-21 20:29:00,"Normal",0,0,5,5,NA,NA,"XXX"
"5",2026-06-21 21:37:00,"Normal",381,0,NA,1.157,NA,NA,"XXX"

Here, the user apparently entered 403 g of carbohydrates at 18:33 by mistake. The corresponding bolus was cancelled (the delivered bolus is zero), and the correct amount was then entered with a new bolus (40g at 18:34). Some platforms, most notably Glooko, export all of these entries, so the same meal would be counted more than once. filter_carbs() detects and removes such cases. Specifically, it removes a carbohydrate entry if

• its bolus is zero. The bolus is found via link_id (see Linking boluses and carbohydrates), so only entries linked to a bolus can be removed. For platforms without links (CareLink, mylife, YourLoops, Tidepool with a Food sheet), nothing is removed;
• another carbohydrate entry of the same format (and with a different link_id) lies at most max_minutes_overwrite minutes (2 minutes by default) before or after it. The order of the two entries does not matter, and the bolus of the other entry is not checked;
• it is not flagged as a hypo treatment or as slowly absorbed carbohydrates.

The zero bolus of a removed entry is removed from the bolus data as well. All removed rows can be inspected with removed_entries(), together with the reason and, for carbohydrates, the entry that overwrote them (matched_to_timestamp, matched_to_value):

path <- system.file("extdata", "glooko_example.zip", package = "aidR")
data_clean_2 <- parse_data(id = "1", paths = path, clean = TRUE)
#> Parsing data for id 1...
data_clean_filter_2 <- filter_carbs(data_clean_2)
removed_entries(data_clean_filter_2)
#> $carbs
#>   id format           timestamp timezone_offset value unit label
#> 1  1 glooko 2026-06-21 22:19:00              NA   403    g    NA
#>   estimated_absorption_duration is_hypo_treatment
#> 1                            NA                NA
#>                               link_id matched_to_timestamp matched_to_value
#> 1 1|2026-06-21 22:19:00|403|0|NA|NA|1  2026-06-21 22:20:00               40
#>                   removed_reason
#> 1 overwritten carbohydrate entry
#> 
#> $bolus
#>   id format           timestamp timezone_offset   type total normal extended
#> 1  1 glooko 2026-06-21 22:19:00              NA normal     0      0        0
#>   unit duration_extended pump_name                             link_id
#> 1    U                NA        NA 1|2026-06-21 22:19:00|403|0|NA|NA|1
#>                                 removed_reason
#> 1 zero bolus of overwritten carbohydrate entry

Iglu

get_iglu_format() reshapes the cgm data frame into the id/time/gl layout expected by the iglu package, so that CGM metrics and visualizations from iglu can be used directly:

iglu_1 <- get_iglu_format(data_clean)
head(iglu_1)
# A tibble: 6 × 3
     id time                   gl
  <int> <dttm>              <dbl>
1     1 2019-01-10 01:01:41   120
2     1 2019-01-10 01:06:41   120
3     1 2019-01-10 01:11:40   118
4     1 2019-01-10 01:16:40   116
5     1 2019-01-10 01:21:40   113
6     1 2019-01-10 01:26:39   112
iglu::plot_agp(iglu_1)

Cgmquantify

get_cgmquantify_format() builds the data frame that the metric functions of the cgmquantify package expect, with columns Time, glucose, Date, time_of_day, and type_of_event. This is the same layout that cgmquantify::readfile() produces from a Dexcom export, so the result can be handed to any cgmquantify function directly and no intermediate file has to be written:

cgmq_1 <- get_cgmquantify_format(data_clean)
head(cgmq_1)
                 Time glucose       Date time_of_day type_of_event
1 2019-01-10 01:01:41     120 2019-01-10    01:01:41             0
2 2019-01-10 01:06:41     120 2019-01-10    01:06:41             0
3 2019-01-10 01:11:40     118 2019-01-10    01:11:40             0
4 2019-01-10 01:16:40     116 2019-01-10    01:16:40             0
5 2019-01-10 01:21:40     113 2019-01-10    01:21:40             0
6 2019-01-10 01:26:39     112 2019-01-10    01:26:39             0
cgmquantify::interdaycv(cgmq_1)
[1] 36.77765
cgmquantify::J_index(cgmq_1)
[1] 50.83163

Note that cgmquantify applies to a single individual only, and get_cgmquantify_format() requires the id argument as soon as data holds more than one id.

DIAX

write_DIAX_format() writes CGM, basal, bolus, carbs, and SMBG data to one JSON file per subject, following the DIAX specification. It accepts either the output of parse_data() for a single subject or the output of merge_all() for several, and writes one file per id to paste0(prefix, id, ".json"):

diax_dir <- tempfile()
dir.create(diax_dir)
write_DIAX_format(combined, prefix = file.path(diax_dir, "diax_id_"))
list.files(diax_dir)
#> [1] "diax_id_1.json" "diax_id_2.json"