The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Getting started with rmoriedata

rmoriedata bundles the open-data fixtures used across the rmorie ecosystem and ships a small set of analyst-facing helpers for releasing aggregate statistics without re-identification risk. Everything shown here runs offline against data that installs with the package – no network required.

The package has four surfaces:

  1. A bundled dataset store you browse and load by slug.
  2. Named loaders for the Chicago crime samples, the Ontario SIU director’s-report corpus, and the CIHI data-table catalogue.
  3. Differential-privacy mechanisms (morie_dp_*).
  4. Re-identification-risk verifiers (kanonvrf, ldivvrf, cellsupp).

1. Browsing the bundled store

Every bundled table ships as a CSV with a bundled column schema, listed in a SHA-256 manifest that is signed with the package’s XMSS key and verified on load (see ). mdcat() lists what’s there, with row/column counts and the original source path.

cat <- mdcat()
table(cat$kind)
#> 
#> dictionary      table 
#>          8         99
head(cat[cat$kind == "table", c("slug", "n_rows", "n_cols")])
#>                                                          slug n_rows n_cols
#> 2 arsau_2020_2022_useofforce_agrregatesummarybyyear_2020_2022      5      6
#> 3        arsau_2020_2022_useofforce_detaileddataset_2020_2022      5    167
#> 5                           arsau_2023_uof_individual_records      5    112
#> 6                                 arsau_2023_uof_main_records      5     23
#> 7                          arsau_2023_uof_probe_cycle_records      5      3
#> 8                   arsau_2023_uof_weapon_records_invaliddata      5      5

Load any table by its slug:

iucr <- mdload("chicago_iucr_codes")
head(iucr)
#>   iucr primary_description         secondary_description index_code active
#> 1 031A             ROBBERY               ARMED - HANDGUN          I   True
#> 2 031B             ROBBERY         ARMED - OTHER FIREARM          I   True
#> 3 033A             ROBBERY       ATTEMPT ARMED - HANDGUN          I   True
#> 4 033B             ROBBERY ATTEMPT ARMED - OTHER FIREARM          I   True
#> 5 041A             BATTERY          AGGRAVATED - HANDGUN          I   True
#> 6 041B             BATTERY    AGGRAVATED - OTHER FIREARM          I   True

Unknown slugs error with guidance rather than failing silently:

mdload("no_such_dataset")
#> Error:
#> ! No dataset 'no_such_dataset'. See mdcat() for valid slugs.

Some tables ship a data dictionary (JSON). Find them via the catalogue’s kind column:

dict_slugs <- cat$slug[cat$kind == "dictionary"]
head(dict_slugs)
#> [1] "arsau_2020_2022_dictionary" "arsau_2023_dictionary"     
#> [3] "arsau_2024_dictionary"      "corrections_uof_dictionary"
#> [5] "cpads_data_provenance"      "otis_dictionary"

2. The named loaders

Chicago crime samples

Two CRAN-safe slices of the City of Chicago open data ship as lazy-loaded data objects, and chiload() wraps them (with an optional full = TRUE network fetch of the complete dataset).

comp <- chiload("complaints")
dim(comp)
#> [1] 25000    16
head(sort(table(comp$primary_type), decreasing = TRUE), 5)
#> 
#>           THEFT         BATTERY CRIMINAL DAMAGE         ASSAULT   OTHER OFFENSE 
#>            5825            4800            2362            1975            1760

arr <- chiload("arrests")
sort(table(arr$charge_type), decreasing = TRUE)
#> 
#>     M           F 
#> 12342  6464  6194

Ontario SIU director’s reports

The first open, machine-readable parse of the Ontario Special Investigations Unit director’s-report corpus – one row per report, 64 structured columns.

en <- siuload(lang = "en")
nrow(en)
#> [1] 2309
head(sort(table(en$police_service), decreasing = TRUE), 5)
#> 
#>          Toronto Police Service       Ontario Provincial Police 
#>                             503                             450 
#>            Peel Regional Police Niagara Regional Police Service 
#>                             200                             102 
#>           Ottawa Police Service 
#>                              99

CIHI data-table catalogue

A catalogue of the public CIHI data-table workbooks, each with a live URL and an Internet Archive fallback so the table stays retrievable if CIHI rotates the file. (cihiget() does the actual download and is a network call, so it isn’t run here.)

tables <- cihitbls()
nrow(tables)
#> [1] 232
head(tables$title, 3)
#> [1] "Injury and Trauma Emergency Department and Hospitalization Statistics, 2024–2025"
#> [2] "Wait Times for Priority Procedures in Canada, 2008 to 2025 — Data Tables"        
#> [3] "Health Workforce in Canada, 2024 — Quick Stats"

3. Data integrity

Verify you’re using the exact data slice the package shipped:

ck <- mdsums()
head(ck[order(-ck$bytes), c("file", "bytes")], 3)
#>                              file   bytes
#> 181  siu_directors_reports.csv.gz 2378925
#> 153 siu_directors_reports.parquet 2053237
#> 34            describe_corpus.Rds 1712072

# The same compiled SHA256 kernel the whole ecosystem uses:
mdsha("abc")
#> [1] "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"

4. Differential privacy

Release counts, means, and histograms with calibrated noise. Smaller epsilon means stronger privacy and more noise.

set.seed(1)

# A private count of records matching a predicate.
dplapcnt(truecnt = 42, epsilon = 1.0)
#> [1] 43

# The mechanism is unbiased -- averaging many releases recovers the truth.
mean(replicate(2000, dplapcnt(42, epsilon = 1.0)))
#> [1] 41.949

# A private mean of bounded data.
x <- runif(1000, 0, 1)
dpgmean(x, lower = 0, upper = 1, epsilon = 1.0)
#> [1] 0.4708987

# A private histogram straight from tabulated data.
counts <- as.integer(table(comp$year))
round(pmax(0, dplaphist(counts, epsilon = 1.0)))
#> [1] 24998     2

5. Re-identification risk

Check a release against the standard disclosure-control thresholds before publishing it.

df <- data.frame(
  age = c(25, 25, 25, 32, 32, 40),
  sex = c("F", "F", "F", "M", "M", "M")
)

# k-anonymity: every quasi-identifier combo must appear >= k times.
kanonvrf(df, c("age", "sex"), k = 2)$summary
#> [1] "k=2: VIOLATED (min class size=1; 1/3 classes below threshold)"

# l-diversity: each class must hold >= l distinct sensitive values.
df2 <- data.frame(
  age = c(25, 25, 25, 25, 32, 32, 32),
  sex = c("F", "F", "F", "F", "M", "M", "M"),
  dx  = c("A", "B", "C", "A", "X", "Y", "Z")
)
ldivvrf(df2, c("age", "sex"), "dx", l = 3)$summary
#> [1] "l=3: SATISFIED (min diversity=3; 0/2 classes below threshold)"

Cell suppression hides small counts and (by default) applies complementary suppression so the hidden values can’t be recovered from the marginals:

tbl <- matrix(c(120, 3, 47, 88, 2, 99, 14, 51, 60), nrow = 3,
              dimnames = list(c("A", "B", "C"), c("X", "Y", "Z")))
res <- cellsupp(tbl, threshold = 5)
res$suppressed
#>    X  Y  Z
#> A NA NA 14
#> B NA NA 51
#> C 47 99 60

Bridging to Python

chiload(..., as = "parquet_path") writes a Parquet file and returns its path – the recommended hand-off to pandas.read_parquet(). The file comes from the package’s own Parquet writer, so it loads natively in pandas, polars, DuckDB and Arrow with no R runtime. The bundled store itself is one CSV per table plus a column schema; read it with mdload() in R or with any CSV reader elsewhere.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.