Data Encryption

The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Data Encryption

Rich FitzJohn

2022-06-20

A group of people are working on a sensitive data set that for practical reasons needs to be stored in a place that we’re not 100% happy with the security (e.g., Dropbox), or we’re concerned that files stored in plain text on users computers (e.g. laptops) may lead to the data being compromised.

If the data can be stored encrypted but everyone in the group can still read and write the data then we’ve improved the situation somewhat. But organising for everyone to get a copy of the key to decrypt the data files is non-trivial. The workflow described here aims to simplify this procedure using lower-level functions in the cyphr package.

Before doing any of this, everyone needs to have ssh keys set up. By default the package will use your ssh keys found at “~/.ssh”; see the main package vignette for how to use this.

For clarity here we will generate two sets of key pairs for two actors Alice and Bob:

These would ordinarily be on different machines (nobody has access to anyone else’s private key) and they would be password protected. In the function calls below, all the path_user arguments would be omitted.

We’ll store data in the directory data; at present there is nothing there (this is in a temporary directory for compliance with CRAN policies but would ordinarily be somewhere persistent and under version control ideally).

First, create a personal set of keys. These will be shared across all projects and stored away from the data. Ideally one would do this with ssh-keygen at the command line, following one of the many guides available. A utility function ssh_keygen (which simply calls ssh-keygen for you) is available in this package though. You will need to generate a key on each computer you want access from. Don’t copy the key around. If you lose your user key you will lose access to the data!

Second, create a key for the data and encrypt that key with your personal key. Note that the data key is never stored directly - it is always stored encrypted by a personal key.

The data key is very important. If it is deleted, then the data cannot be decrypted. So do not delete the directory data_dir/.cyphr! Ideally add it to your version control system so that it cannot be lost. Of course, if you’re working in a group, there are multiple copies of the data key (each encrypted with a different person’s personal key) which reduces the chance of total loss.

This command can be run multiple times safely; if it detects it has been rerun and the data key will not be regenerated.

Third, you can add encrypted data to the directory (or to anywhere really). When run, cyphr::config_data will verify that it can actually decrypt things.

This object can be used with all the cyphr functions (see the “cyphr” vignette; vignette("cyphr"))

Fourth, have someone else join in. Recall that to simulate another person here, I’m going to pass an argument path_user = path_key_bob though to the functions. This contains the path to “Bob”’s ssh keypair. If run on an actually different computer this would not be needed; this is just to simulate two users in a single session for this vignette (see minimal example below where this is simulated). Again, typically this user would also not use the cyphr::ssh_keygen function but use the ssh-keygen command from their shell.

We’re going to assume that the user can read and write to the data. This is the case for my use case where the data are stored on dropbox and will be the case with GitHub based distribution, though there would be a pull request step in here.

This user cannot read the data, though trying to will print a message explaining how you might request access:

The user should the send an email to someone with access and quote the hash in the message above.

Fifth, back on the first computer we can authorise the second user. First, see who has requested access:

We can see the same hash here as above (21549b4ad9d168377ccd9e3180be78fe54298c077e83e21a7b6c5ad89a118098)

…and then grant access to them with the cyphr::data_admin_authorise function.

If you do not specify yes = TRUE will prompt for confirmation at each key added.

Finally, as soon as the authorisation has happened, the user can encrypt and decrypt files:

Minimal example

As above, but with less discussion:

Setup, on Alice’s computer:

cyphr::data_admin_init(data_dir, path_user = path_key_alice)

## Generating data key

## Authorising ourselves

## Adding key 91:e2:bf:0b:1d:d0:0b:c9:81:af:c7:9a:8a:60:c6:0d:ff:ac:17:b6:63:0c:bc:56:e8:cc:cf:c4:bd:cf:70:25
##   user: rich
##   host: wpia-dide136
##   date: 2022-06-20 10:55:37

## Verifying

Get the data key key:

key <- cyphr::data_key(data_dir, path_user = path_key_alice)

Encrypt a file:

cyphr::encrypt(saveRDS(iris, filename), key)

Request access, on Bob’s computer:

hash <- cyphr::data_request_access(data_dir, path_user = path_key_bob)

## A request has been added

## Email someone with access to add you
## 
##     hash: 21:54:9b:4a:d9:d1:68:37:7c:cd:9e:31:80:be:78:fe:54:29:8c:07:7e:83:e2:1a:7b:6c:5a:d8:9a:11:80:98
## 
## If you are using git, you will need to commit and push first:
## 
##     git add .cyphr
##     git commit -m "Please add me to the dataset"
##     git push

Alice authorises this request::

cyphr::data_admin_authorise(data_dir, yes = TRUE, path_user = path_key_alice)

## There is 1 request for access

## Adding key 21:54:9b:4a:d9:d1:68:37:7c:cd:9e:31:80:be:78:fe:54:29:8c:07:7e:83:e2:1a:7b:6c:5a:d8:9a:11:80:98
##   user: rich
##   host: wpia-dide136
##   date: 2022-06-20 10:55:38

## Added 1 key

## If you are using git, you will need to commit and push:
## 
##     git add .cyphr
##     git commit -m "Authorised rich"
##     git push

Bob can get the data key:

key <- cyphr::data_key(data_dir, path_user = path_key_bob)

Bob can read the secret data:

head(cyphr::decrypt(readRDS(filename), key))

##   Sepal.Length Sepal.Width Petal.Length Petal.Width Species
## 1          5.1         3.5          1.4         0.2  setosa
## 2          4.9         3.0          1.4         0.2  setosa
## 3          4.7         3.2          1.3         0.2  setosa
## 4          4.6         3.1          1.5         0.2  setosa
## 5          5.0         3.6          1.4         0.2  setosa
## 6          5.4         3.9          1.7         0.4  setosa

Details & disclosure

Encryption does not work through security through obscurity; it works because we can rely on the underlying maths enough to be open about how things are stored and where.

Most encryption libraries require some degree of security in the underlying software. Because of the way R works this is very difficult to guarantee; it is trivial to rewrite code in running packages to skip past verification checks. So this package is not designed to (or able to) avoid exploits in your running code; an attacker could intercept your private keys, the private key to the data, or skip the verification checks that are used to make sure that the keys you load are what they say they are. However, the data are safe; only people who have keys to the data will be able to read it.

cyphr uses two different encryption algorithms; it uses RSA encryption via the openssl package for user keys, because there is a common file format for these keys so it makes user configuration easier. It uses the modern sodium package (and through that the libsodium library) for data encryption because it is very fast and simple to work with. This does leave two possible points of weakness as a vulnerability in either of these libraries could lead to an exploit that could allow decryption of your data.

Each user has a public/private key pair. Typically this is in ~/.ssh/id_rsa.pub and ~/.ssh/id_rsa, and if found these will be used. Alternatively the location of the keypair can be stored elsewhere and pointed at with the USER_KEY or USER_PUBKEY environment variables. The key may be password protected (and this is recommended!) and the password will be requested without ever echoing it to the terminal.

The data directory has a hidden directory .cyphr in it.

dir(data_dir, all.files = TRUE, no.. = TRUE)

## [1] ".cyphr"   "iris.rds"

This does not actually need to be stored with the data but it makes sense to (there are workflows where data is stored remotely where storing this directory might make sense). The “keys” directory contains a number of files; one for each person who has access to the data.

dir(file.path(data_dir, ".cyphr", "keys"))

## [1] "21549b4ad9d168377ccd9e3180be78fe54298c077e83e21a7b6c5ad89a118098"
## [2] "91e2bf0b1dd00bc981afc79a8a60c60dffac17b6630cbc56e8cccfc4bdcf7025"

names(cyphr::data_admin_list_keys(data_dir))

## [1] "21549b4ad9d168377ccd9e3180be78fe54298c077e83e21a7b6c5ad89a118098"
## [2] "91e2bf0b1dd00bc981afc79a8a60c60dffac17b6630cbc56e8cccfc4bdcf7025"

(the file test is a small file encrypted with the data key used to verify everything is working OK).

Each file is stored in RDS format and is a list with elements:

user: the reported user name of the person who created request for data
host: the reported computer name
date: the time the request was generated
pub: the RSA public key of the user
key: the data key, encrypted with the user key. Without the private key, this cannot be used. With the user’s private key this can be used to generate the symmetric key to the data.

h <- names(cyphr::data_admin_list_keys(data_dir))[[1]]
readRDS(file.path(data_dir, ".cyphr", "keys", h))

## $user
## [1] "rich"
## 
## $host
## [1] "wpia-dide136"
## 
## $date
## [1] "2022-06-20 10:55:38 BST"
## 
## $pub
## [2048-bit rsa public key]
## md5: d85323cf26b481aff3975c96725807ad
## 
## $key
##   [1] ac 36 2f 00 66 02 3a 62 46 83 b4 e6 4f ae eb 79 86 48 81 10 fb c3 8a 03 e1
##  [26] 12 c5 9e 9f 42 c0 4d 5b 69 f8 d2 bd 98 d8 7d 30 66 92 6a 60 0d 99 df e9 67
##  [51] 0e d9 dd ee 55 94 b4 62 db d2 b1 47 58 2b e5 dd ae 5d 83 22 4c ca e4 2b 5f
##  [76] c5 4c fd 46 c3 43 50 91 6b 92 bd 3c d2 11 e5 32 71 21 c9 24 24 64 82 6c cf
## [101] a0 56 c4 42 c5 a1 9c 73 42 d4 6a a2 4d cb 9d 82 b4 05 5b 59 24 ea 16 28 c3
## [126] 8a cc a1 16 4e 13 f3 9f fc ce 75 cb 13 8a 5f 8a 82 5c 03 84 33 db 46 e0 44
## [151] 22 ec 85 32 3a 5e b8 b8 05 6c cf 4b 0c af e1 e1 78 90 4c 8b 2b b0 a6 f5 d5
## [176] df 5f bc 61 48 3f a4 f7 03 f0 09 08 0d 0f cd 4c 1b 1e f4 b1 8f 3d 72 57 36
## [201] db 09 11 e0 46 da 8f e5 90 a9 b9 23 9b 0c ff a4 9a d9 21 b5 1c 70 7f 77 f8
## [226] 24 c8 54 4d b1 b3 7d 18 4e 28 ea 9c 1e 8b 4b a3 a7 bd 62 89 83 83 22 74 c7
## [251] 6d dd 14 2d 2e ad

You can see that the hash of the public key is the same as name of the stored file here (which is used to prevent collisions when multiple people request access at the same time).

## [1] "21549b4ad9d168377ccd9e3180be78fe54298c077e83e21a7b6c5ad89a118098"

When a request is posted it is an RDS file with all of the above except for the key element, which is added during authorisation.

(Note that the verification relies on the package code not being attacked, and given R’s highly dynamic nature an attacker could easily swap out the definition for the verification function with something that always returns TRUE.)

When an authorised user creates the data_key object (which allows decryption of the data) secret will:

read their private user key (probably from ~/.ssh/id_rsa)
read the encrypted data key from the data directory (the $key element from the list above).
decrypt this data key using their user key to yield the the data symmetric key.

Limitations

In the Dropbox scenario, non-password protected keys will afford only limited protection. This is because even though the keys and data are stored separately on Dropbox, they will be in the same place on a local computer; if that computer is lost then the only thing preventing an attacker recovering the data is security through obscurity (the data would appear to be random junk but they will be able to run your analysis scripts as easily as you can). Password protected keys will improve this situation considerably as without a password the data cannot be recovered.

The data is not encrypted during a running R session. R allows arbitrary modification of code at runtime so this package provides no security from the point where the data can be decrypted. If your computer was compromised then stealing the data while you are running R should be assumed to be straightforward.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.