Rraven: Connecting R and Raven Sound Analysis Software

imp_raven

This function imports Raven selection tables. Multiple files can be imported at once. Raven selection tables including data from multiple recordings can also be imported. It returns a single data frame with the information contained in the selection files. We already have 2 Raven selection tables in the working directory:

list.files(path = tempdir(), pattern = "\\.txt$")

[1] "LBH 1 selection table example.txt" "LBH 2 selection table example.txt" "LBH 3 selection table example.txt" "LBH 4 selection table example.txt"

 

This code shows how to import all the data contained in those files into R:

 #providing the name of the column with the sound file names
rvn.dat <- imp_raven(all.data = TRUE)

head(rvn.dat)

 

Note that the ‘waveform’ view data has been removed. It can also be imported as follows (but note that the example selection tables have no waveform data):

rvn.dat <- imp_raven(all.data = TRUE, waveform = TRUE)

 

Raven selections can also be imported in a ‘selection.table’ format so it can be directly input into warbleR functions. To do this you need to set the all.data = FALSE and indicate which column contains the sound file name (using the ‘sound.file.col’ argument):

 #providing the name of the column with the sound file names
rvn.dat <- imp_raven(sound.file.col = "End.File", all.data = FALSE, freq.cols = TRUE)

head(rvn.dat)

sound.files	channel.Channel	channel.channel	selec	start	end	selec.file	bottom.freq	top.freq
Phae.long1.wav	1	1	1	1.1693549	1.3423884	LBH 1 selection table example.txt	2.220105	8.604378
Phae.long1.wav	1	1	2	2.1584085	2.3214565	LBH 1 selection table example.txt	2.169437	8.807053
Phae.long1.wav	1	1	3	0.3433366	0.5182553	LBH 1 selection table example.txt	2.218294	8.756604
Phae.long2.wav	1	1	1	0.1595983	0.2921692	LBH 2 selection table example.txt	2.316862	8.822316
Phae.long2.wav	1	1	2	1.4570585	1.5832087	LBH 2 selection table example.txt	2.284006	8.888027
Phae.long3.wav	1	1	1	0.6265520	0.7577715	LBH 3 selection table example.txt	3.006834	8.822316

 

The data frame contains the following columns: sound.files, channel, selec, start, end, and selec.file. You can also import the frequency range parameters in the ‘selection.table’ by setting ‘freq.cols’ tp TRUE. The data frame returned by “imp_raven” (when in the ‘warbleR’ format) can be input into several warbleR functions for further analysis. For instance, the following code runs additional parameter measurements on the imported selections:

# convert to class selection.table
rvn.dat.st <- make.selection.table(rvn.dat)

sp <- specan(X = rvn.dat, bp = "frange", wl = 150, pb = FALSE, ovlp = 90)

head(sp)

checking selections (step 1 of 2):
all selections are OK 

 

And this code creates song catalogs:

catalog(X = rvn.dat.st[1:9, ], flim = c(1, 10), nrow = 3, ncol = 3, same.time.scale = F,
 ovlp = 90, parallel = 1, mar = 0.01, wl = 200, pal = reverse.heat.colors, width = 20,  labels = c("sound.files", "selec"), legend = 1, 
 tag.pal = list(terrain.colors), tags = "sound.files")

extract_ts

The function extracts parameters encoded as time series in Raven selection tables. The resulting data frame can be directly input into functions for time series analysis of acoustic signals as in the warbleR function dfDTW. The function needs an R data frame, so the data should have been previously imported using imp_raven. This example uses the selection_file.ts example data that comes with Rraven:

#remove previous raven data files
unlink(list.files(pattern = "\\.txt$"))

#save Raven selection table in the temporary directory
writeLines(selection_files[[5]], con = names(selection_files)[5])

rvn.dat <- imp_raven(all.data = TRUE) 

# Peak freq contour dif length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)")
 
head(fcts)

 

Note that these sequences are not all of equal length (one has NAs at the end). extract_ts can also interpolate values so all time series have the same length:

# Peak freq contour equal length
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)",  equal.length = TRUE)

#look at the last rows wit no NAs
head(fcts)

 

And the length of the series can also be specified:

# Peak freq contour equal length 10 measurements
fcts <- extract_ts(X = rvn.dat, ts.column = "Peak Freq Contour (Hz)",
equal.length = T, length.out = 10)  

head(fcts)

sound.files	selec	PFC..1	PFC..2	PFC..3	PFC..4	PFC..5	PFC..6	PFC..7	PFC..8	PFC..9	PFC..10
Phae.long1.wav	1	6943.4	7431.633	5449.200	6533.200	6376.933	4736.333	6767.600	7207.000	7119.100	7119.1
Phae.long1.wav	2	6767.6	7402.333	5263.633	6650.400	6357.411	5898.467	4951.167	7041.011	7207.000	7207.0
Phae.long1.wav	3	6943.4	4921.900	7792.933	7236.333	6347.633	5000.011	6767.600	7040.967	7128.867	7119.1
Phae.long2.wav	4	5888.7	5263.633	5292.967	6533.200	7109.367	5986.344	5742.167	7363.267	7822.300	6152.3
Phae.long2.wav	5	4570.3	5097.700	5888.700	4570.300	6855.500	5712.900	6240.200	6943.400	7558.600	5537.1
Phae.long3.wav	6	4218.8	6416.000	6533.200	6679.700	6650.400	6943.333	7470.700	7617.200	7470.733	6855.5

 

The time series data frame can be directly input into the dfDTW warbleR function to calculate Dynamic Time Warping distances:

dfDTW(ts.df = fcts)

 

---

relabel_colms

This is a simple function to relabel columns so they match the selection table format used in warbleR:

#to simplify the example select a subset of the columns 
st1 <- rvn.dat[ ,1:7]

#check original column names
st1

# Relabel the basic columns required by warbleR
relabel_colms(st1)

 

Additional columns can also be relabeled:

# 2 additional column 
relabel_colms(st1, extra.cols.name = "View",
              extra.cols.new.name = "Raven view")

selec	Raven view	Channel	start	end	bottom.freq	top.freq
1	Spectrogram 1	1	1.169355	1.342388	2220.1	8604.4
2	Spectrogram 1	1	2.158408	2.321457	2169.4	8807.1
3	Spectrogram 1	1	0.343337	0.518255	2218.3	8756.6
4	Spectrogram 1	1	2.659643	2.792214	2316.9	8822.3
5	Spectrogram 1	1	3.957103	4.083253	2284.0	8888.0
6	Spectrogram 1	1	4.826641	4.957860	3006.8	8822.3
7	Spectrogram 1	1	6.174302	6.304481	2776.8	8888.0
8	Spectrogram 1	1	4.323453	4.454670	2316.9	9315.2
9	Spectrogram 1	1	7.916900	8.062325	2514.0	9216.6
10	Spectrogram 1	1	9.332781	9.476967	2579.7	10235.1
11	Spectrogram 1	1	6.545487	6.690585	2579.7	9742.3

 

---

imp_corr_mat

The function imports the output of a batch correlation routine in Raven. Both the correlation and lag matrices contained in the output ‘.txt’ file are read and both waveform and spectrogram (cross-correlation) correlations can be imported.

This example shows how to input the sound files into Raven and how to bring the results back to R. First, the selections need to be cut as single sound files for the Raven correlator to be able to read it. We can do this using the cut_sels function from warbleR:

#create new folder to put cuts
dir.create("cuts")

# add a rowname column to be able to match cuts and selections
selec.table$rownames <- sprintf("%02d",1:nrow(selec.table))

# cut files
cut_sels(X = selec.table, mar = 0.05, path = tempdir(), dest.path = file.path(tempdir(), "cuts"), labels = c("rownames", "sound.files", "selec"), pb = FALSE)

#list cuts
list.files(path = file.path(tempdir(), "cuts"))

 

Every selection is in its own sound file (labeled as paste("rownames", "sound.files", "selec")). Now open Raven and run the batch correlator on the ‘cuts’ folder as follows:

 

And then import the output file into R:

# Import output (change the name of the file if you used a different one)
xcorr.rav <- imp_corr_mat(file = "BatchCorrOutput.txt", path = tempdir())

 

The function returns a list containing the correlation matrix:

xcorr.rav$correlation

 

 

and the time lag matrix:

xcorr.rav$`lag (s)`

 

 

This output is ready for stats. For instance, the following code runs a mantel test between cross-correlation (converted to distances) and warbleR spectral parameter pairwise dissimilarities:

#convert cross-corr to distance
xcorr.rvn <- 1- xcorr.rav$correlation

#sort matrix to match selection table
xcorr.rvn <- xcorr.rvn[order(rownames(xcorr.rvn)), order(colnames(xcorr.rvn))]

#convert it to distance matrix
xcorr.rvn <- as.dist(xcorr.rvn)

# measure acoustic parameters
sp.wrblR <- specan(selec.table, bp = c(1, 11), wl = 150, pb = FALSE)

#convert them to distance matrix
dist.sp.wrblR <- dist(sp.wrblR)

Warning in dist(sp.wrblR): NAs introduced by coercion

vegan::mantel(xcorr.rvn, dist.sp.wrblR)

Mantel statistic based on Pearson's product-moment correlation 

Call:
vegan::mantel(xdis = xcorr.rvn, ydis = dist.sp.wrblR) 

Mantel statistic r: 0.259 
      Significance: 0.019 

Upper quantiles of permutations (null model):
  90%   95% 97.5%   99% 
0.145 0.182 0.230 0.294 
Permutation: free
Number of permutations: 999

 

There is actually a good match between the two methods!

---

exp_raven

exp_raven saves a selection table in ‘.txt’ format that can be directly opened in Raven. No objects are returned into the R environment. The following code exports a data table from a single sound file:

# Select data for a single sound file
st1 <- selec.table[selec.table$sound.files == "Phae.long1.wav",]

# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE)

 

If the path to the sound file is provided, the functions exports a ‘sound selection table’ which can be directly open by Raven (and which will also open the associated sound file):

# Select data for a single sound file
st1 <- selec.table[selec.table$sound.files == "Phae.long1.wav",]

# Export data of a single sound file
exp_raven(st1, file.name = "Phaethornis 1", khz.to.hz = TRUE, sound.file.path = tempdir())

 

This is useful to add new selections or even new measurements:

 

If several sound files are available, users can either export them as a single selection file or as multiple selection files (one for each sound file). This example creates a multiple sound file selection:

exp_raven(X = selec.table, file.name = "Phaethornis multiple sound files", 
sound.file.path = tempdir(), single.file = TRUE)

 

These type of tables can be opened as a multiple file display in Raven:

 

---

run_raven

The function opens multiple sound files simultaneously in Raven. When the analysis is finished (and the Raven window is closed) the data can be automatically imported back into R using the ‘import’ argument. Note that Raven, unlike R, can also handle files in ‘mp3’, ‘flac’ and ‘aif’ format .

# here replace with the path where Raven is install in your computer
raven.path <- "PATH_TO_RAVEN_DIRECTORY_HERE" 

# run function 
run_raven(raven.path = raven.path, sound.files = c("Phae.long1.wav", "Phae.long2.wav", "Phae.long3.wav", "Phae.long4.wav"), import = TRUE, 
 all.data = TRUE)  

 

See imp_raven above for more details on additional settings when importing selections.

---

raven_batch_detec

As the name suggests, raven_batch_detec runs Raven detector on multiple sound files (sequentially). Batch detection in Raven can also take files in ‘mp3’, ‘flac’ and ‘aif’ format (although this could not be further analyzed in R!).

This is example runs the detector on one of the example sound files that comes by default with Raven:

detec.res <- raven_batch_detec(raven.path = raven.path, 
sound.files = "BlackCappedVireo.aif", path = file.path(raven.path, "Examples"))

 

---

Please report any bugs here. The Rraven package should be cited as follows:

Araya-Salas, M. (2017), Rraven: connecting R and Raven bioacoustic software. R package version 1.0.0.

---

Session information

R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.10

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.3.3.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=C               LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] kableExtra_0.9.0   knitr_1.20         Rraven_1.0.5       warbleR_1.1.16     NatureSounds_1.0.1 seewave_2.1.0      tuneR_1.3.3       
[8] maps_3.3.0        

loaded via a namespace (and not attached):
 [1] pbapply_1.4-0     lattice_0.20-35   colorspace_1.3-2  htmltools_0.3.6   viridisLite_0.3.0 yaml_2.2.0        mgcv_1.8-24       rlang_0.3.1      
 [9] pracma_2.2.2      pillar_1.3.0      jpeg_0.1-8        stringr_1.3.1     munsell_0.5.0     rvest_0.3.2       evaluate_0.12     permute_0.9-4    
[17] fftw_1.0-4        parallel_3.5.1    highr_0.7         Rcpp_1.0.0        readr_1.3.1       backports_1.1.3   scales_1.0.0      vegan_2.5-3      
[25] soundgen_1.3.2    Sim.DiffProc_4.3  Deriv_3.8.5       rjson_0.2.20      hms_0.4.2         digest_0.6.18     stringi_1.2.4     dtw_1.20-1       
[33] rprojroot_1.3-2   grid_3.5.1        tools_3.5.1       bitops_1.0-6      magrittr_1.5      RCurl_1.95-4.11   proxy_0.4-22      tibble_1.4.2     
[41] cluster_2.0.7-1   crayon_1.3.4      pkgconfig_2.0.2   Matrix_1.2-14     MASS_7.3-50       xml2_1.2.0        rmarkdown_1.10    httr_1.4.0       
[49] rstudioapi_0.9.0  iterators_1.0.10  R6_2.3.0          signal_0.7-6      nlme_3.1-137      compiler_3.5.1