In this Vignette, I’ll illustrate the new functionality of the OpenImageR package (Gabor Feature Extraction). The Gabor features have been used extensively in image analysis and processing (Character and Face recognition). Gabor (Nobel prize winner, an electrical engineer, and physicist) used the following wording which I think it’s worth mentioning in this Vignette, “You can’t predict the future, but you can invent it.” (source: https://www.it.lut.fi/project/gabor/).
gabor_filter_bank
The gabor_filter_bank method “… generates a custom-sized Gabor filter bank. It creates a UxV cell array, whose elements are MxN matrices; each matrix being a 2-D Gabor filter” according to the author of the Matlab code. The following code chunk shows how this works in R,
library(OpenImageR)
init_gb = GaborFeatureExtract$new()
#------------------
# gabor-filter-bank
#------------------
gb_f = init_gb$gabor_filter_bank(scales = 5, orientations = 8, gabor_rows = 39,
gabor_columns = 39, plot_data = TRUE)
#-----------------------------------------------
# plot of the real part of the gabor-filter-bank
#-----------------------------------------------
plt_f = init_gb$plot_gabor(real_matrices = gb_f$gabor_real, margin_btw_plots = 0.65,
thresholding = FALSE)
For the gabor_filter_bank I use 5 scales and 8 orientations to build filters of size 39 x 39. The output of the method is a list of length 3,
str(gb_f)
List of 3
$ gaborArray :List of 40
..$ : cplx [1:39, 1:39] -1.58e-12-0.00i 0.00-5.02e-12i 1.50e-11-0.00i ...
..$ : cplx [1:39, 1:39] 4.86e-08-3.96e-08i 1.02e-07+4.63e-08i 6.31e-09+1.93e-07i ...
..$ : cplx [1:39, 1:39] 6.24e-06-6.24e-06i 1.18e-05+0.00i 1.10e-05+1.10e-05i ...
..$ : cplx [1:39, 1:39] -6.69e-05-3.18e-05i -4.63e-05-7.20e-05i -1.60e-06-9.81e-05i ...
..$ : cplx [1:39, 1:39] 1.40e-04+5.81e-05i 1.15e-04+1.15e-04i 6.68e-05+1.61e-04i ...
.......
$ gabor_imaginary:List of 40
..$ : num [1:39, 1:39] -4.65e-27 -5.02e-12 -1.10e-26 4.21e-11 -2.99e-25 ...
..$ : num [1:39, 1:39] -3.96e-08 4.63e-08 1.93e-07 1.53e-07 -3.04e-07 ...
..$ : num [1:39, 1:39] -6.24e-06 4.84e-20 1.10e-05 2.01e-05 1.81e-05 ...
..$ : num [1:39, 1:39] -3.18e-05 -7.20e-05 -9.81e-05 -9.58e-05 -5.78e-05 ...
..$ : num [1:39, 1:39] 5.81e-05 1.15e-04 1.61e-04 1.86e-04 1.83e-04 ...
.......
$ gabor_real :List of 40
..$ : num [1:39, 1:39] -1.58e-12 5.54e-27 1.50e-11 -4.12e-26 -1.11e-10 ...
..$ : num [1:39, 1:39] 4.86e-08 1.02e-07 6.31e-09 -2.85e-07 -4.28e-07 ...
..$ : num [1:39, 1:39] 6.24e-06 1.18e-05 1.10e-05 -8.11e-20 -1.81e-05 ...
..$ : num [1:39, 1:39] -6.69e-05 -4.63e-05 -1.60e-06 5.73e-05 1.12e-04 ...
..$ : num [1:39, 1:39] 1.40e-04 1.15e-04 6.68e-05 -3.77e-19 -7.57e-05 ...
.......
The first sublist (gaborArray) consists of 40 matrices (5 scales x 8 orientations) of type complex, where each matix is of dimension 39 x 39 (gabor filter). The second sublist (gabor_imaginary) is the imaginary part (numeric) whereas the third sublist is the real part (gabor_real). The real part (numeric) is used to plot the gabor filters.
The documentation includes more details for each of the parameters used.
gabor_feature_engine
The gabor_feature_engine method is an extension of the initial Matlab code and allows the user to extract gabor features from multiple images. This method works in the same way as the HOG_apply method, which takes a matrix of images - such as the mnist data set - and after processing it returns the features. The following example illustrates how to use the gabor_feature_engine method with the mnist data set,
# for instance downloading of the mnist data on a linux OS
#----------------------------------------------------------
system("wget https://raw.githubusercontent.com/mlampros/DataSets/master/mnist.zip")
# or just navigate to "https://github.com/mlampros/DataSets/blob/master/mnist.zip" and click the download button
#-------------------------------------------------------------------------------------------
mnist <- read.table(unz("mnist.zip", "mnist.csv"), nrows = 70000, header = T,
quote = "\"", sep = ",")
mnist = as.matrix(mnist)
x = mnist[, -ncol(mnist)]
y = mnist[, ncol(mnist)] + 1
dat = init_gb$gabor_feature_engine(img_data = x, img_nrow = 28, img_ncol = 28,
scales = 4, orientations = 8, gabor_rows = 13,
gabor_columns = 13, downsample_gabor = FALSE,
downsample_rows = NULL, downsample_cols = NULL,
normalize_features = FALSE, threads = 6,
verbose = TRUE)
Time to complete : 4.111672 mins
> str(dat)
List of 2
$ magnitude : num [1:70000, 1:3136] 0 0 0 0 0 0 0 0 0 0 ...
$ energy_aptitude: num [1:70000, 1:64] 2682 2576 1399 1178 2240 ...
The dat object is a list of length 2. The first sublist corresponds to magnitude whereas the second sublist to local-energy and mean-aptitude. The first thing to do before calculating the accuracy for the mnist data is to reduce the dimensionality of the magnitude feature (I’ll use the irlba package for this purpose),
svd_irlb = irlba::irlba(as.matrix(dat$magnitude), nv = 100, nu = 100, verbose = T)
new_x = as.matrix(dat$magnitude) %*% svd_irlb$v
and I’ll create a centered-scaled matrix of the reduced magnitude and the energy-aptitude data,
dat = ClusterR::center_scale(cbind(new_x, dat$energy_aptitude))
dim(dat)
[1] 70000 164
then I’ll utilize the nmslibR library (approximate method ‘hnsw’) to calculate the accuracy for the mnist data,
M = 30
efC = 100
num_threads = 6
index_params = list('M'= M, 'indexThreadQty' = num_threads, 'efConstruction' = efC,
'post' = 0, 'skip_optimized_index' = 1 )
efS = 100
query_time_params = list('efSearch' = efS)
fit_gab = nmslibR::KernelKnnCV_nmslib(dat, y, k = 20, folds = 4, h = 1,
weights_function = 'biweight_triangular_tricube_MULT',
Levels = sort(unique(y)), Index_Params = index_params,
Time_Params = query_time_params, space = "cosinesimil",
space_params = NULL, method = "hnsw",
data_type = "DENSE_VECTOR", dtype = "FLOAT",
index_filepath = NULL, print_progress = T,
num_threads = 6, seed_num = 1)
time to complete : 30.82252 secs
acc_fit_gab = unlist(lapply(1:length(fit_gab$preds),
function(x) {
mean(y[fit_gab$folds[[x]]] == max.col(
fit_gab$preds[[x]], ties.method = 'random')) }))
mean(acc_fit_gab)
[1] 0.9752714
I’ll do the same using the HOG_apply function,
# hog-features for the mnist data
#--------------------------------
hog = OpenImageR::HOG_apply(x, cells = 6, orientations = 9, rows = 28, columns = 28, threads = 6)
fit_hog = nmslibR::KernelKnnCV_nmslib(hog, y, k = 20, folds = 4, h = 1,
weights_function = 'biweight_triangular_tricube_MULT',
Levels = sort(unique(y)), Index_Params = index_params,
Time_Params = query_time_params, space = "cosinesimil",
space_params = NULL, method = "hnsw",
data_type = "DENSE_VECTOR", dtype = "FLOAT",
index_filepath = NULL, print_progress = T,
num_threads = 6, seed_num = 1)
time to complete : 41.00309 secs
acc_fit_hog = unlist(lapply(1:length(fit_hog$preds),
function(x) {
mean(y[fit_hog$folds[[x]]] == max.col(
fit_hog$preds[[x]], ties.method = 'random')) }))
mean(acc_fit_hog)
[1] 0.9787429
By averaging both the gabor and the HoG features the mean accuracy increases to 98.34 % which is very close to the score of the HoG + brute force method of KernelKnn (98.4),
acc_fit_gab_hog = unlist(lapply(1:length(fit_gab$preds),
function(x) {
mean(y[fit_gab$folds[[x]]] == max.col(
(fit_gab$preds[[x]] + fit_hog$preds[[x]]) / 2,
ties.method = 'random')) }))
mean(acc_fit_gab_hog)
[1] 0.9833857
References :
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
