---
title: "Using Self-Organizing Maps with SOMbrero for contingency tables"
date: "`r format(Sys.time(), '%d %B, %Y')`"
output:
  html_document:
    theme: journal
    toc: yes
    toc_float:
      collapsed: no
---

<!--
%\VignetteEngine{knitr}
%\VignetteIndexEntry{SOM for contingency tables}
%\VignettePackage{SOMbrero}
-->

To limit this documentation size, most figures are not displayed in the version
of the vignette included in the package. To see all figures, you can rerun
the vignettes, changing the following option to `TRUE`:
```{r setup, include=TRUE}
knitr::opts_chunk$set(include = FALSE)
```

Alternatively, this compilation is also available at:
http://sombrero.nathalievialaneix.eu/articles/d-doc-korrespSOM.html

## Basic package description

`SOMbrero` implements different variants of the Self-Organizing Map algorithm
(also called Kohonen's algorithm). To process a given dataset with the SOM
algorithm, you can use the function `trainSOM()`.

__This documentation only considers the case of contingency tables.__

```{r loading, results='hide', echo=FALSE, warning=FALSE, message=FALSE}
library("ggplot2")
library("SOMbrero")
```

### Arguments

The ```trainSOM``` function has several arguments, but only the first one is
required. This argument is ```x.data``` which is the dataset used to train the 
SOM. In this documentation, it is passed to the function as a matrix or a data
frame and encodes a contingency tables (the entries are the frequencies of joint
observations for two factors). Column and row names must be supplied to ease the
interpretation.

The function handles over options, which are the same as the ones passed to
```initSOM``` (they are parameters defining the algorithm, see 
```help(initSOM)``` for further details).

### Outputs

The ```trainSOM``` function returns an object of class ```somRes``` (see 
```help(trainSOM)``` for further details on this class).

## Case study: the ```presidentielles2002``` data set

The ```presidentielles2002``` data set provides the number of votes for the 
first round of the 2002 French presidential election for each of the 16 
candidates in all of the 106 French administrative districts called 
"départements". Further details about this data set and the 2002 French 
presidential election are given with ```help(presidentielles2002)```.

```{r loadData, include=TRUE}
data(presidentielles2002)
apply(presidentielles2002, 2, sum)
```
(the two candidates that ran the second round of the election were Jacques
Chirac and the far-right candidate Jean-Marie Le Pen)

### Training the SOM

```{r presiTrain, cache=TRUE, include=TRUE}
set.seed(01091407)
korresp.som <- trainSOM(x.data = presidentielles2002, dimension = c(8,8),
                        type = "korresp", scaling = "chi2", nb.save = 10,
                        topo = "hexagonal", maxit = 500)
korresp.som
```

As the energy is registered during the intermediate backups, we can take a look
at its evolution
```{r energyPresi, include=TRUE}
plot(korresp.som, what = "energy")
```

which has approximately stabilized at iteration 500.

### Resulting clustering

The clustering component contains the final classification of the dataset. As 
both row and column variables are classified, the length of the resulting 
vector is equal to the sum of the number of rows and the number of columns.

__NB: The clustering component shows first the column variables (here, the 
candidates) and then the row variables (here, the départements).__

```{r presiClusters, include=TRUE}
korresp.som$clustering
```

The following table indicates which graphics are available for a korresp SOM.

| What <br> <br> SOM or SC<br>Type | SOM<br>Energy  <br> <br> <br>| Obs  <br> <br> <br> <br>| Prototypes  <br> <br> <br> <br>| Add  <br> <br> <br> <br>| SuperCluster<br>(no what) <br> <br> <br> | Obs <br> <br> <br> <br> | Prototypes <br> <br> <br> <br> | Add <br> <br> <br> <br> |
|:------------|:--------|:-----|:------------|:-----|:-------------|:-----|:------------|:-----|
| (no type)   | x       |      |             |      |              |      |             |      |
| hitmap      |         | x    |             |      |              | x    |             |      |
| color       |         |      | x           |      |              |      | x           |      |
| lines       |         |      | x           |      |              |      | x           |      |
| barplot     |         |      | x           |      |              |      | x           |      |
| 3d          |         |      | x           |      |              |      |             |      |
| poly.dist   |         |      | x           |      |              |      | x           |      |
| umatrix     |         |      | x           |      |              |      |             |      |
| smooth.dist |         |      | x           |      |              |      |             |      |
| mds         |         |      | x           |      |              |      | x           |      |
| grid.dist   |         |      | x           |      |              |      |             |      |
| names       |         | x    |             |      |              |      |             |      |
| grid        |         |      |             |      |              |      | x           |      |
| dendrogram  |         |      |             |      | x            |      |             |      |
| dendro3d    |         |      |             |      | x            |      |             |      |

The resulting distribution of the clustering on the map can also be visualized
by a hitmap:
```{r presiHitmap, include=TRUE}
plot(korresp.som, what = "obs", type = "hitmap", show.names = FALSE)
```

For a more precise view, ```"names"``` plot is implemented: the names of the 
values assigned to every neuron is displayed in the corresponding cluster. In 
`korresp` SOM, both row and column names are displayed.

```{r presiGraphObs, warning=FALSE, fig.width=12, fig.height=12, include=TRUE}
plot(korresp.som, what="obs", type="names")
```

The map is divided into two main parts: minor candidates are classified at its 
top left hand side whereas the first main candidates CHIRAC, LE PEN and JOSPIN
are classified at the bottom right hand side of the map, in three different 
parts of this corner. Some striking facts are:

* most of rural départements (Corrèze, Creuse, Jura, Cantal, Ariège, ...) are
classified in the bottom right corner, in between CHIRAC and LE PEN, who have
high number of votes (compared to the other candidates) in these département;

* CHIRAC is characterized by higher votes for La Réunion (oversee département)
whereas LE PEN has higher votes for Indre Et Loire, Aisne, Loiret, Côte d'Or;

* some well known associations, like HUE (communist party) in the Nord, are also
visible on the map.


### Clustering interpretation

Some graphics from the numeric SOM algorithm are still available in the korresp 
case. They are detailed below. As the resulting clustering provides the 
classification for both rows and columns, a new argument ```view``` is used to 
specify which one should be considered. Its possible values are either 
```"r"``` for row variables (the default value) or ```"c"``` for column 
variables.

#### Graphics on prototype values

Three representations are available:

* with lines: either all rows or all columns are displayed (```view``` argument 
is used)
```{r presiProtoL}
# plot the line prototypes (106 French departements)
plot(korresp.som, what = "prototypes", type = "lines", view = "r", 
     show.names = TRUE) +    
  guides(color = guide_legend(keyheight = 0.5, ncol = 2,
                              label.theme = element_text(size = 4))) + 
  theme(axis.text.x = element_blank(), axis.ticks.x = element_blank())
# plot the column prototypes (16 candidates)
plot(korresp.som, what = "prototypes", type = "lines", view = "c", 
     show.names = TRUE) +
  guides(color = guide_legend(keyheight = 0.5, ncol = 1, 
                              label.theme = element_text(size = 6))) + 
  theme(axis.text.x = element_blank(), axis.ticks.x = element_blank())
```

The département profiles are much flatter (and with low values) in the top left 
corner of the map than in the bottom right corner which shows more differences 
between département and globally higher number of votes.

On the contrary, the candidate profiles are flatter, with globally lower values
in the bottom right corner of the map. 

A more precise individual view are given with the graphics "color" and "3d",
here drawn, as an example for the candidate "LE PEN" and for the département
"La Réunion".
* in color: one of the row or column variable (the variable is chosen with the 
argument ```variable```) is represented on the map;
* in 3d, which handling is similar to ```"color"```.
```{r presiProtoC3d, fig.width=12, fig.height=6, include=TRUE}
plot(korresp.som, what = "prototypes", type = "color", variable = "LE_PEN")
```

```{r presiProtoC3e, fig.width=12, fig.height=6}
plot(korresp.som, what = "prototypes", type = "3d", variable = "la_reunion")
```

The first graphic shows that LE_PEN obtained more votes in the departements 
located at the top left corner of the map. The second graphic shows that the 
candidates that obtained the highest scores in La Réunion are located at the 
bottom of the map (like Chirac).

The graphics can also be drawn by giving the variable number and its type,
either "r" or "c" (here, as an example, CHIRAC who is the 5th candidate, 5th column):
```{r presiProtoNumber, fig.width=12, fig.height=6}
plot(korresp.som, what = "prototypes", type = "color", variable = 5, view = "c")
plot(korresp.som, what = "prototypes", type = "3d", variable = 5, view = "c")
```

Hence CHIRAC obtained more votes in departement located at the left hand side of
the map.


#### Graphic on prototype distances

These graphics are exactly the same as in the numerical case and provide various
way to display the distance between prototypes on the grid.
```{r presiGraphProto2, include=TRUE}
plot(korresp.som, what = "prototypes", type = "poly.dist", show.names = FALSE)
```

```{r presiGraphProto3}
plot(korresp.som, what = "prototypes", type = "umatrix")
plot(korresp.som, what = "prototypes", type = "smooth.dist")
plot(korresp.som, what = "prototypes", type = "mds")
plot(korresp.som, what = "prototypes", type = "grid.dist")
```

All these graphics show a clear separation between the top left corner of the
map and the bottom right corner of the map. 


### Analyze the projection quality

The quality of the projection is provided by the function `quality` that 
outputs the same quality criteria than in the numeric case.

```{r presiQuality, include=TRUE}
quality(korresp.som)
```

### Building super classes from the resulting SOM

In the SOM algorithm, the number of clusters is necessarily close to the number 
of neurons on the grid (not necessarily equal as some neurons may have no 
observations assigned to them). This - quite large - number may not suit the 
original data for a clustering purpose.

A usual way to address clustering with SOM is to perform a hierarchical
clustering on the prototypes. This clustering is directly available in the
package ```SOMbrero``` using the function ```superClass```. To do so, you can
first have a quick overview to decide on the number of super clusters which 
suits your data.

```{r presiSC1, include=TRUE}
plot(superClass(korresp.som))
```

By default, the function plots both a dendrogram and the evolution of the
percentage of explained variance. Here, 3 super clusters seem to be a good
choice. The output of ```superClass``` is a ```somSC``` class object.
Basic functions have been defined for this class:
```{r presiSC2, include=TRUE}
my.sc <- superClass(korresp.som, k = 3)
summary(my.sc)
plot(my.sc, plot.var = FALSE)
```

Like ```plot.somRes```, the function ```plot.somSC``` has an argument 
```'type'``` which offers many different plots and can thus be combined with 
most of the graphics produced by ```plot.somSC```:

* Case ```"grid"``` fills the grid with colors according to the super clustering 
(and can provide a legend).
* Case ```"dendro3d"``` plots a 3d dendrogram.
```{r presiSC3, include=TRUE}
plot(my.sc, type = "grid")
```

```{r presiSC3b}
plot(my.sc, type = "dendro3d")
```

The three super-clusters correspond to most voted candidates (blue), less
voted candidates (green) and, in between, départments with intermediate votes in 
which BAYROU (from one of the center party) are classified.

A couple of plots from ```plot.somRes``` are also available for the super 
clustering. Some identify the super clusters with colors:
```{r presiSC4}
plot(my.sc, what = "obs", type = "hitmap")
plot(my.sc, what = "prototypes", type = "lines", show.names = TRUE, view = "c")
plot(my.sc, what = "prototypes", type = "poly.dist")
plot(my.sc, what = "prototypes", type = "mds")
```

And some others identify the super clusters with titles:
```{r presiSC5}
plot(my.sc, what = "prototypes", type = "color", view = "r", 
     variable = "correze")
plot(my.sc, what = "prototypes", type = "color", view = "c", 
     variable = "JOSPIN")
```


### Session information

This vignette has been computed with the following environment:
```{r sessionInfo, include=TRUE}
sessionInfo()
```