Introduction to tardis

The default dictionaries

To be completed…

Fixing false positives: Ed’s little bed

A simple children’s rhyme shows one pitfall of relying on a fixed dictionary. Here we see the sad story of Ed, whose bed is too small:

library(tardis)
library(dplyr)
library(knitr)

text <- c("This is not good.", 
          "This is not right.", 
          "My feet stick out of bed all night.", 
          "And when I pull them in, oh dear!", 
          "My feet stick out of bed up here!")

tardis::tardis(text) %>%
  dplyr::select(sentences, score) %>%
  knitr::kable()

Tardis has correctly noted that “not good” is negative, but has incorrectly classified the fourth sentence as positive because it contains the affectionate term “dear.” To fix this, we can add a new row to our default dictionary classifying “oh dear” as a negative term.

custom_dictionary <- dplyr::add_row(tardis::dict_tardis_sentiment,
                                    token = "oh dear", score = -1)

tardis::tardis(text, dict_sentiments = custom_dictionary) %>%
  dplyr::select(sentences, score) %>%
  knitr::kable()

Of course, our choice to assign “oh dear” a sentiment value of -1 was arbitrary, but with this change tardis correctly flags the fourth sentence as negative. This demonstrates how easy it is to adapt tardis’s dictionaries to a specific context.

Identifying sarcasm in online communications

Here are three two-sentence texts that have similarly neutral mean sentiments, but very different meanings.

text <- c("I guess so, that might be fine. I don't know.",
          "Wow, you're really smart. MORON!",
          "It's the worst idea I've ever heard 😘" )

tardis::tardis(text) %>%
  knitr::kable()

Only the first sentence is genuinely neutral; the second two express two wildly different sentiments that on average are neutral, but to most human readers imply a strong emotional value. Tardis also returns the standard deviation and ranges of within-text sentence sentiments, and we can see that the ranges for the two sarcastic texts are much larger than for the truly neutral text. Of course, these examples are blunt and not particularly funny, but they show the use of looking beyond the mean when studying sentiment in informal online communications.

Simple counts

In some cases, researchers may have pre-built dictionaries and be interested in simply detecting those words, without worrying about any of the more complex rules described above. For this use case, tardis has a convenience parameter simple_count which, when TRUE, disables most of the logic and returns simple sums of token values. Tardis also sends the user a warning to confirm this is the expected behaviour.

For example:

dict_cats <- dplyr::tibble(token = c("cat", "cats"), score = c(1, 1))

text <- c("I love cats.", "Not a cat?!?!", "CATS CATS CATS!!!")

tardis::tardis(text, dict_sentiments = dict_cats, simple_count = TRUE) %>%
  dplyr::select(sentences, score) %>%
  knitr::kable()
#> Warning in tardis::tardis(text, dict_sentiments = dict_cats, simple_count =
#> TRUE): Parameter simple_count = TRUE overrides most other parameters. Make sure
#> this is intended!

Note that the column names are unchanged, although the interpretation differs.

The algorithm in excruciating detail

• Texts are decomposed into sentences.
  • By default, we count the number of exclamation and question marks in each sentence for future use.
  • Emojis are treated as sentences.
• Sentences are decomposed into tokens, which are usually words but can also be emojis (e.g. “:)”) or multi-word strings (e.g. “oh dear”).
• Tokens are then checked against three dictionaries and classified into one of four groups:
  • Sentiment-bearing tokens have positive or negative sentiment values. For example, “happy” might have a value of +1 and “sad” might have a value of -1.
  • Modifiers amplify or damp the value of subsequent sentiment-bearing tokens. For example, “very” might amplify future values and “somewhat” might damp them.
  • Negators reverse the direction and reduce the amplitude of subsequent sentiment-bearing tokens. The intuitive justification is that “not bad” is positive, but not as positive as “bad” is negative.
  • Unidentified tokens that fall into none of the preceding categories have no direct effect.

Once a text has been broken down into sentences and tokens, scores are built back up starting from the tokens.

• If our sentence is a string of \(n\) tokens, then a sentiment-bearing token \(t_j\) at index \(j\) has a sentiment that is a function of itself and its three preceding tokens.
  • If \(t_j\) is in ALL CAPS, then its sentiment is scaled up by a multiplicative factor.
  • Modifiers work by scaling a token’s score up or down by a user-supplied multiplier. Modifiers are less influential the farther back they are, and scale down to 95% and 90% efficacy at two and three steps back respectively.
  • Negators reverse a token’s sign and decrease its amplitude by a user-supplied multiplier. The intuition is that “not bad” is in the direction of “good,” but not quite as good as “bad” is bad.
• Sentence scores are computed from their tokens and punctuation.
  • Adjusted token scores are summed.
  • Sentence sums are then adjusted based on exclamation and question marks. A single question mark has no effect, but otherwise we count the number of exclamation marks and punctuation marks up to three, and multiply the sentence sum by the punctuation factor that many times.
  • Finally, we optionally scale the punctuation-adjusted sentence sums with a sigmoid function to map the values to the range \((-1,1)\).
• Texts
  • Texts are scored based on their constitutive sentences.
  • By default, we take the mean of the sentence scores. However, users can also use other summary functions (e.g. median, max, min).
  • The standard deviation and range of each text’s sentence scores is also returned.