The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

Open Scoring API Client for R

R-CMD-check CRAN/METACRAN Version Codecov test coverage

Creativity research involves the need to score open-ended problems. Usually done by humans, automatic scoring using AI becomes more and more accurate. This package provides a simple interface to the ‘Open Scoring’ API, leading creativity scoring technology by Organiscak et al. (2023). With it, you can score your own data directly from an R script.

Installation

Install the released version of openscoring from CRAN with:

install.packages("openscoring")

You can install the development version of openscoring from GitHub with:

# install.packages("pak")
pak::pak("jakub-jedrusiak/openscoring")

Example

library(openscoring)

df <- tibble::tibble(
  stimulus = c("brick", "hammer", "sponge"),
  response = c(
    "butter for trolls",
    "make Thor jealous",
    "make it play in a kids show"
  )
)

df
#> # A tibble: 3 × 2
#>   stimulus response                   
#>   <chr>    <chr>                      
#> 1 brick    butter for trolls          
#> 2 hammer   make Thor jealous          
#> 3 sponge   make it play in a kids show

scored_df <- ocsai(df, stimulus, response, model = "2-xs")

scored_df
#> # A tibble: 3 × 4
#>   stimulus response                    .originality .confidence
#>   <chr>    <chr>                              <dbl>       <dbl>
#> 1 brick    butter for trolls                   2.5         0.49
#> 2 hammer   make Thor jealous                   3.34        0.47
#> 3 sponge   make it play in a kids show         2.97        0.48

The "1.5" model works for multiple languages:

df_polish <- tibble::tibble(
  stimulus = c("cegła", "młotek", "gąbka"),
  response = c(
    "masło dla trolli",
    "wywoływanie zazdrości u Thora",
    "postać w programie dla dzieci"
  )
)

ocsai(df_polish, stimulus, response, model = "2", language = "Polish")
#> # A tibble: 3 × 4
#>   stimulus response                      .originality .confidence
#>   <chr>    <chr>                                <dbl>       <dbl>
#> 1 cegła    masło dla trolli                      2.55        0.54
#> 2 młotek   wywoływanie zazdrości u Thora         3.08        0.84
#> 3 gąbka    postać w programie dla dzieci         3.21        0.62

Confidence scores

By default, logprob_scoring is TRUE, which adds a .confidence column alongside the originality scores. Set logprob_scoring = FALSE to disable it, or rename the column with confidence_col:

ocsai(df, stimulus, response, model = "2-xs", confidence_col = "confidence")
#> # A tibble: 3 × 4
#>   stimulus response                    .originality confidence
#>   <chr>    <chr>                              <dbl>      <dbl>
#> 1 brick    butter for trolls                   2.5        0.49
#> 2 hammer   make Thor jealous                   3.34       0.47
#> 3 sponge   make it play in a kids show         2.97       0.48

API key

The API works without authentication for moderate usage. To increase rate limits, pass your key with the api_key argument (it is sent as the X-API-KEY header):

ocsai(
  df,
  stimulus,
  response,
  model = "2",
  api_key = Sys.getenv("OPENSCORING_API_KEY")
)

For safety, keep keys out of scripts and set them as environment variables.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.