The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

chatRater 1.3.1

A Tool for Rating Text/Image/Audio Stimuli via Large Language Models

lifecycle

What’s New in 1.3.1

CRAN maintenance release — no new user-facing features. 1.3.1 fixes two issues surfaced by R CMD check on CRAN incoming:

stats::aggregate() import added to NAMESPACE (was flagged as undefined global function in rate_openai_weighted()).
rank argument of alignment() is now documented in man/alignment.Rd (was triggering a Codoc mismatches WARNING).
inst/CITATION is now a single APA 7 entry pointing at the PsyArXiv preprint (doi:10.31234/osf.io/mje6w_v1).

R CMD check is clean on CRAN incoming (Windows Server 2022, R-devel r90190, ucrt): Status: OK.

The probability-weighted scoring mode (method = "weighted", top_logprobs, include_probs) introduced in 1.3.0 remains the headline feature of this release line — see the What’s New in 1.3.0 section below for details.

Key Features

Multi-modal input: Plain text, local image files, image URLs, and audio files (transcribed via Whisper)
20+ LLM providers: OpenAI, Anthropic, DeepSeek, Groq, Mistral, Ollama, LM Studio, OpenRouter, and any OpenAI-compatible endpoint
Flexible output: Extract numeric ratings, full text responses, or raw API output
Batch processing: Rate multiple stimuli in one call
Local model support: Run entirely offline with Ollama or LM Studio (no API key needed)

Supported Providers

Provider	Description	API Key Required?
openai	OpenAI GPT models	Yes
anthropic	Anthropic Claude models	Yes
ollama	Local models via Ollama	No
lmstudio	Local models via LM Studio	No
deepseek	DeepSeek models	Yes
groq	Groq inference	Yes
mistral	Mistral models	Yes
openrouter	Unified access to many models	Yes
openai_compatible	Custom endpoints (vLLM, etc.)	Depends

Installation

# Install from CRAN (production version)
install.packages("chatRater")
pak::pkg_install("chatRater")

# Install from GitHub (development version)
remotes::install_github("ShiyangZheng/chatRater")

Quick Start

Using Cloud Providers (OpenAI, Anthropic, etc.)

library(chatRater)

# Basic usage with OpenAI
stim <- 'The early bird catches the worm'
res <- generate_ratings(
  model = 'gpt-4o',
  stim = stim,
  provider = 'openai',
  api_key = Sys.getenv("OPENAI_API_KEY"),
  prompt = 'You are an expert in figurative language.',
  question = 'Rate the creativity of this phrase on a scale of 1-10:',
  scale = '1-10'
)

# Using Anthropic Claude
res <- generate_ratings(
  model = 'claude-sonnet-4-20250514',
  stim = stim,
  provider = 'anthropic',
  api_key = Sys.getenv("ANTHROPIC_API_KEY"),
  scale = '1-5'
)

Using Local Models (No API Key Needed!)

# Make sure Ollama is running first
# Download from: https://ollama.com

res <- generate_ratings(
  stim = 'Bite the bullet',
  provider = 'ollama',
  model = 'llama3.2',
  scale = '1-7',
  n_iterations = 3
)

# Or with LM Studio (run on port 1234 by default)
res <- generate_ratings(
  stim = 'Hit the nail on the head',
  provider = 'lmstudio',
  model = 'your-model-name',
  scale = '1-5'
)

Batch Processing

stim_list <- c('Kick the bucket', 'Beat around the bush', 'Cut to the chase')

results <- generate_ratings_for_all(
  stim_list = stim_list,
  provider = 'ollama',
  model = 'llama3.2',
  scale = '1-7',
  n_iterations = 5
)

Image Rating

# Rate an image from URL
res <- generate_ratings(
  stim = 'https://example.com/image.jpg',
  provider = 'openai',
  model = 'gpt-4o',
  api_key = Sys.getenv("OPENAI_API_KEY"),
  question = 'Rate the visual quality:',
  scale = '1-10'
)

# Rate a local image file
res <- generate_ratings(
  stim = '/path/to/image.png',
  provider = 'anthropic',
  api_key = Sys.getenv("ANTHROPIC_API_KEY"),
  scale = '1-5'
)

Audio Rating (New in 1.3.0)

# Audio is transcribed via OpenAI Whisper, then rated
res <- generate_ratings(
  stim = '/path/to/audio.mp3',
  provider = 'openai',
  model = 'gpt-4o',
  api_key = Sys.getenv("OPENAI_API_KEY"),
  prompt = 'You are rating audio transcripts.',
  question = 'Rate the formality of this speech:',
  scale = '1-10'
)

Controlling Return Type (New in 1.3.0)

# Numeric (default): extracts numbers from LLM response
res <- generate_ratings(
  stim = 'test',
  provider = 'ollama',
  model = 'llama3.2',
  return_type = 'numeric',
  scale = '1-10'
)

# Text: returns full LLM response text
res <- generate_ratings(
  stim = 'test',
  provider = 'ollama',
  model = 'llama3.2',
  return_type = 'text'
)

# Raw: returns raw API response
res <- generate_ratings(
  stim = 'test',
  provider = 'ollama',
  model = 'llama3.2',
  return_type = 'raw'
)

Configuration

Setting API Keys

# Option 1: Set environment variables in .Renviron
# OPENAI_API_KEY=sk-...
# ANTHROPIC_API_KEY=sk-ant-...

# Option 2: Pass directly
generate_ratings(
  api_key = 'sk-...',
  ...
)

# Option 3: Use Sys.getenv()
generate_ratings(
  api_key = Sys.getenv("OPENAI_API_KEY"),
  ...
)

Custom Endpoints

# For vLLM or other OpenAI-compatible servers
res <- generate_ratings(
  stim = 'test',
  provider = 'openai_compatible',
  base_url = 'http://localhost:8080/v1',
  api_key = NULL,  # or your API key if required
  model = 'your-model'
)

Citation

If you use chatRater in your research, please cite the associated preprint (APA 7):

Zheng, S. (2026, May 16). chatRater: Validating LLM-Generated Psycholinguistic Norms with Probability-Weighted Scoring [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/mje6w_v1

You can also get these citations from within R:

citation("chatRater")

Dependencies

chatRater 1.3.1 depends on: - llmcoder (>= 1.2.0): LLM integration backend - base64enc: For encoding local files - tools: For file extension detection - httr2: For HTTP requests - curl: For file uploads - jsonlite: For JSON parsing

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.