Package {pressfreedom.data}

Type:	Package
Title:	Download and Process Reporters Without Borders Press Freedom Index Data
Version:	0.3.0
Maintainer:	Peter Baumgartner <petzi53@gmail.com>
Description:	Download press freedom index data from Reporters Without Borders (RSF) with period-aware encoding handling. Data are downloaded from the RSF website (https://rsf.org/en/index). Provides infrastructure for data cleaning and ISO 3166 standardization in downstream phases.
License:	MIT + file LICENSE
URL:	https://github.com/petzi53/pressfreedom.data, https://www.peter-baumgartner.net/pressfreedom.data/
BugReports:	https://github.com/petzi53/pressfreedom.data/issues
Encoding:	UTF-8
LazyData:	true
Imports:	readr, here, rlang, countrycode, dplyr, stringr, cli, fs, glue, purrr, tibble, stringi
Suggests:	usethis, knitr, rmarkdown, ggplot2, tidyr, forcats, patchwork, scales, sf, rnaturalearth, testthat (≥ 3.0.0), covr
VignetteBuilder:	knitr
Config/roxygen2/version:	8.1.0
Config/testthat/edition:	3
Depends:	R (≥ 4.1.0)
NeedsCompilation:	no
Packaged:	2026-08-20 09:57:27 UTC; petzi
Author:	Peter Baumgartner [aut, cre]
Repository:	CRAN
Date/Publication:	2026-08-24 12:40:02 UTC

Commit Changes to Git

Description

Commit Changes to Git

Usage

.commit_update(years_downloaded, years_cleaned, verbose = TRUE)

Validate Update Results

Description

Validate Update Results

Usage

.validate_update(
  standardized_path,
  rows_before,
  rows_after,
  years_downloaded,
  verbose = TRUE
)

Apply Column Name Overrides to a Mapping

Description

Updates a period column mapping (target_col -> raw_col) with user-provided overrides, so a renamed raw column can still be found.

Usage

apply_column_overrides(mapping, overrides)

Arguments

Value

The (possibly updated) mapping list.

Batch Clean All RSF Years

Description

Processes all available raw CSV files and produces cleaned RDS files. Automatically detects period for each year and applies appropriate transformations.

Usage

clean_all_rwb_years(input_dir, output_dir)

Arguments

Details

This function: 1. Lists all rwbYYYY.csv files in input_dir 2. Extracts years and detects periods 3. Calls clean_rwb_single() for each year 4. Catches and logs errors for failed years 5. Creates period-specific subdirectories (period_1/, period_2/, period_3/) 6. Returns summary of processing results

Processing proceeds in year order (2002-2026, excluding 2011). Skips years without data files.

Value

Data frame with processing summary: - year: Year processed - period: Period (1, 2, or 3) - status: "success" or "error" - message: Details or error message - output_file: Path to cleaned RDS (if successful)

Clean Period 1 Data (2002-2012)

Description

Normalizes Period 1 raw data to the unified 20-column structure. Handles ISO-8859-1 encoding, decimal separator conversion, and column mapping.

Usage

clean_period_1(filepath, year)

Arguments

Details

Processing steps: 1. Detect file encoding (RSF's 2002-2021 exports are actually UTF-8, not ISO-8859-1 as originally assumed) and read accordingly 2. Convert decimal separators (comma -> period) 3. Rename columns per Period 1 mapping 4. Convert iso, country_en, zone to character 5. Convert numeric columns to numeric type 6. Handle year_n for 2012 (raw data contains "2011-12" text value) 7. Add NA columns for dimensions and score_evolution 8. Reorder to target 20-column structure

Value

Data frame with 20 columns, standardized types (character, numeric)

Clean Period 2 Data (2013-2021)

Description

Normalizes Period 2 raw data to the unified 20-column structure. Identical structure to Period 1 (same columns, encoding). Scores are comparable across 2013-2021 due to methodology introduced in 2013.

Usage

clean_period_2(filepath, year)

Arguments

Value

Data frame with 20 columns, standardized types (character, numeric)

Clean Period 3 Data (2022-2026)

Description

Normalizes Period 3 raw data to the unified 20-column structure. Handles UTF-8 encoding, RSF column renames (including year-specific score column naming), dimension columns, and decimal separator conversion for score_evolution.

Usage

clean_period_3(filepath, year)

Arguments

Details

Processing steps: 1. Detect file encoding (RSF has silently switched between UTF-8 and ISO-8859-1 across Period 3 years, e.g. 2025-2026 exports arrived as Latin-1 even though 2022-2024 were UTF-8) and read accordingly 2. Drop problematic columns (Situation, etc.) 3. Apply any column-name overrides for years where RSF renamed a raw column (e.g. "Score" -> "Score 2025"; see 'load_column_overrides()') 4. Validate that every expected raw column is present, aborting with a clear error (rather than silently producing all-NA columns) if RSF has renamed something not covered by an override 5. Rename columns per Period 3 mapping 6. Convert iso, country_en, zone to character 7. Convert numeric columns to numeric type 8. Apply decimal separator conversion to all numeric columns (handles score_evolution) 9. Set score_n_1 and score_evolution to NA for 2022 10. Reorder to target 20-column structure

Value

Data frame with 20 columns, standardized types (character, numeric)

Clean Single Year of RSF Data

Description

Dispatcher function that routes to appropriate period-specific cleaner. Reads raw CSV, applies period-specific transformations, saves as RDS.

Usage

clean_rwb_single(filepath, year, output_dir)

Arguments

Details

This function: 1. Detects period from year using get_period() 2. Routes to clean_period_1(), clean_period_2(), or clean_period_3() 3. Validates output has 20 columns in correct order 4. Saves as RDS file in output_dir 5. Returns invisible filepath (for logging/progress tracking)

Value

Invisible. Writes RDS file to output_dir. Filename format: rwbYYYY_cleaned.rds

Combine Cleaned RDS Files from All Periods

Description

Reads all cleaned RDS files from period_1, period_2, and period_3 directories and combines them into a single data frame.

Usage

combine_cleaned_periods(input_dir, output_file)

Arguments

Value

Invisibly returns the path to the output file

Consolidate and Standardize Country Names and Assign ISO Codes

Description

Generic consolidation engine that applies country name consolidations, removes diacritics, and assigns ISO 3-letter codes.

Usage

consolidate_and_standardize_countries(
  combined_df,
  consolidation_mapping = NULL
)

Arguments

Value

Data frame with standardized country names, ISO codes, and metadata

Convert Factor Columns to Character

Description

Converts specified factor columns to character vectors. Used to standardize iso, country_en, and zone columns to character type.

Usage

convert_factors_to_character(df, cols)

Arguments

Value

Data frame with specified columns converted to character

Detect CSV File Encoding

Description

Guesses whether a raw RSF CSV file is UTF-8 or a Latin-1 variant. Used by clean_period_1(), clean_period_2(), and clean_period_3() because RSF has switched export encoding across years without notice (e.g. 2002-2021 exports are UTF-8 despite once being assumed ISO-8859-1, and 2025-2026 arrived as ISO-8859-1 while 2022-2024 were UTF-8).

Usage

detect_csv_encoding(filepath)

Arguments

Details

Uses readr::guess_encoding(), which ranks candidate encodings by confidence. Explicitly detects UTF-8/US-ASCII and the ISO-8859-1 family (ISO-8859-1, windows-1252, latin1) and normalizes all to one of these two outcomes. Raises an error if: - readr::guess_encoding() returns zero candidates (truly indeterminate), or - the top candidate is neither UTF-8/ASCII nor the ISO-8859-1 family (an unexpected encoding, typically indicating data corruption or a source format change).

This explicit design prevents silent misidentification: if RSF ever introduces a third encoding, or a file is corrupted, the error is visible at parse time rather than allowing bad data downstream.

Value

Character. Either "UTF-8" or "ISO-8859-1"

Download RSF Press Freedom Index Data

Description

Downloads press freedom index CSV files from Reporters Without Borders (RSF) for specified years. Files are saved as-is (unmodified) with consistent naming (rwb<year>.csv).

Usage

download_rwb_data(years = 2002:2026, output_dir, skip_missing = TRUE)

Arguments

Details

RSF publishes press freedom index data at a per-year URL, e.g. for 2024: https://rsf.org/sites/default/files/import_classement/2024.csv Substitute the target year for 2024 to get other years' data.

**Encoding Handling:** Files are downloaded as raw bytes (utils::download.file(mode = "wb")) and written to disk unmodified. No parsing, re-encoding, or re-serialization happens at download time, so whatever bytes RSF serves (RSF has used both UTF-8 and ISO-8859-1/Latin-1 depending on the year, without warning) are preserved exactly as-is. Encoding is detected later, per file, when the data is cleaned (see detect_csv_encoding()).

An earlier version of this function read each file with readr::read_delim() using a period-based encoding guess and then re-wrote it with readr::write_delim(). That round-trip silently corrupted the 2002-2021 files (which are actually UTF-8, not ISO-8859-1 as assumed): text was double-encoded and numeric columns like Score N were mis-parsed (commas treated as thousands separators, e.g. "92,48" became 9248). Downloading raw bytes avoids this class of bug entirely.

Year 2011 is not available; no imputation is performed. If skip_missing = TRUE, the function automatically filters out 2011 before downloading.

Error Handling: - Connection failures are logged but don't stop the function - HTTP 404 errors (missing years) are logged as warnings - File write permission errors are caught and reported

Value

Invisibly returns a named list where names are years and values indicate success/failure status. Called for side effects (downloading files).

Detect RSF Data Period by Year

Description

Determines which structural period a given year belongs to based on known changes in RSF's data format and calculation methodology.

Usage

get_period(year)

Arguments

Details

This function determines the structural period a year belongs to, which drives column mapping and normalization logic.

**Important:** Encoding is NOT period-based. Use detect_csv_encoding() to determine per-file encoding. Period 1-2 (2002-2021) are UTF-8 despite the structural naming; Period 3 (2022-2026) mixed UTF-8 (2022-2024) and ISO-8859-1 (2025-2026) without warning.

**Period 1 (2002-2012):** - 16 columns with fixed structure - Delimiter: semicolon (;) - Scores not comparable across years (within-year ranks only)

**Period 2 (2013-2021):** - 16 columns, same structure as Period 1 - Delimiter: semicolon (;) - Scores comparable across years (new calculation method introduced)

**Period 3 (2022-2026):** - 22-25 columns (varies by year) - Delimiter: semicolon (;) - Major restructuring: columns reordered, score dimensions added - Column names vary by year (e.g., "Score" vs "Score 2026")

Value

Character. One of: "period_1" (2002-2012), "period_2" (2013-2021), or "period_3" (2022-2026). Returns NA for year 2011 (no official RSF data).

Get the appropriate mapping for a given period and year

Description

Get the appropriate mapping for a given period and year

Usage

get_period_mapping(period, year)

Arguments

Value

List with column mappings

Get Years That Need Downloading

Description

Compares available CSV files in the input directory against a full list of years to identify which years are missing. Automatically excludes 2011 (no official RSF data).

Usage

get_years_to_download(input_dir, all_years = 2002:2026)

Arguments

Details

This is a utility function useful for incremental updates. For example, when a new year of data becomes available at RSF, use this function to detect which years need downloading without re-downloading existing data.

Note: Year 2011 is never included in the returned vector, even if it's in the all_years range.

Value

Integer vector of years that don't have corresponding CSV files. Returns empty vector if all years are present.

Load User-Provided Column Name Overrides

Description

Reads 'period3_column_overrides.csv', if present, and returns any override mappings for the given year as a named list suitable for 'apply_column_overrides()'.

Usage

load_column_overrides(year, overrides_file = NULL)

Arguments

Details

This is the general-purpose safety net for RSF column renames – both the ones already seen (e.g. '"Score"' -> '"Score 2025"') and any future, unpredictable ones (e.g. '"Economic Context"' -> '"Economy"'). There is no special-cased detection logic for any single column, including 'score': every rename, however likely, is handled the same way, via this override file. When 'validate_column_names_exist()' aborts because an expected column is missing, add a row to the CSV:

“' year,target_col,expected_col,actual_col 2027,economic_context,Economic Context,Economy “'

'target_col' is the unified output column name (from 'target_columns'); 'expected_col' documents what the mapping originally expected (for human readability only, not used programmatically); 'actual_col' is the raw column name actually found in that year's CSV. Append future rows to the same file rather than creating a new file per year.

Value

Named list ('target_col = "actual_col"', ...) of overrides for ‘year', or 'NULL' if the file doesn’t exist or has no rows for 'year'.

Normalize Column Names to Target Structure

Description

Applies period-specific column mappings to raw data. Renames columns and adds NA columns for missing data.

Usage

normalize_column_names(df, mapping)

Arguments

Details

This function: 1. Renames raw columns to target names 2. Adds NA columns for missing data 3. Reorders to match target column order

Value

Data frame with normalized column names in target order

Column Mapping for Period 1 (2002-2012)

Description

Period 1 raw columns mapped to unified 20-column structure. Periods 1-2 omit dimension columns and score history columns (set as NA). Named in target column order.

Usage

period_1_mapping

Column Mapping for Period 2 (2013-2021)

Description

Period 2 raw columns mapped to unified 20-column structure. Identical to Period 1 except methodology changed in 2013 (scores comparable).

Usage

period_2_mapping

Column Mapping for Period 3 (2022-2026)

Description

Period 3 raw columns mapped to unified 20-column structure. Note: 'score' maps to the plain "Score" raw name, which is what RSF has used in most Period 3 years (2022-2024). Years where RSF appended the year to the column name instead (2025: "Score 2025", 2026: "Score 2026") are handled via 'inst/extdata/period3_column_overrides.csv', the same generic mechanism used for any other unpredictable RSF rename – see 'load_column_overrides()'. This mapping is intentionally not special-cased for "Score", since only 2 of the 5 Period 3 years so far have used the year-suffixed name. Named in target column order.

Usage

period_3_mapping

Print Method for rwb_update Results

Description

Print Method for rwb_update Results

Usage

## S3 method for class 'rwb_update'
print(x, ...)

Arguments

Value

The input object x, returned invisibly. Called for its side effect of printing a formatted summary of the update report to the console.

Repair Mojibake and Normalize Text to ASCII

Description

Repairs character strings that were corrupted by one or more rounds of incorrect Latin-1-as-UTF-8 decoding (a recurring artifact in RSF's source files), then transliterates any remaining accented characters to their closest ASCII equivalent. Detection and repair are byte-level and generic, so this handles corruption depth (single or double mojibake) and new corrupted values automatically, without needing a hand-maintained list of known-bad strings.

Usage

repair_and_asciify(x, max_passes = 3)

Arguments

Value

Character vector, ASCII-only

Resolve Missing-Trailing-Zero Score Scaling

Description

RSF's 2013+ exports store score/dimension percentages (0-100, 2 implied decimal places) as bare digit strings with no decimal point (e.g. "9189" means 91.89). Values ending in one or two zeros have those trailing zeros silently dropped somewhere in RSF's own export pipeline (e.g. "844" means 84.40, not 8.44; "87" means 87.00, not 0.87), which is indistinguishable from a genuinely low sub-10 score for the worst-ranked countries (e.g. "46" for a rank-180 country can legitimately mean 0.46).

Usage

resolve_percent_scaling(raw_chr, rank_chr)

Arguments

Details

Values with 4 or more digits are unambiguous ("confirmed"): a 4-digit value is divided by 100 (2 implied decimals), and a small number of 2025 rows have 5-digit values (RSF apparently breaks near-tied ranks with a third decimal place, e.g. "65487" means 65.487, not 654.87), so n-digit confirmed values are divided by '10^(n_digits - 2)' generally. For shorter (2-3 digit) values, two candidates are computed: right-padding with zeros to 4 digits before dividing by 100 (the "dropped trailing zero" interpretation), and dividing the raw digits by 100 directly (the "already complete, genuinely low score" interpretation). The candidate closer to a rank-based linear interpolation of neighboring confirmed values is selected. Missing rank or missing value inputs fall back to the zero-padded interpretation.

Value

Numeric vector of resolved percentages (0-100 scale).

Standardized Reporters Without Borders Press Freedom Index (2002-2026)

Description

A comprehensive dataset of press freedom indicators from Reporters Without Borders (RSF: Reporters Sans Frontieres, the organization's French name and legal identity; RWB: Reporters Without Borders, its common English name – used for the 'rwb_' prefix throughout this package), standardized and cleaned by the pressfreedom.data package pipeline.

Usage

rwb_standardized

Format

A data frame with 4,192 rows and 20 columns:

year_n

Numeric year (2002-2026, excluding 2011)

iso

ISO 3166-1 alpha-3 country code

country_en

Standardized country name in English

score

Press freedom score (0-100; higher = more free). Comparable only within periods.

rank

Rank within the year (1 = most free)

political_context

Sub-index: Political context (if available)

rank_pol

Rank within political context (if available)

economic_context

Sub-index: Economic context (if available)

rank_eco

Rank within economic context (if available)

legal_context

Sub-index: Legal context (if available)

rank_leg

Rank within legal context (if available)

social_context

Sub-index: Social context (if available)

rank_soc

Rank within social context (if available)

safety

Sub-index: Safety (if available)

rank_saf

Rank within safety (if available)

zone

Geographic zone assigned by RSF

rank_n_1

Previous year's rank (year_n - 1)

rank_evolution

Change in rank from previous year

score_n_1

Previous year's score (year_n - 1)

score_evolution

Change in score from previous year

Details

## Data Cleaning Pipeline

This dataset is the output of a comprehensive 4-phase data pipeline implemented in the pressfreedom.data package:

- **Phase A (Download):** Raw CSV files from RSF website - **Phase B (Normalize):** Column names and data types standardized across periods - **Phase C (Combine):** All periods merged into unified structure - **Phase D (Standardize):** Country names consolidated, ISO codes assigned, duplicates resolved

## Important Notes

**Audit Trail:** The full audit trail (including 'country_name_original' and 'consolidation_flag' columns) is preserved in the source RDS file ('data/processed/rwb_standardized.rds') within the pressfreedom.data package. This exported dataset contains only the 20 core columns for analysis.

**Score Comparability:** Scores are only comparable within their respective periods: - **Period 1 (2002-2012):** Non-comparable scores; use ranks for trends - **Period 2 (2013-2021):** Comparable scores (0-100 scale) - **Period 3 (2022-2026):** New methodology; different dimensions tracked

**Missing Data:** Sub-indices (political, economic, legal, social, safety contexts) are only available in Period 3 (2022-2026). Periods 1-2 have NA values for these columns.

**Cyprus:** Tracked as two separate entities: - "Cyprus" (ISO: CYP) - Republic of Cyprus - "Northern Cyprus" (ISO: CXX) - Turkish Republic of Northern Cyprus

Source

Reporters Sans Frontieres, https://rsf.org

Examples

# Load the dataset
data(rwb_standardized)

# Basic summary
head(rwb_standardized)

# Countries included
length(unique(rwb_standardized$country_en))

# Years covered
range(rwb_standardized$year_n)

Standardize Decimal Separators

Description

Converts comma decimal separators to periods for Period 1-2 data (ISO-8859-1 encoded data used commas as decimal separators).

Usage

standardize_decimal_separators(df, cols)

Arguments

Details

This function targets numeric columns that may contain comma separators. It is primarily for Period 1-2 data where European number formatting was used.

Value

Data frame with decimal separators converted from comma to period

Standardize RSF Country Data

Description

Main wrapper function for Phase D standardization pipeline. Loads combined data, applies consolidations, assigns ISO codes, and saves standardized output.

Usage

standardize_rwb_countries(input_file, output_file, mapping_file = NULL)

Arguments

Value

Invisibly returns the path to the output file

Target output structure (20 columns in order)

Description

Defines the unified output structure for all periods.

Usage

target_columns

Update Reporters Without Borders Press Freedom Data

Description

Orchestrates the full yearly update workflow: downloads missing years, cleans them, recombines all periods, re-standardizes countries, exports to package format, and validates the result.

Usage

update_rwb_data(
  years = NULL,
  download = TRUE,
  clean = TRUE,
  combine = TRUE,
  standardize = TRUE,
  validate = TRUE,
  verbose = TRUE,
  auto_commit = TRUE
)

Arguments

Details

This function implements the "minimal disruption principle": only new years are downloaded and cleaned. Phases C-D (combine, standardize) always run because evolution columns depend on year N-1's data.

**Workflow Overview:**

1. **Detection (Phase A pre-check)** - If years = NULL, detect missing years via get_years_to_download() - If no years missing, report and return early (unless standardize = TRUE)

2. **Download (Phase A)** - Only if download = TRUE - Downloads CSVs for detected missing years to inst/extdata/ - Validates each CSV before saving - On download failure: aborts update with error message

3. **Clean (Phase B)** - Only if clean = TRUE and years detected - Cleans newly downloaded years via clean_rwb_single() - Outputs normalized RDS to data/cleaned/period_X/ - On cleaning failure: aborts update and reports which year failed

4. **Combine (Phase C)** - Always runs (required) - Recombines all cleaned periods via combine_cleaned_periods() - Recalculates evolution columns (rank_n_1, score_n_1, etc.) - Output: data/processed/rwb_combined.rds - Cost: ~1-2 seconds

5. **Standardize (Phase D)** - Always runs (required) - Re-standardizes all rows via standardize_rwb_countries() - Applies consolidation rules from inst/extdata/consolidation_mapping.csv - Output: data/processed/rwb_standardized.rds - Cost: ~2-3 seconds

6. **Export** - Regenerates rwb_standardized.rda via data-raw/rwb_standardized.R - Cost: <1 second

7. **Validation** - Only if validate = TRUE - Checks row count increase matches expectations - Verifies no duplicate rows - Ensures all required columns present - On validation failure: reports issues but doesn't abort

8. **Git Commit** - Only if auto_commit = TRUE - Stages updated RDS files - Creates commit with message describing what changed - On commit failure: reports warning but doesn't abort update

**Intelligent Defaults:** - combine = TRUE, standardize = TRUE: Cannot be overridden (always required) - validate = TRUE: Recommended for production workflows - auto_commit = TRUE: Recommended; provides git history of updates - verbose = TRUE: Recommended for interactive use

**Error Handling:** - Download fails -> Aborts with error - Cleaning fails -> Aborts and reports which year failed - Combine/Standardize fail -> Aborts (indicates data corruption) - Validation fails -> Reports issues; doesn't abort - Git commit fails -> Reports warning; doesn't abort update

**Example: Minimal yearly update** “'r # Run once per year when new RSF data available result <- update_rwb_data() # Auto-detects missing years, downloads, cleans, combines, standardizes print(result) “'

**Example: Testing without download** “'r # Test combining/standardizing without network calls result <- update_rwb_data(years = NULL, download = FALSE, clean = FALSE) “'

Value

Invisible list with class "rwb_update" containing: - status: "success", "partial", or "failed" - years_downloaded: Integer vector of years downloaded - years_cleaned: Integer vector of years cleaned - rows_before: Row count in combined RDS before update - rows_after: Row count in combined RDS after update - consolidations_applied: Number of consolidation rules applied - validation_passed: Logical, TRUE if all checks pass - messages: Character vector of progress messages - git_commit: Commit hash if auto-committed; NA otherwise

See Also

- download_rwb_data for Phase A details - clean_rwb_single for Phase B details - combine_cleaned_periods for Phase C details - standardize_rwb_countries for Phase D details - get_years_to_download for missing year detection

Validate That Expected Raw Columns Exist in a CSV

Description

Fails loudly if any expected raw column name is missing from the data frame, rather than letting downstream renaming silently produce all-NA columns. Intended to be called right after reading a raw CSV and resolving the column mapping (including any overrides), before any renaming happens.

Usage

validate_column_names_exist(df, expected_raw_cols, year)

Arguments

Details

This exists because RSF has renamed export columns before without notice (e.g. "Score" -> "Score 2025") and may rename others in the future in ways that cannot be predicted ahead of time (e.g. "Economic Context" -> "Economy"). Without this check, a renamed column silently resolves to 'NA' for every row via 'normalize_column_names()', and the corrupted data can ship undetected. See 'load_column_overrides()' for how to fix a failure this raises without changing package code.

Value

Invisible 'NULL'. Called for its side effect (aborting on failure).

Validate Standardization Output

Description

Check that standardization preserved data integrity and produced expected results.

Usage

validate_standardization(standardized, original_row_count)

Arguments

Value

Invisibly returns TRUE if all checks pass