Validation Against WHO Official Fact Sheets

Acknowledgements

This validation relies on several resources developed and maintained by the World Health Organization (WHO):

• WHO STEPS instrument and methodology: The STEPwise Approach to NCD Risk Factor Surveillance (STEPS) was developed by the WHO Department of Noncommunicable Diseases. The instrument, survey manuals, and analysis guidelines (including the STEPS Part 4: Guide to Physical Measurements and Part 5: Data Management and Analysis) define the standard definitions and indicator computations used throughout this package.

• WHO official STEPS analysis scripts: The R scripts maintained by WHO for STEPS data processing and databook generation (available at https://github.com/WorldHealthOrganization) served as the authoritative reference for indicator derivation logic, survey design setup, weight handling, and data quality filters (smk_cln, smkless_cln). Key methodological decisions in stepssurvey — such as not trimming sampling weights, setting skip-pattern NAs to zero for denominator alignment, and applying screening-question logic for GPAQ — were directly informed by studying these scripts.

• WHO STEPS country fact sheets: Published fact sheet values for nine countries (see table below) were used as the gold-standard reference estimates. These fact sheets are publicly available from the WHO NCD Microdata Repository (https://extranet.who.int/ncdsmicrodata).

• Licensed STEPS microdata: The underlying survey microdata were accessed under license from the WHO NCD Microdata Repository. Use of these data is subject to the WHO STEPS data access agreement, and the microdata files are not redistributable.

We gratefully acknowledge the national STEPS survey teams and coordinators in all nine validation countries for designing, conducting, and managing these surveys and for making the data available for research use through the WHO NCD Microdata Repository. The quality and completeness of these datasets made systematic validation possible.

We also acknowledge the WHO NCD Surveillance team for developing and maintaining the STEPS methodology, analysis tools, and data sharing infrastructure that underpin this work.

Overview

The stepssurvey package has been validated against WHO-published fact sheet values for nine national STEPS surveys spanning different WHO regions, time periods, and STEPS instrument versions:

For each country, the package pipeline (import -> detect -> clean -> survey design -> indicator computation) was run end-to-end and results compared with the “Both Sexes” estimates published in the corresponding WHO STEPS country fact sheet.

Validation criteria

Two complementary criteria were used to judge agreement:

1. Point-estimate match — the package estimate falls within 1.0 percentage point (pp) of the fact sheet value for proportions, or 1.0 unit for continuous measures (BMI, SBP, cholesterol).
2. Confidence-interval overlap — the 95 % confidence intervals from the package and the fact sheet share at least one common value.

The 1 pp tolerance accounts for rounding at different stages and minor differences in the treatment of edge cases (e.g. “don’t know” responses coded as 77 or 88).

Indicators compared

Up to 17 indicators were compared per country, covering all three STEPS Steps:

Step 1 — Behavioural risk factors: current tobacco use, current smoking, second-hand smoke exposure (home and workplace), current alcohol use (past 30 days), heavy episodic drinking, insufficient fruit and vegetable consumption, insufficient physical activity.

Step 2 — Physical measurements: mean BMI, overweight (BMI >= 25), obesity (BMI >= 30), mean systolic blood pressure, raised blood pressure or on medication.

Step 3 — Biochemical measurements: mean total cholesterol, raised cholesterol or on medication, raised fasting glucose or on medication, impaired fasting glucose.

Not all indicators were available in every country fact sheet. Some fact sheets omit second-hand smoke, and the Bahamas fact sheet reports Step 3 biochemical measurements as unweighted estimates (response rate < 60 %), so those were excluded from comparison.

Results summary

Overall, 123 of 128 indicators (96 %) match within 1 pp, and all 128 (100 %) have overlapping confidence intervals. The five remaining mismatches are small (1.0–2.2 pp) and all have overlapping CIs, suggesting they reflect minor methodological differences rather than errors in the package.

Detailed results by country

Analysis of mismatches

Across all nine countries, only five of 128 indicators exceed the 1 pp tolerance. The mismatches are:

Current tobacco use (Moldova –2.2 pp): the largest remaining mismatch. The confidence intervals overlap, suggesting a slightly different indicator definition in the published fact sheet (e.g. whether smokeless tobacco is included).

Raised blood pressure or on medication (Ukraine +1.9, Bahamas –1.4): the mismatches are in opposite directions, suggesting country-specific differences in how the medication question is coded rather than a systematic package issue.

Raised cholesterol or on medication (Ecuador –1.0 pp): right at the 1 pp threshold boundary. The CIs overlap.

Insufficient fruit/veg (Cabo Verde –1.7 pp): the CIs overlap.

All five mismatches have overlapping 95 % confidence intervals, confirming that the differences are not statistically significant at conventional levels.

Key methodological alignments

During validation, several methodological details were identified where alignment with the WHO official analysis scripts was essential for reproducibility:

Survey design. Sampling weights are used as-is without trimming, matching the WHO official STEPS analysis scripts. Lone PSUs are handled with options(survey.lonely.psu = "adjust").

Alcohol skip patterns. Non-drinkers who skip the heavy-episodic drinking question (A9) are coded as FALSE (not NA), ensuring the denominator covers the total population rather than only current drinkers.

Physical activity screening questions. GPAQ screening questions (P1, P4, P7, P10, P13) where the respondent answers “No” set the domain contribution to zero, rather than leaving it as NA. Special codes 77 and 88 (“don’t know” / “refused”) are cleaned to NA.

Diet zero-days handling. When a respondent reports eating fruit or vegetables on zero days per week, the servings-per-day variable (which is skipped and therefore NA) is set to zero rather than excluded from the denominator.

Configurable blood-pressure thresholds. The package supports custom SBP/DBP thresholds via clean_steps_data(bp_sbp_threshold, bp_dbp_threshold), needed because some countries (e.g. Mongolia) use thresholds other than the standard 140/90.

Cholesterol unit conversion and threshold alignment. Some country datasets store cholesterol in mg/dl rather than mmol/L. The package auto-detects units and converts to mmol/L. For validation, the raised-cholesterol threshold was set to 4.914 mmol/L (= 190 mg/dl, the WHO standard) rather than the default 5.0 mmol/L, aligning with the WHO fact sheet definition.

Physical activity data quality (P_clean). Following the WHO GPAQ Analysis Guide, respondents with inconsistent physical activity data are excluded from all PA computations. Specifically, if a respondent answers “Yes” to a GPAQ screening question (e.g. “Do you do vigorous work?”) but has missing or invalid follow-up data for that domain, all PA variables are set to NA. Additionally, if any single domain reports more than 960 minutes (16 hours) per day, all PA data for that respondent is invalidated. Without this cleaning, partial domain data would be silently dropped by rowSums(na.rm = TRUE), underestimating total PA and systematically inflating the insufficient_pa indicator.

Data quality filters. The WHO smk_cln filter is applied: logically inconsistent tobacco respondents (e.g. “I currently smoke” combined with “I never smoked in the past”) are excluded from the tobacco denominator.

Tobacco indicator mapping. WHO fact sheets vary in which tobacco indicator they report under “current smoking”: some report daily smoking (Mongolia, Georgia), while others report any-frequency tobacco smoking (Afghanistan, Algeria, Ukraine, Ecuador, Cabo Verde, Bahamas). The package computes all variants and the appropriate one is used for each country comparison.

Column mapping. Country datasets often use non-standard variable names. The package’s read_column_mapping() function and Excel mapping template were used to handle datasets where auto-detection required manual overrides.

Running the validation yourself

The full validation script is included in the package at inst/validation/validate_all.R. To run it, you need the licensed STEPS microdata files placed in a STEPS Licensed Datasets directory alongside the package:

library(stepssurvey)
## Run from the package source directory
source(system.file("validation", "validate_all.R", package = "stepssurvey"))

Per-country results are saved as CSV files in inst/validation/.

Reproducibility notes

The validation was performed with R 4.5.1, using survey package version 4.5, and stepssurvey version 0.1.0.

The licensed STEPS microdata files are not redistributable and are therefore not included in the package. Researchers with access to the WHO STEPS microdata repository can request datasets for the nine countries listed above and reproduce these results independently.

References

1. World Health Organization. (2017). WHO STEPS Surveillance Manual: The WHO STEPwise Approach to Noncommunicable Disease Risk Factor Surveillance. Geneva: WHO. https://www.who.int/teams/noncommunicable-diseases/surveillance/systems-tools/steps

2. World Health Organization. WHO NCD Microdata Repository: STEPS Survey Data. https://extranet.who.int/ncdsmicrodata

3. World Health Organization. STEPS data analysis scripts (R). GitHub repository. https://github.com/WorldHealthOrganization

4. Republic of Moldova STEPS Survey 2021. WHO NCD Country Fact Sheet. Geneva: WHO, 2022.

5. Mongolia STEPS Survey 2019. WHO NCD Country Fact Sheet. Geneva: WHO, 2020.

6. Georgia STEPS Survey 2016. WHO NCD Country Fact Sheet. Geneva: WHO, 2017.

7. Afghanistan STEPS Survey 2018. WHO NCD Country Fact Sheet. Geneva: WHO, 2019.

8. Algeria STEPS Survey 2016. WHO NCD Country Fact Sheet. Geneva: WHO, 2017.

9. Ukraine STEPS Survey 2019. WHO NCD Country Fact Sheet. Geneva: WHO, 2020.

10. Ecuador STEPS Survey 2018. WHO NCD Country Fact Sheet. Geneva: WHO, 2019.

11. Cabo Verde STEPS Survey 2020. WHO NCD Country Fact Sheet. Geneva: WHO, 2021.

12. Bahamas STEPS Survey 2019. WHO NCD Country Fact Sheet. Geneva: WHO, 2020.

13. Lumley, T. (2004). Analysis of complex survey samples. Journal of Statistical Software, 9(8), 1–19. R package survey version 4.5.