--- title: "CAPS-5 workflow" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{CAPS-5 workflow} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 4 ) ``` This vignette compares a PCL-5 symptom definition against the CAPS-5 diagnosis, and shows how to optimize a definition against the CAPS-5 diagnosis directly. ## Why use the CAPS-5 The PCL-5 is completed by the patient; the CAPS-5 is administered by a clinician. Both rate the same 20 DSM-5-TR symptoms on the same 0 to 4 scale, so when both are available for the same participants the CAPS-5 diagnosis is the closest available reference to a clinical ground truth. That makes it the natural standard against which to judge a self-report definition. ## The bundled paired data The general-population dataset `simulated_ptsd_genpop` ships both instruments for the same participants: the PCL-5 items `S1`–`S20` and paired CAPS-5 severity ratings `C1`–`C20`, simulated so the two total scores correlate about 0.8 (the level usually reported empirically). Because both come from one data frame, the PCL-5 and CAPS-5 views describe the same people in the same row order, which is exactly what a paired-instrument comparison requires. To analyse one instrument we standardise its 20 columns and park everything else, including the other instrument's columns, in `id_col`, so `rename_*` sees exactly 20 item columns. We use a 120-row subset for speed. ```{r setup, message = FALSE} library(PTSDdiag) data("simulated_ptsd_genpop") gp <- simulated_ptsd_genpop[1:120, ] demo <- c("patient_id", "age", "sex") # PCL-5 view: rename S1..S20; carry the CAPS-5 columns through untouched ptsd <- rename_ptsd_columns(gp, id_col = c(demo, paste0("C", 1:20))) # CAPS-5 view: rename C1..C20; carry the PCL-5 columns through untouched caps5 <- rename_caps5_columns(gp, id_col = c(demo, paste0("S", 1:20))) ``` ## Computing the CAPS-5 diagnosis `create_caps5_diagnosis()` applies the DSM-5-TR algorithm to the CAPS-5 severity scores, using the same clusters and the same presence threshold of 2 or higher as the PCL-5 diagnosis. It returns a single logical column, `PTSD_caps5`. ```{r caps5-dx} caps5_dx <- create_caps5_diagnosis(caps5) mean(caps5_dx$PTSD_caps5) * 100 ``` ## Comparing the PCL-5 and CAPS-5 diagnoses `compare_diagnostic_systems()` builds one summary table that scores each diagnostic system against a chosen reference. Setting `reference = "caps5"` makes the CAPS-5 the standard, so the PCL-5 diagnosis and ICD-11 are evaluated against it. Each row reports sensitivity, specificity, PPV, NPV, accuracy, and balanced accuracy against the reference, together with the counts of false positives, false negatives, and total misclassifications. ```{r compare} compare_diagnostic_systems( ptsd, caps5_data = caps5, icd11 = TRUE, reference = "caps5" ) ``` Because the two instruments are correlated here rather than random, the self-report PCL-5 diagnosis agrees substantially with the clinician CAPS-5 reference. Sensitivity is the share of CAPS-5-positive participants the PCL-5 rule also calls positive, accuracy is the overall share classified the same way, and balanced accuracy averages performance in the CAPS-5-positive and CAPS-5-negative groups. ## Optimizing against the CAPS-5 diagnosis To make the CAPS-5 the optimization target rather than a row in a comparison, pass the clinician diagnosis to `compare_optimizations()` as a fixed criterion. The optimized PCL-5 definitions are then reported alongside it, and the symptom-frequency tools apply as in the [Comparing diagnostic criteria](comparing-criteria.html) vignette. ```{r caps5-as-fixed} comp <- compare_optimizations( ptsd, scenarios = list( "4/6 Hierarchical" = list(n_symptoms = 6, n_required = 4, hierarchical = TRUE), "4/6 Non-hierarchical" = list(n_symptoms = 6, n_required = 4, hierarchical = FALSE), "CAPS-5 (reference)" = list(type = "fixed", criterion = caps5_dx$PTSD_caps5, symptoms = 1:20) ), n_top = 5, score_by = "balanced_accuracy", show_progress = FALSE ) summarize_top_combinations(comp, top_n = 3, as_percent = TRUE) ``` ## Validating derived definitions against the CAPS-5 diagnosis At a validation site the definitions arrive already derived (see [Validating a shared definition across sites](multi-site-validation.html)), and the question becomes how each performs against the local clinician diagnosis. `evaluate_definitions()` takes the clinician standard through its `reference` argument: a logical vector, a 0/1-coded column, or the name of a carry-through column in the data. Participants without a clinician interview — `NA` in the reference — are excluded automatically, with a message reporting how many. A `"Full 20-item PCL-5"` ceiling row is added by default: the complete PCL-5 diagnosis scored against the same reference. That row separates the cost of dropping items from the intrinsic PCL-5-vs-CAPS-5 disagreement, and no reduced definition can be expected to beat it. Here we extract the top three combinations per optimized rule from the comparison above and request the tidy table layout. ```{r evaluate-vs-caps5} definitions <- extract_definitions(comp, n = 3) evaluate_definitions(ptsd, definitions, reference = caps5_dx$PTSD_caps5, tidy = TRUE) ``` The `Combination` column is `NA` for the ceiling row, which applies all 20 items rather than a subset. Comparing each definition's balanced accuracy with the ceiling shows how much of the disagreement with the CAPS-5 is attributable to simplification and how much the full instrument shares. ## Interpreting agreement and disagreement When the PCL-5 and CAPS-5 diagnoses agree closely, as they do in this paired sample, a simplified PCL-5 definition that reproduces the full PCL-5 diagnosis also largely reproduces the clinician diagnosis, so the choice of reference matters little. When the instruments diverge, the choice of reference matters, and reporting performance against both is the honest course; the fixed-criterion mechanism above makes that choice explicit. ## See also - [Getting started](getting-started.html) for the single-cohort derivation workflow. - [Comparing diagnostic criteria](comparing-criteria.html) for multi-rule comparison and the symptom-frequency heatmap. - [Validating abbreviated symptom definitions](validation.html) for internal and external validation.