The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
make_split_plan(group = , batch = ) directly?stratum column mean, and does splitGraph stratify?make_split_plan(group = , batch = ) directly?For a dataset with one subject column and one batch column you should: bioLeak’s make_split_plan() reaches the same grouping in one call, and splitGraph adds a hop. splitGraph earns its place when the structure is not one clean column per axis:
time_index that contradicts timepoint_precedes: validate_graph() reports these before any fold exists. Column-based grouping silently produces folds from inconsistent metadata.mode = "composite" merges samples into connected components over all chosen relations. bioLeak’s combined mode instead picks a primary axis and deletes training rows sharing a secondary level with the test set. Same metadata, different partitions; splitGraph makes the choice explicit and reproducible.relatedness_edges_from_kinship() and spatial_edges_from_coords() threshold them into edges and the derivation takes connected components. No categorical column can express this.split_spec records which relations, thresholds and strategy produced each group, and travels as schema-checked JSON to Python or any other consumer.Strict composite grouping is transitive closure. If sample A shares a subject with B, and B shares a batch with C, then A, B and C land in one group even though A and C share nothing directly. With a few large batches this collapses most of the dataset into one component:
meta <- data.frame(
sample_id = paste0("S", 1:6),
subject_id = c("P1", "P1", "P2", "P2", "P3", "P3"),
batch_id = c("B1", "B2", "B2", "B3", "B3", "B1"),
stringsAsFactors = FALSE
)
g <- graph_from_metadata(meta)
table(grouping_vector(derive_split_constraints(g, "composite", via = c("subject", "batch"))))
#>
#> component_1
#> 6Every subject bridges two batches, so all six samples form a single group and no split is possible. Three remedies, in order of preference:
spec$block_vars).strategy = "rule_based": each sample is grouped by the first relation in priority that is available to it, so relations do not chain.compare_split_constraints() to compare candidate modes (group counts, effective group counts, \(k\)-fold feasibility, and severed vs. remaining leakage risks), find_dependency_bridges() to pinpoint which specific entities (e.g. a shared batch or site) merge components, and summarize_leakage_risks() to inspect the severed column per mode.compare_split_constraints(g, k = 2)
#> <split_constraint_comparison> (k = 2 folds)
#> mode strategy n_samples n_groups max_group_size max_group_share
#> subject subject 6 3 2 0.3333333
#> batch batch 6 3 2 0.3333333
#> composite strict 6 1 6 1.0000000
#> effective_n_groups singleton_share k_fold_feasible n_risks_severed
#> 3 0 TRUE 3
#> 3 0 TRUE 0
#> 1 0 FALSE 3
#> n_risks_remaining min_stratum_groups
#> 0 NA
#> 3 NA
#> 0 NA
find_dependency_bridges(g, via = c("Subject", "Batch"))
#> <graph_query_result> find_dependency_bridges
#> Rows: 6
grouping_vector(derive_split_constraints(g, "composite", strategy = "rule_based",
via = c("subject", "batch"),
priority = c("subject", "batch")))
#> S1 S2 S3
#> "composite_subject:P1" "composite_subject:P1" "composite_subject:P2"
#> S4 S5 S6
#> "composite_subject:P2" "composite_subject:P3" "composite_subject:P3"relatedness_edges_from_kinship(pairs, threshold) keeps a pair when its kinship is at least the threshold; spatial_edges_from_coords(coords, radius) keeps a pair when its distance is at most the radius. The derivation then forms connected components over the kept edges. Two consequences:
graph$metadata$edge_sources and spec$metadata$threshold, so a reader of the spec can see which cut produced the groups.pairs <- data.frame(id1 = c("P1", "P2"), id2 = c("P2", "P3"), kinship = c(0.26, 0.13))
meta <- data.frame(sample_id = c("S1", "S2", "S3"), subject_id = c("P1", "P2", "P3"))
build <- function(threshold) {
g <- build_dependency_graph(
list(create_nodes(meta, "Sample", "sample_id"), create_nodes(meta, "Subject", "subject_id")),
list(create_edges(meta, "sample_id", "subject_id", "Sample", "Subject", "sample_belongs_to_subject"),
relatedness_edges_from_kinship(pairs, threshold = threshold))
)
grouping_vector(derive_split_constraints(g, "relatedness"))
}
build(0.25) # only P1~P2 pass: {S1,S2}, {S3}
#> S1 S2 S3
#> "relatedness:component_1" "relatedness:component_1" "relatedness:component_2"
build(0.10) # P2~P3 also passes and chains: {S1,S2,S3}
#> S1 S2 S3
#> "relatedness:component_1" "relatedness:component_1" "relatedness:component_1"You can also build the graph once at a permissive threshold and vary min_weight (for relatedness) or max_distance (for spatial) directly inside derive_split_constraints() without rebuilding the graph.
stratum column mean, and does splitGraph stratify?No. stratum is an annotation: by default (stratify_by = "outcome"), the outcome level attached to each sample (from sample_has_outcome, or the subject’s outcome via subject_has_outcome). You can also pass stratify_by = "batch" (or "study", "site", "region", "platform", "assay", or "none") to as_split_spec() to populate stratum from a blocking variable instead. It is exposed through spec$stratum_var so a consumer such as scikit-learn’s StratifiedGroupKFold or bioLeak’s stratify = TRUE can balance folds. Balancing is execution and belongs downstream; splitGraph only describes.
schema_version (currently 0.3.0) is independent of the package version. It describes the on-disk JSON contract.NA and ignore unknown ones. A differing major loads with a warning suggesting migrate_dependency_graph_json() / migrate_split_spec_json().inst/schema/<schema_version>/, and every written file carries a $schema URL pointing at its own version, so the reference stays valid after later bumps. validate_graph_json() and validate_split_spec_json() (or read_*(validate = TRUE)) check a file against the installed schema without a JSON Schema engine.The R reader is permissive at that boundary, which is worth seeing rather than taking on trust:
tiny <- data.frame(sample_id = c("S1", "S2"), subject_id = c("P1", "P2"),
stringsAsFactors = FALSE)
g_tiny <- graph_from_metadata(tiny)
p <- tempfile(fileext = ".json")
write_split_spec(as_split_spec(derive_split_constraints(g_tiny, "subject"),
graph = g_tiny), p)
# Pretend the file was written by a future splitGraph with a different major.
raw <- jsonlite::fromJSON(p, simplifyVector = FALSE)
raw$schema_version <- "1.0.0"
writeLines(jsonlite::toJSON(raw, auto_unbox = TRUE, null = "null"), p)
back <- withCallingHandlers(
read_split_spec(p),
warning = function(w) {
message("warning: ", conditionMessage(w))
invokeRestart("muffleWarning")
}
)
#> warning: Reading split_spec: JSON schema_version `1.0.0` differs in major version from installed splitGraph schema_version `0.3.0`. Loading anyway; consider `migrate_split_spec_json()` to upgrade the file.
class(back)
#> [1] "split_spec"
unlink(p)One asymmetry to know about if you consume the format outside R: the shipped Python reader is stricter. Where R warns and loads, splitspec raises a ValueError naming the version and refuses the file, on the grounds that a non-interactive consumer is better off failing than silently misreading a format it does not know. Both implementations accept every schema sharing major 0.
Yes. graph_from_metadata() is an S3 generic. The SummarizedExperiment method reads colData() as the metadata table (and MultiAssayExperiment reads sampleMap() and colData(), mapping colname \(\rightarrow\) sample_id, primary \(\rightarrow\) subject_id, and assay \(\rightarrow\) assay_id). When colData has no sample_id column the assay column names are used, so a typical Bioconductor object needs no preparation at all. Pass sample_id_col = to name a different column, and columns = to map your own names onto the canonical ones exactly as for a data frame.
meta <- data.frame(
sample_id = c("S1", "S2", "S3", "S4"),
subject_id = c("P1", "P1", "P2", "P2"),
batch_id = c("B1", "B2", "B1", "B2"),
stringsAsFactors = FALSE
)
se <- SummarizedExperiment::SummarizedExperiment(
assays = list(counts = matrix(0, nrow = 3, ncol = 4,
dimnames = list(NULL, meta$sample_id))),
colData = meta[, c("subject_id", "batch_id")]
)
g_se <- graph_from_metadata(se, graph_name = "from-se")
grouping_vector(derive_split_constraints(g_se, "subject"))
#> S1 S2 S3 S4
#> "subject:P1" "subject:P1" "subject:P2" "subject:P2"
# identical to building from the data frame directly
identical(
grouping_vector(derive_split_constraints(g_se, "subject")),
grouping_vector(derive_split_constraints(graph_from_metadata(meta), "subject"))
)
#> [1] TRUEEvery error inherits from splitgraph_error and carries a code; see ?splitgraph_conditions for the subclasses (splitgraph_schema_error, splitgraph_reference_error, splitgraph_ambiguity_error, splitgraph_validation_error, splitgraph_io_error).
g <- graph_from_metadata(data.frame(sample_id = c("S1", "S2"), subject_id = c("P1", "P2")))
tryCatch(
query_neighbors(g, "sample:S9"),
splitgraph_reference_error = function(e) e$code
)
#> [1] "unknown_node_ids"Every step is linear in nodes plus edges. On a 20,000-sample synthetic cohort the shipped benchmark (inst/bench/pipeline.R) builds the graph, validates it, derives a default composite constraint, enriches a spec and writes both JSON files in about twelve seconds on a laptop. More than half of that is writing the graph JSON (~6 s); the specification itself writes in well under a second, and the derivation steps are hundredths of a second. If you only need the handoff artifact, skip write_dependency_graph().
Two caveats about the guard rail. tests/testthat/test-performance.R is a wall-clock budget at 5,000 samples with deliberately generous limits, not a benchmark; only the composite derivation is additionally checked for scaling, by timing 1,000 against 4,000 samples and failing if the ratio exceeds 8 (a quadratic step would give roughly 16). Other steps could therefore degrade somewhat without tripping it.
The one inherently quadratic output is the explicit sample-pair table of detect_shared_dependencies() and detect_dependency_components()$metadata$projection_edges, whose size is the number of pairs sharing a target; grouping itself never enumerates pairs.
These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.