---
title: "Adapter cookbook: from split_spec to native resamples"
author: "Selçuk Korkmaz"
date: "`r Sys.Date()`"
output:
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{Adapter cookbook: from split_spec to native resamples}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  message = FALSE,
  warning = FALSE,
  eval = TRUE
)

package_root <- if (file.exists("../DESCRIPTION")) ".." else "."
if (requireNamespace("pkgload", quietly = TRUE) &&
    file.exists(file.path(package_root, "DESCRIPTION"))) {
  pkgload::load_all(package_root, export_all = FALSE, helpers = FALSE, quiet = TRUE)
} else {
  library(splitGraph)
}
```

## What this vignette is for

`splitGraph` ends at a `split_spec` object. It deliberately knows nothing
about `rsample`, `tidymodels`, or any other resampling engine. The handoff
contract is the `sample_data` table inside the spec plus a few scalar
fields (`group_var`, `block_vars`, `time_var`, `stratum_var`,
`ordering_required`, `recommended_resampling`), together with provenance the
adapter can inspect to choose a strategy (`constraint_mode`,
`constraint_strategy`).

You do not always have to write this glue yourself. The reference downstream
consumer, [**bioLeak**](https://github.com/selcukorkmaz/bioLeak), takes a
`split_spec` directly — `bioLeak::as_leaksplits(spec, data, outcome)` builds an
executable, leakage-audited split plan from it. The released bioLeak (0.3.8)
reads the `subject`, `batch`, `study` and `time` modes; for the others,
including `composite`, hand it the grouping column instead
(`bioLeak::make_split_plan(joined, outcome, mode = "subject_grouped",
group = "group_id")`), which produces the same split by a different route.

This cookbook is for the other case: when you want to feed a `split_spec` into a
different engine, or understand exactly what a consumer has to honor. It shows
three small, self-contained adapters that turn a `split_spec` into something a
downstream workflow can use:

1. A **base-R adapter** that returns a list of `(train, test)` row-index
   pairs — runnable here, no extra dependencies.
2. An **`rsample::group_vfold_cv()`** adapter for grouped cross-validation
   keyed to `group_id`.
3. An **`rsample::rolling_origin()`** adapter for ordered evaluation keyed
   to `order_rank`.

Adapters 2 and 3 are evaluated when `rsample` is installed (it is in
`Suggests`, never in `Imports`: `splitGraph` itself has no resampling
dependency) and shown as code otherwise.

The same pattern works for any other resampling library you happen to use.

## Build a split_spec to work with

```{r build-spec}
meta <- data.frame(
  sample_id    = c("S1", "S2", "S3", "S4", "S5", "S6"),
  subject_id   = c("P1", "P1", "P2", "P2", "P3", "P3"),
  batch_id     = c("B1", "B2", "B1", "B2", "B1", "B2"),
  timepoint_id = c("T0", "T1", "T0", "T1", "T0", "T1"),
  time_index   = c(0, 1, 0, 1, 0, 1),
  outcome_id   = c("ctrl", "case", "ctrl", "case", "case", "ctrl"),
  stringsAsFactors = FALSE
)

g <- graph_from_metadata(meta, graph_name = "cookbook")
subject_constraint <- derive_split_constraints(g, mode = "subject")
spec <- as_split_spec(subject_constraint, graph = g)
spec
```

The `sample_data` table is the contract:

```{r show-sample-data}
as.data.frame(spec)[, c("sample_id", "group_id", "batch_group", "order_rank")]
```

## Adapter 1 — base R: leave-one-group-out folds

This is the simplest meaningful adapter. It groups by whatever
`split_spec$group_var` says is the split unit, and returns one held-out
group per fold.

```{r adapter-base-r}
logo_folds <- function(spec, observation_data, sample_id_col = "sample_id") {
  stopifnot(inherits(spec, "split_spec"))
  if (!sample_id_col %in% names(observation_data)) {
    stop("`observation_data` must contain a `", sample_id_col, "` column.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$group_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )
  joined$.row <- seq_len(nrow(joined))
  groups <- split(joined$.row, joined[[spec$group_var]])

  lapply(names(groups), function(g) {
    list(
      group   = g,
      train   = unlist(groups[setdiff(names(groups), g)], use.names = FALSE),
      assess  = groups[[g]]
    )
  })
}

# Pretend we have an observation frame keyed by sample_id.
set.seed(1)
obs <- data.frame(
  sample_id = meta$sample_id,
  x = rnorm(nrow(meta)),
  y = rbinom(nrow(meta), 1, 0.5)
)

folds <- logo_folds(spec, obs)
length(folds)
folds[[1]]
```

That is the entire downstream contract: take `spec`, take an observation
frame, return train/assess index lists. Anything more complicated is
specific to a resampling library.

### Honoring block variables

`group_var` is the primary split unit, but `split_spec` also advertises coarser
`block_vars` — dependency axes that should ideally not straddle a fold even when
they are not the grouping unit. They are per-sample columns aligned to
`sample_id`, so an adapter reads them exactly like `group_var`:

```{r block-vars}
spec$block_vars
head(spec$sample_data[, c("sample_id", spec$group_var, spec$block_vars)])
```

A block-aware adapter can pass these to a resampler's blocking/strata argument,
or simply audit its folds. Here we check whether any batch straddles the
train/assess boundary — a leak a subject-only split does not prevent, and
exactly what carrying `batch_group` on the spec lets a consumer catch:

```{r block-audit}
block <- spec$block_vars[[1]]
block_of <- setNames(spec$sample_data[[block]], spec$sample_data$sample_id)

do.call(rbind, lapply(folds, function(f) {
  data.frame(
    held_out_group    = f$group,
    straddling_batches = paste(
      intersect(block_of[obs$sample_id[f$train]],
                block_of[obs$sample_id[f$assess]]),
      collapse = ", "
    )
  )
}))
```

Every batch appears on both sides, because grouping by subject does not also
block by batch. Whether that matters is a scientific decision — the point is
that the spec carries enough information for the adapter to make it.

### Honoring the stratum annotation

`stratum_var` names one more column: the outcome level each sample carries.
`splitGraph` records it and stops there — it never balances folds itself — so
stratification is the adapter's job, and the spec hands it the input.

```{r stratum}
spec$stratum_var
spec$sample_data[, c("sample_id", spec$group_var, spec$stratum_var)]
```

There is a catch worth knowing before you wire it into a resampler's `strata`
argument. The annotation is *per sample*, while the split unit is the group, and
the two need not agree: here every subject contributes one `case` and one
`ctrl`, so no group has a single stratum at all.

```{r stratum-constant}
tapply(spec$sample_data$stratum, spec$sample_data$group_id,
       function(x) length(unique(x)) == 1L)
```

A grouped resampler that stratifies needs one stratum per *group*, and says so
plainly when it does not get one:

```{r stratum-rsample, eval = requireNamespace("rsample", quietly = TRUE)}
joined_s <- merge(obs, spec$sample_data[, c("sample_id", "group_id", "stratum")],
                  by = "sample_id", sort = FALSE)

tryCatch(
  rsample::group_vfold_cv(joined_s, group = "group_id", v = 3, strata = "stratum"),
  error = function(e) conditionMessage(e)
)
```

So the adapter has to decide how to get there — summarise the annotation to the
group level (the group's only label when it has one, a majority label otherwise),
stratify on a blocking variable that *is* constant within the group, or accept
unbalanced folds. `splitGraph` deliberately does not pick for you; it records
what each sample is so the choice is visible instead of silent.

## Adapter 2 — `rsample::group_vfold_cv()`

Grouped CV keyed to `group_id`. The downstream package would typically
ship something like this; the adapter is short enough that you can paste
it into your own analysis script.

```{r adapter-rsample-group, eval = requireNamespace("rsample", quietly = TRUE)}
spec_to_group_vfold <- function(spec, observation_data,
                                v = NULL,
                                sample_id_col = "sample_id") {
  stopifnot(inherits(spec, "split_spec"))
  if (!requireNamespace("rsample", quietly = TRUE)) {
    stop("Install rsample to use this adapter.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$group_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )

  n_groups <- length(unique(joined[[spec$group_var]]))
  if (is.null(v)) v <- n_groups

  rsample::group_vfold_cv(
    data  = joined,
    group = !!spec$group_var,
    v     = v
  )
}
```

`v = NULL` (the default above) gives leave-one-group-out, which is the
right default when `splitGraph` has already grouped samples by their
deepest leakage-relevant unit (e.g. subject). Pick a smaller `v` for
k-fold-style grouped CV.

```{r adapter-rsample-group-run, eval = requireNamespace("rsample", quietly = TRUE)}
grouped <- spec_to_group_vfold(spec, obs)
grouped
# Every assessment set is exactly one subject's samples:
vapply(grouped$splits, function(s) {
  paste(sort(unique(rsample::assessment(s)$group_id)), collapse = ", ")
}, character(1))
```

## Adapter 3 — `rsample::rolling_origin()`

When `spec$ordering_required` is `TRUE` (or `spec$time_var` is set), the
right downstream object is an ordered split rather than a grouped one.

```{r adapter-rsample-rolling, eval = requireNamespace("rsample", quietly = TRUE)}
spec_to_rolling_origin <- function(spec, observation_data,
                                   sample_id_col = "sample_id",
                                   initial = NULL,
                                   assess = 1L) {
  stopifnot(inherits(spec, "split_spec"))
  if (is.null(spec$time_var)) {
    stop("This split_spec has no `time_var`; ordered evaluation is not available.")
  }
  if (!requireNamespace("rsample", quietly = TRUE)) {
    stop("Install rsample to use this adapter.")
  }

  joined <- merge(
    observation_data,
    spec$sample_data[, c("sample_id", spec$time_var)],
    by.x = sample_id_col, by.y = "sample_id", sort = FALSE
  )
  ordered <- joined[order(joined[[spec$time_var]]), , drop = FALSE]

  if (is.null(initial)) initial <- max(1L, floor(nrow(ordered) * 0.6))
  rsample::rolling_origin(ordered, initial = initial, assess = assess)
}
```

```{r adapter-rsample-rolling-run, eval = requireNamespace("rsample", quietly = TRUE)}
rolling <- spec_to_rolling_origin(spec, obs, initial = 3, assess = 1)
rolling
# No analysis sample comes after any assessment sample:
vapply(rolling$splits, function(s) {
  max(rsample::analysis(s)$order_rank) <= min(rsample::assessment(s)$order_rank)
}, logical(1))
```

The key idea: `splitGraph` puts ordering information on the spec; the
adapter is just a thin shim that consumes it.

### Two things to check before trusting an ordered adapter

**`order_rank` has ties, and row-wise slicing ignores them.** `order_rank` is a
rank over *timepoints*, not over rows, so every sample collected at the same
timepoint shares a value — here six samples carry just two distinct ranks. A
resampler that slices by row position will therefore put part of a timepoint in
the analysis set and the rest in the assessment set:

```{r rolling-ties, eval = requireNamespace("rsample", quietly = TRUE)}
length(unique(spec$sample_data$order_rank))   # distinct ranks
nrow(spec$sample_data)                        # rows

vapply(rolling$splits, function(s) {
  shared <- intersect(rsample::analysis(s)$order_rank,
                      rsample::assessment(s)$order_rank)
  paste(shared, collapse = ", ")
}, character(1))
```

Slices 2 and 3 share rank `2`: one `T1` sample is being predicted while another
`T1` sample is in training. If your evaluation requires that a whole timepoint
be held out, cut on rank boundaries rather than row counts — choose `initial` so
it falls at a change of `order_rank`, or group the rows by rank first. The spec
gives you the rank; only you know whether ties are acceptable.

**`rolling_origin()` is superseded.** It still works and is not deprecated, but
rsample now steers users to `sliding_window()` / `sliding_index()` /
`sliding_period()`, where active development happens. The equivalent call is:

```{r sliding-window, eval = requireNamespace("rsample", quietly = TRUE)}
joined  <- merge(obs, spec$sample_data[, c("sample_id", spec$time_var)],
                 by = "sample_id", sort = FALSE)
ordered <- joined[order(joined[[spec$time_var]]), , drop = FALSE]

sliding <- rsample::sliding_window(
  ordered,
  lookback     = Inf,   # cumulative analysis window, like rolling_origin()
  assess_stop  = 1,
  complete     = FALSE,
  skip         = 2      # start where `initial = 3` did
)

identical(
  lapply(sliding$splits, function(s) rsample::assessment(s)$sample_id),
  lapply(rolling$splits, function(s) rsample::assessment(s)$sample_id)
)
```

Either way the adapter is the same shim; only the rsample entry point changes.

## Going across language boundaries via JSON

If the downstream consumer is not in R, write the spec to JSON and let the
consumer interpret it. The on-disk format is a formal, versioned contract: it
has a JSON Schema (Draft 2020-12) shipped in `inst/schema/<schema_version>/`,
each file names it
via a `$schema` key, and `validate_split_spec_json()` checks a file against it
before you consume it.

```{r serialize, eval = requireNamespace("jsonlite", quietly = TRUE)}
tmp <- tempfile(fileext = ".json")
write_split_spec(spec, tmp)

# The file opens with its $schema reference and schema_version.
cat(readLines(tmp, n = 5), sep = "\n")

# Validate the file against the shipped JSON Schema, then read it back exactly.
validate_split_spec_json(tmp)$valid
spec2 <- read_split_spec(tmp)
identical(spec$sample_data$group_id, spec2$sample_data$group_id)

unlink(tmp)
```

You do not have to write a JSON parser to consume this from Python: the package
ships a pure-Python reference reader (`inst/python/splitspec`) that recovers the
same grouping and ordering and drives scikit-learn `GroupKFold` /
`TimeSeriesSplit`. The *cross-language-handoff* vignette walks the full
R → JSON → Python → scikit-learn path:

```{r xlang-pointer, eval = FALSE}
vignette("cross-language-handoff", package = "splitGraph")
```

The same read/write pair exists for `dependency_graph`
(`write_dependency_graph()` / `read_dependency_graph()`, validated with
`validate_graph_json()`). Both formats are documented under `?write_split_spec`
and `?write_dependency_graph`. Because `schema_version` follows a documented
major-compatibility policy, a file written by an older splitGraph still loads;
`migrate_split_spec_json()` upgrades it to the current version in place.

## Letting the spec pick the adapter

The three adapters above cover different shapes of split. You do not have to
choose between them by hand: `split_spec` carries `recommended_resampling`, so a
single dispatcher can route each spec to the right one. This makes a pipeline
that handles subject, batch, time, and composite specs uniformly.

```{r dispatch}
recommend_adapter <- function(spec) {
  switch(
    spec$recommended_resampling,
    grouped_cv          = "group_vfold_cv (group = group_id)",
    blocked_cv          = "group_vfold_cv (group = group_id)",
    custom_grouped_cv   = "group_vfold_cv (group = group_id)",
    leave_one_group_out = "leave-one-group-out over group_id",
    ordered_split       = "rolling_origin (order by order_rank)",
    "group_vfold_cv (default)"
  )
}

# The subject spec recommends grouped CV; a time-mode spec recommends ordering.
recommend_adapter(spec)
time_spec <- as_split_spec(derive_split_constraints(g, mode = "time"), graph = g)
recommend_adapter(time_spec)
```

Those five branches are the whole vocabulary: `recommended_resampling` takes one
of exactly `grouped_cv`, `blocked_cv`, `leave_one_group_out`, `ordered_split` and
`custom_grouped_cv`, whatever the mode. Eleven modes map onto them like this:

| `recommended_resampling` | Modes that produce it |
|---|---|
| `grouped_cv` | `subject`, `region`, `assay`, `relatedness`, `spatial`, `composite` (rule-based) |
| `blocked_cv` | `batch`, `platform` |
| `leave_one_group_out` | `study`, `site` |
| `ordered_split` | `time` |
| `custom_grouped_cv` | `composite` (strict) |

```{r dispatch-exhaustive}
# One graph carrying every direct relation, so each mode has something to group.
g_all <- graph_from_metadata(transform(
  meta,
  site_id     = c("N", "N", "N", "B", "B", "B"),
  region_id   = "ctx",
  platform_id = "il",
  assay_id    = "rna"
))

modes <- c("subject", "batch", "study", "time",
           "site", "region", "platform", "assay")
vapply(modes, function(m) {
  as_split_spec(derive_split_constraints(g_all, mode = m),
                graph = g_all)$recommended_resampling
}, character(1))

# And the two composite strategies, which supply the fifth value.
c(
  strict = as_split_spec(derive_split_constraints(
    g_all, mode = "composite", strategy = "strict",
    via = c("Subject", "Batch")), graph = g_all)$recommended_resampling,
  rule_based = as_split_spec(derive_split_constraints(
    g_all, mode = "composite", strategy = "rule_based",
    priority = c("subject", "batch")), graph = g_all)$recommended_resampling
)
```

So a dispatcher with those five branches cannot be surprised by a spec, and the
`default` arm above is belt and braces rather than a real fallback.
`recommended_resampling` is still only a hint — your adapter is free to override
it — but it lets one entry point serve every constraint mode without inspecting
the graph.

## When you need a custom adapter

The only assumptions an adapter has to honor:

- `split_spec$sample_data` is keyed by `sample_id` (character).
- `split_spec$group_var` is the column that holds the splitting unit.
- `split_spec$block_vars` are present-but-coarser blocking columns. Depending on
  the graph these include `batch_group`, `study_group`, `site_group`,
  `region_group`, `platform_group`, and `assay_group` — an adapter can block or
  stratify on any that are present.
- `split_spec$time_var`, when non-`NULL`, defines the ordering. Its values are
  ranks over timepoints, so ties are expected and meaningful.
- `split_spec$stratum_var`, when non-`NULL`, names the per-sample outcome level.
  It is an annotation: `splitGraph` never balances folds, and the value is not
  guaranteed constant within a group.
- `split_spec$recommended_resampling` is a hint, not a contract — your
  adapter is free to ignore it. It is one of five documented strings.
  `constraint_mode` / `constraint_strategy` are available if you want to branch
  (e.g. treat `"time"` specs as ordered).

That is the whole interface, and it is stable: `bioLeak::as_leaksplits()`
consumes exactly these fields, and a contract test in splitGraph pins the seam
so it cannot drift. As long as those fields are honored, anything is a valid
downstream consumer.