
Cross-Validate Conditional Choices in Complete Matched Sets
Source:R/cv_choice.R
cross_validate_clogit.RdCross-validate a step-selection model or another conditional logistic model. In each fold, refit the coefficients, and optionally the scales of effect, using the analysis rows, then score the probability the model gives the used alternative among all alternatives of each held-out choice set. A choice set is one used alternative together with the available alternatives it was chosen from, for example one observed step and its random steps.
Usage
cross_validate_clogit(
formula,
data,
folds,
kernel_inputs = NULL,
backend = c("conditional_logit", "refit"),
method = c("exact", "efron", "breslow"),
cox_control = survival::coxph.control(),
optim_args = list(),
keep_fits = FALSE,
keep_selection = FALSE
)
# S3 method for class 'multiScaleR_choice_cv'
print(x, ...)Arguments
- formula
Two-sided formula. The response is a numeric or logical column coded 1 for the used alternative and 0 for the available ones. The right side holds the predictors and one bare
strata(choice_id)term naming the column that identifies each choice set. Factors, interactions, offsets, and the data-dependent termsscale(),poly(),splines::ns(), andsplines::bs()are allowed. A dot (.) expands to the columns ofdata, so remove the set ID from it, for examplecase_ ~ . - step + strata(step).- data
Data frame with one row per alternative, holding the case column, the set ID, and every predictor. Its row names are the row IDs used by
foldsand, when supplied, bykernel_inputs. They must be unique.- folds
Named list of explicit splits of row IDs, as in
cross_validate(). Every choice set must lie wholly in the analysis rows, wholly in the assessment rows, or wholly outside the split.grouped_cv_folds()applied to the set IDs, or to animal IDs, does this. See "Build folds that keep choice sets whole".- kernel_inputs
Optional
multiScaleR_dataobject fromkernel_prep()orkernel_prep_by_group()with the cell-level data kept, one prepared row per alternative. The defaultNULLfits an ordinary conditional logistic reference model without scales. When it is supplied, each fold also estimates the scales, and learns the standardization and distance bins, from its analysis rows only.- backend
Scale-fitting route used with
kernel_inputs, either"conditional_logit"(default) or"refit". Supplyoptim_args = list(radius_grid = ...)with"refit"to profile hard radii, together with any smooth scales, in each training split. The usual kernel, metric, and numerical checks ofmultiScale_optim()apply. Ignored whenkernel_inputsisNULL.- method
Partial-likelihood method passed to
survival::clogit(), one of"exact"(default),"efron", or"breslow". These are rules for tied event times. With one used alternative per set they give the same likelihood, but the method is recorded, andcv_tab()andselect_cv()compare only candidates that declare the same method.- cox_control
Named list of
survival::coxph.control()settings for the native training fit. The default keeps the settings ofcoxph.control().- optim_args
Named list of extra
multiScale_optim()arguments used in every fold, such asparorprofile_control. Supplyradius_gridwithbackend = "refit"to profile hard radii, together with any smooth scales, in each training split. Supplycovariate_transform = "raw"withbackend = "refit"to keep optimized covariates in their own units (see "Keep optimized covariates in their own units"). Arguments that supply data, the starting model, a custom refit function, parallel workers,backend, orverboseare reserved. Every fold runs on one worker. The defaultlist()keeps the backend's own defaults. Choose starting values without using the assessment outcomes.- keep_fits
Logical. The default
FALSEkeeps the predictions, scores, fold estimates, and compact diagnostics.TRUEalso keeps the full fitted model of every fold infits, which can use much more memory. You need full fits only to inspect the fold models or to haveselect_cv()return the selected fits.- keep_selection
Logical.
TRUEkeeps a compact record of each fold's training likelihood and parameter count inselection_records, which is whatselect_cv()needs whenkeep_fits = FALSE. The records come from the fits already run, so they add no fitting time. The default isFALSE.- x
A
multiScaleR_choice_cvresult, forprint().- ...
Not used. Reserved for future print controls.
Value
A list of class multiScaleR_choice_cv, which inherits from
multiScaleR_cv. Its summary holds the negative log score of the
used alternative, averaged over held-out choice sets, and predictions
holds one row per alternative with its probability within its set. The
other elements follow cross_validate(). The conditional-choice section
of multiScaleR_cv lists every difference. print() shows how
many folds were scored and the pooled score, and returns x invisibly.
Build folds that keep choice sets whole
Before any fitting, the function rejects a split that divides a choice
set. Holding out whole choice sets measures transfer to new sets from the
same animals. To measure transfer to new animals, build the folds from
animal IDs instead. If step labels repeat across animals, first create a
globally unique set ID, such as interaction(animal, step, drop = TRUE),
and use that column in strata(). For spatial blocks, pass the set IDs or
the animal IDs as group to spatial_cv_folds(), so that no set crosses
a split boundary.
What the model can contain
Each set needs exactly one used alternative and at least one available
alternative. Weights, missing values, robust clustering (cluster()),
frailty and other special or penalized Cox terms, and general censored
survival responses are outside this function, and no rows are dropped
silently. Predictors that are constant within every set have no
identifiable coefficient. Native alias and convergence problems are kept
as failures rather than resolved by dropping terms or alternatives, and a
warning from the native training fit makes the fold fail. Every scored
fold checks that its native fit is unweighted and unpenalized, uses the
analysis rows and complete sets, and used the declared method, whether
or not fits or records are kept. Learned transformations follow the rules
described in cross_validate(). Their training constants add no stratum
effects and do not change the event count.
Estimate scales within the folds
With kernel_inputs, each fold rebuilds the spatial covariates from the
raw cells of its analysis rows, learns the standardization and the
representative bin distances there, and estimates the scales. The
assessment rows reuse those frozen references. Inputs from
kernel_prep_by_group() are fixed covariate maps and add no random
effects. Raw covariate units (covariate_transform = "raw" in
optim_args) require backend = "refit".
vignette("v04-landscape-metrics", package = "multiScaleR") shows a
search over a grid of radii on a full data set.
Read the scores
The score of a held-out set is the negative natural log of the probability the fitted model gives its used alternative, where the probabilities of a set's alternatives sum to 1. The pooled score averages over sets. A model without covariates or offsets gives each of k alternatives a probability of 1/k, so its score is log(k), or 1.099 for three alternatives. A useful covariate lowers the score. Scores are computed on the log scale, so a probability that prints as zero still has a finite score. Predictions use the saved training terms and assign no effect to the labels of new sets. These within-set probabilities differ from Bernoulli probabilities of use and from unnormalized relative selection strengths.
For a model with no estimated coefficients, survival's logLik() can
return NA. The function then checks survival's stored null
log-likelihood against the exact complete-set likelihood before using it,
with K = 0, for training-criterion selection. Missing or inconsistent
values remain failures. Other native fits are not altered.
Compare and select models
Use cv_tab() with metric = "neg_log_score" to rank choice models fitted
to the same sets, folds, and method. Use select_cv() to score a
training AIC, AICc, or BIC rule. Its sample size is the number of choice
sets (one event each), and K counts the coefficients and any estimated
scales and shapes. Choice scores cannot be mixed with scores from
Bernoulli GLMs. They treat fitted parameters as known, so they do not
integrate parameter or selection uncertainty, give intervals, or establish
calibration. vignette("v08-cross-validation", package = "multiScaleR")
includes a step-selection example.
Examples
if (requireNamespace("survival", quietly = TRUE)) {
# 24 choice sets of three alternatives. The first of each set was used.
i <- 1:72
choices <- data.frame(step = (i - 1) %/% 3 + 1,
used = as.numeric(i %% 3 == 1),
habitat = sin(i * 1.7))
folds <- grouped_cv_folds(choices$step, k = 3) # whole sets held out
models <- list(
null = cross_validate_clogit(used ~ strata(step), choices, folds,
keep_selection = TRUE),
habitat = cross_validate_clogit(used ~ habitat + strata(step), choices,
folds, keep_selection = TRUE))
print(models$habitat) # natural-log loss per held-out choice set
# The null model scores log(3) = 1.099. Here habitat is unrelated to use,
# so it does not lower the score.
print(cv_tab(models, metric = "neg_log_score"))
# Model picked by training AICc in each fold
select_cv(models, criterion = "AICc")$selection$chosen
}
#> Conditional-choice CV: 3 of 3 folds scored.
#> metric estimate unit n_assessment n_scored weight_sum complete
#> neg_log_score 1.126067 choice_set 24 24 24 TRUE
#> n_alternatives
#> 72
#> Modnames Rank Score Delta Metric Unit N_assessment
#> 1 null 1 1.098612 0.00000000 neg_log_score choice_set 24
#> 2 habitat 2 1.126067 0.02745496 neg_log_score choice_set 24
#> N_scored Weight_sum Warnings
#> 1 24 24 0
#> 2 24 24 0
#> fold1 fold2 fold3
#> "null" "null" "null"