Skip to content

17/ New classifiers: aorsf, glmboost (boosted Cox), LiblineaR and ncvreg (Suggests) - #143

Draft
ellispatrick wants to merge 2 commits into
review/16-registryfrom
review/17-new-classifiers
Draft

ellispatrick wants to merge 2 commits into
review/16-registryfrom
review/17-new-classifiers

Conversation

@ellispatrick

Copy link
Copy Markdown
Member

Part 17 (#140 → #141 → #142 → this). Based on #142 (the registry). The four classifiers are added through the registry. Each is in Suggests, and existing results are unchanged.

Summary

Four new classifier keywords, each one interface file and one registry entry:

Keyword Library Outcomes Model
"aorsf" aorsf classes, survival oblique random forest (linear-combination splits), 500 trees
"glmboost" mboost survival componentwise boosted Cox model, mstop = 100, nu = 0.1
"LiblineaR" LiblineaR classes logistic regression with an L2 (default) or L1 penalty; cost chosen by cross-validation
"ncvreg" ncvreg two classes, survival MCP-penalised logistic or Cox regression (SCAD and lasso available); lambda chosen by cross-validation

Why

Ellis Patrick's decision on Q27–Q30 (add all four), ProjectClassifyR docs/DECISIONS.md, 10 October 2026. The survey in docs/CLASSIFIER_LIBRARIES.md found:

  • aorsf gave the best forest C-index on METABRIC and fitted 14–80× faster than randomForestSRC.
  • glmboost gave the best boosting C-index on Colocalisation.
  • LiblineaR was the fastest and most accurate linear model on asthma.
  • ncvreg offers MCP and SCAD penalties.

What changes

  • R/interfaceObliqueForest.R:
    • One interface for both outcomes.
    • Class probabilities, or predicted mortality as the risk score.
    • Features are ranked by ANOVA importance.
    • It uses one thread by default (aorsf's own default is every core), and set.seed makes it reproducible.
  • R/interfaceBoostedCox.R:
    • The linear predictor is the risk score, and the chosen coefficients give the selected features.
    • glmboost's warning that a Cox model has no intercept is muffled.
  • R/interfaceLiblineaR.R:
    • Features are standardised by the training samples, and classes get inverse class size weights.
    • Without a cost, the cost is chosen from 10^(-3:2) by the balanced error of five-fold stratified cross-validation of the training samples (the smallest cost among ties).
    • Only the logistic types (0, 6, 7) are accepted, because they give class probabilities.
  • R/interfaceNcvreg.R:
    • cv.ncvreg (binomial) or cv.ncvsurv, five folds, at lambda.min (ncvreg's default measure, the deviance).
    • More than two classes give a clear error.
  • Registry entries: each has tuning presets for tuneParams = "auto", as listed in the presets vignette.
  • Tests: the registry test fits and predicts each for every outcome it accepts. Further tests check reproducibility, that higher risk scores mean earlier events, the feature extractors, and the errors.

Evidence

20 × 5 cross-validation, shared folds, seed 1, 4 cores (ProjectClassifyR scripts/structure/new_classifiers_bench.R, results/structure/new_classifiers/tables.txt).

Asthma (190 samples, 124 vs 66), classes.

Features Classifier Time (s) Balanced accuracy AUC Median features used
t-test top 20 DLDA 6.4 0.787 0.876 20
randomForest 8.1 0.781 0.889 20
aorsf 9.4 0.799 0.898 20
elasticNetGLM 7.4 0.816 0.892 11.5
LASSOGLM 7.7 0.810 0.892 8
LiblineaR 5.9 0.810 0.894 20
ncvreg 11.9 0.785 0.888 3
SVM 5.3 0.776 0.874 20
XGB 5.4 0.781 0.877 20
all 2000 randomForest 14.0 0.833 0.934 1643
aorsf 51.4 0.829 0.937 630
elasticNetGLM 14.9 0.909 0.976 85
LiblineaR 24.7 0.942 0.989 2000
ncvreg 21.9 0.864 0.947 8

METABRIC (165 patients, 39 relapses), relapse-free survival, CoxPH top 20 features of each view; C-index.

Classifier clinical RNA Microarray Type Proportions Type Protein Means Colocalisation Domain Proportions Proportions of Parent Time (s), all views
CoxPH 0.653 0.481 0.432 0.560 0.604 0.588 0.609 12.1
CoxNet 0.619 0.520 0.521 0.608 0.637 0.560 0.643 37.7
randomSurvivalForest 0.680 0.585 0.595 0.664 0.617 0.622 0.592 56.8
XGB 0.550 0.544 0.597 0.672 0.621 0.592 0.574 20.2
aorsf 0.661 0.564 0.565 0.655 0.642 0.603 0.622 30.9
glmboost 0.623 0.516 0.523 0.627 0.646 0.588 0.621 21.8
ncvreg 0.524 0.494 0.506 0.547 0.604 0.507 0.538 37.1
  • aorsf: close to the best forest on every view and about half randomSurvivalForest's time. With all 2000 asthma features it is 3.7× slower than ranger, at the same accuracy.
  • glmboost: the best C-index on Colocalisation, as in the survey.
  • LiblineaR: the best asthma model with all features. With 20 features it matches the penalised GLMs and is the fastest of them.
  • ncvreg: weak on METABRIC survival. With 39 events, its deviance-tuned MCP lambda gives very sparse models: 0–3 of the 20 Colocalisation features in 20 resampled training sets, sometimes none. It is offered as an option and isn't recommended as a default for small survival cohorts.
  • Existing results: the regression harness (18 scenarios) is identical to 16/ One registry of classifiers and feature selection methods (no change to results) #142's build.
  • Tests and check: R CMD check (tests, examples, vignettes) gives the same single NOTE as 15/ Defaults and dependencies: no rebalancing and serial by default; five packages to Imports, three dropped #141. Its test run passes 567 expectations, none failing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TfsjwkbDbHkX4kJZNe4ER4

ellispatrick and others added 2 commits October 14, 2026 10:49
…BLINEAR logistic regression and ncvreg (Suggests)

- "aorsf": oblique random forests of classes or survival (aorsf::orsf), 500 trees, one thread, features ranked by
  ANOVA importance; reproducible with set.seed.
- "glmboost": componentwise boosted Cox model (mboost::glmboost with CoxPH()), mstop 100, nu 0.1.
- "LiblineaR": logistic regression with an L2 (type 0, default) or L1 (type 6) penalty on standardised features,
  inverse class size weights, and the cost chosen by the balanced error of five-fold cross-validation of the
  training samples.
- "ncvreg": MCP (or SCAD, lasso) penalised logistic regression of two classes or Cox regression (cv.ncvreg,
  cv.ncvsurv), lambda chosen by five-fold cross-validation.
- Each is one interface file and one registry entry, with tuning presets for tuneParams = "auto". The registry test
  fits and predicts each for every outcome it accepts; further tests check reproducibility, the direction of risk
  scores and the feature extractors. The vignette tables list them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TfsjwkbDbHkX4kJZNe4ER4
@ellispatrick
ellispatrick force-pushed the review/17-new-classifiers branch from 606c8da to df94aa4 Compare October 14, 2026 00:20

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant