[ellispatrick]

Based on #134; independent of the other post-#134 PRs. One commit per change, so each can be reverted on its own.

Summary

Seven speed changes decided by Ellis Patrick (ProjectClassifyR DECISIONS.md, 10 Oct 2026, Q1, Q2, Q5, Q15, Q25, Q31, Q32).

• Identical results: the merge ranking cache (Q5) and, on asthma, the likelihood-ratio ranking (Q25).
• Results change: Q1 (tie-breaking only), Q2, Q15, Q31 and Q32. Details and sizes are below.

Change	Commit	Results	Timing (4 cores)
Q1 random forest ranks by the impurity importance of its own forest; no second forest	4b4244a	rankings change; vote shares identical; only tied votes can flip (random tie-break)	asthma, all 2000 features: 135.8 → 85.0 s; Procedure 1: 34.0 → 27.7 s
Q2 random forests try √p features per split by default (ranger's and randomForestSRC's default); mTryProportion still sets a proportion	77b82fb	change	asthma, all features: 85.0 → 16.1 s; Procedure 1: 27.7 → 26.3 s
Q31 penalised GLM fits binomial for two classes	f7073c2	change	asthma, 3 GLMs + LASSO on all features: 37.7 → 23.9 s
Q25 likelihood-ratio ranking in closed form	2d58035	identical on asthma	ranking 3.0 → 0.13 s (23×); asthma CV 101.4 → 8.1 s
Q32 XGB fits 100 rounds by default	c1b0cce	change	asthma: 6.3 → 7.0 s
Q5 merge reuses each assay's selection across combinations	b7ff7cb	identical	Procedure 2 step 7: 330.6 → 305.6 s; asthma 3-assay merge: 17.5 → 15.4 s
Q15 nested CV: 1 × 5 inner scheme, serial, no unused final fit	fb4d47e	change (nested tuning only)	asthma, nested tuning at 20 × 5: 66.7 → 11.3 s

Why

• Ellis's answers to the open questions (DECISIONS.md, 10 Oct 2026). Q1, Q2, Q31 and Q32 change results, so they are changes to defaults, not speed-ups. Users can still override them.
• AUDIT.md (branch audit-2026-10) has the background: M5 (merge repeats selection), M6 (nested CV), 1B (likelihood ratio) and 1C (random forest).

What changes

• Q1: randomForestTrainInterface grows one forest with importance = "impurity", unless the user passes importance. forestFeatures ranks by that forest's importance, and still reads the old forImportance attribute of models saved by earlier versions. runTests no longer strips that attribute.

• Q2: mTryProportion = NULL by default in randomForestTrainInterface and rfsrcTrainInterface. NULL leaves mtry to the package's default: √p, floored by ranger and rounded up by randomForestSRC. The presets vignette says so.

• Q31: penalisedGLMtrainInterface uses family = "binomial" for two classes. Prediction builds both class columns from the probability of the second class. Feature ranking reads the coefficient matrix. Scores for one test sample stay a 1-row matrix.

• Q25: likelihoodRatioRanking computes each feature's normal log-likelihood in closed form, with colSums over a matrix, instead of mapply/dnorm per feature. It also works for one feature and for a class absent from the training set.

• Q32: extremeGradientBoostingTrainInterface(nrounds = 100). The presets vignette says so. The "auto" tuning preset (nrounds = c(5, 10)) is unchanged.

• Q5: .doSelection reuses an earlier selection while .runTestsSplits runs, through a per-process cache:

  • a selection is reused only for identical data, outcome and selection settings;
  • when the number of features is tuned, the classifier settings, tuning mode and metric must also match;
  • a selection that drew random numbers is reused only from the identical random number state, and the state after it is restored;
  • only the current training set's selections are kept (tasks are ordered by split);
  • merge's own combination-level call is not cached.

  Forked workers each have their own cache; socket workers (Windows) don't use one.

• Q15: CrossValParams gains innerPermutations = 1 and innerFolds = 5. Nested CV, for selection or training parameters, runs that scheme on each training set with SerialParam. It skips the final model of all inner training samples (.prepareTests(finalModel = FALSE)). The ensemble-selection branch of .selectFeatures (Q26, broken) is untouched.

Evidence

Scripts: ProjectClassifyR scripts/speed_defaults/ (bench.R, lr_ranking.R, results in results/). Each build is installed from its commit, run with 4 cores, set.seed(1). Asthma is the package data set (190 × 2000), 20 × 5 CV unless stated; Procedure 1 is the paper's (GHIST, random forest, 100 × 5).

Q1 (base 9bff739 → 4b4244a)

• Vote shares (class scores) are identical in every prediction.
• Class predictions changed only where two classes had the same vote share, because ranger breaks ties with a random number and one fewer forest is grown first:
  • Procedure 1: 16 of 16,200;
  • asthma: 2 of 3,800.

	Balanced accuracy	AUC
Procedure 1 clinical	0.6786 → 0.6796	0.6470 → 0.6470
Procedure 1 protein	0.7672 → 0.7670	0.8250 → 0.8250
Procedure 1 RNA	0.7999 → 0.7995	0.8693 → 0.8693
asthma, all features	0.8671 → 0.8664	0.9475 → 0.9475

Q2 (4b4244a → 77b82fb)

	Balanced accuracy	AUC
Procedure 1 clinical	0.6796 → 0.6796	0.6470 → 0.6470
Procedure 1 protein	0.7670 → 0.7896	0.8250 → 0.8293
Procedure 1 RNA	0.7995 → 0.7898	0.8693 → 0.8702
asthma, all 2000 features	0.8664 → 0.8333	0.9475 → 0.9345

With feature selection (Procedure 1 keeps 20 features) the change is small and mixed. With all 2000 asthma features, √p is 5.3× faster but loses 0.033 balanced accuracy and 0.013 AUC.

Q31 (77b82fb → f7073c2): asthma.

• The ridge model is most affected: 84 of 3,800 class predictions changed. Multinomial and binomial models scale the penalty differently, so the same lambda path is not the same model.

Model	Balanced accuracy	AUC
ridge (t-test 20)	0.8241 → 0.8186	0.9010 → 0.8975
elastic net (t-test 20)	0.8116 → 0.8117	0.8870 → 0.8885
LASSO (t-test 20)	0.8129 → 0.8126	0.8855 → 0.8855
LASSO (all features)	0.9008 → 0.9012	0.9715 → 0.9715

Q25 (f7073c2 → 2d58035)

• Orders on all 2000 asthma features are identical for each of the three location/scale alternatives.
• Ranking time: 3.0–3.2 s → 0.12–0.13 s.
• The asthma CV (DLDA) results are identical: balanced accuracy 0.7988, AUC 0.8865.
• Features whose statistics differ only by rounding can swap, which none did here.

Q32 (2d58035 → c1b0cce): asthma balanced accuracy 0.7759 → 0.7811, AUC 0.8740 → 0.8760, 6.3 → 7.0 s.

Q5 (c1b0cce → b7ff7cb)

• Predictions, rankings and selections are identical:
  • Procedure 2 step 7 (31 combinations, CoxPH ranking, CoxNet; mean C-index 0.6017 before and after);
  • an asthma merge of three 500-feature assays (limma, DLDA, 7 combinations);
  • all 18 regression-harness scenarios.
• The saving is modest here because ranking is a small share of these runs: CoxNet fitting dominates step 7. In the test, a three-assay merge makes 3 selections per split instead of 7.

Q15 (b7ff7cb → fb4d47e): asthma, DLDA, nFeatures = c(5, 10, 25), nested tuning.

Outer scheme	Time	Balanced accuracy	AUC
20 × 5	66.7 → 11.3 s	0.7986 → 0.7976	0.8825 → 0.8820
5 × 5	8.8 → 5.9 s	0.8042 → 0.8017	0.8840 → 0.8900

Tests: 575 expectations pass, with 0 failures. New tests cover:

• impurity ranking from the prediction forest;
• √p and proportion mtry;
• binomial and multinomial GLM, including one test sample;
• the likelihood ratio against summed dnorm;
• 100 XGB rounds;
• cache reuse and identical results with and without random numbers;
• the nested-CV fit count.

Regression harness (18 scenarios, 1 core, base → this branch without Q15):

• changes in exactly the scenarios using random forest or GLM;
• identical elsewhere.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TfsjwkbDbHkX4kJZNe4ER4