Repository navigation
fit_full_models: read training data in a fixed order - #64
Conversation
DuckDB returns the rows of trans_pred_data_v() in no fixed order, and learners such as ranger fit a different model on reordered rows even with a fixed seed, so full models (and their potentials) did not replay. fit_partial_models() already ordered its data when seeded; fit_full_models() now always does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
|
@claude At this point, I think the added complexity of keeping the argument is unnecessary; the cost of reordering is too small. Let's just drop the |
The fixed order now comes from an `order by` in trans_pred_data.sql instead of an optional setkeyv(), so the partial and full model workers and get_pred_filter_score() no longer take `ordered` / `ordered_pred_data`: the cost of sorting is small next to fitting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
|
Done in fac76ad.
A new check in One caller outside this repo: Generated by Claude Code |
… into claude/gifted-lamport-41q8gv-06-deterministic-training-order
Stack 6/6, on top of #63.
What
DuckDB returns the rows of
trans_pred_data_v()in no fixed order. Learners such as ranger fit a different model on reordered rows even with a fixed seed. Full models, and the potentials predicted from them, therefore did not replay. On the PIE benchmark, two runs of the same seeded random forest gave potentials differing by up to 0.1.fit_partial_models()already asked forordered = TRUEwhen seeded;fit_full_models()now always does. The cost is one keyed sort of the training table.Tests
test_integ_trans_models_t.R: a seededclassif.rangerfitted twice throughfit_full_models()gives identical split values. On PIE, the random forest potentials of two runs are now bit-identical; they differed before.🤖 Generated with Claude Code
https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
Generated by Claude Code