Skip to content

fit_full_models: read training data in a fixed order - #64

Merged
mmyrte merged 4 commits into
claude/gifted-lamport-41q8gv-05-dinamica-temp-dir-docsfrom
claude/gifted-lamport-41q8gv-06-deterministic-training-order
Oct 10, 2026
Merged

mmyrte merged 4 commits into
claude/gifted-lamport-41q8gv-05-dinamica-temp-dir-docsfrom
claude/gifted-lamport-41q8gv-06-deterministic-training-order

Conversation

@mmyrte

@mmyrte mmyrte commented Oct 5, 2026

Copy link
Copy Markdown
Member

Stack 6/6, on top of #63.

What

DuckDB returns the rows of trans_pred_data_v() in no fixed order. Learners such as ranger fit a different model on reordered rows even with a fixed seed. Full models, and the potentials predicted from them, therefore did not replay. On the PIE benchmark, two runs of the same seeded random forest gave potentials differing by up to 0.1.

fit_partial_models() already asked for ordered = TRUE when seeded; fit_full_models() now always does. The cost is one keyed sort of the training table.

Tests

test_integ_trans_models_t.R: a seeded classif.ranger fitted twice through fit_full_models() gives identical split values. On PIE, the random forest potentials of two runs are now bit-identical; they differed before.

🤖 Generated with Claude Code

https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf


Generated by Claude Code

claude added 2 commits October 3, 2026 00:15
DuckDB returns the rows of trans_pred_data_v() in no fixed order, and learners
such as ranger fit a different model on reordered rows even with a fixed seed,
so full models (and their potentials) did not replay. fit_partial_models()
already ordered its data when seeded; fit_full_models() now always does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf
@mmyrte
mmyrte added this pull request to stack #67 October 6, 2026 09:43
@mmyrte

mmyrte commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

@claude At this point, I think the added complexity of keeping the argument is unnecessary; the cost of reordering is too small. Let's just drop the ordered argument (also for the partial learner), and make it the default by moving the ordering from the data.table::setkeyv to the SQL query.

The fixed order now comes from an `order by` in trans_pred_data.sql instead of an optional
setkeyv(), so the partial and full model workers and get_pred_filter_score() no longer take
`ordered` / `ordered_pred_data`: the cost of sorting is small next to fitting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D1LLswXe3jLjGTFrBsxgPf

mmyrte commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Done in fac76ad. trans_pred_data.sql now ends with order by tr.id_coord, tr.id_period, and the setkeyv() in trans_pred_data_v() is gone, together with its ordered argument. The callers no longer pass an ordering argument:

  • the partial and full model workers;
  • get_pred_filter_score() / pred_filter_worker(), which lose ordered_pred_data.

A new check in test_db_evoland.R confirms that the rows come back sorted. The full suite passes with at_home = TRUE. I merged the change forward into #65 and #66 without conflicts; the suite passes on #66 as well (651 tests).

One caller outside this repo: evoland-experiments/2026-05-ssp-ch/050-covariate-selection.qmd still passes ordered_pred_data = TRUE. It won't break. It passes a Filter object rather than a string, so the argument lands in ... and is ignored. The line can be dropped whenever that script's evoland pin is bumped.


Generated by Claude Code

… into claude/gifted-lamport-41q8gv-06-deterministic-training-order
@mmyrte
mmyrte merged commit 0c32c57 into develop Oct 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants