Skip to content

Validate forecasts only twice in transform_forecasts() (#1236) - #1238

Open
nikosbosse wants to merge 3 commits into
mainfrom
fix/1236-transform-forecasts-validation
Open

nikosbosse wants to merge 3 commits into
mainfrom
fix/1236-transform-forecasts-validation

Conversation

@nikosbosse

@nikosbosse nikosbosse commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

CLAUDE: Closes #1236.

What

transform_forecasts() updated the forecast object with :=, and [.forecast re-validates the whole object after every such update. One call therefore ran assert_forecast() six times with append = TRUE and three times with append = FALSE. Multivariate forecasts paid for one more validation on top, because the method rebuilt an object that had just been built.

The transformation now lives in an internal helper, transform_forecast_data(). It validates the input once (clean_forecast()) and does all := updates and the rbind() on a plain data.table. Each method then builds the forecast object once with the matching as_forecast_<type>(). That makes two validations per call for every forecast type and both values of append. With append = FALSE, multivariate forecasts keep their existing .mv_group_id, since the rows and groups don't change.

The smaller items in the issue (check_observed_constant(), check_number_per_forecast(), get_duplicate_forecasts()) are left for a separate PR.

Timing

On the reprex in #1236 (120k quantile forecasts, 2.76M rows):

main this PR
append = TRUE 19.9 s 9.5 s
append = FALSE 7.3 s 5.0 s

The remaining time is mostly the two validations; the output of append = TRUE has twice as many rows as the input. Skipping input validation would save about another 2 s, but then invalid input would fail inside fun() with an obscure error.

Behaviour change

With append = FALSE, a transformation that produced an invalid forecast used to return it, with "Error in validating forecast object" warnings. It now errors, as it already did with append = TRUE. For example, fun = as.character now errors, and so does a fun that returns all NA. This is in NEWS.

The most likely way to hit this is calling append = FALSE on an already-transformed forecast with a label that is already in scale. That includes the default "log" after a default transformation. That call duplicates forecasts, so it now gets its own clear error ("Can't transform with label 'log' ...") instead of a duplicate-forecast error. The label docs now say this.

Checks

  • For quantile, sample, point and multivariate sample forecasts, outputs are identical to main's for append = TRUE, append = FALSE, and append = FALSE on an already-transformed object with a new label.
  • New tests cover:
    • the validation count for quantile and multivariate forecasts (they fail on main, with 6 and 3 for quantile);
    • that the input is not modified, with and without an existing scale column;
    • the errors on invalid output and on a reused label;
    • multivariate grouping with append = FALSE.
  • An independent review found no correctness bugs. Its findings (reused label, multivariate grouping, a redundant copy, test breadth) are addressed in the second commit.
  • devtools::test() passes. lintr reports no lints on the changed files. Coverage of R/transform-forecasts.R stays at 100%.

🤖 Generated with Claude Code

transform_forecasts() updated the forecast object with :=, and
`[.forecast` re-validates after every such update, so the full
validation ran six times per call. The transformation now runs on a
plain data.table in an internal helper, and each method rebuilds the
forecast object once at the end. Closes #1236.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.18%. Comparing base (8b18f07) to head (24d5b02).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1238      +/-   ##
==========================================
- Coverage   99.18%   99.18%   -0.01%     
==========================================
  Files          41       41              
  Lines        2333     2331       -2     
==========================================
- Hits         2314     2312       -2     
  Misses         19       19              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

nikosbosse and others added 2 commits October 26, 2026 13:46
- Error clearly when append = FALSE reuses a label already in 'scale'
- Keep .mv_group_id for multivariate forecasts with append = FALSE
  instead of re-inferring joint_across, which fails when every forecast
  unit column is constant within groups
- Drop a redundant copy of the data
- Broaden tests to multivariate forecasts and existing 'scale' columns

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@nikosbosse nikosbosse left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seabbs looks good to me

@nikosbosse
nikosbosse requested a review from seabbs October 26, 2026 15:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

transform_forecasts() validates the forecast six times, via [.forecast on its internal := updates

1 participant