fix(forecast): train load models on closed history intervals

[frahlg]

Problem

Core worked out household load three ways:

Use	Evidence before this PR
Daily Home use and History	Each source integrated separately into 1-minute energy, balanced per interval (history_source_energy)
Energyplan forecast labels	Its own ForecastAccumulator: 15-minute trapezoid means of aligned instant readings, any invalid instant drops the quarter
Built-in load model (loadmodel, mpc.Load)	One instant HouseholdW reading per minute

So the models learned a different quantity from the Home use people see, and the built-in model learned from single instant readings. Fredrik asked for one common basis.

Two layers

Fredrik set the direction on 19 October: Home use has two layers.

• Display (live and History, daily Home use) is lenient: a simple residual of grid, PV, battery and EV, with gaps bridged as well as possible and marked as estimates.
• Training (the load models behind forecasts) is strict: only intervals measured in full.

The contract between them is the history_quality mask. Anything the display layer bridges or estimates carries EstimatedLoad; training leaves out every interval with MissingPower, BalanceMismatch or EstimatedLoad. PV at 0 W while the sun is well below the horizon is physics, not an estimate, so a PV inverter asleep all night must not remove the night from training (being settled in #1755).

Change

• state.HouseholdIntervals returns closed, fully measured intervals from the same source energy as daily totals. An interval is closed once committed source progress passes its end, so a returned value never changes. Intervals with a missing source, a recording gap, an estimate or a negative balance are left out. It also returns where the closed range ends, so callers move past intervals left out.
• loadmodel.Service trains once per closed quarter-hour on its mean instead of one instant reading per minute. The model already averages within a day per weekly hour, so the hourly means keep their meaning. Cold-day counters keep the minute units used by stored models: each new quarter adds 15 minute units to the count and both sums. This preserves partial-day evidence and the six-hour bound without resetting learning. It persists after each trained quarter.
• The Energyplan tracker takes load labels from the same quarters (Quality: source_interval_balance_v1) and drops its household accumulator. PV keeps its accumulator, since PV labels need the curtailment flag. Observations are immutable per quarter, so PV waits until history decides the quarter (at most 2 minutes), and each quarter is archived once.
• Both start at the next whole quarter after start or reconfiguration, so a quarter observed under an earlier revision never becomes a label.

Model state, feature hash and learning revisions are unchanged, so no site loses learning.

This follows docs/roadmap.md: household Load history balances source energy over the same interval, and every use is checked against the same stored evidence.

Relation to other work

• fix(history): keep Home use through device dropouts and restarts #1753 (issue Repair historical Home use gaps from retained source measurements #1750) and fix(history): count unknown PV as 0 W after sunset #1755 build the display layer: short gaps bridged and PV after dark. They do not touch these files. Intervals they mark as estimated show in Home use and stay out of training.
• fix(history): repair Home use and Solar gaps with audited estimates #1751 (Today subtotal) is display only and unaffected.

Evidence

• go test ./internal/state ./internal/loadmodel ./cmd/ftw ./internal/api ./internal/telemetry: all pass.
• New tests:
  • TestHouseholdIntervalsMatchDailyTotal: the sum of training intervals equals DailyEnergy.LoadWh.
  • TestHouseholdIntervalsLeaveOutIncompleteEvidence: missing EV, a 3-minute recording gap and a negative balance give no label; neighbours keep theirs.
  • TestHouseholdIntervalsWaitForCommittedProgress: an open quarter is never returned.
  • TestTrainsOnceOnEachClosedValidHouseholdInterval, TestLearningCutoffSkipsIntervalsStartedBeforeIt, TestIntervalReadInFlightIsDiscarded for the built-in model.
  • TestForecastPVWaitsForTheHouseholdDecision, reworked TestForecastConfigChangeBreaksInterval and TestForecastIndependentPVIntervalsAndCurtailment for the tracker. Readings carry a wrong instant house value; labels must come from history.
• Removed tests that asserted per-minute telemetry sampling in loadmodel; history now owns those validity rules and the state tests cover them.
• TestRestoredMinuteHeatingBoundsKeepLearning: restores both learned profiles and the active profile unchanged from minute-based state, persists and reloads them after a new quarter, rejects 30 old minutes as a full cold day, and accepts 345 old minutes plus one new quarter as exactly six hours. It fails before the fix and passes after it.
• go test -race ./internal/loadmodel: passed.
• make verify: passed (vet, tests, build).
• TestHouseholdIntervalsLeaveOutEstimatedMinutes: an estimated minute keeps its quarter out of training while daily Home use still shows it.
• Codex found that cold-day heating counters on disk are in minutes; 44297c4 keeps that unit (a quarter counts 15 minutes), so an update in the middle of a cold day cannot close it after half an hour.
• Home box (Fredrik's Pi 4): test builds from 18:08 and 19:27 on 19 October trained the built-in model and Energyplan once per closed quarter (e.g. 19:30–19:45 at 19:45:21 and 19:46:01), with no label or archive errors.
• Simulation of 70 days with daily faults (restarts, unknown PV/EV/battery/meter, meter lag, bad values), same seed: labels 87.4 % here vs 86.7 % for the old collector, mean label error 3.3 W vs 4.2 W, load forecast error unchanged (185 vs 184 W).

Watch after the beta: loadmodel sample counts grow per quarter, and the Energyplan learning status stays healthy.

🤖 Generated with Claude Code