Repository navigation
fix(performance): replace untrusted airspeeds instead of dropping rows (audit I1, I9, I14) - #83
Merged
Merged
Conversation
…s (audit I1, I9, I14) The low-speed filter removed rows whose TAS was below the BADA stall speed, and had three defects: - I1: BADA4Adapter used the BADA3 VStall call; pyBADA 0.1.12 BADA4 needs (mass, HLid, LG), so it raised TypeError, which run_by_bada_version swallowed: BADA4 flights were never checked. - I14: it compared a TAS with a CAS stall speed (TAS/CAS is 1.62 at 30,000 ft at stall speeds). - I9: segment_duration, rocd and acceleration were not recomputed on the rows kept, so deleted rows took their flight time with them and CO2 != 3.159 x fuel (BADA3: 34.7 h deleted from fuel, 43.0 h from CO2 on the NM sample; 998 of 1,050 BADA3 flights off by more than 0.1 %). Replaced by an airspeed trust step (steps/performance/airspeed_trust.py), maintainer decisions of 2026-10-23: - BADA4Adapter.v_stall_cas: (HLid, LG) = getAeroConfig(config), then VStall(mass, HLid, LG, nz=1, theta=1, delta=1) (CAS at ISA sea level); None for a configuration missing from the BADA4 file. The missing-stall-speed warning names the BADA model. - A row is untrusted when its TAS is not finite, its CAS is below the stall CAS at MTOW (new parameter airspeed_stall_config, "TO" or "LD", default "TO") or above the aircraft's maximum operating speed VMO, or Mach >= 1. CAS and Mach from pyBADA at the row's pressure altitude and delta_tau. - VMO (maintainer decision of 2026-10-23, option 1 then b): new vmo_cas() on BaseBADAAdapter, BADA4Adapter and BADA3Adapter, from the BADA model already loaded (pyBADA 0.1.12 VMO, kt CAS: BADA4 XML KLM/vmo, BADA3 OPF flight envelope), returned in m/s, None when missing or not a positive number. airspeed_trust.max_trusted_cas uses it with no margin, and falls back to DEFAULT_MAX_TRUSTED_CAS (400 kt) with a warning naming the BADA model. All 132 BADA codes of the NM sample (43 BADA4, 89 BADA3) have a VMO, 250 to 370 kt: no fallback. Reason: with 400 kt, "catch-up" minutes after a frozen position passed the limit, became the first or last trusted row, and the end rule copied their CAS onto the take-off or landing rows: 86 flights took off with > 250 kt CAS below FL10 (26 > 300 kt; ETD75F 374, QTR75N 372, EZY68GP 369, RYR6RJ 369 kt at FL0-30), 10 flights held > 300 kt CAS in an end run, TOM902P held 372 kt CAS down to FL13 (+17.9 % fuel against dev). Typical VMO is 330-350 kt. - The Mach limit stays Mach >= 1 (DEFAULT_MAX_TRUSTED_MACH), not the BADA MMO: TAS is the ground speed minus the modelled wind (10 kt of wind error at FL370 is about Mach 0.017) and some BADA models stand for faster types (business jets on a GLEX model). On the NM sample, MMO with no margin untrusts 15,628 level cruise minutes at FL245 and above on about 2,000 flights (median 0.018 above MMO) and fails one flight on the 80 % rule; MMO + 0.08 leaves 1,205, with a tuned margin. - No row is removed and no column is added. An untrusted TAS between two trusted rows is replaced by linear interpolation in time. Before the first and after the last trusted row, it takes the CAS of that trusted row, converted back to TAS with the row's own altitude and delta_tau (hold_end_cas), so a descent after a cruise anchor keeps a constant CAS and its TAS falls with the altitude. acceleration is recomputed; segment_duration and rocd stay. - A computed TAS is judged and filled on the unsmoothed 1-min TAS (segment_true_airspeed(smooth=False), same winds), then smoothed with the same Savitzky-Golay filter (window 7, polyorder 1, now one constant DEFAULT_TRUE_AIR_SPEED_SMOOTHING_POLYORDER for both). Only the rows whose smoothing window holds an untrusted row take the new smoothed value; all other rows keep the TAS from the smoothed ground speed bit for bit. The end runs get the held CAS again after the smoothing (the edge fit would extrapolate a climb or descent over them); the last row (NaN, no forward segment) takes the TAS of the row before it. A provided TAS is not smoothed. - The output schema does not change: true_airspeed is the TAS used. apply_airspeed_trust returns the untrusted mask to its caller only. - The step runs before every path, with the stall speed and VMO of each attempt's adapter. The flight fails with a non-retryable PerformanceStepError (no BADA3 retry, no altitude fallback) when the step raises, and when more than 80 % of its time is untrusted or no row is trusted, except when the AO provides fuel flow, engine efficiency and aircraft mass (the early exit, _has_ao_fuel_and_efficiency, where the BADA fuel flow is not used): there the TAS is still replaced (kept as is when no row is trusted) and the flight is kept. - filter_low_speed_points and speed_filter.py are removed. Effect, NM sample 2025-07-09 (4,959 flights; this commit and its parent against dev 5aba2f9, TO threshold, VMO limit): - the same 4,303 flights computed, the 656 failures unchanged; fleet fuel -0.77 % (49,591.5 t -> 49,212.1 t; BADA4 -1.10 %, BADA3 +1.26 %, mostly deleted time coming back), CO2 -0.74 %; CO2 = 3.159 x fuel on all 4,303 flights (dev: 3,305 within 0.1 %); 10 flights move from BADA3 to BADA4 (6 with Mach >= 1 rows, 4 NaN-hole flights). - 20,300 untrusted rows on 3,935 flights (last row excluded); maximum untrusted time 62 % (NACHO20), so the 80 % rule fails no flight. - flights whose only untrusted row is the last one keep TAS, fuel burn and mass bit for bit on every other row. - holds now burn 0.83-1.00x an ICAO holding-speed BADA reference (BAW36, BAW196E, UAL146; dev 3.6-4.1x); THY6RA's end run (FL370 to FL28) keeps 248 kt CAS; take-offs above 300 kt CAS below FL10: 21 flights (dev 43). - after the step no row has Mach >= 1; 26 rows on 10 flights are at most 14 kt above VMO (interpolation in TAS while the altitude changes, last row copies the row before). - VMO (not 400 kt) adds 3,773 untrusted rows on 1,489 flights (median 11 kt above VMO, mostly one-minute spikes), including 784 level rows at FL245 and above on 150 flights (median Mach 0.907, mostly B777 and A350 models), probably real cruise with a wind error. MMO is not used: it would flag about 15,600 real cruise minutes on about 2,000 flights. - output keys of every runner identical to dev on the golden cases. Known limits, not changed: a catch-up minute between 250 kt and VMO stays trusted and an end run copies its CAS; the anchor row of an end run can be re-smoothed while the run holds its unsmoothed CAS, so the CAS can step at the boundary (median 22 kt at starts, 7 kt at ends, up to about 100 kt) with a one-row acceleration spike. Golden outputs regenerated with this code (scripts/create_golden_outputs.py). The inputs derived from the baseline performance (am, ff, ee, tas, takeoff_mass) change with it; outputs change in values only, with CO2 = 3.159 x fuel on all 99 records. Tests: tests/test_airspeed_trust.py (trust mask incl. CAS vs TAS at altitude and FAST, CAS limit = VMO without margin, 400 kt fallback with warning, Mach limit 1 not MMO, take-off catch-up minute not an anchor, fill, CAS hold in a descent and a climb and with delta_tau, smoothing identical to pycontrails, smoothing window mask, clean flight unchanged, rows outside the window unchanged, interior gap, end runs not extrapolated and at constant CAS after the smoothing, > 80 % and no trusted row fail or, when not asked to fail, replace or keep the TAS, LD threshold, no column added); tests/test_bada_model_airspeed_trust.py (integration, early exit not failed by the 80 % rule, partial AO data still failed, no retry on failure); BADA4/BADA3 VStall in tests/test_bada_adapters_vstall.py; vmo_cas in tests/test_bada_adapters_vmo.py (kt to m/s, missing or invalid gives None; real BADA4 A320-214, B738W26, B744GE, B788GE67 and BADA3 A320, AT76, equal to pyBADA maxCAS / VMax); the TC_PERF "provided columns are preserved" test keeps the trusted provided TAS; the fleet golden test checks that the output keys equal the golden ones; tests that patched filter_low_speed_points now run the real step; tests/test_guardrails.py gives its fake adapter a stall speed and a VMO. Docs: input-prioritization.md, mrv-specification.md, output-format.md (true_airspeed can be replaced), audit I1, I9, I14 and status table. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #82 (I13, merged). All numbers are against
dev5aba2f9 (before #82), NM sample 2025-07-09.Problem. The low-speed filter (
speed_filter.py) deleted the rows whose TAS was below the BADA stall speed:HLid, LG), soall 3,253 BADA4 flights ran unfiltered, at 20-100 kt CAS where the data is bad. BAW36 (B788)
burns 1,825 kg in 13 min in the BIG hold, 3.65 times a reference at ICAO holding speed.
572 slow rows on 338 BADA3 flights were kept (PGT023 at FL362-390: CAS 50-66 m/s, stall 66.3).
fuel and 43.0 h from CO2. CO2 differs from 3.159 x fuel by more than 0.1 % on 998 of the 1,050
BADA3 flights (TALE77: 1.386). BAW4LV gets 0 kg of fuel for a 19-min hold.
Causes. I classified each of the 17,567 suspect rows by cause.
plausible stall speed, are already slow on the raw track: sparse holds (BAW36: a position every
90-108 s, ~1 km apart), frozen positions (IBS15XJ: 8.7 min at one position, FL219 to FL156),
duplicate timestamps (116 in 31 computed flights; BAW12's climb under one timestamp, 10,483 kt GS).
(708 rows), and its edge fit and the 1-min chords cut turns (1,478 rows).
Change . New step
steps/performance/airspeed_trust.py;the filter is removed.
true_airspeedis the TAS used. Anunexpected error in the step fails the flight.
parameter
airspeed_stall_config,TO/LD, defaultTO; BADA4 call fixed); its CAS isabove the VMO of the loaded BADA model (
vmo_cas(); 400 kt only without one, none in thesample: 132 codes, 250-370 kt); or its Mach is 1 or more. MMO is deliberately not used: TAS is
ground speed minus a modelled wind, and MMO flags ~15,600 real cruise minutes on ~2,000 flights.
a flight, the CAS of the nearest trusted row is held.
accelerationis recomputed.again; every other row keeps its TAS bit for bit.
except when the operator gives fuel flow, efficiency and mass. There is no limit on
consecutive untrusted rows: long runs are often real holds. An operator TAS below stall is
replaced for BADA.
Effect (fleet run of this code, with the I13 fix, against
dev5aba2f9; same 4,959 flights):1 or more), and RYR4WR, RYR6EF, EZY87KM and VLG90AB (the NaN-mass flights of I13).
untrusted); the highest share is 62 % (NACHO20). Flights with only the last row untrusted keep
TAS, fuel and mass bit for bit.
(43 before).
Whole flights: VLG1254 2,727 -> 3,365 kg; IBS15XJ 5,229 -> 5,725 kg; THY6RA 5,221 -> 4,151 kg;
TALE77 758 -> 1,064 kg (37 frozen minutes no longer deleted); TOM902P -4.7 %.
Known limits. A catch-up minute between 250 kt and VMO stays trusted and an end run copies its
CAS (the 21 take-offs). The CAS can step where a start run meets the trusted rows (median 22 kt;
7 kt at end runs) with a one-row acceleration spike. 26 rows on 10 flights end at most 14 kt
above VMO. VMO untrusts 784 level cruise rows on 150 flights (median Mach 0.907).
Golden outputs regenerated (11 outputs, 8 inputs; the inputs derive
am,ff,ee,tas,takeoff_massfrom a baseline performance run, as in #80). Same keys and list lengths in all 19files, only values change; CO2 = 3.159 x fuel on all 99 records.
Schema unchanged. Fleet and Flight runner outputs compared with
devon 33 goldencase/runner pairs: no key added or removed. In the PerformanceOnly outputs the last point of
true_airspeed,fuel_flowandengine_efficiencyis a number instead of null, as documented.Tests. New:
test_airspeed_trust.py(87),test_bada_model_airspeed_trust.py(9),test_bada_adapters_vmo.py(22),test_bada_adapters_vstall.py(13); the fleet golden test alsochecks the output keys. Full suite: 454 passed, 4 skipped, 23 xfailed. ruff and mypy are clean.