feat(probe): validate HDR10+ and add probe limits and cancellation - #583
Conversation
Add opt-in whole-probe input, packet and deadline limits with caller cancellation for URL and custom readers. Validate codec metadata framing and ST 2094-40 payloads before confirming HDR10+, preserving Dolby Vision precedence and positive-only detail semantics. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exercise oversized read results through real media and assert the exact clamped read and caller ownership. Snapshot completed native seek results outside testing macros. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Wait for size and finite-range request callbacks before releasing probe origin slots. Cover cancellation and deadlines with isolated callback-queue gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve upstream's request-target credential filtering alongside the controlled probe request session and cancellation drain. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
f55789d
into
superuser404notfound:main
|
Merged as f55789d. Thank you for this one, it is a substantial piece of work and the two ideas in it are both right: the validator belongs in the engine and shared by playback and probing, and a probe that reads a hostile source needs bounds the caller sets rather than bounds the engine assumes. Two things verified here before merging, for the record. The bit accounting in Four follow-ups are in #586, all of the same shape: a bound meant to limit what the probe SPENDS was also deciding what it may REPORT.
Tests that encoded the old contracts were rewritten rather than dropped: every case where nothing was validated still fails closed, and the cases where a confirmed message precedes the damage moved to a test that pins the new answer. |
|
Released in 7.10.0, together with the #586 follow-ups: https://github.com/superuser404notfound/AetherEngine/releases/tag/7.10.0 Both downstream consumers are already pinned to it. Thanks again for the contribution. |
…g it Follow-up to PR superuser404notfound#583, whose validator and whole-probe controls are right in the core. Four places where a bound meant to limit COST was also deciding what the probe may report. The HDR10+ validator discarded a message it had already parsed in full whenever anything later in the same packet failed the structural walk. Verified against the scan directly: a valid ST 2094-40 SEI followed by a NAL with its forbidden bit set, or by an SEI missing rbsp_trailing_bits, came back negative, where the byte scan this replaced said yes. That is ordinary shape on real media, and it reaches the playback path through HLSSegmentProducer, not just the probe. The walks now stop at the damage and report what they validated. Nothing can invent a positive there: only validT35 ever sets one. The HDR10+ and Atmos detail passes retracted a detection they had already made when their soft wall-clock budget expired mid-pass. The caller cannot tell a withheld confirmation apart from a source that carries none, so an overrun on a slow origin silently became "no Atmos", on the existing API with no opt-in. An overrun now retires the pass without taking its answer back. The whole-probe deadline from superuser404notfound#583 is untouched: that one throws, so the caller knows. Passing `limits` switched the open to the still extractor's profile, so probe(url:limits:) read 2 MiB / 2 s where probe(url:) reads 50 MiB / 60 s and could report fewer streams for it. The caller's own limits are the only thing that should bind: the profile is built from .playback with probesize clamped to maxInputBytes, and the deadline plus the FFmpeg interrupt callback bound the analysis. The recordless Dolby Vision audit stays out of reach, and docs/api.md now says so in terms of what a host observes: that audit opens the source a second time by URL, outside this probe's budget and cancellation. A controlled HTTP probe took the origin slot with tryAcquire and failed with sourceBusy the instant any other request held it, which is the ordinary shape when a host probes several items off one server. It waits for the slot until its own deadline instead, since a slot wait is the one wait the watchdog cannot interrupt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015PM3xUJB6ZQyqnmGK1fp6F
Summary
The HDR10+ scan currently accepts a marker anywhere in packet bytes, so compressed picture data can be mistaken for metadata. The existing HDR10+/Atmos scan budgets do not cover opening, stream analysis or seeking, and cancelling an awaiting Swift task does not stop synchronous probe I/O.
This PR validates actual HDR10+ metadata and adds optional probe-wide limits and cancellation, shared by both detail passes.
What changed
ProbeLimitsandProbeCancellationto URL and custom-reader probes. Input and time budgets cover opening, stream analysis, seeks and both passes; inspection packets share packet-count and packet-size limits.Limits are cooperative, not hard network or memory caps. Input accounting measures bytes delivered by the reader, not wire traffic; packet-size checks happen after FFmpeg allocation. Custom readers must implement cancellation to unblock I/O, and the probe waits for native work and cleanup before returning. An unconfirmed detail is not proof of absence. Atmos detection remains E-AC-3 JOC, not TrueHD Atmos.
Test plan
swift buildand the full test suite passed after incorporating current upstream main. Four HTTP callback-drain regression cases pass with the fix and fail without it. The Xcode 26.3 fixture simplification passed all 41 targeted HDR tests and byte-equivalence checks.a1d9529b: Xcode 26.3 macOS tests and iOS/tvOS/visionOS Simulator builds, Xcode 27 macOS/tvOS builds, and documentation links. Full CI tests: 3,296 Swift Testing tests and 642 XCTest cases passed; the existing live AES-128 test was skipped becauseAETHER_LIVE_URLwas unset.Checklist
CHANGELOG.mdupdated