Repository navigation
Harden the lost P-frame and Copy diagnostics tests - #138
Merged
Merged
Conversation
Frames sent right behind the lost one let the reassembler and the decoder see the loss on two queues within a millisecond, and the 500 ms request spacing keeps whichever asks first, so the first reason was decodeError on a loaded runner. The IDR followed a fixed run of 17 frames, which a starved sender stretched past the 2 s Resume. The test now waits for the reassembler's request before sending more frames, and sends the IDR right after the decoder's third request instead of after the frames.
The upload status line above the buttons grows from one line to three when it loads. On a loaded runner it loaded between XCTest finding Copy and its tap, which then landed on the row's old position and missed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two iPad tests failed on CI in runs that didn't change the code they cover. Both failures came from test timing. Neither is an app defect.
StreamRecoveryTests.testALostPFrameHoldsThePictureAndAsksUntilTheNextIDR
Failure: run 40969597183 (candidate head fd9b791, scripts and workflows only), caps 0 iteration:
decodeError, notgapLoss. The 1.9 s window held 4 × decodeError and no gapLoss.resumearrived after the IDR.Root cause, from the CI log in the evidence artifact:
gapLoss. The decoder rejects frame 4 on itsframe_numor seq gap and asks withdecodeError.ControlChannelkeeps only the first request in any 500 ms. On a loaded runner the decoder's queue won, so gapLoss was never sent. The PC handles both reasons the same way:handle_keyframe_requestonly logs the reason, then forces an IDR. Either way one request goes out at once, so this is test over-specification, not a defect.Fix (
ios/EternalMonitorTests/StreamRecoveryTests.swift:154-176,FakeUDPHost.swift:129-143): each step now waits for the iPad's own requests instead of a fixed time.gapLoss. The test then waits 600 ms, past the 500 ms spacing.decodeError.pictureRestoringis waited for instead of read once. It is refreshed at 4 Hz.Every other assertion is unchanged: nothing after the loss is shown, the hold exists, retry spacing, no requests after the IDR, the hold ends, and no -12909.
ConnectScreenTests.testDiagnosticsCanBeCopiedAndSharedWithoutAConnection
Failure: run 40969661205 (#127's merge head 5b11562; #127 doesn't touch this test or DiagnosticsView). Line 244: "Diagnostics copied" never appeared.
Root cause, from the evidence artifact's synthesized events and UI tree:
DiagnosticsUpload.status()returns from its queue. On the loaded runner that happened between XCTest finding Copy and its tap, so the tap missed.Fix (
ios/EternalMonitorUITests/ConnectScreenTests.swift:243-251): wait until "Checking automatic upload" is gone, then letswipeUp(toReveal:)confirm the Copy row is hittable and has stopped moving, then tap once. The copy itself is checked exactly as before.Results (Xcode 16.4, iOS 18.6, iPad Pro 11-inch (M4) simulator, ten
yesloops running)Reproducing the CI failures with added delays (uncommitted)
The delays:
usleepbetween the reassembler delivering a frame and dropping the older partial one;The fixed tests still fail when the feature is broken (uncommitted)
FrameAssembler.requestKeyframe)[]vs[gapLoss]), both capsVideoDecoderkeepshold)[decodeError, resume]after the IDR) and 182 (hold still set), both capsadmittedreturns true)copied = trueremoved)Failure history
All attempts, reruns included, of the last 400 CI runs since each test existed (10-18 and 10-29). Failed iOS jobs that list these tests in "Failing tests":
The earlier Diagnostics failures were share-sheet and app-termination waits, which later commits already lengthened or fixed. The lost Copy tap appears only in run 40969661205. The lost P-frame test failed once, in 40969597183.
Known app issue, not changed here
The Diagnostics screen's Export and Copy rows jump about 30 pt down when the upload status line loads: one line ("Checking automatic upload") becomes three. A tap made at that moment can miss Copy or land on Export. It only shows when
DiagnosticsUpload.status()is slow to return, as on the loaded runner. This PR changes only the test.