Repository navigation
ci: fix the symbolication, iOS first-launch, ingest 5xx and burst.pid flakes - #2911
Draft
bitsandfoxes wants to merge 7 commits into
Draft
bitsandfoxes wants to merge 7 commits into
bitsandfoxes wants to merge 7 commits into
Conversation
This was referenced Oct 26, 2026
bitsandfoxes
force-pushed
the
fix/ci-stable
branch
from
October 26, 2026 18:27
2aab1c2 to
068663a
Compare
bitsandfoxes
force-pushed
the
fix/ci-stable
branch
from
October 26, 2026 18:51
068663a to
08490ff
Compare
bitsandfoxes
force-pushed
the
fix/ci-stable
branch
2 times, most recently
from
October 26, 2026 19:57
17dd6c7 to
bc44cac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the flakes that still failed CI runs on top of the stack, each traced to its cause.
sentry-cli debug-files uploadreturns before sentry.io has processed the files. In run 39645739252 the macOS GameAssembly debug companion was uploaded at 17:54:36 and listed by the server at 17:57:27; the crash event came in at 17:56:02, was symbolicated without it and never reprocessed.assert-symbol-upload.ps1now pollsfiles/dsyms/?debug_id=until each uploaded debug companion, IL2CPP mapping and source bundle is listed (portable PDBs count via their embeddedsourcesfeature), for at most 300 s. A timeout, a missing token or a 401/403 is a::warning::, not a failure. The assert steps getSENTRY_AUTH_TOKENfor that (Compile iOS gets it as a newworkflow_callsecret).simctl bootstatusdoesn't help (Skip the iOS simulator terminate right after boot and bound the others app-runner#65). Run iOS now starts with Build iOS, boots the simulator while Compile iOS runs, and waits for the app artifact withwait-for-artifact. When the artifact is already there (a rerun), the first launch waits until 150 s after boot. Job timeout 25 → 45 min to cover the wait.Received response status code: 503in run 40264124350), so the crash event never arrives.crash-capturenow reruns once per job when the send output shows a 5xx.Library/burst.pidat startup. The project comes from Create in another container, where that PID was 259; in the build container PID 259 is thexvfb-runwrapping the editor (run 40275305182). The orphaned editor then failed the retry with "another Unity instance is running".build-project.ps1removesLibrary/*.pidbefore building.OptionsConfigurationItem.SetScriptre-created the configuration asset on everyConfigureOptions. On 6000.6AssetDatabase.CreateAssetdestroys the old object, and??=kept the destroyed reference; on 2021.3 the reference stays live but no longer points to the asset. Either way, a secondConfigureOptionssavedOptionsConfiguration/CliOptionsConfigurationas{fileID: 0}, so the build had no auth token (seen in run 40314070230 attempt 10, when theshutdown-crashretry re-ran that launch).SetScriptnow reuses the asset already at the path; EditMode tests cover both references, and fail before on both Unity versions. Users runningConfigureOptionstwice hit the same bug, so it has a CHANGELOG entry.File upload/processing complete, which is all a rebuild prints when every chunk is already on sentry.io, and fails onsome symbols did not process correctly. Theshutdown-crashretry now also fires on exit 133 (SIGTRAP) afterBatchmode quit successfully invoked, for a shutdown stall that kills Unity before it logs the signal.Stacked on #2910.
CI evidence: run 40314070230, rerun in full back to back on 383a6c3 (before the last three commits): 8 of 10 attempts green on the first try, p50 32.0 / p90 35.2 min for the green ones (baseline 41 / 80). All 20 symbol waits in attempt 1 finished in 0–14 s, none timed out. The two red attempts:
Request cancelled while reading the binlog) reported as compiler errors without compiling. First sighting in 26 runs.shutdown-crashretry of the configure-and-build launch ranConfigureOptionsa second time.OptionsConfigurationItem.SetScriptre-creates the configuration assets, and??=keeps the deleted objects, so both references were saved as{fileID: 0}and the build had no auth token. Fixed in this PR (see above).A throwaway commit (since dropped) that ran the configure-and-build launch twice in every build job, as a retry would: run 40386193023 is green in 33.2 min, and every build called
CliConfiguration::Configure()on the second launch with noAuth Token missing. The clean head: run 40391327148 is green on the first attempt in 27.3 min.🤖 Generated with Claude Code