[Backport wrynose] ci: make the shared FSx build cache robust under parallel jobs - #3263
Merged
Ricardo Salveti (ricardosalveti) merged 9 commits intoOct 3, 2026
Conversation
Use GitHub's dedicated self-repository syntax: | use '$/...' instead of './...' Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com> (cherry picked from commit d34bb87) Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
Secrets unconditionally inherited by called workflow: this reusable workflow Signed-off-by: Jose Quaresma <jose.quaresma@oss.qualcomm.com> (cherry picked from commit c7d6e50) Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com>
zizmor (excessive-permissions) flags the jobs that run with the default token permissions because neither they nor their workflow declare any: the compile jobs in build-yocto.yml, the test jobs in test.yml, and the markdownlint and repolinter workflows. Declare them explicitly: the permissions the callers already grant for the compile and test jobs, and read-only contents for the linters. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit 953fb75)
zizmor (artipacked) flags the checkouts that leave the token in .git/config, where an artifact upload could leak it. None of these jobs run git against the remote after checking out, so set persist-credentials: false. The backport workflow keeps them, with an inline ignore, as backport-action pushes the backport branch with plain git. Assisted-by: Claude Code:claude-opus-5-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit 8a2c827)
Jobs on different hosts update the shared git clone dirs in DL_DIR concurrently, and bitbake's lock does not reliably serialize them over the network mount. On FSx this corrupted clone dirs (zstd do_unpack failed in 42 jobs of one PR run), and since the fetcher never cleans up a broken clone dir, every later build of that recipe fails until it is removed by hand. Keep the clone dirs job-local and share git history through the git2_*.tar.gz mirror tarballs, which the fetcher writes atomically and restores from automatically, as the Yocto autobuilders do. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit 9cbe0a4)
The qcom_fitimage selftests fetch kernel sources into the same shared git clone dirs, and hit the same corruption in staging validation (do_unpack: "clone directory not available or not up to date"). Add an opt-in GIT_CLONEDIR_JOB_LOCAL to ci/oe-selftest.sh with the ci/ci.yml settings, and enable it in the checks job through KAS_OPTS, which the kas-container shell helper now forwards. Local runs keep using DL_DIR/git2. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit b1cf7b5)
All qcom kernel flavors fetch the same multi-gigabyte repository. The fetcher keeps its lock in DL_DIR even with job-local clone dirs, and a job without a clone takes it exclusively to untar the full mirror tarball, so a kernel revision bump queues the whole matrix behind it. On 2026-09-13 most of the 457 jobs killed at the six-hour limit were still waiting in a linux-qcom do_fetch or do_unpack. Fetch these kernels with BB_GIT_SHALLOW in the image builds and in the oe-selftest job-local mode, so each job unpacks a depth-1 tarball of a few hundred megabytes, generated once per SRCREV, under a shared lock. With no tags in the shallow clone, setlocalversion stops adding the git-describe commit count: linux-qcom-next in CI reports 7.2.0-gd49c33864d06 instead of 7.2.0-01462-gd49c33864d06. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit 3882c51)
On a cold cache all warm_up jobs start together and each builds the common core (native tools, toolchain, generic world) on its own; in the FSx staging validation this held the matrix back for almost five hours. Build the core once in a compile_stage1 job (qcom-armv8a nodistro world) and start the seeds after it. Warm runs pay the core job's duration, about nine minutes, before the seeds start. Rename the seed and main jobs to compile_stage2 and compile_stage3 to match. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit 7529e6a)
Cold-cache builds run right at GitHub's 360-minute default job timeout: in the 2026-09-19 weekly build, 22 debug jobs were killed at six hours while still compiling and ten more finished between 348 and 359 minutes. A killed job publishes none of its remaining sstate, so the next run repeats the work. Set the compile job timeout to 480 minutes. Assisted-by: Claude Code:claude-fable-5 Signed-off-by: Ricardo Salveti <ricardo.salveti@oss.qualcomm.com> (cherry picked from commit f4355f2)
Test Results 119 files ± 0 715 suites ±0 7h 14m 19s ⏱️ - 1h 46m 16s For more details on these failures, see this check. Results for commit 6e22ae3. ± Comparison against base commit a1a92a3. ♻️ This comment has been updated with latest results. |
Dmitry Baryshkov (lumag)
approved these changes
Sep 29, 2026
Koen Kooi (koenkooi)
approved these changes
Sep 30, 2026
Jose Quaresma (quaresmajose)
approved these changes
Sep 30, 2026
Ricardo Salveti (ricardosalveti)
merged commit Oct 3, 2026
8e5a1e0
into
qualcomm-linux:wrynose
209 of 210 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #3170, #3222 and #3221 to
wrynose.#3170 and #3222 go first so #3221 applies unchanged; their conflicts were
only surrounding lines that differ on wrynose, and files that exist only on
master.