ci: test job summary fixes - #3137
Merged
Dmitry Baryshkov (lumag) merged 10 commits intoSep 29, 2026
Merged
Dmitry Baryshkov (lumag) merged 10 commits into
Dmitry Baryshkov (lumag) merged 10 commits into
Conversation
Contributor
Author
Milosz Wasilewski (mwasilew)
marked this pull request as ready for review
September 11, 2026 21:30
Milosz Wasilewski (mwasilew)
changed the title
Test Results 119 files + 51 715 suites +396 9h 55m 19s ⏱️ + 5h 24m 23s For more details on these failures, see this check. Results for commit b727b1f. ± Comparison against base commit 5c98a78. ♻️ This comment has been updated with latest results. |
The LAVA hostname was hardcoded in every REST API request and results link, so the action could only summarise jobs run on lava.infra.foundries.io. Add a lava_base_url input taking the base URL of the instance, defaulting to that one, and build the URLs from it. Ported from qualcomm/qcom-reusable-workflows commit f0aa804a071e. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
The resource_uri field returned by the LAVA REST API points to the API endpoint of the test case, so the links in the results table open raw JSON. Build the link to the test case page from the test case ID instead. Ported from qualcomm/qcom-reusable-workflows commit e0307f791bd1. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
The boot result was emitted from inside the loop over the job test suites. A boot job that has no suites, for example one cancelled before it started, was listed in the all jobs table but missing from the results table and the totals. Boot result only depends on the job state and health, so report it once per job without fetching the suites. Ported from qualcomm/qcom-reusable-workflows commit 8d69917c7905. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
curl -s exits with success on HTTP errors, and errexit is not inherited by the command substitutions most of the requests run in. A LAVA outage therefore produced a summary with missing results that looked like a successful run. Make curl fail on HTTP errors, retry transient ones, and set errexit and pipefail with inherit_errexit so that a failed request fails the step. Send the requests through a helper. A busy LAVA server can be slow to respond, so the helper repeats a failed request once after a 30 second wait. If the request fails again, the helper reports the request URL and the curl exit code as an error annotation, so it's clear why the step failed. Ported from qualcomm/qcom-reusable-workflows commit 4338fbcc5aec. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
Add and amend descriptions of the action parameters and action itself. Ported from qualcomm/qcom-reusable-workflows commit a1ef819360b3. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
The LAVA hostname was repeated in each lava-action step that submits test jobs, while the test job summary relied on the default of its lava_base_url input to read the results back. Nothing tied these together, so changing the LAVA instance in one place would leave the summary reading results from a different one. Define the hostname once as a workflow environment variable and pass it to the job submission and test job summary steps. Assisted-by: Claude Code:opus-5 Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
A test that is known to fail keeps the summary red for as long as the failure stands, which hides the regressions the summary exists to show. There was no way to say that a failure is expected. Take a list of them. A listed failure is reported as a known failure and counted as a pass; a listed test that passes is reported as an unexpected pass and counted as a failure, so an entry that outlived its bug gets noticed rather than quietly suppressing a test. The comment of each entry, which is where the issue it is tracked in belongs, is carried into a table of its own. An entry may name the kernel flavours it applies to, for a caller whose list covers a build that is made with several. The list is optional: without known_failures_file the summary is what it was, and the known failures section is not rendered at all. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
The rows and columns came out in whatever order the results happened to be read in, which for a table that is read across boards and down tests makes it hard to find anything, and moves things between runs. Sort both. boot is the exception: it is the one result every job has and the one that explains all the others when it fails, so it stays the top row whatever it sorts as. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
A cell for a listed failure said "known failure", which the tick beside it already says. What it did not say is why the failure is accepted, which is the one thing the reader of the table wants and which the list already records. Put the note there instead, and point the cell at the URL in it when it has one, so that a cell about a tracked failure links to the issue rather than to the test case. A note without a URL still links where it did. Long notes are cut to keep the board columns on the page, and a pipe in one is escaped so it cannot end the cell early. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
A listing answers with a page of results and a link to the rest, fifty long unless asked otherwise, and only the first page was read. A job with more suites than that, or a suite with more test cases, was reported on the strength of its first fifty with the remainder dropped and nothing said about it. Ask for a large page to keep the round trips down and follow the link until it runs out, so the size of the listing stops mattering. The results are handed back as one response, so the callers are unchanged. Signed-off-by: Milosz Wasilewski <milosz.wasilewski@oss.qualcomm.com>
Milosz Wasilewski (mwasilew)
force-pushed
the
test-job-summary-fixes
branch
from
September 24, 2026 19:33
62fa91b to
b727b1f
Compare
Contributor
Author
|
I rebased on top of master and ported features from reusable-workflows and qcom-deb-images |
This was referenced Sep 24, 2026
Closed
Ricardo Salveti (ricardosalveti)
approved these changes
Sep 25, 2026
Dmitry Baryshkov (lumag)
approved these changes
Sep 27, 2026
Koen Kooi (koenkooi)
approved these changes
Sep 28, 2026
Dmitry Baryshkov (lumag)
merged commit Sep 29, 2026
d4cccd3
into
qualcomm-linux:master
210 of 211 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix issues found during migration to reusable workflow: