Skip to content

Flaky api and ocpp tests fail CI on unrelated PRs #1712

Closed

Description

@frahlg

On 17 October 2026, flaky tests on master failed CI on four PRs whose changes did not touch the failing packages (#1701, #1703, #1706 and the #1674 follow-ups). Master requires an up-to-date branch, so each flaky run costs a full CI cycle before a merge. Re-running the failed jobs passed every time.

Seen today

go/internal/api, in the race detector job (and so in the required go test + vet):

  • TestAViewerMayStillRead, api_viewer_writes_test.go:212: "a viewer's read of /api/system/info answered error, want a head". One run gave the same for /api/mode.
  • TestReadingTheConfigIsAllowedForAnyRole/owner, api_passthrough_test.go:588: "the answer was not the handler's JSON: unexpected end of JSON input".
  • TestAViewerIsRefusedAtEveryWrite, api_viewer_writes_test.go:143: "POST /api/ev/chargers was refused with "E_UNAVAILABLE", which is not a refusal to write".

Runs: 39045010631 (#1703), 39045384675 attempt 1 (#1701), 39047533237 attempt 1 (#1706).

go/internal/ocpp, in core (Go):

Proposed fix in flight

Draft #1709 (fix-api-flaky-race-tests) addresses the api failures: the box frees its passthrough-queue slot before it sends an answer's last frame. That changes go/internal/appproto/passthrough.go, so it needs review as a product change, not a test fix. Its session reports 5 failures in 4,000 runs on master and 0 on the branch. It does not cover the ocpp panic.

Done when

  • The api tests above pass reliably under -race (for example 4,000 runs with no failure), with fix(appproto): free the passthrough slot before an answer's last frame #1709 or another fix reviewed and merged.
  • The ocpp test no longer panics: stop the test's charge point before its client closes, or avoid writing on a closed client, with a repeat count that shows it.
  • Until then, re-running failed jobs is the workaround. Do not add skips or retries to hide the failures.

🤖 Generated with Claude Code

Activity

  1. frahlg commented on Oct 18, 2026

    @frahlg
    MemberAuthor

    The API fixes merged in #1709 (8f937e7e) and the affected OCPP cleanup fix in #1717 (2dde2463). On the current combined candidate, the three API tests named here passed 100 repeats with the race detector, and TestPerChargerCredentialOverTheWire passed 50 repeats with it. Full make verify also passed.

    Closing the listed failures. This does not claim that every test in Core is free of flakes: #1691 retains its uplink follow-up, #1719 carries the plan-push revision wait, and intentional mid-test OCPP disconnects still retain the upstream library's narrow close/write window noted in #1717.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions