Skip to content

storage-sqlite: a restore slot test checks the slot before the copy frees it #872

Open

Description

@LeeroyHannigan

backup::restore_tests::reconciler_fencing::a_running_restore_holds_its_slot_and_is_not_listed_dead_by_its_heartbeat (crates/storage-sqlite/src/backup.rs) failed in run-tests (stable) of job 127447427993, on #868 on 5 November 18:33 UTC, with the slot is freed with the claim when the copy ends. #868 does not change storage-sqlite, and the rerun of the job passed. In the 184 completed runs of test.yml before it, from 3 to 5 November, this test did not fail.

The test calls wait_for_copy, which returns when the target's table_status is ACTIVE. Then it asserts at once that job_slots no longer holds the restore's slot. The copy commits ACTIVE in its own transaction and returns. The slot is freed later, when the job's claim (crates/storage-sqlite/src/backup_catalog.rs) is dropped. So a check that runs between the two sees the slot held.

A fix that keeps the assertion is to wait, with a time limit, for the slot to be freed after wait_for_copy returns. Another test in the same module already waits that way, in a loop on engine.job_slots.holds(&slot).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci:stabilitysqliteRelated to the SQLite backend specifically.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions