Skip to content

Close a stuck request on retry when its local owner process is gone - #16

Merged
hyperdrivebeep merged 1 commit into
mainfrom
public-20261020
Oct 22, 2026
Merged

hyperdrivebeep merged 1 commit into
mainfrom
public-20261020

Conversation

@hyperdrivebeep

Copy link
Copy Markdown
Owner

A short command interrupted when THOTH exited stayed RUNNING forever;
sending the same request again only replayed RUNNING. In local mode,
repeating the same request now closes it when that is safe.

Owner record

  • New table operation_owners (one migration, existing tables and rows
    untouched): worker id, process id, process marker and a host token,
    written in the same transaction as the claim. It is internal only and
    never appears in GET or RPC responses or events.
  • The host token is an HMAC of the OS tag, machine id and boot id; only
    the hash is stored. If any input cannot be read, no owner row is
    written. Hosted review mode reads nothing.

Closing on retry

  • Only for the 67 commands measured to write nothing when repeated, an
    operation with an owner row, the same machine and boot, an owning
    worker confirmed dead (alive or unknown never closes), and while this
    process holds the workspace lock.
  • The operation is re-read and closed in one transaction with its
    binding and operation.failed event, after the access check. If the
    original handler completed first, its stored result is replayed. The
    answer is OPERATION_OWNER_GONE with remote_observation UNKNOWN and one
    Korean and one English line asking the user to check whether the
    change was saved before sending a new request.
  • No time-based expiry, automatic rerun or sweep. Research commands,
    commands outside the list, old rows and hosted review are unchanged.
  • When completion or failure loses the compare-and-set to another
    writer, the command bus now replays the stored winner instead of
    appending its own succeeded or failed events.
  • docs/KNOWN_LIMITATIONS describes the scope and the case where a saved
    change may miss its notification when the close wins.

Verification: the full local check (Python 5345 tests, web tests, lint,
typecheck, build, architecture and module budget, README smoke) passed
with no failures, including real child process kill tests and the
completion race tests.

A short command interrupted when THOTH exited stayed RUNNING forever;
sending the same request again only replayed RUNNING. In local mode,
repeating the same request now closes it when that is safe.

Owner record

- New table operation_owners (one migration, existing tables and rows
  untouched): worker id, process id, process marker and a host token,
  written in the same transaction as the claim. It is internal only and
  never appears in GET or RPC responses or events.
- The host token is an HMAC of the OS tag, machine id and boot id; only
  the hash is stored. If any input cannot be read, no owner row is
  written. Hosted review mode reads nothing.

Closing on retry

- Only for the 67 commands measured to write nothing when repeated, an
  operation with an owner row, the same machine and boot, an owning
  worker confirmed dead (alive or unknown never closes), and while this
  process holds the workspace lock.
- The operation is re-read and closed in one transaction with its
  binding and operation.failed event, after the access check. If the
  original handler completed first, its stored result is replayed. The
  answer is OPERATION_OWNER_GONE with remote_observation UNKNOWN and one
  Korean and one English line asking the user to check whether the
  change was saved before sending a new request.
- No time-based expiry, automatic rerun or sweep. Research commands,
  commands outside the list, old rows and hosted review are unchanged.
- When completion or failure loses the compare-and-set to another
  writer, the command bus now replays the stored winner instead of
  appending its own succeeded or failed events.
- docs/KNOWN_LIMITATIONS describes the scope and the case where a saved
  change may miss its notification when the close wins.

Verification: the full local check (Python 5345 tests, web tests, lint,
typecheck, build, architecture and module budget, README smoke) passed
with no failures, including real child process kill tests and the
completion race tests.
@hyperdrivebeep
hyperdrivebeep merged commit 0ad7fb5 into main Oct 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant