Skip to content

sandbox: release the token only when it was acquired - #48

Merged
nalgeon merged 1 commit into
nalgeon:mainfrom
justskiv:sandbox-release-token
Aug 21, 2026
Merged

sandbox: release the token only when it was acquired#48
nalgeon merged 1 commit into
nalgeon:mainfrom
justskiv:sandbox-release-token

Conversation

@justskiv

Copy link
Copy Markdown
Contributor

Exec registers defer semaphore.Release() before checking whether
Acquire() returned ErrBusy, so a request rejected as busy puts back a
token it never took. The next caller acquires it and starts a worker beyond
pool_size.

I measured it with pool_size: 1 and a burst of 8 requests sleeping 3 seconds,
sampling docker ps alongside:

binary admitted busy peak containers
v0.14.0 4 4 4
with the fix 1 7 1

The sandbox and the container runtime do not matter — the defect is in
internal/sandbox, above the engine.

Sequential requests never show it: Release is non-blocking, so at rest the
semaphore is back to pool_size. It takes a burst, where every rejection hands
out one more slot: for K requests hitting a pool with s free tokens, up to
(K + s) / 2 get in instead of s. The ceiling follows the load, not the
setting.

To see it fail without Docker, put the defer back above the ErrBusy check
and run go test ./internal/sandbox/:

sandbox_test.go:96:  want 0, got 1
sandbox_test.go:100: want error, got <nil>

The busy subtest now checks that a rejection leaves no free tokens and that
the request right after it is rejected too — the contract the doc comment on
Exec already promises. The exec subtest checks that a finished worker
returns its token.

The Exec body is unchanged since the first public commit (8447197), so
every release is affected. I did not find this in the tracker.


Unrelated: pool_size has no default and is not validated, so a config
without the key gives NewSemaphore(0) and a busy response to everything.

The deferred Release was registered above the ErrBusy check, so a
rejected request put back a token it never took and the next caller
started a worker beyond pool_size. With pool_size 1, a burst of 8
requests ran 4 containers side by side.

Tokens do not accumulate, so this is not a leak: Release is
non-blocking and the channel holds at most pool_size of them. The
excess lives inside a burst, where every rejection hands out one
more slot, so the ceiling follows the load instead of the setting.

The doc comment on Exec already promises no more than pool.Size()
concurrent workers, so this brings the code to what is documented
rather than changing the design.
@nalgeon
nalgeon merged commit fe5920e into nalgeon:main Aug 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants