Skip to content

DRU-1005 -- Write sandbox uploads in blocks the SFTP server accepts - #1131

Merged
czpython merged 1 commit into
czpython:mainfrom
p-r-a-v-i-n:dru-1005-sftp-upload-block-size
Nov 9, 2026
Merged

czpython merged 1 commit into
czpython:mainfrom
p-r-a-v-i-n:dru-1005-sftp-upload-block-size

Conversation

@p-r-a-v-i-n

@p-r-a-v-i-n p-r-a-v-i-n commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Host.upload_file passes block_size=255 * 1024 to sftp.put. With the default block_size=-1, asyncssh sizes each write from the max_write_len the server advertises. OpenSSH's sftp-server drops the session on a packet over 256 KiB and advertises a max-write-length of 255 KiB, which leaves room for the packet header. The drukbox gateway in front of it advertises more, so every upload of a file of 256 KiB or more failed: chat thread files, workspace files, and browser state. An explicit block_size is used as given, so no advertised limit can raise it. The gateway issue is czpython/drukbox#85 and its fix is czpython/drukbox#86; this keeps Druks inside what sftp-server takes whatever a gateway advertises.

  • Host.upload_file raises SandboxUploadError, beside SandboxDownloadError, when its connect or transfer fails with an asyncssh.Error or an OSError. The error names the remote path and the cause, for example failed to upload /…/files/F1/plan.pdf: SFTPConnectionLost: Connection lost. A refused or timed-out connect is an OSError, not an asyncssh.Error, and the upload connects again because the host that get_running_sandbox returns has its SSH connection closed.
  • chat_read_file answers 502 with that message on a SandboxUploadError. Before, the error left the route as a bare 500, and the agent could not tell the person which file failed or why.
  • SandboxUploadError is a SandboxError. An agent call uploads its prompt as the run's stdin file, so a failed prompt upload now becomes a HarnessSandboxError in Host._start_run, which retries, as other sandbox failures at the run's start do. Before, the call failed without a retry.
  • The fake SFTP client in test_sandbox_host.py records the block_size of each put, and the upload test asserts 255 KiB. A new host test covers a lost SFTP session and a refused connect; the route test covers the 502.

@p-r-a-v-i-n
p-r-a-v-i-n requested a review from czpython as a code owner November 9, 2026 05:51

@czpython czpython left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The block-size change is right. Two fixes and a nit are inline.

The PR body has the same error as the comment in host.py. sftp-server drops a packet over 256 KiB, and 255 KiB is the max-write-length it advertises. The body also calls czpython/drukbox#85 the gateway fix. That is the issue. The fix is czpython/drukbox#86.

Comment thread backend/druks/chat/routes.py Resolved
Comment thread backend/druks/sandbox/host.py Resolved
Comment thread backend/tests/test_sandbox_host.py Resolved
@p-r-a-v-i-n
p-r-a-v-i-n force-pushed the dru-1005-sftp-upload-block-size branch from f874e69 to d89c2a5 Compare November 9, 2026 09:35
@p-r-a-v-i-n
p-r-a-v-i-n requested a review from czpython November 9, 2026 09:55

@czpython czpython left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more change, inline. Sorry, we missed it in the first review.

Comment thread backend/druks/chat/routes.py Outdated
path = await upload_file(session, conversation, file_id, name, content)
try:
path = await upload_file(session, conversation, file_id, name, content)
except (asyncssh.Error, OSError) as error:

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of catching these errors on the route, catch them in Host.upload_file. Raise a SandboxUploadError there, beside SandboxDownloadError in sandbox/exceptions.py. This way the route does not know the details of the upload: SandboxUploadError is all it catches, and routes.py drops its asyncssh import. The two error cases then belong in a Host.upload_file test, and the route test needs one case.

@p-r-a-v-i-n
p-r-a-v-i-n force-pushed the dru-1005-sftp-upload-block-size branch from d89c2a5 to 5927e72 Compare November 9, 2026 14:14
@p-r-a-v-i-n
p-r-a-v-i-n requested a review from czpython November 9, 2026 14:24
@czpython
czpython merged commit 3e59a04 into czpython:main Nov 9, 2026
6 checks passed
@p-r-a-v-i-n
p-r-a-v-i-n deleted the dru-1005-sftp-upload-block-size branch November 10, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants