Repository navigation
DRU-1005 -- Write sandbox uploads in blocks the SFTP server accepts - #1131
Conversation
czpython
left a comment
There was a problem hiding this comment.
The block-size change is right. Two fixes and a nit are inline.
The PR body has the same error as the comment in host.py. sftp-server drops a packet over 256 KiB, and 255 KiB is the max-write-length it advertises. The body also calls czpython/drukbox#85 the gateway fix. That is the issue. The fix is czpython/drukbox#86.
f874e69 to
d89c2a5
Compare
czpython
left a comment
There was a problem hiding this comment.
One more change, inline. Sorry, we missed it in the first review.
| path = await upload_file(session, conversation, file_id, name, content) | ||
| try: | ||
| path = await upload_file(session, conversation, file_id, name, content) | ||
| except (asyncssh.Error, OSError) as error: |
There was a problem hiding this comment.
Instead of catching these errors on the route, catch them in Host.upload_file. Raise a SandboxUploadError there, beside SandboxDownloadError in sandbox/exceptions.py. This way the route does not know the details of the upload: SandboxUploadError is all it catches, and routes.py drops its asyncssh import. The two error cases then belong in a Host.upload_file test, and the route test needs one case.
d89c2a5 to
5927e72
Compare
Host.upload_filepassesblock_size=255 * 1024tosftp.put. With the defaultblock_size=-1, asyncssh sizes each write from themax_write_lenthe server advertises. OpenSSH'ssftp-serverdrops the session on a packet over 256 KiB and advertises a max-write-length of 255 KiB, which leaves room for the packet header. The drukbox gateway in front of it advertises more, so every upload of a file of 256 KiB or more failed: chat thread files, workspace files, and browser state. An explicitblock_sizeis used as given, so no advertised limit can raise it. The gateway issue is czpython/drukbox#85 and its fix is czpython/drukbox#86; this keeps Druks inside whatsftp-servertakes whatever a gateway advertises.Host.upload_fileraisesSandboxUploadError, besideSandboxDownloadError, when its connect or transfer fails with anasyncssh.Erroror anOSError. The error names the remote path and the cause, for examplefailed to upload /…/files/F1/plan.pdf: SFTPConnectionLost: Connection lost. A refused or timed-out connect is anOSError, not anasyncssh.Error, and the upload connects again because the host thatget_running_sandboxreturns has its SSH connection closed.chat_read_fileanswers 502 with that message on aSandboxUploadError. Before, the error left the route as a bare 500, and the agent could not tell the person which file failed or why.SandboxUploadErroris aSandboxError. An agent call uploads its prompt as the run's stdin file, so a failed prompt upload now becomes aHarnessSandboxErrorinHost._start_run, which retries, as other sandbox failures at the run's start do. Before, the call failed without a retry.test_sandbox_host.pyrecords theblock_sizeof each put, and the upload test asserts 255 KiB. A new host test covers a lost SFTP session and a refused connect; the route test covers the 502.