Problem
S3FileSystem.clear_multipart_uploads() (and AioS3FileSystem.clear_multipart_uploads(), which calls it) lists the in-progress uploads and then aborts each of them in parallel (pyathena/filesystem/s3.py :2997-3022 on master 911492c). An upload that is completed or aborted between the listing and its abort, for example by the writer that owns it or by another clear_multipart_uploads(), makes AbortMultipartUpload answer NoSuchUpload, which PyAthena raises as FileNotFoundError. future.result() re-raises it, so the whole call fails although that upload is gone, which is what the call wants, and the results of the other aborts are not checked after the first error.
Reproduction
An abort of an upload that does not exist raises FileNotFoundError (measured against S3 with S3Core.abort_multipart_upload() of #1069, which sends the same request as clear_multipart_uploads() on master 911492c):
core.abort_multipart_upload(path, "nonexistent-upload-id")
# FileNotFoundError: The specified upload does not exist. The upload ID may be
# invalid, or the upload may have been aborted or completed.
The race itself (an upload completed between list_multipart_uploads() and the abort) was not reproduced.
Environment
PyAthena master 911492c, botocore 1.43.102, Python 3.13.
Proposed fix (optional)
Treat FileNotFoundError from the abort of a listed upload as already cleared, and keep raising other errors. Validate with an offline test whose abort answers NoSuchUpload for one of the listed uploads.
Problem
S3FileSystem.clear_multipart_uploads()(andAioS3FileSystem.clear_multipart_uploads(), which calls it) lists the in-progress uploads and then aborts each of them in parallel (pyathena/filesystem/s3.py:2997-3022 on master 911492c). An upload that is completed or aborted between the listing and its abort, for example by the writer that owns it or by anotherclear_multipart_uploads(), makesAbortMultipartUploadanswerNoSuchUpload, which PyAthena raises asFileNotFoundError.future.result()re-raises it, so the whole call fails although that upload is gone, which is what the call wants, and the results of the other aborts are not checked after the first error.Reproduction
An abort of an upload that does not exist raises
FileNotFoundError(measured against S3 withS3Core.abort_multipart_upload()of #1069, which sends the same request asclear_multipart_uploads()on master 911492c):The race itself (an upload completed between
list_multipart_uploads()and the abort) was not reproduced.Environment
PyAthena master 911492c, botocore 1.43.102, Python 3.13.
Proposed fix (optional)
Treat
FileNotFoundErrorfrom the abort of a listed upload as already cleared, and keep raising other errors. Validate with an offline test whose abort answersNoSuchUploadfor one of the listed uploads.