Skip to content

fix(s3): correct S3 block size error messages to match checks #984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Jah-yee wants to merge 1 commit into from
Closed

fix(s3): correct S3 block size error messages to match checks #984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pyathena/filesystem/s3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1275,7 +1275,7 @@ def _copy_object_with_multipart_upload(
block_size < self.MULTIPART_UPLOAD_MIN_PART_SIZE
or block_size > self.MULTIPART_UPLOAD_MAX_PART_SIZE
):
raise ValueError("Block size must be greater than 5MiB and less than 5GiB.")
raise ValueError("Block size must be at least 5 MiB (5242880 bytes) and at most 5 GiB (5368709120 bytes).")

copy_source = {
"Bucket": bucket1,
Expand Down Expand Up @@ -2322,7 +2322,7 @@ def __init__(
if "r" not in mode and block_size < self.fs.MULTIPART_UPLOAD_MIN_PART_SIZE:
# When writing occurs, the block size should not be smaller
# than the minimum size of a part in a multipart upload.
raise ValueError(f"Block size must be >= {self.fs.MULTIPART_UPLOAD_MIN_PART_SIZE}MB.")
raise ValueError(f"Block size must be >= 5 MiB ({self.fs.MULTIPART_UPLOAD_MIN_PART_SIZE} bytes).")

self.append_block = False
if "a" in mode and self.fs.exists(path):
Expand Down
2 changes: 1 addition & 1 deletion pyathena/filesystem/s3_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ async def _copy_object_with_multipart_upload(
block_size < S3FileSystem.MULTIPART_UPLOAD_MIN_PART_SIZE
or block_size > S3FileSystem.MULTIPART_UPLOAD_MAX_PART_SIZE
):
raise ValueError("Block size must be greater than 5MiB and less than 5GiB.")
raise ValueError("Block size must be at least 5 MiB (5242880 bytes) and at most 5 GiB (5368709120 bytes).")

copy_source: dict[str, Any] = {
"Bucket": bucket1,
Expand Down