Problem
S3FileSystem.find(path, withdirs=True) does not include path itself when it is a directory.
fsspec's AbstractFileSystem.find() adds the root (if withdirs and path != "" and self.isdir(path)), "needed for posix glob compliance".
Both branches of S3FileSystem._find() omit it: the maxdepth branch adds only the listed directories (pyathena/filesystem/s3.py:741 on master e0e85da), and the unlimited branch adds only the parents derived from the listed keys below the path (s3.py:763-765).
As a consequence, glob() patterns that should match the root directory (for example glob("s3://bucket/dir/**"), which fsspec matches against find(..., withdirs=True)) miss it.
Found by the independent review of #956 (#933).
Reproduction
from fsspec.implementations.memory import MemoryFileSystem
mfs = MemoryFileSystem()
mfs.pipe("/d/direct", b"")
mfs.pipe("/d/sub/nested", b"")
mfs.find("/d", maxdepth=1, withdirs=True)
# ['/d', '/d/direct', '/d/sub']
fs.find("s3://BUCKET/d", maxdepth=1, withdirs=True)
# PyAthena: ['BUCKET/d/sub', 'BUCKET/d/direct'] (no 'BUCKET/d')
mfs.glob("/d/**")
# ['/d', '/d/direct', '/d/sub', '/d/sub/nested']
fs.glob("s3://BUCKET/d/**")
# PyAthena: no 'BUCKET/d'
Environment
- PyAthena master e0e85da, fsspec 2026.9.0.
Proposed fix (optional)
Include the root directory when withdirs is true and the path is a directory, in both branches, and check the effect on glob(), expand_path(), rm(), and copy(), which call find(..., withdirs=True).
Problem
S3FileSystem.find(path, withdirs=True)does not includepathitself when it is a directory.fsspec's
AbstractFileSystem.find()adds the root (if withdirs and path != "" and self.isdir(path)), "needed for posix glob compliance".Both branches of
S3FileSystem._find()omit it: themaxdepthbranch adds only the listed directories (pyathena/filesystem/s3.py:741on master e0e85da), and the unlimited branch adds only the parents derived from the listed keys below the path (s3.py:763-765).As a consequence,
glob()patterns that should match the root directory (for exampleglob("s3://bucket/dir/**"), which fsspec matches againstfind(..., withdirs=True)) miss it.Found by the independent review of #956 (#933).
Reproduction
Environment
Proposed fix (optional)
Include the root directory when
withdirsis true and the path is a directory, in both branches, and check the effect onglob(),expand_path(),rm(), andcopy(), which callfind(..., withdirs=True).