read_xarray doesn't work on all xarray datasets

[ahuang11]

ds = xr.open_zarr(
  'gs://gcp-public-data-arco-era5/ar/full_37-1h-0p25deg-chunk-1.zarr-v3',
  chunks=None,  # Turn dask off
  storage_options={'token': 'anon'}  # Anonymous read from the public GCS bucket — no auth required.
)
xql.read_xarray(ds)

File ~/repos/xarray-sql/xarray_sql/reader.py:96, in XarrayRecordBatchReader.__init__(self, ds, chunks, batch_size, _iteration_callback)
     94 fst = next(iter(ds.values())).dims
     95 if not all(da.dims == fst for da in ds.values()):
---> [96](https://file+.vscode-resource.vscode-cdn.net/Users/ahuang/repos/xarray-sql/~/repos/xarray-sql/xarray_sql/reader.py:96)     raise ValueError(
     97         "All dimensions must be equal. Please filter data_vars in the Dataset."
     98     )

ValueError: All dimensions must be equal. Please filter data_vars in the Dataset.

So, I tried:

xql.read_xarray(ds, data_vars=["surface"])

But data_vars isn't a kwarg.

Should read_xarray have an option to pass data_vars?

[alxmrs]

xql.read_xarray(ds, data_vars=["surface"])

This is intended to be called like so:

xql.read_xarray(ds[["2m_temperature", ...]])

Tbh, this probably should be an internal API.

[ahuang11]

Yeah I think we should follow the zen of python or else it might end up like pandas with multitude of ways:

There should be one-- and preferably only one --obvious way to do it.

[ahuang11]

However, I think it's tedious that users have to initialize the SessionContext first before reading it.

[JHON12091986]

/claim #180

I'll work on optimizing read_xarray to handle datasets with many chunks more efficiently.
Will include:

• Analysis of current O(num_chunks) behavior
• Implementation of batched or chunked reading
• Performance improvements for finely chunked stores
• Tests for large datasets

PR ready in 12-24 hours.