Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀[FEA]: Package system doesn't natively work with Zarr Stores #115

Open
dallasfoster opened this issue Aug 16, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@dallasfoster
Copy link
Collaborator

dallasfoster commented Aug 16, 2024

Version

main

On which installation method(s) does this occur?

Pip, Source

Describe the issue

The package .resolve method uses fsspec._open under the hood, which fails if the file in question is interpreted as a directory, as is the case with Zarr stores.

store = package.resolve('metadata.zarr')
package.resolve("metadata.zarr")
  File "/usr/local/lib/python3.10/dist-packages/earth2studio/models/auto.py", line 319, in resolve
    with self.open(file_path) as file:
  File "/usr/local/lib/python3.10/dist-packages/earth2studio/models/auto.py", line 301, in open
    return self.fs.open(full_path, callback=callback)
  File "/usr/local/lib/python3.10/dist-packages/fsspec/spec.py", line 1293, in open
    f = self._open(
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 197, in _open
    return LocalFileOpener(path, mode, fs=self, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 322, in __init__
    self._open()
  File "/usr/local/lib/python3.10/dist-packages/fsspec/implementations/local.py", line 327, in _open
    self.f = open(self.path, mode=self.mode)
IsADirectoryError: [Errno 21] Is a directory: '/path/to/file/metadata.zarr'
@dallasfoster dallasfoster added bug Something isn't working ? - Needs Triage Need team to review and classify labels Aug 16, 2024
@NickGeneva NickGeneva added enhancement New feature or request and removed bug Something isn't working ? - Needs Triage Need team to review and classify labels Sep 24, 2024
@NickGeneva
Copy link
Collaborator

Updating to feature request, doing this on NGC will potentially be quite challenging due to how file access works... specifically for authenticated users.

@NickGeneva NickGeneva changed the title 🐛[BUG]: Package system doesn't natively work with Zarr Stores 🚀[FEA]: Package system doesn't natively work with Zarr Stores Sep 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants