Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature request: tokio::io::AsyncSeek support for Reader #57

Open
rbtcollins opened this issue Oct 3, 2023 · 5 comments
Open

feature request: tokio::io::AsyncSeek support for Reader #57

rbtcollins opened this issue Oct 3, 2023 · 5 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@rbtcollins
Copy link

Use case: I have zip files being added to a cacache instance, and zip is a non-streamable format - the index of a zip is at the end, so it is somewhat inefficient to read everything, then scan the index, then reopen and re-read to extract the actual content (using async_zip)

@zkat zkat added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers labels Oct 4, 2023
@zkat
Copy link
Owner

zkat commented Oct 4, 2023

sure, this sounds good.

Ideally, we'll also have the async-std version of this done at the same time, though.

@baszalmstra
Copy link

What do you think the API would look like @zkat ? The Reader in cacache also performs integrity checks, if we could seek in the file this becomes much harder.

@zkat
Copy link
Owner

zkat commented Nov 30, 2023

yeah I have no idea how that would work tbh

@rbtcollins
Copy link
Author

I think you could read through to do the verification, then let the caller use seek to read the bits they need; the main point is to not have to buffer the entire file in RAM or make a temp copy of it, when using a non-streamable layer on top

@zkat
Copy link
Owner

zkat commented Apr 15, 2024

Yeah. And we can have the unchecked and checked versions, in case you don't want to read the whole thing because, e.g. the file is enormous.

Ok.

I'll take a PR for this if anyone's interested, but it would need to be implemented for tokio, async-std, and sync. I don't want feature drift between the backends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants