Asynchronous fetching of dist files

[mmstick]

For Pop!_OS, I've just written async-fetcher (Crates.io), which could greatly enhance your current capabilities. It has been implemented with fetching apt dist files in mind, as can be seen from the example that fetches a few files from our debian repository.

Some of the key features are:

• Checking if a file requires to be downloaded based on the modified time if no checksum is provided.
• Comparing the checksum of a local file to determine if a GET request is required.
• Enabling the caller to provide a checksum to check against the downloaded file's contents.
• Enabling the caller to define how the downloaded file is processed into the destination, such as decompression.
• Enabling the caller to use a different checksum to compare the checksum of the destination file.
• Setting the file time of the fetched files to the modified time on the server.
• Supports execution in a parallel tokio runtime.

This would be useful to fetch a compressed dist file, compare the checksum of the compressed checksum, decompress it if it's fine, and then check the checksum of the decompressed files.

[mmstick]

apt-fetcher is gradually getting closer to completion, based on async-fetcher. The goal is to delegate all apt-related network requests to apt-fetcher, so that all apt-related projects can simply integrate with it to fetch source lists and packages either asynchronously, or asynchronously and in parallel.

If you need a crate for reading debian archives, deb-archive supports reading / extracting files from the inner control and data archives. We use this in debrep to construct the Packages / Contents dist archives.