You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the ingestor is using credentials for the upload. In order to avoid having credentials stored, the archiver backend server should provide presigned urls to upload data
Change s3 upload to use presigned urls form archiver backend
create separate s3 upload module
verify error handling
add checksum verification
use openapi based client
add retries
expose parameters (chunksize, concurrent files, pool size) and find sensible defaults
add expiration date to presigned urls
verify full files by incremental hashing
The text was updated successfully, but these errors were encountered:
Currently, the ingestor is using credentials for the upload. In order to avoid having credentials stored, the archiver backend server should provide presigned urls to upload data
The text was updated successfully, but these errors were encountered: