Skip to content

Commit

Permalink
Encode PUT data before sending POST request to Codecov (#144)
Browse files Browse the repository at this point in the history
When testing ATS in the sentry fork I have 3 out of 5 uploads timing out when pushing to storage.

I noticed that we are generating the PUT payload *after* receiving the response from Codecov.
Generating PUT payload includes compressing the report file, that might be pretty big,
and considering that the pre-signed put URL has a TTL of 10s this time might be influential.

So now we are just prepping the data completely before starting the upload process,
in the hopes that this will save us enough time to get all uploads into storage in time.
  • Loading branch information
giovanni-guidini authored May 9, 2023
1 parent 6bfcd84 commit 509312f
Showing 1 changed file with 9 additions and 8 deletions.
17 changes: 9 additions & 8 deletions codecov_cli/services/upload/upload_sender.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,19 +46,20 @@ def send_upload_data(
"job_code": job_code,
}

# Data to upload to Codecov
headers = get_token_header_or_fail(token)
encoded_slug = encode_slug(slug)
url = f"https://api.codecov.io/upload/{git_service}/{encoded_slug}/commits/{commit_sha}/reports/{report_code}/uploads"
resp = send_post_request(url=url, data=data, headers=headers)

if resp.status_code >= 400:
return resp
# Data that goes to storage
reports_payload = self._generate_payload(upload_data, env_vars)

resp_json_obj = json.loads(resp.text)
resp_from_codecov = send_post_request(url=url, data=data, headers=headers)
if resp_from_codecov.status_code >= 400:
return resp_from_codecov
resp_json_obj = json.loads(resp_from_codecov.text)
put_url = resp_json_obj["raw_upload_location"]
reports_payload = self._generate_payload(upload_data, env_vars)
resp = send_put_request(put_url, data=reports_payload)
return resp
resp_from_storage = send_put_request(put_url, data=reports_payload)
return resp_from_storage

def _generate_payload(
self, upload_data: UploadCollectionResult, env_vars: typing.Dict[str, str]
Expand Down

0 comments on commit 509312f

Please sign in to comment.