-
DescriptionWe have two local repositories where the trivy-db is mirroring (scan task doesn't have internet access). Sometime the first repo is too busy and send the response code 429. We are expecting trivy will try the second repo, but not. Desired BehaviorTo fall over the second repo when the first is too busy. Actual BehaviorA Fatal error and an exit code. Reproduction StepsTo reproduce the "Too Many Requests", install for exemple an NGINX server ain local who always return 429 error.
Execute a scan :
$ ./trivy image --no-progress --format table --exit-code 1 --output report_trivy.md --db-repository 127.0.0.1/trivy,xxx/aquasecurity/trivy-db --cache-dir .trivy-cache xxx/test:latest
2025-01-13T08:42:08Z INFO Adding schema version to the DB repository for backward compatibility repository="127.0.0.1/trivy:2"
2025-01-13T08:42:08Z INFO Adding schema version to the DB repository for backward compatibility repository="xxx/aquasecurity/trivy-db:2"
2025-01-13T08:42:08Z INFO [vulndb] Need to update DB
2025-01-13T08:42:08Z INFO [vulndb] Downloading vulnerability DB...
2025-01-13T08:42:08Z INFO [vulndb] Downloading artifact... repo="127.0.0.1/trivy:2"
2025-01-13T08:42:08Z FATAL Fatal error init error: DB error: failed to download vulnerability DB: OCI artifact error: failed to download vulnerability DB: failed to download artifact from 127.0.0.1/trivy:2: OCI repository error: 1 error occurred:
* Get "https://127.0.0.1/v2/": dial tcp 127.0.0.1:443: connect: connection refused; GET http://127.0.0.1/v2/: unexpected status code 429 Too Many Requests
If we change the error code from 429 to 500 in the first repository, we have :
$ ./trivy image --no-progress --format table --exit-code 1 --output report_trivy.md --db-repository 127.0.0.1/trivy,xxx/trivy-db --cache-dir .trivy-cache xxx/test:latest
2025-01-13T08:43:01Z INFO Adding schema version to the DB repository for backward compatibility repository="127.0.0.1/trivy:2"
2025-01-13T08:43:01Z INFO Adding schema version to the DB repository for backward compatibility repository="xxx/aquasecurity/trivy-db:2"
2025-01-13T08:43:01Z INFO [vulndb] Need to update DB
2025-01-13T08:43:01Z INFO [vulndb] Downloading vulnerability DB...
2025-01-13T08:43:01Z INFO [vulndb] Downloading artifact... repo="127.0.0.1/trivy:2"
2025-01-13T08:43:02Z ERROR [vulndb] Failed to download artifact repo="127.0.0.1/trivy:2" err="OCI repository error: 1 error occurred:\n\t* Get \"https://127.0.0.1/v2/\": dial tcp 127.0.0.1:443: connect: connection refused; GET http://127.0.0.1/v2/: unexpected status code 500 Internal Server Error\n\n"
2025-01-13T08:43:02Z INFO [vulndb] Trying to download artifact from other repository...
2025-01-13T08:43:02Z INFO [vulndb] Downloading artifact... repo="xxx/aquasecurity/trivy-db:2"
2025-01-13T08:43:06Z INFO [vulndb] Artifact successfully downloaded repo="xxx/aquasecurity/trivy-db:2"
2025-01-13T08:43:06Z INFO [vuln] Vulnerability scanning is enabled
With this error, the second repo is used.
Tell me if my analyse is wrong. In the code at https://github.com/aquasecurity/trivy/blob/main/pkg/oci/artifact.go#L237, the trying another repo is conditioned to a temporary error from the library https://github.com/google/go-containerregistry/blob/main/pkg/v1/remote/transport/error.go#L147. But there isn't the 429 error in it. This error is not considered temporary so...
May be this code in artifact.go could be better to try an other repo :
if terr.Temporary() || terr.StatusCode == 429 {
return true
}
We need your advice on this issue. TargetContainer Image ScannerVulnerability Output FormatTable ModeNone Debug Output2025-01-13T09:27:04Z DEBUG No plugins loaded
2025-01-13T09:27:04Z DEBUG Default config file "file_path=trivy.yaml" not found, using built in values
2025-01-13T09:27:04Z DEBUG Cache dir dir=".trivy-cache"
2025-01-13T09:27:04Z DEBUG Cache dir dir=".trivy-cache"
2025-01-13T09:27:04Z INFO Adding schema version to the DB repository for backward compatibility repository="127.0.0.1/trivy:2"
2025-01-13T09:27:04Z INFO Adding schema version to the DB repository for backward compatibility repository="index.docker.io/xxx/aquasecurity/trivy-db:2"
2025-01-13T09:27:04Z DEBUG Parsed severities severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2025-01-13T09:27:04Z DEBUG Ignore statuses statuses=[]
2025-01-13T09:27:04Z DEBUG [vulndb] There is no valid metadata file err="unable to open a file: open .trivy-cache/db/metadata.json: no such file or directory"
2025-01-13T09:27:04Z INFO [vulndb] Need to update DB
2025-01-13T09:27:04Z DEBUG [vulndb] No metadata file
2025-01-13T09:27:04Z INFO [vulndb] Downloading vulnerability DB...
2025-01-13T09:27:04Z INFO [vulndb] Downloading artifact... repo="127.0.0.1/trivy:2"
2025-01-13T09:27:04Z FATAL Fatal error
- init error:
github.com/aquasecurity/trivy/pkg/commands/artifact.Run
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:367
- DB error:
github.com/aquasecurity/trivy/pkg/commands/artifact.NewRunner
/home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:119
- failed to download vulnerability DB:
github.com/aquasecurity/trivy/pkg/commands/operation.DownloadDB
/home/runner/work/trivy/trivy/pkg/commands/operation/operation.go:40
- OCI artifact error:
github.com/aquasecurity/trivy/pkg/db.(*Client).Download
/home/runner/work/trivy/trivy/pkg/db/db.go:163
- failed to download vulnerability DB:
github.com/aquasecurity/trivy/pkg/db.(*Client).downloadDB
/home/runner/work/trivy/trivy/pkg/db/db.go:212
- failed to download artifact from 127.0.0.1/trivy:2:
github.com/aquasecurity/trivy/pkg/oci.Artifacts.Download
/home/runner/work/trivy/trivy/pkg/oci/artifact.go:238
- OCI repository error:
github.com/aquasecurity/trivy/pkg/oci.(*Artifact).populate
/home/runner/work/trivy/trivy/pkg/oci/artifact.go:97
- 1 error occurred:
* Get "https://127.0.0.1/v2/": dial tcp 127.0.0.1:443: connect: connection refused; GET http://127.0.0.1/v2/: unexpected status code 429 Too Many Requests: Operating Systemimage VersionVersion: 0.58.1 Checklist
|
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 2 replies
-
hi @econogit your analysis seems correct. @nikpivkin @knqyf263 why do we limit ourselves to falling back only on temporary errors? https://github.com/aquasecurity/trivy/blob/main/pkg/oci/artifact.go#L257-L263 |
Beta Was this translation helpful? Give feedback.
-
Hello @econogit Thanks for your report! IIUC, Your test server returns error that doesn't match to distribution. I set the following setting:
and it works correctly for me: 2025-01-14T13:14:16+06:00 ERROR [vulndb] Failed to download artifact repo="127.0.0.1/trivy:2" err="OCI repository error: 1 error occurred:\n\t* Get \"https://127.0.0.1/v2/\": dial tcp 127.0.0.1:443: connect: connection refused; GET http://127.0.0.1/v2/: TOOMANYREQUESTS: too many requests\n\n"
2025-01-14T13:14:16+06:00 INFO [vulndb] Trying to download artifact from other repository...
2025-01-14T13:14:16+06:00 INFO [vulndb] Downloading artifact... repo="ghcr.io/aquasecurity/trivy-db:1"
Regards, Dmitriy |
Beta Was this translation helpful? Give feedback.
-
Thank you very @DmitriyLewen for your quick answer. I actually thought that the return code 429 was enough to switch. But you are right, you also need the json. Thanks again for your help and the clear information that we need to check. |
Beta Was this translation helpful? Give feedback.
hi @econogit your analysis seems correct.
@nikpivkin @knqyf263 why do we limit ourselves to falling back only on temporary errors? https://github.com/aquasecurity/trivy/blob/main/pkg/oci/artifact.go#L257-L263