Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Unable to pull the full SWE-Bench test set(0.19.0) #6136

Open
1 task done
jianzzz opened this issue Jan 8, 2025 · 1 comment
Open
1 task done

[Bug]: Unable to pull the full SWE-Bench test set(0.19.0) #6136

jianzzz opened this issue Jan 8, 2025 · 1 comment
Labels
bug Something isn't working evaluation Related to running evaluations with OpenHands

Comments

@jianzzz
Copy link

jianzzz commented Jan 8, 2025

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Describe the bug and reproduction steps

When trying to pull the full test set using:

evaluation/benchmarks/swe_bench/scripts/docker/pull_all_eval_docker.sh instance full

It fails with this error:

Using namespace: xingyaoww
Pulling docker images for [instance] level
Pattern: sweb.base\|sweb.env\|sweb.eval
Image file: evaluation/benchmarks/swe_bench/scripts/docker/all-swebench-full-instance-images.txt
...
Pulling xingyaoww/sweb.eval.x86_64.astropy_s_astropy-7746:latest into sweb.eval.x86_64.astropy_s_astropy-7746:latest
latest: Pulling from xingyaoww/sweb.eval.x86_64.astropy_s_astropy-7746
Digest: sha256:0b84adba522d3ac37fb8857de814d726dc5a330106797539afebcca455d2927b
Status: Image is up to date for xingyaoww/sweb.eval.x86_64.astropy_s_astropy-7746:latest
docker.io/xingyaoww/sweb.eval.x86_64.astropy_s_astropy-7746:latest
Pulling xingyaoww/sweb.env.x86_64.502d8fc6ebccd881244091:latest into sweb.env.x86_64.502d8fc6ebccd881244091:latest
Error response from daemon: pull access denied for xingyaoww/sweb.env.x86_64.502d8fc6ebccd881244091, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
(openhands) [root@VM-89-45-tencentos OpenHands]# docker pull sweb.env.x86_64.502d8fc6ebccd881244091:latest
Error response from daemon: pull access denied for sweb.env.x86_64.502d8fc6ebccd881244091, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

I have ensured I have logged in with docker.

OpenHands Installation

Docker command in README

OpenHands Version

0.19.0

Operating System

Linux

Logs, Errors, Screenshots, and Additional Context

No response

@jianzzz jianzzz added the bug Something isn't working label Jan 8, 2025
@mamoodi mamoodi added the evaluation Related to running evaluations with OpenHands label Jan 8, 2025
@mamoodi
Copy link
Collaborator

mamoodi commented Jan 8, 2025

CC @xingyaoww

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working evaluation Related to running evaluations with OpenHands
Projects
None yet
Development

No branches or pull requests

2 participants