Coordinates are all over the place for local testing, but works perfectly on your gradio webapp

[pentatonictritones]

Hi, excited to see this performance.

The gradio space has very accurate performance
https://huggingface.co/spaces/Aheader/gui_test_app

So i tried running it locally but the coordinate boxes once i scale it by the image width and height is just all over the place. I get very poor results locally.

Could you share more details about the gradio implementation?

I tried using both the prompt examples in your docs but it is awful compared to the webapp - which gives perfect results everytime

Thanks!!

[pentatonictritones]

For reference, my user prompt is

"Click the edit button"

Locally:

On gradio:

[AHEADer]

Hi
We deploy a cloud server following the deployment guide in readme. The space app just make requests to the server, you can check the app's source code to check the prompt. The prompt should be "Output only the coordinate of one box in your response. "

[liy9426]

Same here. Is there any difference deploying locally? I did the same instructions as in the guide but it outputs different results every time. Thanks

[pentatonictritones]

Hi, thanks, i checked out the notion file but unfortunately at least with the 2b and 7b model, i get very poor results locally

I've tried running it locally with all different frameworks and the methods but the coordinates continue to have 0% accuracy while the gradio model is very accurate.

Also, i tried out many different prompts, but it definitely isn't the issue

It would be great if anyone has any success running this locally.

[JjjFangg]

Have you tried converting the output coordinates using the transformation described in Coordinate Mapping Section

[pentatonictritones]

Have you tried converting the output coordinates using the transformation described in Coordinate Mapping Section

hi!

Yes. I am certain it is not a coordinate conversion issue, because if i prompt it 5 times, i get 5 vastly different coordinates

[AHEADer]

Hi, thanks, i checked out the notion file but unfortunately at least with the 2b and 7b model, i get very poor results locally

I've tried running it locally with all different frameworks and the methods but the coordinates continue to have 0% accuracy while the gradio model is very accurate.

Also, i tried out many different prompts, but it definitely isn't the issue

It would be great if anyone has any success running this locally.

Are you using Ollama? Or vLLM?

[pentatonictritones]

Hi, thanks, i checked out the notion file but unfortunately at least with the 2b and 7b model, i get very poor results locally
I've tried running it locally with all different frameworks and the methods but the coordinates continue to have 0% accuracy while the gradio model is very accurate.
Also, i tried out many different prompts, but it definitely isn't the issue
It would be great if anyone has any success running this locally.

Are you using Ollama? Or vLLM?

ollama as the doc suggested, as well as transformers

[liy9426]

Also noticed that it outputs some texts that are definitely not in the pic I provide. Wondering what's the reason for that

[JjjFangg]

It seems that the issue might be related to the precision of the model deployed with Ollama. We are currently working on optimizing it. In the meantime, you can prioritize testing with vLLM or the cloud deployment.

[pentatonictritones]

It seems that the issue might be related to the precision of the model deployed with Ollama. We are currently working on optimizing it. In the meantime, you can prioritize testing with vLLM or the cloud deployment.

thanks