Autogen with Huggingface Llama 3.1 70B does not work with tool calling. Autogen Studio works however.

[redpanda77]

I'm running code with two agents. Proxy Agent and an agent with a tool to write text files. When using Llama 3.1 70B and the python code I get an error related to the use of the tool. When used exactly the same code for the tool/skill in Autogen Studio it worked perfectly.

On jupyter I get this error:

***** Response from calling tool (0) *****
Error: Expecting value: line 1 column 1 (char 0)
The argument must be in JSON format.

---

I tried the same python code with the gpt-4-turbo model instead of llama 3.1 70B and it runs perfectly.

Why does this happen? What's the difference between Autogen library imports to a python file and autogen studio?

[rlam3]

@redpanda77 did you find a workaround to use huggingface with autogen

[ekzhu]

We will release v0.4 this week. See migration guide: https://aka.ms/autogen-migrate.

To use hugging face, you might want to start a local proxy server like LiteLLM. Or just use Ollama instead.