Tracing Assistant Chat

[asheeshgarg]

In order to run tracing I was able to run the following.

runtime = SingleThreadedAgentRuntime(tracer_provider=configure_oltp_tracing())
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument()

When I was looking at the BaseGroupChat the SingleThreadedAgentRuntime() is not started with tracing. Is there a way to start tracing for AssitantAgent using some environment.

[jackgerrits]

Currently the runtime is not accessible for agentchat, we're planning on addressing this shortly in 0.4.1. The issue is here: #4075

[jackgerrits]

0.4 will be releasing this week. Don't yet have a timeline for 0.4.1 but we should be doing more regular releases going forward. The 0.4 one was big and tricky

[asheeshgarg]

Thanks for tracing the chat history are we planning to add some support to it. We can do save state but it will be ideal to add traces as message passing is going on.

[jackgerrits]

Tracing is available at the level of detail of the core messages. Are you looking for traces for the specific agentchat messages?

[asheeshgarg]

Yeah in order to understand what is the context in which certain results are produced by agents and any way to replay/optimize etc

[ekzhu]

The AgentChat uses the runtime for publish and send messages. So you can definitely trace those messages, although some of those messages will carry different schema from the messages that AgentChat produces at its API level -- they are more lower level and have to do with orchestration. You will also see more messages types.

Perhaps set up a filter to look only for a subset of messages that envelope the AgentChat message types.