[Question] Memory deleted at the end of the request

[JanHuege]

Hi,

I am currently experimenting with the quarkus langchain4j features and I was wondering why the memory is not working. So I decided to implement a RedisMemoryStore myself. Now it seems to be used but after each request all messages are deleted since quarkus seems to call the deleteMessages Method on my Implementation of ChatMemoryStore

@Default
@ApplicationScoped
public class RedisMemoryStore implements ChatMemoryStore {

    private final ValueCommands<String, List<ChatMessage>> valueCommands;
    private final KeyCommands<String> keyCommands;

    public RedisMemoryStore(RedisDataSource redisDataSource) {
        this.valueCommands = redisDataSource.value(new TypeReference<List<ChatMessage>>(){});
        this.keyCommands = redisDataSource.key();
    }

    @Override
    public List<ChatMessage> getMessages(Object memoryId) {
        if(keyCommands.exists(memoryId.toString())){
            return valueCommands.get(memoryId.toString());
        }
        {
            return Collections.emptyList();
        }
    }

    @Override
    public void updateMessages(Object memoryId, List<ChatMessage> messages) {
        valueCommands.set(memoryId.toString(), messages);
    }

    @Override
    public void deleteMessages(Object memoryId) {
        keyCommands.del(memoryId.toString());
    }

}

Behavior:

1. all the RAG functionality is run
2. deleteMessages with the specified MemoryId is called
3. updateMessages is called for all the messages (systemmessage, query, retrival, tools...)
4. deleteMessages is called again resulting in the deletion of all messages in redis

Can someone explain this behavior?

[geoand]

If the AiService is request scoped (the default), we delete the memory when the request is completed. The docs provide more information

[JanHuege]

But the AiService is an interface. I do not understand what the docs are trying to tell me here.

I cannot

@ApplicationScoped
@RegisterAiService(
        tools = { ExampleTool.class },
        retrievalAugmentor = ExampleRetriever.class
)
@SystemMessage("...")
public interface Bot {
    String chat(@MemoryId String memoryId, @UserMessage String question);
}

[geoand]

Adding @ApplicationScoped on an AiService works just fine

[JanHuege]

Ok then IntelliJ has a bug here right?

But now I am running into this:

How to prevent the AIService from trying to execute tools that are not there?

Also why does a toolmessage appear if all its properties are null?

[geoand]

Ok then IntelliJ has a bug here right?

It's just a warning, you can safely ignore it.

As for the tools issue, I would need to have a reproducer that I can in order to be able to give you an answer

[JanHuege]

https://github.com/JanHuege/MinimumReproTool/tree/main

I added a few .stream.filters to remove the empty entries. What I did is I ingested a couple of .pdfs and then ran a few requests using llama 3.2 locally.

I used a persistent redis and llama on my local docker instead if using the devservices. I hope this helps.

[JanHuege]

I was expecting you to use the langchain4j ollama package internally but it seems like this is not the case.

The code expects all tool-calls from the LLM response are actual tools that are available. I guess langchain4j ollama should filter these out. Also if no tool is called this should not result in empty ToolExecutionRequests I guess.

https://github.dev/quarkiverse/quarkus-langchain4j/blob/c7c75610176de7406953c075661010dffb7c362e/model-providers/ollama/runtime/src/main/java/io/quarkiverse/langchain4j/ollama/OllamaChatLanguageModel.java#L182

I think this issue also exists in langchain4j:

toToolExecutionRequests(chatResponse.getMessage().getToolCalls())

https://github.dev/langchain4j/langchain4j/blob/64ca980d63b438ef9e9bea65f26330459c881417/langchain4j-ollama/src/main/java/dev/langchain4j/model/ollama/OllamaChatModel.java#L172

[geoand]

Nice catch!

Would you like to provide a fix for this?

[JanHuege]

I am not sure if I understand this well enough at the moment since I just started trying it out. But maybe I'll try and look into it tomorrow no guarantees.

[JanHuege]

Created a PR @geoand . Feel free to critique.