Docs should clarify how Claude Desktop treats resources

[jackdeansmith]

The docs state:

Resources are designed to be application-controlled, meaning that the client application can decide how and when they should be used.
For example, one application may require users to explicitly select resources, while another could automatically select them based on heuristics or even at the discretion of the AI model itself.

I don't think it's immediately clear how Claude Desktop treats resources. From playing around with it, it seems like it requires explicit user selection at least for now. I think it would be useful to make this explicit in the documentation so servers authors know what to expect when debugging.

[dsp-ant]

That sounds reasonable. We should make sure it reflects that Claude Desktop is purely an example. There are other MCP clients. Maybe, if you have a chance, would you give it a shot to make that change and send a PR? I'd much appreciate that.

[evalstate]

@dsp-ant. My testing so far shows:

• Prompts and Resources are only available in the first message of a chat, are added to the message as attachments.
• Resource Templates aren't exposed/understood by the Claude Desktop application.

Is that right? I'm referring to https://modelcontextprotocol.io/clients as well to set my expectations :)

(edit: Tool Results returned as Resources don't seem to pass the URI back to the Model Context; I assume this is a client implementation choice?)

(edit: **all messages can have attachments - I was confused by the UX moving the icon from the left to the right and the slide-out behaviour).

[burningion]

Just want to comment here and say that Tool Results should be able to pass URI's back to Model Context for inclusion, maybe via another prompt for permission?

[evalstate]

In the spec, Resource URI's are mandatory in Tool Responses.

https://github.com/modelcontextprotocol/specification/blob/bb5fdd282a4d0793822a569f573ebc36804d38f8/schema/schema.ts#L482-L509

The Client would receive that but it's not specified exactly how the Resource would be presented to the LLM.

The Server can suggest an Audience [ User | Assistant ] and a Priority for each Resource. At the moment I'm interpreting "User" as being "Display to User" and "Assistant" as "Include in Context" with none specified as "Both".

I think a common case may be the Server wanting to return a set of Resource Templates to the Client, so that the User or Assistant can get them on demand if wanted in the next turn. That's obviously not prohibited in the spec.

There's a similar case where the User may wish to upload a binary to send to a tool, for the result to go in context (e.g. Image -> OCR Tool -> Markdown). That's purely a Client implementation question. I'm tempted to tabulate these scenarios/options. There's also what Claude Desktop actually does vs. other Client implementations.

There's a couple of other discussions that seem relevant so I'll paste them here:
https://github.com/orgs/modelcontextprotocol/discussions/54
modelcontextprotocol/specification#90

[burningion]

Thanks, this is helpful context too.

Now that I've got Resources being returned from my API, I'd ideally have multi-modal instances of my Resource responses.

For example, right now I'm returning Video Files from a search, which are then query able, based upon a separate, analysis pipeline I've built happening in my API.

Ideally I'd be able to pass in a set of embedded images in responses like this:

And show thumbnails inline for each of these detected scenes.

Maybe I'm thinking of a Resource in the wrong way? Or maybe I should include a hierarchy of Resources in my response? IE imply that a set of Resources belong to a single video?

It seems the protocol specifies a single mimetype for each Resource:

[evalstate]

Claude Desktop as a client has limited support for any resource types beyond images. I'm doing quite a lot of testing here at the moment so will report back with what I learn... but planning to integrate with a different Client soon.

[cstroliadavis]

Not sure if it's because I'm on a Mac, but Claude Desktop does not seem to be aware that I have resources (template or otherwise, nor prompts). So far, all I've been able to get it to use or show are the tools. Let me know if I need to add this to a different ticket.

I have been able to get resources to work in roo-cline (and presumably cline), when I use sonnet 3.5 as the AI (other AIs -- I have not tested extensively, though -- seem to have 0 clue that these are there, but don't seem to have issues with tools). I have not yet tested prompts in roo-cline.

[burningion]

@cstroliadavis Claude Desktop does show resources for me. Here's some images for how to find it:

You've got to hover over the paperclip attachment, and then the MCP resources become available as an option. (The connection looking icon.)

From there you need to click choose an Integration:

Which then shows you Resources and Prompts that are availabe, sorted by MCP servers:

Whatever you pick then gets attached, similar to a file.

[cstroliadavis]

Ah, thanks. That was not intuitive for me, I guess.

Is that in the docs? I must have missed it.

[burningion]

No worries! It was confusing to me too, and I'm sure other people.

And no, it's not in the docs (yet).

[evalstate]

https://llmindset.co.uk/posts/2024/12/mcp-build-notes/ <-- I documented some of this stuff here. I'm probably going to follow up looking at resource handling in the protocol specifically (would that be helpful?)