Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue]: When we generate reports using LLM, how do we avoid the reports carrying LLM's own data? #1347

Open
1 of 3 tasks
mggger opened this issue Nov 4, 2024 · 1 comment
Labels
triage Default label assignment, indicates new issue needs reviewed by a maintainer

Comments

@mggger
Copy link
Contributor

mggger commented Nov 4, 2024

Do you need to file an issue?

  • I have searched the existing issues and this bug is not already filed.
  • My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
  • I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the issue

The current report data generated by LLM includes information from both indexed and non-indexed files. Even when additional instructions are added to the prompt, the impact is still not significant.

Remember:

  • Every single statement must be explicitly supported by input data
  • Include exact quotes in the validation section
  • Track coverage of statements and their support
  • If information isn't in the input data, don't include it
  • Flag any statement that can't be directly tied to source data

Steps to reproduce

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

  • GraphRAG Version:
  • Operating System:
  • Python Version:
  • Related Issues:
@mggger mggger added the triage Default label assignment, indicates new issue needs reviewed by a maintainer label Nov 4, 2024
@Deng-Xian-Sheng
Copy link

You use what LLM?

#1543

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
triage Default label assignment, indicates new issue needs reviewed by a maintainer
Projects
None yet
Development

No branches or pull requests

2 participants