[Question]: Token indices sequence length is longer than the specified maximum sequence length for this model (614 > 512). Running this sequence through the model will result in indexing errors

[lifengyu2005]

Describe the issue

I use the following configuration, why is it throwing an error? I see a lot of 512 configurations in the llmlingua installation path. Do I need to retrain the model, or is it an issue with the llmlingua version?

self.model_compress = PromptCompressor(
model_name="/xxx/llmlingua/llmlingua-2-xlm-roberta-large-meetingbank",
use_llmlingua2=True, # Whether to use llmlingua-2
llmlingua2_config={
"max_batch_size": 100,
"max_force_token": 4096,
},
)

llmlingua ver 0.2.2

[iofu728]

Hi @lifengyu2005, thanks for your support. These logs appear to be warnings. Did your program crash because of these warnings? Please provide more details to help us identify the issue.

[cornzz]

@lifengyu2005 This warning comes from the tokenizer, not the model itself, you can reproduce this as shown below.
The model used in LLMLingua-2 can only handle input lengths of up to 512 tokens, the compressor divides the prompt into 512 token length chunks and compresses those chunks separately. So even when you see this warning, everything is working as intended.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("microsoft/llmlingua-2-xlm-roberta-large-meetingbank")
tokens = tokenizer.encode("Loooong prompt...")
# Output: Token indices sequence length is longer than the specified maximum sequence length for this model (1500 > 512). Running this sequence through the model will result in indexing errors