[Bug]: LLMLingua-2 uses wrong special tokens by default

[cornzz]

Describe the bug

The TokenClfDataset is initialized without a model_name parameter and therefore defaults to bert-base-multilingual-cased, meaning that incorrect special tokens are used in llmlingua-2, i.e.

    if "bert-base-multilingual-cased" in model_name:
            self.cls_token = "[CLS]"
            self.sep_token = "[SEP]"
            self.unk_token = "[UNK]"
            self.pad_token = "[PAD]"
            self.mask_token = "[MASK]"

instead of

    elif "xlm-roberta-large" in model_name:
            self.bos_token = "<s>"
            self.eos_token = "</s>"
            self.sep_token = "</s>"
            self.cls_token = "<s>"
            self.unk_token = "<unk>"
            self.pad_token = "<pad>"
            self.mask_token = "<mask>"

The tokenizer simply treats these wrong special tokens (bos/eos/pad) as unknown tokens, I don't know what effect that has exactly. The difference in compression is not very significant, but there is some difference.

Steps to reproduce

Add print(tokenized_text) in line 57 in utils.py to see the wrong tokens used for the xlm-robert-large based compression model.

Expected Behavior

The correct special tokens should be used for the respective compression model.

Additional Information

• LLMLingua version: 0.2.2

[iofu728]

Hi @cornzz, thanks for your feedback. I will review this PR ASAP.

[cornzz]

Hi @iofu728, any update, did you get a chance to check if this finding was valid? 😅