version 0.2.1 iteration plan

[iofu728]

Estimated Release Date: 3/19
Release Manager: @suiguoxin
Schedule:

• Design Review: 3/19
• Coding: 3/10
• Testing: 3/19

Features

• P0 LLMLingua-2 @QianhuiWu

Backlog

• P1 exp: target comp ratio v.s. real comp ratio on specific data
• >token level, < sentence level, list different mappings and design interface P1 word level compression When I use chinese llama, the compressed prompt has garbled code #4
• P1 Support more / faster engines Support for llama.cpp or exl2 #41, including llama_cpp, FasterTransformer, vLLM ETA: TBD
  • survey which engines to support
• P2 Documentation and examples
  • Supported models and experiment results (with compressor throughput) after a faster engine supported

[iofu728]

v0.2.1 released.
move to v0.2.2.