[BFCL] proprietary_model deploied in local，what should i do？

[Marcher-lam]

If I deploy a proprietary model locally and use OSS model to access it, do I need to rewrite the file? Or can you reuse the proprietary_model Handler

[HuanzhiMao]

If your model is already privately hosted somewhere, something similar to the funcitonary handler should be enough. https://github.com/ShishirPatil/gorilla/blob/main/berkeley-function-call-leaderboard/bfcl/model_handler/proprietary_model/functionary.py

If you need the BFCL inference pipeline to spin up the vllm/sglang server to host the model, then you will need to implement a oss_model handler.

Don't worry about the naming of the proprietary_model handler vs oss_model handler; it's a bit confusing and will be addressed in #859. proprietary_model should mean API-based inference while oss_model means local-hosting-based inference.