Can we use the Engine (SRT Engine without HTTP server) while using choices method?

[sushah13]

Hi Team,

I am trying to implement a scoring method in my inference calls using SGLang's choices: https://sgl-project.github.io/frontend/choices_methods.html. Where I have a prompt (combined prompt for A + B) and I want to determine if B is a good pick for A. I want the model to return Yes or No.

In my python code, I am initializing Engine:
sgl.Engine(model_path=self.model_path, tokenizer_path=self.tokenizer_path)

At inference time, I am doing a prompt_function.run(). However, I get a "Please specify a backend" error. How do I use the Engine (or equivalent SRT engine which doesn't require me to spin up a HTTP server layer) ? How do I define the backend to use for these choices function calls? Or is there another way to define the choices via Engine