Skip to content

Commit

Permalink
[vLLM Snippet] Fix escaping (#952)
Browse files Browse the repository at this point in the history
Follow up to #693

There was trailing space in vlmm snippet which made the snippet
unusable. When you try the snippet, vllm server would respond with
error:
```
mishig@machine:~$ # Call the server using curl:
curl -X POST "http://localhost:8000/v1/chat/completions" \ 
        -H "Content-Type: application/json" \ 
        --data '{
                "model": "meta-llama/Llama-3.2-3B-Instruct",
                "messages": [
                        {"role": "user", "content": "Hello!"}
                ]
        }'

{"object":"error","message":"[{'type': 'missing', 'loc': ('body',), 'msg': 'Field required', 'input': None}]","type":"BadRequestError","param":null,"code":400}curl: (3) URL using bad/illegal format or missing URL
-H: command not found
--data: command not found
mishig@machine:~$ 
```

Explanation:  trailing space was breaking the escaping
  • Loading branch information
mishig25 authored Oct 7, 2024
1 parent 4e7a89a commit ada8548
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions packages/tasks/src/local-apps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -145,8 +145,8 @@ const snippetLocalAI = (model: ModelData, filepath?: string): LocalAppSnippet[]
const snippetVllm = (model: ModelData): LocalAppSnippet[] => {
const runCommand = [
"# Call the server using curl:",
`curl -X POST "http://localhost:8000/v1/chat/completions" \\ `,
` -H "Content-Type: application/json" \\ `,
`curl -X POST "http://localhost:8000/v1/chat/completions" \\`,
` -H "Content-Type: application/json" \\`,
` --data '{`,
` "model": "${model.id}",`,
` "messages": [`,
Expand Down

0 comments on commit ada8548

Please sign in to comment.