Skip to content

Commit

Permalink
Groq Cloud API looks to match OpenAI
Browse files Browse the repository at this point in the history
  • Loading branch information
drnic committed Apr 20, 2024
1 parent ee52a00 commit 1d40341
Showing 1 changed file with 34 additions and 1 deletion.
35 changes: 34 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,33 @@ Groq Cloud runs LLM models fast and cheap. Llama 3, Mixtrel, Gemma, and more at

Speed and pricing at 2024-04-21.

You can interact with their API using any Ruby HTTP library by following their documentation at <https://console.groq.com/docs/quickstart>.
## Groq Cloud API

You can interact with their API using any Ruby HTTP library by following their documentation at <https://console.groq.com/docs/quickstart>

The Groq Cloud API looks to be copying a subset of the OpenAI API. For example, you perform chat completions at `https://api.groq.com/openai/v1/chat/completions` with the same POST body schema as OpenAI. The Tools support looks to have the same schema for defining tools/functions.

So you can write your own Ruby client code to interact with the Groq Cloud API.

Or you can use this convenience RubyGem with some nice helpers to get you started.

```ruby
client = Groq::Client.new
client.chat("Hello, world!")
=> {"role"=>"assistant", "content"=>"Hello there! It's great to meet you!"}

include Groq::Helpers
client.chat([
U("Hi"),
A("Hello back. Ask me anything. I'll reply with 'cat'"),
U("Favourite food?")
])
# => {"role"=>"assistant", "content"=>"Um... CAT"}
# => {"role"=>"assistant", "content"=>"Not a cat! It's a pizza!"}
# => {"role"=>"assistant", "content"=>"Pizza"}
# => {"role"=>"assistant", "content"=>"Cat"}
```

## Installation

Install the gem and add to the application's Gemfile by executing:
Expand Down Expand Up @@ -129,6 +152,16 @@ Assistant reply with model gemma-7b-it:
{"role"=>"assistant", "content"=>"Hello to you too! 👋🌎 It's great to hear from you. What would you like to talk about today? 😊"}
```

### Tools (coming soon)

LLMs are increasingly supporting deferring to tools or functions to fetch data, perform calculations, or store structured data. Groq Cloud in turn then supports their tool implementations through its API.

See the [Using Tools](https://console.groq.com/docs/tool-use) documentation for the list of models that currently support tools.

```ruby
client = Groq::Client.new(model_id: "mixtral-8x7b")
```

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
Expand Down

0 comments on commit 1d40341

Please sign in to comment.