Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(plugin-ai): Adding command 'ai:models:call' #21

Merged
merged 5 commits into from
Sep 28, 2024
Merged

Conversation

sbosio
Copy link
Contributor

@sbosio sbosio commented Sep 28, 2024

Description

This PR implements the ai:models:call command with tests somewhat according to the design outlined in the UX design doc, the MIA API reference doc and discussions around the behavior for that command in the following Slack threads:

How to test

Prepare your environment for testing

  • Fetch this branch.
  • Run yarn && yarn build.
  • Set the HEROKU_INFERENCE_ADDON to the production canary add-on via export HEROKU_INFERENCE_ADDON="inference-staging"
  • Set the HEROKU_INFERENCE_HOST to the production canary add-on default host via export HEROKU_INFERENCE_HOST="staging.inference.herokai.com"
  • Create a test app: heroku apps:create test-cli-plugin-ai

Actual testing

  • Create a chat completion model resource: ./bin/run ai:models:create claude-3-sonnet -a test-cli-plugin-ai --as completion
  • Create an image generation model resource: ./bin/run ai:models:create stable-diffusion-xl -a test-cli-plugin-ai --as image
  • Create an embeddings generation model resource: ./bin/run ai:models:create cohere-embed-multilingual -a test-cli-plugin-ai --as embeddings
  • Verify that help looks ok: ./bin/run ai:models:call --help. Keep in mind that more option flags were added where appropriate, and it will differ from the design doc.
  • Generate a chat completion with: ./bin/run ai:models:call completion -a test-cli-plugin-ai -p "<your prompt here>". The command should succeed and get a response for your prompt.
  • Repeat the previous command with different combinations of flags --json and --output option. When using --json the full JSON response should be displayed or redirected to the output file when using --output.
  • Generate an image with: ./bin/run ai:models:call image -a test-cli-plugin-ai -p "<your prompt here>" --opts='{"response_format":"base64"}'. The command should succeed and a Base64-encoded image content should be written to the standard outpu.
  • Repeat the previous command redirecting the output to a file with ./bin/run ai:models:call image -a test-cli-plugin-ai -p "<your prompt here>" --opts='{"response_format":"base64"}' --output=/tmp/output-image.png. Verify that the command succeeds and when you run open /tmp/output-image.png you should see the generated image.
  • Repeat the previous command but requesting JSON output with ./bin/run ai:models:call image -a test-cli-plugin-ai -p "<your prompt here>" --opts='{"response_format":"base64"}' --output=/tmp/output-image.json. Verify that the command succeeds and when you run open /tmp/output-image.json you should see the full JSON response.
  • Generate an embedding with: ./bin/run ai:models:call embeddings -a test-cli-plugin-ai -p "<your prompt here>". The command should succeed and a comma-separated list of float numbers (embedding vector) should be displayed.
  • Repeat the previous command with different combinations of flags --json and --output option. When using --json the full JSON response should be displayed or redirected to the output file when using --output.
  • Remove your test app to get rid of all add-ons: heroku apps:destroy -a test-cli-plugin-ai

SOC2 Compliance

GUS Work Item

@sbosio sbosio requested a review from a team as a code owner September 28, 2024 00:02
@sbosio sbosio changed the title Sbosio/ai models call feat(plugin-ai): Adding command 'ai:models:call' Sep 28, 2024
Copy link
Contributor

@zwhitfield3 zwhitfield3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is absolutely fantastic! Great work @sbosio!

@sbosio sbosio merged commit 01d0207 into main Sep 28, 2024
6 checks passed
@sbosio sbosio deleted the sbosio/ai-models-call branch September 28, 2024 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants