Skip to content

This provides sample app that shows the ASP.NET Core streaming API capability with Semantic Kernel

License

Notifications You must be signed in to change notification settings

devkimchi/semantic-kernel-streaming-api-sample

Repository files navigation

Semantic Kernel Streaming API Sample

This provides sample app that shows the ASP.NET Core streaming API capability with Semantic Kernel.

Prerequisites

Getting Started

  1. Get the Azure OpenAI access endpoint and key.

  2. Clone this repository.

    gh repo fork devkimchi/semantic-kernel-streaming-api-sample --clone
  3. Change the directory to the repository.

    cd semantic-kernel-streaming-api-sample
  4. Open src/SKStreaming.ApiApp/appsettings.json and update either the Azure:OpenAI section with your Azure OpenAI access endpoint and key or the GitHub:Models section with your GitHub PAT and Model ID.

    {
      "Azure": {
        "OpenAI": {
          "Endpoint": "{{AZURE_OPENAI_ENDPOINT}}",
          "ApiKey": "{{AZURE_OPENAI_API_KEY}}",
          "DeploymentName": "{{AZURE_OPENAI_DEPLOYMENT_NAME}}"
        }
      },
    
      "GitHub": {
        "Models": {
          "Endpoint": "https://models.inference.ai.azure.com",
          "GitHubToken": "{{GITHUB_PAT}}",
          "ModelId": "{{GITHUB_MODEL_ID}}"
        }
      }
    }

    NOTE: DO NOT commit this file back to the repository.

  5. In the same appsettings.json file, choose which LLM you will use. You can choose either aoai or github. As a default, it's set to github.

    {
      "SemanticKernel": {
        "ServiceId": "github"
      }
    }
  6. Build the sample apps.

    dotnet restore && dotnet build
  7. Run the API app.

    dotnet run --project ./src/SKStreaming.ApiApp
  8. Open a new terminal and run the console app.

    # Simple chat completion
    dotnet run --project ./src/SKStreaming.ConsoleApp -- -t chat
    
    # Streaming chat completion
    dotnet run --project ./src/SKStreaming.ConsoleApp -- -t chat-streaming

    When you're asked, enter any prompt you want to chat, and see the difference between simple chat completion and streaming chat completion.

  9. For auto function calling, run the console app with the bookings option.

    # Book a meeting room
    dotnet run --project ./src/SKStreaming.ConsoleApp -- -t bookings

    When you're asked, enter a prompt like "Please book a meeting room", and see the auto function calling in action.

About

This provides sample app that shows the ASP.NET Core streaming API capability with Semantic Kernel

Topics

Resources

License

Stars

Watchers

Forks