docs/user_guides/multi_config_api/multi_config_api.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Multi-config API\n",
    "\n",
    "This guide describes how to use multiple configurations as part of the same server API call. \n",
    "\n",
    "## Motivation\n",
    "\n",
    "When running a guardrails server, it is convenient to create *atomic configurations* which can be reused across multiple \"complete\" configurations. In this guide, we use [these example configurations](../../../examples/server_configs/atomic):\n",
    "1. `input_checking`: which uses the self-check input rail.\n",
    "2. `output_checking`: which uses the self-check output rail.\n",
    "3. `main`: which uses the `gpt-3.5-turbo-instruct` model with no guardrails. "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "# Get rid of the TOKENIZERS_PARALLELISM warning\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:15:47.277081Z",
     "start_time": "2024-02-27T13:15:47.274169Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Prerequisites\n",
    "\n",
    "1. Install the `openai` package:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "outputs": [],
   "source": [
    "!pip install openai"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "2. Set the `OPENAI_API_KEY` environment variable:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [],
   "source": [
    "!export OPENAI_API_KEY=$OPENAI_API_KEY    # Replace with your own key"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:15:54.140879Z",
     "start_time": "2024-02-27T13:15:54.028776Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "3. If you're running this inside a notebook, patch the AsyncIO loop."
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:22:09.852260Z",
     "start_time": "2024-02-27T13:22:09.846303Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Setup\n",
    "\n",
    "In this guide, the server is started programmatically, as shown below. This is equivalent to (from the root of the project):\n",
    "\n",
    "```bash\n",
    "nemoguardrails server --config=examples/server_configs/atomic\n",
    "```"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [],
   "source": [
    "import os\n",
    "from nemoguardrails.server.api import app\n",
    "from threading import Thread\n",
    "import uvicorn\n",
    "\n",
    "def run_server():\n",
    "    current_path = %pwd \n",
    "    app.rails_config_path = os.path.normpath(os.path.join(current_path, \"..\", \"..\", \"..\", \"examples\", \"server_configs\", \"atomic\"))\n",
    "    \n",
    "    uvicorn.run(app, host=\"127.0.0.1\", port=8000, log_level=\"info\")\n",
    "\n",
    "# Start the server in a separate thread so that you can still use the notebook\n",
    "thread = Thread(target=run_server)\n",
    "thread.start()"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:22:13.519377Z",
     "start_time": "2024-02-27T13:22:11.291463Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can check the available configurations using the `/v1/rails/configs` endpoint:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'id': 'output_checking'}, {'id': 'main'}, {'id': 'input_checking'}]\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "\n",
    "base_url = \"http://127.0.0.1:8000\"\n",
    "\n",
    "response = requests.get(f\"{base_url}/v1/rails/configs\")\n",
    "print(response.json())"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:25:33.220071Z",
     "start_time": "2024-02-27T13:25:33.213609Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "You can make a call using a single config as shown below: "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [
    {
     "data": {
      "text/plain": "Fetching 7 files:   0%|          | 0/7 [00:00<?, ?it/s]",
      "application/vnd.jupyter.widget-view+json": {
       "version_major": 2,
       "version_minor": 0,
       "model_id": "61d861c7936e46989c33d9b038653753"
      }
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [{'role': 'assistant', 'content': 'I apologize if I have given you that impression. I am an AI assistant designed to assist and provide information. Is there something specific you would like me to help you with?'}]}\n"
     ]
    }
   ],
   "source": [
    "response = requests.post(f\"{base_url}/v1/chat/completions\", json={\n",
    "  \"config_id\": \"main\",\n",
    "  \"messages\": [{\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"You are stupid.\"\n",
    "  }]\n",
    "})\n",
    "print(response.json())"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:25:37.759668Z",
     "start_time": "2024-02-27T13:25:35.146250Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "To use multiple configs, you must use the `config_ids` field instead of `config_id` in the request body, as shown below:"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'messages': [{'role': 'assistant', 'content': \"I'm sorry, I can't respond to that.\"}]}\n"
     ]
    }
   ],
   "source": [
    "response = requests.post(f\"{base_url}/v1/chat/completions\", json={\n",
    "  \"config_ids\": [\"main\", \"input_checking\"],\n",
    "  \"messages\": [{\n",
    "    \"role\": \"user\",\n",
    "    \"content\": \"You are stupid.\"\n",
    "  }]\n",
    "})\n",
    "print(response.json())"
   ],
   "metadata": {
    "collapsed": false,
    "ExecuteTime": {
     "end_time": "2024-02-27T13:26:20.861796Z",
     "start_time": "2024-02-27T13:26:20.119092Z"
    }
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "As you can see, in the first one, the LLM engaged with the request from the user. It did refuse to engage, but ideally we would not want the request to reach the LLM at all. In the second call, the input rail kicked in and blocked the request. "
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Conclusion\n",
    "\n",
    "This guide showed how to make requests to a guardrails server using multiple configuration ids. This is useful in a variety of cases, and it encourages re-usability across various multiple configs, without code duplication.  "
   ],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}