Skip to content
This repository has been archived by the owner on Nov 13, 2024. It is now read-only.

Commit

Permalink
docs: Qdrant reference library.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Anush008 committed Mar 12, 2024
1 parent d239e12 commit 6a35ae5
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 3 deletions.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Canopy has two flows: knowledge base creation and chat. In the knowledge base cr
1. **Canopy Core Library** - The library has 3 main classes that are responsible for different parts of the RAG workflow:
* **ChatEngine** - Exposes a chat interface to interact with your data. Given the history of chat messages, the `ChatEngine` formulates relevant queries to the `ContextEngine`, then uses the LLM to generate a knowledgeable response.
* **ContextEngine** - Performs the “retrieval” part of RAG. The `ContextEngine` utilizes the underlying `KnowledgeBase` to retrieve the most relevant documents, then formulates a coherent textual context to be used as a prompt for the LLM.
* **KnowledgeBase** - Manages your data for the RAG workflow. It automatically chunks and transforms your text data into text embeddings, storing them in a Pinecone vector database. Given a text query - the `KnowledgeBase` will retrieve the most relevant document chunks from the database.
* **KnowledgeBase** - Manages your data for the RAG workflow. It automatically chunks and transforms your text data into text embeddings, storing them in a Pinecone(Default)/Qdrant vector database. Given a text query - the knowledge base will retrieve the most relevant document chunks from the database.


> More information about the Core Library usage can be found in the [Library Documentation](docs/library.md)
Expand Down Expand Up @@ -67,11 +67,12 @@ pip install canopy-sdk
### Extras

| Name | Description |
|----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `grpc` | To unlock some performance improvements by working with the GRPC version of the [Pinecone Client](https://github.com/pinecone-io/pinecone-python-client) |
| `torch` | To enable embeddings provided by [sentence-transformers](https://www.sbert.net/) |
| `transformers` | If you are using Anyscale LLMs, it's recommended to use `LLamaTokenizer` tokenizer which requires transformers as dependency |
| `cohere` | To use Cohere reranker or/and Cohere LLM |
| `qdrant` | To use [Qdrant](http://qdrant.tech/) as an alternate knowledge base |

</details>

Expand Down
34 changes: 33 additions & 1 deletion docs/library.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,39 @@ You can always verify the connection to the Pinecone index with the `verify_inde
kb.verify_index_connection()
```

To learn more about customizing the KnowledgeBase and its inner components, see [understanding knowledgebase workings section](#understanding-knowledgebase-workings).
#### Using Qdrant as a knowledge base

Canopy supports [Qdrant](https://qdrant.tech) as an alternative knowledge base. To use Qdrant with Canopy, install the `qdrant` extra.

```bash
pip install canopy-sdk[qdrant]
```

The Qdrant knowledge base is accessible via the `QdrantKnowledgeBase` class.

```python
from canopy.knowledge_base import QdrantKnowledgeBase

kb = QdrantKnowledgeBase(collection_name="<YOUR_COLLECTION_NAME>")
```

The constructor accepts additional [options](https://github.com/qdrant/qdrant-client/blob/eda201a1dbf1bbc67415f8437a5619f6f83e8ac6/qdrant_client/qdrant_client.py#L36-L61) to customize your connection to Qdrant.

To create a new Qdrant collection and connect it to the knowledge base, use the `create_canopy_collection` method:

```python
kb.create_canopy_collection()
```

The method accepts additional [options](https://github.com/qdrant/qdrant-client/blob/c63c62e6df9763591622d1921b3dfcc486666481/qdrant_client/qdrant_remote.py#L2137-L2150) to configure the collection to be created.

You can always verify the connection to the collection with the `verify_index_connection` method:

```python
kb.verify_index_connection()
```

To learn more about customizing the KnowledgeBase and its inner components, see [understanding knowledge ebase workings section](#understanding-knowledgebase-workings).

### Step 3: Upsert and query data

Expand Down

0 comments on commit 6a35ae5

Please sign in to comment.