Skip to content

Commit

Permalink
docs: Superduper integration (#1179)
Browse files Browse the repository at this point in the history
* docs: Superduper integration

* Update _index.md
  • Loading branch information
Anush008 authored Nov 27, 2024
1 parent 728e9c6 commit e06d364
Show file tree
Hide file tree
Showing 2 changed files with 88 additions and 0 deletions.
1 change: 1 addition & 0 deletions qdrant-landing/content/documentation/frameworks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ partition: build
| [Rig-rs](/documentation/frameworks/rig-rs/) | Rust library for building scalable, modular, and ergonomic LLM-powered applications. |
| [Semantic Router](/documentation/frameworks/semantic-router/) | Python library to build a decision-making layer for AI applications using vector search. |
| [Spring AI](/documentation/frameworks/spring-ai/) | Java AI framework for building with Spring design principles such as portability and modular design. |
| [Superduper](/documentation/frameworks/superduper/) | Framework for building flexible, compositional AI apps which may be applied directly to databases. |
| [Swarm](/documentation/frameworks/swarm/) | Python framework for managing multiple AI agents that can work together. |
| [Sycamore](/documentation/frameworks/sycamore/) | Document processing engine for ETL, RAG, LLM-based applications, and analytics on unstructured data. |
| [Testcontainers](/documentation/frameworks/testcontainers/) | Framework for providing throwaway, lightweight instances of systems for testing |
Expand Down
87 changes: 87 additions & 0 deletions qdrant-landing/content/documentation/frameworks/superduper.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
title: Superduper
---

# Superduper

[Superduper](https://superduper.io/) is a framework for building flexible, compositional AI applications which may be applied directly to databases using a declarative programming model. These applications declare and maintain a desired state of the database, and use the database directly to store outputs of AI components, meta-data about the components and data pertaining to the state of the system.

Qdrant is available as a vector search provider in Superduper.

## Installation

```bash
pip install superduper-framework
```

## Setup

- To use Qdrant for vector search layer, create a `settings.yaml` file with the following config.

> settings.yaml
```yaml
cluster:
vector_search:
type: qdrant

vector_search_kwargs:
url: "http://localhost:6333"
api_key: "<YOUR_API_KEY>
# Supports all parameters of qdrant_client.QdrantClient
```
- Set the `SUPERDUPER_CONFIG` env value to the path of the config file.
```bash
export SUPERDUPER_CONFIG=path/to/settings.yaml
```
That's all. You can now use Superduper backed by Qdrant.
## Example
Here's an example to run vector search using the configured Qdrant index.
```python
import json
import requests
from superduper import superduper, Document
from superduper.ext.sentence_transformers import SentenceTransformer
r = requests.get('https://superduperdb-public-demo.s3.amazonaws.com/text.json')
with open('text.json', 'wb') as f:
f.write(r.content)
with open('text.json', 'r') as f:
data = json.load(f)
db = superduper('mongomock://test')
_ = db['documents'].insert_many([Document({'txt': txt}) for txt in data]).execute()
model = SentenceTransformer(
identifier="test",
predict_kwargs={"show_progress_bar": True},
model="all-MiniLM-L6-v2",
device="cpu",
postprocess=lambda x: x.tolist(),
)

vector_index = model.to_vector_index(select=db['documents'].find(), key='txt')

db.apply(vector_index)

query = db['documents'].like({'txt': 'Tell me about vector-search'}, vector_index=vector_index.identifier, n=3).find()
cursor = query.execute()

for r in cursor:
print('=' * 100)
print(r.unpack()['txt'])
print('=' * 100)
```
## 📚 Further Reading
- Superduper [Intro](https://docs.superduper.io/docs/intro)
- Superduper [Source](https://github.com/superduper-io/superduper)

0 comments on commit e06d364

Please sign in to comment.