Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract model outputs into a Result class #58

Merged
merged 38 commits into from
Nov 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
624020d
rename utils to optional imports
dnth Nov 8, 2024
2e243ff
format
dnth Nov 8, 2024
9f3de05
create a types file and move data types into this
dnth Nov 8, 2024
ad0153d
use results in timm model infer
dnth Nov 8, 2024
4a2c378
apply to batch inference
dnth Nov 8, 2024
7c40b9f
simplify timm class by using parse images
dnth Nov 8, 2024
068b8ea
update test serve
dnth Nov 8, 2024
bfed4e1
add test for timm
dnth Nov 8, 2024
24004b3
update moondream to use Result class
dnth Nov 8, 2024
00e0f83
update moondream test
dnth Nov 8, 2024
b33412b
hide warning from huggingface
dnth Nov 8, 2024
d3c6d20
update vision2seq model to use Result class
dnth Nov 8, 2024
226a2c1
update llama32
dnth Nov 8, 2024
dc68df4
update
dnth Nov 8, 2024
862fe43
update
dnth Nov 8, 2024
dafbb9d
autocast
dnth Nov 8, 2024
3b28a2c
autocast
dnth Nov 8, 2024
b7c753d
update joy caption to use Result class output
dnth Nov 8, 2024
cc062a2
update florence model
dnth Nov 8, 2024
72daf0e
update florence test
dnth Nov 8, 2024
ecf842e
update ollama
dnth Nov 8, 2024
7d2118d
update molmo
dnth Nov 8, 2024
798af84
update phi3
dnth Nov 8, 2024
dc3d506
fix yolo cls to use Result
dnth Nov 8, 2024
0631ca1
use Result in detection model
dnth Nov 8, 2024
830e621
apply to mask
dnth Nov 8, 2024
2ff6f7e
apply to pose estimation
dnth Nov 9, 2024
d11d985
update testing
dnth Nov 9, 2024
5dec60d
clean up
dnth Nov 9, 2024
aa91ff9
update nbs
dnth Nov 9, 2024
1535b1f
joycaption to use parse images
dnth Nov 9, 2024
f718bc6
update notebooks
dnth Nov 9, 2024
90b3a88
update gradio demo
dnth Nov 9, 2024
19d6daa
update nb
dnth Nov 9, 2024
0527784
clean up
dnth Nov 9, 2024
c043644
update readme prompt -> text
dnth Nov 9, 2024
96ba1ce
update quickstart
dnth Nov 9, 2024
b8633f6
BaseModel to BaseXinferModel
dnth Nov 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ repos:
# Run the linter.
- id: ruff
types_or: [ python, pyi ]
args: [ --fix, --ignore=F403,F405 ]
args: [ --fix, --ignore=F401,F403,F405 ]
# Run the formatter.
- id: ruff-format
types_or: [ python, pyi ]
types_or: [ python, pyi ]

10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ model.infer_batch(images, prompts) # Run batch inference
model.launch_gradio() # Launch Gradio interface
```

Have a custom model? Create a class that implements the `BaseModel` interface and register it with x.infer. See [Add Your Own Model](#add-your-own-model) for more details.
Have a custom model? Create a class that implements the `BaseXInferModel` interface and register it with x.infer. See [Add Your Own Model](#add-your-own-model) for more details.

## 🚀 Quickstart

Expand Down Expand Up @@ -273,7 +273,7 @@ curl -X 'POST' \
-H 'Content-Type: application/json' \
-d '{
"image": "https://raw.githubusercontent.com/dnth/x.infer/main/assets/demo/00aa2580828a9009.jpg",
"infer_kwargs": {"prompt": "Caption this image"}
"infer_kwargs": {"text": "Caption this image"}
}'
```

Expand All @@ -290,7 +290,7 @@ headers = {
payload = {
"image": "https://raw.githubusercontent.com/dnth/x.infer/main/assets/demo/00aa2580828a9009.jpg",
"infer_kwargs": {
"prompt": "Caption this image"
"text": "Caption this image"
}
}

Expand All @@ -300,7 +300,7 @@ print(response.json())

### Add Your Own Model

+ **Step 1:** Create a new model class that implements the `BaseModel` interface.
+ **Step 1:** Create a new model class that implements the `BaseXInferModel` interface.

+ **Step 2:** Implement the required abstract methods `load_model`, `infer`, and `infer_batch`.

Expand All @@ -309,7 +309,7 @@ print(response.json())
For example:
```python
@register_model("my-model", "custom", ModelInputOutput.IMAGE_TEXT_TO_TEXT)
class MyModel(BaseModel):
class MyModel(BaseXInferModel):
def load_model(self):
# Load your model here
pass
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ model.infer_batch(images, prompts) # Run batch inference
model.launch_gradio() # Launch Gradio interface
```

Have a custom model? Create a class that implements the `BaseModel` interface and register it with x.infer. See [Add Your Own Model](#add-your-own-model) for more details.
Have a custom model? Create a class that implements the `BaseXInferModel` interface and register it with x.infer. See [Add Your Own Model](#add-your-own-model) for more details.

## 🌟 Key Features
<div align="center">
Expand Down
143 changes: 51 additions & 92 deletions nbs/florence-2.ipynb

Large diffs are not rendered by default.

41 changes: 2 additions & 39 deletions nbs/gradio_demo.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,46 +2,9 @@
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"* Running on local URL: http://127.0.0.1:7860\n",
"\n",
"To create a public link, set `share=True` in `launch()`.\n"
]
},
{
"data": {
"text/html": [
"<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"1000\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m2024-10-25 23:54:24.405\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m63\u001b[0m - \u001b[1mModel: bat_resnext26ts.ch_in1k\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:24.409\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m64\u001b[0m - \u001b[1mDevice: cuda\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:24.410\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m65\u001b[0m - \u001b[1mDtype: float16\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:27.348\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m63\u001b[0m - \u001b[1mModel: bat_resnext26ts.ch_in1k\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:27.350\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m64\u001b[0m - \u001b[1mDevice: cuda\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:27.352\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m65\u001b[0m - \u001b[1mDtype: float16\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:38.268\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m63\u001b[0m - \u001b[1mModel: bat_resnext26ts.ch_in1k\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:38.270\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m64\u001b[0m - \u001b[1mDevice: cuda\u001b[0m\n",
"\u001b[32m2024-10-25 23:54:38.271\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mxinfer.models\u001b[0m:\u001b[36m__init__\u001b[0m:\u001b[36m65\u001b[0m - \u001b[1mDtype: float16\u001b[0m\n"
]
}
],
"outputs": [],
"source": [
"import xinfer\n",
"\n",
Expand Down
Loading
Loading