Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
dnth committed Oct 24, 2024
1 parent 9d02602 commit 656b628
Showing 1 changed file with 9 additions and 9 deletions.
18 changes: 9 additions & 9 deletions docs/examples/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ This is the <code>init_notebook_mode</code> cell from ITables v2.2.2<br>



<table id="itables_a97359bf_88ed_49b6_a617_9c6d5e62e7d9" class="display nowrap" data-quarto-disable-processing="true" style="table-layout:auto;width:auto;margin:auto;caption-side:bottom">
<table id="itables_9467fa67_b8b4_4ac0_a402_03f795e9781a" class="display nowrap" data-quarto-disable-processing="true" style="table-layout:auto;width:auto;margin:auto;caption-side:bottom">
<thead>
<tr style="text-align: right;">

Expand Down Expand Up @@ -329,7 +329,7 @@ Loading ITables v2.2.2 from the <code>init_notebook_mode</code> cell...
<script type="module">
const { DataTable, jQuery: $ } = await import(window._datatables_src_for_itables_2_2_2);

document.querySelectorAll("#itables_a97359bf_88ed_49b6_a617_9c6d5e62e7d9:not(.dataTable)").forEach(table => {
document.querySelectorAll("#itables_9467fa67_b8b4_4ac0_a402_03f795e9781a:not(.dataTable)").forEach(table => {
if (!(table instanceof HTMLTableElement))
return;

Expand Down Expand Up @@ -358,9 +358,9 @@ Since we have GPU available, let's create the model on the GPU and use `float16`
model = xinfer.create_model("vikhyatk/moondream2", device="cuda", dtype="float16")
```

2024-10-24 16:07:14.739 | INFO  | xinfer.models:__init__:63 - Model: vikhyatk/moondream2
2024-10-24 16:07:14.739 | INFO  | xinfer.models:__init__:64 - Device: cuda
2024-10-24 16:07:14.740 | INFO  | xinfer.models:__init__:65 - Dtype: float16
2024-10-24 16:17:09.355 | INFO  | xinfer.models:__init__:63 - Model: vikhyatk/moondream2
2024-10-24 16:17:09.356 | INFO  | xinfer.models:__init__:64 - Device: cuda
2024-10-24 16:17:09.356 | INFO  | xinfer.models:__init__:65 - Dtype: float16
PhiForCausalLM has generative capabilities, as `prepare_inputs_for_generation` is explicitly overwritten. However, it doesn't directly inherit from `GenerationMixin`. From 👉v4.50👈 onwards, `PreTrainedModel` will NOT inherit from `GenerationMixin`, and this model will lose the ability to call `generate` and other related functions.
- If you're using `trust_remote_code=True`, you can get rid of this warning by loading the model with an auto class. See https://huggingface.co/docs/transformers/en/model_doc/auto#auto-classes
- If you are the owner of the model architecture code, please modify your model class such that it inherits from `GenerationMixin` (after `PreTrainedModel`, otherwise you'll get an exception).
Expand Down Expand Up @@ -438,8 +438,8 @@ model.print_stats()
│<span style="color: #008080; text-decoration-color: #008080"> Device </span>│<span style="color: #800080; text-decoration-color: #800080"> cuda </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Dtype </span>│<span style="color: #800080; text-decoration-color: #800080"> torch.float16 </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Number of Inferences </span>│<span style="color: #800080; text-decoration-color: #800080"> 2 </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Total Inference Time (ms) </span>│<span style="color: #800080; text-decoration-color: #800080"> 2029.0934 </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Average Latency (ms) </span>│<span style="color: #800080; text-decoration-color: #800080"> 1014.5467 </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Total Inference Time (ms) </span>│<span style="color: #800080; text-decoration-color: #800080"> 2021.5498 </span>│
│<span style="color: #008080; text-decoration-color: #008080"> Average Latency (ms) </span>│<span style="color: #800080; text-decoration-color: #800080"> 1010.7749 </span>│
╰───────────────────────────┴─────────────────────╯
</pre>

Expand Down Expand Up @@ -467,13 +467,13 @@ For convenience, you can also launch a Gradio interface to interact with the mod
model.launch_gradio()
```

* Running on local URL: http://127.0.0.1:7862
* Running on local URL: http://127.0.0.1:7861

To create a public link, set `share=True` in `launch()`.



<div><iframe src="http://127.0.0.1:7862/" width="100%" height="500" allow="autoplay; camera; microphone; clipboard-read; clipboard-write;" frameborder="0" allowfullscreen></iframe></div>
<div><iframe src="http://127.0.0.1:7861/" width="100%" height="1000" allow="autoplay; camera; microphone; clipboard-read; clipboard-write;" frameborder="0" allowfullscreen></iframe></div>


That's it! You've successfully run inference with x.infer.
Expand Down

0 comments on commit 656b628

Please sign in to comment.