Tasks: Add image-text-to-text pipeline and inference API to task page

packages/tasks/src/tasks/image-text-to-text/about.md

@@ -32,39 +32,57 @@ Vision language models can recognize images through descriptions. When given det
 
 ## Inference
 
-You can use the Transformers library to interact with vision-language models. You can load the model like below.
+You can use the Transformers library to interact with vision-language models. Specifically `pipeline` makes it easy to infer models.
+
+Initialize the pipeline first.
+
+```python
+from transformers import pipeline
+
+pipe = pipeline("image-text-to-text", model="llava-hf/llava-interleave-qwen-0.5b-hf")
+```
+
+We will use chat templates to format the text input. We can also pass the image as URL in context part of the user role in our chat template.
 
 ```python
-from transformers import LlavaNextProcessor, LlavaNextForConditionalGeneration
-import torch
-
-device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
-processor = LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf")
-model = LlavaNextForConditionalGeneration.from_pretrained(
-    "llava-hf/llava-v1.6-mistral-7b-hf",
-    torch_dtype=torch.float16
-)
-model.to(device)
+messages = [
+     {
+         "role": "user",
+         "content": [
+             {
+                 "type": "image",
+                 "image": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/bee.jpg",
+             },
+             {"type": "text", "text": "Describe this image."},
+         ],
+     },
+     {
+         "role": "assistant",
+         "content": [
+             {"type": "text", "text": "There's a pink flower"},
+         ],
+     },
+ ]
+
 ```
 
-We can infer by passing image and text dialogues.
+We can now directly pass in the messages to pipeline to infer. `return_full_text` is a flag to include the full prompt including the user input. Here we pass as `False` to only return the generated part.
 
 ```python
-from PIL import Image
-import requests
+outputs = pipe(text=messages, max_new_tokens=20, return_full_text=False)
 
-# image of a radar chart
-url = "https://github.com/haotian-liu/LLaVA/blob/1a91fc274d7c35a9b50b3cb29c4247ae5837ce39/images/llava_v1_5_radar.jpg?raw=true"
-image = Image.open(requests.get(url, stream=True).raw)
-prompt = "[INST] <image>\nWhat is shown in this image? [/INST]"
+outputs[0]["generated_text"]
+# with a yellow center in the foreground. The flower is surrounded by red and white flowers with green stems
+```
 
-inputs = processor(prompt, image, return_tensors="pt").to(device)
-output = model.generate(**inputs, max_new_tokens=100)
+You can also use Inference API to play with image-text-to-text models.
 
-print(processor.decode(output[0], skip_special_tokens=True))
-# The image appears to be a radar chart, which is a type of multivariate chart that displays values for multiple variables represented on axes
-# starting from the same point. This particular radar chart is showing the performance of different models or systems across various metrics.
-# The axes represent different metrics or benchmarks, such as MM-Vet, MM-Vet, MM-Vet, MM-Vet, MM-Vet, MM-V
+```bash
+curl https://api-inference.huggingface.co/models/meta-llama/Llama-3.2-11B-Vision-Instruct \
+	-X POST \
+	-d '{"inputs": "Can you please let us know more details about your "}' \
+	-H 'Content-Type: application/json' \
+	-H "Authorization: Bearer hf_***"
 ```
 
 ## Useful Resources