diff --git a/README.md b/README.md index b05770fafbd..920e2e26fa1 100644 --- a/README.md +++ b/README.md @@ -273,44 +273,44 @@ For a quicker start with NNCF-powered compression, try sample notebooks and scri A collection of ready-to-run Jupyter* notebooks tutorials and demos are available to explain and display NNCF compression algorithms for optimizing models for inference with the OpenVINO Toolkit. -| Notebook Tutorial Name | Compression Algorithm | Backend | Domain | -|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------:|:----------:|:-----------------------------------:| -| [BERT Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/105-language-quantize-bert)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/105-language-quantize-bert/105-language-quantize-bert.ipynb) | Post-Training Quantization | OpenVINO | NLP | -| [MONAI Segmentation Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/110-ct-segmentation-quantize)
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F110-ct-segmentation-quantize%2F110-ct-scan-live-inference.ipynb) | Post-Training Quantization | OpenVINO | Segmentation | -| [PyTorch Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/112-pytorch-post-training-quantization-nncf) | Post-Training Quantization | PyTorch | Image Classification | -| [TensorFlow Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/301-tensorflow-training-openvino) | Post-Training Quantization | Tensorflow | Image Classification | -| [Quantization with Accuracy Control](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/122-quantizing-model-with-accuracy-control) | Post-Training Quantization with Accuracy Control | OpenVINO | Speech-to-Text,
Object Detection | -| [PyTorch Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/302-pytorch-quantization-aware-training) | Training-Time Compression | PyTorch | Image Classification | -| [TensorFlow Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/301-tensorflow-training-openvino) | Training-Time Compression | Tensorflow | Image Classification | -| [Joint Pruning, Quantization and Distillation for BERT](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/116-sparsity-optimization) | Joint Pruning, Quantization and Distillation | OpenVINO | NLP | +| Notebook Tutorial Name | Compression Algorithm | Backend | Domain | +|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------:|:----------:|:-----------------------------------:| +| [BERT Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/language-quantize-bert)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/language-quantize-bert/language-quantize-bert.ipynb) | Post-Training Quantization | OpenVINO | NLP | +| [MONAI Segmentation Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/ct-segmentation-quantize)
[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2Fct-segmentation-quantize%2Fct-scan-live-inference.ipynb) | Post-Training Quantization | OpenVINO | Segmentation | +| [PyTorch Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-post-training-quantization-nncf) | Post-Training Quantization | PyTorch | Image Classification | +| [TensorFlow Model Quantization](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Post-Training Quantization | Tensorflow | Image Classification | +| [Quantization with Accuracy Control](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/quantizing-model-with-accuracy-control) | Post-Training Quantization with Accuracy Control | OpenVINO | Speech-to-Text,
Object Detection | +| [PyTorch Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/pytorch-quantization-aware-training) | Training-Time Compression | PyTorch | Image Classification | +| [TensorFlow Training-Time Compression](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/tensorflow-training-openvino) | Training-Time Compression | Tensorflow | Image Classification | +| [Joint Pruning, Quantization and Distillation for BERT](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sparsity-optimization) | Joint Pruning, Quantization and Distillation | OpenVINO | NLP | Below is a list of notebooks demonstrating OpenVINO conversion and inference together with NNCF compression for models from various domains. -| Demo Model | Compression Algorithm | Backend | Domain | -|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------:|:---------:|:--------------------------------------------------------------------:| -| [YOLOv8](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/230-yolov8-optimization)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/230-yolov8-optimization/230-yolov8-object-detection.ipynb) | Post-Training Quantization | OpenVINO | Object Detection,
KeyPoint Detection,
Instance Segmentation | -| [YOLOv7](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/226-yolov7-optimization) | Post-Training Quantization | OpenVINO | Object Detection | -| [EfficientSAM](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/274-efficient-sam) | Post-Training Quantization | OpenVINO | Image Segmentation | -| [Segment Anything Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/237-segment-anything) | Post-Training Quantization | OpenVINO | Image Segmentation | -| [OneFormer](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/249-oneformer-segmentation) | Post-Training Quantization | OpenVINO | Image Segmentation | -| [InstructPix2Pix](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/231-instruct-pix2pix-image-editing) | Post-Training Quantization | OpenVINO | Image-to-Image | -| [CLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/228-clip-zero-shot-image-classification) | Post-Training Quantization | OpenVINO | Image-to-Text | -| [BLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/233-blip-visual-language-processing) | Post-Training Quantization | OpenVINO | Image-to-Text | -| [Segmind-VegaRT](https://github.com/openvinotoolkit/openvino_notebooks/blob/main/notebooks/248-stable-diffusion-xl/248-segmind-vegart.ipynb) | Post-Training Quantization | OpenVINO | Text-to-Image | -| [Latent Consistency Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/263-latent-consistency-models-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image | -| [Würstchen](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/265-wuerstchen-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image | -| [ControlNet QR Code Monster](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/264-qrcode-monster) | Post-Training Quantization | OpenVINO | Text-to-Image | -| [SDXL-turbo](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/271-sdxl-turbo) | Post-Training Quantization | OpenVINO | Text-to-Image,
Image-to-Image | -| [DeepFloyd IF](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/238-deepfloyd-if) | Post-Training Quantization,
Weight Compression | OpenVINO | Text-to-Image,
Image-to-Image | -| [ImageBind](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/239-image-bind) | Post-Training Quantization | OpenVINO | Multi-Modal Retrieval | -| [Distil-Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/267-distil-whisper-asr) | Post-Training Quantization | OpenVINO | Speech-to-Text | -| [Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/main/notebooks/227-whisper-subtitles-generation/227-whisper-convert.ipynb) | Post-Training Quantization | OpenVINO | Speech-to-Text | -| [MMS Speech Recognition](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/255-mms-massively-multilingual-speech) | Post-Training Quantization | OpenVINO | Speech-to-Text | -| [Grammar Error Correction](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/214-grammar-correction) | Post-Training Quantization | OpenVINO | NLP, Grammar Correction | -| [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/275-llm-question-answering) | Weight Compression | OpenVINO | NLP, Instruction Following | -| [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following) | Weight Compression | OpenVINO | NLP, Instruction Following | -| [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/273-stable-zephyr-3b-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot | -| [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot | +| Demo Model | Compression Algorithm | Backend | Domain | +|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------:|:---------:|:--------------------------------------------------------------------:| +| [YOLOv8](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/yolov8-optimization/yolov8-object-detection.ipynb) | Post-Training Quantization | OpenVINO | Object Detection,
KeyPoint Detection,
Instance Segmentation | +| [YOLOv7](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov7-optimization) | Post-Training Quantization | OpenVINO | Object Detection | +| [EfficientSAM](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/efficient-sam) | Post-Training Quantization | OpenVINO | Image Segmentation | +| [Segment Anything Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/segment-anything) | Post-Training Quantization | OpenVINO | Image Segmentation | +| [OneFormer](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/oneformer-segmentation) | Post-Training Quantization | OpenVINO | Image Segmentation | +| [InstructPix2Pix](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/instruct-pix2pix-image-editing) | Post-Training Quantization | OpenVINO | Image-to-Image | +| [CLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/clip-zero-shot-image-classification) | Post-Training Quantization | OpenVINO | Image-to-Text | +| [BLIP](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/blip-visual-language-processing) | Post-Training Quantization | OpenVINO | Image-to-Text | +| [Segmind-VegaRT](https://github.com/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/stable-diffusion-xl/segmind-vegart.ipynb) | Post-Training Quantization | OpenVINO | Text-to-Image | +| [Latent Consistency Model](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/latent-consistency-models-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image | +| [Würstchen](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/wuerstchen-image-generation) | Post-Training Quantization | OpenVINO | Text-to-Image | +| [ControlNet QR Code Monster](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/qrcode-monster) | Post-Training Quantization | OpenVINO | Text-to-Image | +| [SDXL-turbo](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/sdxl-turbo) | Post-Training Quantization | OpenVINO | Text-to-Image,
Image-to-Image | +| [DeepFloyd IF](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/deepfloyd-if) | Post-Training Quantization,
Weight Compression | OpenVINO | Text-to-Image,
Image-to-Image | +| [ImageBind](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/image-bind) | Post-Training Quantization | OpenVINO | Multi-Modal Retrieval | +| [Distil-Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/distil-whisper-asr) | Post-Training Quantization | OpenVINO | Speech-to-Text | +| [Whisper](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/whisper-subtitles-generation)
[![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_notebooks/blob/latest/notebooks/whisper-subtitles-generation/whisper-convert.ipynb) | Post-Training Quantization | OpenVINO | Speech-to-Text | +| [MMS Speech Recognition](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/mms-massively-multilingual-speech) | Post-Training Quantization | OpenVINO | Speech-to-Text | +| [Grammar Error Correction](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/grammar-correction) | Post-Training Quantization | OpenVINO | NLP, Grammar Correction | +| [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering) | Weight Compression | OpenVINO | NLP, Instruction Following | +| [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/dolly-2-instruction-following) | Weight Compression | OpenVINO | NLP, Instruction Following | +| [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/stable-zephyr-3b-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot | +| [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot) | Weight Compression | OpenVINO | NLP, Chat Bot | ### Post-Training Quantization Examples diff --git a/docs/compression_algorithms/CompressWeights.md b/docs/compression_algorithms/CompressWeights.md index bc53948441b..766ab0fb97a 100644 --- a/docs/compression_algorithms/CompressWeights.md +++ b/docs/compression_algorithms/CompressWeights.md @@ -408,7 +408,7 @@ This modification applies only for patterns `MatMul-Multiply-MatMul` (for exampl List of notebooks demonstrating OpenVINO conversion and inference together with NNCF weight compression for models from various domains: -- [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/275-llm-question-answering) -- [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following) -- [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/273-stable-zephyr-3b-chatbot) -- [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot) +- [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering) +- [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/dolly-2-instruction-following) +- [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/stable-zephyr-3b-chatbot) +- [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot) diff --git a/examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/README.md b/examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/README.md index 906c9ae483f..57e1c028cdd 100644 --- a/examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/README.md +++ b/examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/README.md @@ -44,4 +44,4 @@ python deploy.py ## See also -- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization) +- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization) diff --git a/examples/post_training_quantization/openvino/yolov8/README.md b/examples/post_training_quantization/openvino/yolov8/README.md index f861b3c5b15..78a85325634 100644 --- a/examples/post_training_quantization/openvino/yolov8/README.md +++ b/examples/post_training_quantization/openvino/yolov8/README.md @@ -34,4 +34,4 @@ python main.py ## See also -- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization) +- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization) diff --git a/examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md b/examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md index f5649efbdf8..5dcc75450a9 100644 --- a/examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md +++ b/examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md @@ -36,4 +36,4 @@ python main.py ## See also -- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization) +- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)