Merge branch 'develop' into ad/constraints

.github/workflows/nightly.yml

@@ -10,4 +10,4 @@ jobs:
     runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v4
-      - uses: AlexanderDokuchaev/md-dead-link-check@v0.6
+      - uses: AlexanderDokuchaev/md-dead-link-check@v0.8

.github/workflows/pre-commit-linters.yml

@@ -23,4 +23,4 @@ jobs:
     runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v4
-      - uses: AlexanderDokuchaev/md-dead-link-check@v0.6
+      - uses: AlexanderDokuchaev/md-dead-link-check@v0.8

.mypy.ini

@@ -1,5 +1,5 @@
 [mypy]
-files = nncf/common/sparsity
+files = nncf/common/sparsity, nncf/common/graph
 follow_imports = silent
 strict = True
 

.pre-commit-config.yaml

@@ -17,7 +17,7 @@ repos:
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: v0.1.3
+    rev: v0.3.7
     hooks:
       - id: ruff
 

docs/Installation.md

@@ -12,14 +12,6 @@ NNCF can be installed as a regular PyPI package via pip:
 pip install nncf
 ```
 
-If you want to install both NNCF and the supported PyTorch version in one line, you can do this by simply running:
-
-```bash
-pip install nncf[torch]
-```
-
-Other viable options besides `[torch]` are `[tf]`, `[onnx]` and `[openvino]`.
-
 ## As a package built from a checked-out repository
 
 Install the package and its dependencies by running the following command in the repository root directory:
@@ -28,20 +20,6 @@ Install the package and its dependencies by running the following command in the
 pip install .
 ```
 
-Use the same `pip install` syntax as above to install NNCF along with the backend package version in one go:
-
-```bash
-pip install .[<BACKEND>]
-```
-
-List of supported backends: `torch`, `tf`, `onnx` and `openvino`.
-
-For development purposes install extra packages by
-
-```bash
-pip install .[dev,tests]
-```
-
 _NB_: For launching example scripts in this repository, we recommend setting the `PYTHONPATH` variable to the root of the checked-out repository once the installation is completed.
 
 NNCF is also available via [conda](https://anaconda.org/conda-forge/nncf):
@@ -65,7 +43,7 @@ as well as the supported versions of Python:
 
 | NNCF      | OpenVINO   | PyTorch  | ONNX     | TensorFlow | Python |
 |-----------|------------|----------|----------|------------|--------|
-| `develop` | `2024.4.0` | `2.2.1`  | `1.13.1` | `2.12.0`   | `3.8`  |
+| `develop` | `2024.4.0` | `2.2.1`  | `1.16.0` | `2.12.0`   | `3.8`  |
 | `2.9.0`   | `2024.4.0` | `2.1.2`  | `1.13.1` | `2.12.0`   | `3.8`  |
 | `2.8.1`   | `2023.3.0` | `2.1.2`  | `1.13.1` | `2.12.0`   | `3.8`  |
 | `2.8.0`   | `2023.3.0` | `2.1.2`  | `1.13.1` | `2.12.0`   | `3.8`  |

docs/compression_algorithms/CompressWeights.md

@@ -408,7 +408,7 @@ This modification applies only for patterns `MatMul-Multiply-MatMul` (for exampl
 
 List of notebooks demonstrating OpenVINO conversion and inference together with NNCF weight compression for models from various domains:
 
-- [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/275-llm-question-answering)
-- [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/240-dolly-2-instruction-following)
-- [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/273-stable-zephyr-3b-chatbot)
-- [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/254-llm-chatbot)
+- [LLM Instruction Following](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-question-answering)
+- [Dolly 2.0](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/dolly-2-instruction-following)
+- [Stable-Zephyr-3b](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/stable-zephyr-3b-chatbot)
+- [LLM Chat Bots](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/llm-chatbot)

docs/compression_algorithms/post_training/Quantization.md

@@ -89,3 +89,5 @@ for data_item in val_loader:
 
 NNCF provides the examples of Post-Training Quantization where you can find the implementation of data transformation
 function: [PyTorch](../../../examples/post_training_quantization/torch/mobilenet_v2/README.md), [TensorFlow](../../../examples/post_training_quantization/tensorflow/mobilenet_v2/README.md), [ONNX](../../../examples/post_training_quantization/onnx/mobilenet_v2/README.md), and [OpenVINO](../../../examples/post_training_quantization/openvino/mobilenet_v2/README.md)
+
+In case the Post-Training Quantization algorithm could not reach quality requirements you can fine-tune a quantized pytorch model. Example of the Quantization-Aware training pipeline for a pytorch model could be found [here](../../../examples/quantization_aware_training/torch/resnet18/README.md).

docs/styleguide/PyGuide.md

@@ -316,10 +316,10 @@ inline comments.
 
 #### 4.2.1 Modules
 
-Every file should contain a license boilerplate.
+Every file should contain a license boilerplate, where [YYYY] should be replaced to current year.
 
 ```python
-# Copyright (c) 2023 Intel Corporation
+# Copyright (c) [YYYY] Intel Corporation
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
@@ -803,7 +803,7 @@ def log_current_time(log_stream: LogStream):
 
 ```python3
 class CheckpointConverter:
-    # ... 
+    # ...
     def convert(self, ckpt: CheckpointType) -> AnotherCheckpointType:
         pass
 ```

examples/post_training_quantization/onnx/mobilenet_v2/main.py

@@ -44,7 +44,7 @@ def download_model() -> Path:
     return download_url(MODEL_URL, Path(MODEL_PATH).resolve())
 
 
-def validate(path_to_model: str, validation_loader: torch.utils.data.DataLoader) -> float:
+def validate(path_to_model: Path, validation_loader: torch.utils.data.DataLoader) -> float:
     predictions = []
     references = []
 
@@ -61,7 +61,7 @@ def validate(path_to_model: str, validation_loader: torch.utils.data.DataLoader)
     return accuracy_score(predictions, references)
 
 
-def run_benchmark(path_to_model: str, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
+def run_benchmark(path_to_model: Path, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
     command = f"benchmark_app -m {path_to_model} -d CPU -api async -t 15"
     if shape is not None:
         command += f' -shape [{",".join(str(x) for x in shape)}]'
@@ -79,7 +79,7 @@ def run_benchmark(path_to_model: str, shape: Optional[List[int]] = None, verbose
 
 normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
 val_dataset = datasets.ImageFolder(
-    root=f"{dataset_path}/val",
+    root=dataset_path / "val",
     transform=transforms.Compose(
         [
             transforms.Resize(256),
@@ -127,11 +127,11 @@ def transform_fn(data_item):
 ###############################################################################
 # Benchmark performance and validate accuracy
 
-fp32_model_path = f"{ROOT}/mobilenet_v2_fp32.onnx"
+fp32_model_path = ROOT / "mobilenet_v2_fp32.onnx"
 onnx.save(model, fp32_model_path)
 print(f"[1/7] Save FP32 model: {fp32_model_path}")
 
-int8_model_path = f"{ROOT}/mobilenet_v2_int8.onnx"
+int8_model_path = ROOT / "mobilenet_v2_int8.onnx"
 onnx.save(onnx_quantized_model, int8_model_path)
 print(f"[2/7] Save INT8 model: {int8_model_path}")
 

examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/README.md

@@ -44,4 +44,4 @@ python deploy.py
 
 ## See also
 
-- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization)
+- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)

examples/post_training_quantization/onnx/yolov8_quantize_with_accuracy_control/main.py

@@ -195,20 +195,17 @@ def validation_ac(
         preset=nncf.QuantizationPreset.MIXED,
         ignored_scope=nncf.IgnoredScope(
             types=["Mul", "Sub", "Sigmoid"],  # ignore operations
-            names=[
-                "/model.22/dfl/conv/Conv",  # in the post-processing subgraph
-                "/model.22/Add",
-                "/model.22/Add_1",
-                "/model.22/Add_2",
-                "/model.22/Add_3",
-                "/model.22/Add_4",
-                "/model.22/Add_5",
-                "/model.22/Add_6",
-                "/model.22/Add_7",
-                "/model.22/Add_8",
-                "/model.22/Add_9",
-                "/model.22/Add_10",
-                "/model.22/Add_11",
+            subgraphs=[
+                nncf.Subgraph(
+                    inputs=[
+                        "/model.22/Concat_3",
+                        "/model.22/Concat_6",
+                        "/model.22/Concat_24",
+                        "/model.22/Concat_5",
+                        "/model.22/Concat_4",
+                    ],
+                    outputs=["/model.22/Concat_29"],
+                )
             ],
         ),
     )

examples/post_training_quantization/openvino/mobilenet_v2/main.py

@@ -56,7 +56,7 @@ def validate(model: ov.Model, val_loader: torch.utils.data.DataLoader) -> float:
     return accuracy_score(predictions, references)
 
 
-def run_benchmark(model_path: str, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
+def run_benchmark(model_path: Path, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
     command = f"benchmark_app -m {model_path} -d CPU -api async -t 15"
     if shape is not None:
         command += f' -shape [{",".join(str(x) for x in shape)}]'
@@ -67,7 +67,7 @@ def run_benchmark(model_path: str, shape: Optional[List[int]] = None, verbose: b
     return float(match.group(1))
 
 
-def get_model_size(ir_path: str, m_type: str = "Mb", verbose: bool = True) -> float:
+def get_model_size(ir_path: Path, m_type: str = "Mb", verbose: bool = True) -> float:
     xml_size = os.path.getsize(ir_path)
     bin_size = os.path.getsize(os.path.splitext(ir_path)[0] + ".bin")
     for t in ["bytes", "Kb", "Mb"]:
@@ -90,7 +90,7 @@ def get_model_size(ir_path: str, m_type: str = "Mb", verbose: bool = True) -> fl
 
 normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
 val_dataset = datasets.ImageFolder(
-    root=f"{dataset_path}/val",
+    root=dataset_path / "val",
     transform=transforms.Compose(
         [
             transforms.Resize(256),
@@ -136,12 +136,12 @@ def transform_fn(data_item):
 ###############################################################################
 # Benchmark performance, calculate compression rate and validate accuracy
 
-fp32_ir_path = f"{ROOT}/mobilenet_v2_fp32.xml"
+fp32_ir_path = ROOT / "mobilenet_v2_fp32.xml"
 ov.save_model(ov_model, fp32_ir_path, compress_to_fp16=False)
 print(f"[1/7] Save FP32 model: {fp32_ir_path}")
 fp32_model_size = get_model_size(fp32_ir_path, verbose=True)
 
-int8_ir_path = f"{ROOT}/mobilenet_v2_int8.xml"
+int8_ir_path = ROOT / "mobilenet_v2_int8.xml"
 ov.save_model(ov_quantized_model, int8_ir_path, compress_to_fp16=False)
 print(f"[2/7] Save INT8 model: {int8_ir_path}")
 int8_model_size = get_model_size(int8_ir_path, verbose=True)

examples/post_training_quantization/openvino/yolov8/README.md

@@ -34,4 +34,4 @@ python main.py
 
 ## See also
 
-- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization)
+- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)

examples/post_training_quantization/openvino/yolov8/main.py

@@ -122,20 +122,12 @@ def transform_fn(data_item: Dict):
         quantization_dataset,
         preset=nncf.QuantizationPreset.MIXED,
         ignored_scope=nncf.IgnoredScope(
-            types=["Multiply", "Subtract", "Sigmoid"],  # ignore operations
-            names=[
-                "/model.22/dfl/conv/Conv",  # in the post-processing subgraph
-                "/model.22/Add",
-                "/model.22/Add_1",
-                "/model.22/Add_2",
-                "/model.22/Add_3",
-                "/model.22/Add_4",
-                "/model.22/Add_5",
-                "/model.22/Add_6",
-                "/model.22/Add_7",
-                "/model.22/Add_8",
-                "/model.22/Add_9",
-                "/model.22/Add_10",
+            types=["Multiply", "Subtract", "Sigmoid"],
+            subgraphs=[
+                nncf.Subgraph(
+                    inputs=["/model.22/Concat", "/model.22/Concat_1", "/model.22/Concat_2"],
+                    outputs=["output0/sink_port_0"],
+                )
             ],
         ),
     )

examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/README.md

@@ -36,4 +36,4 @@ python main.py
 
 ## See also
 
-- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization)
+- [YOLOv8 Jupyter notebook](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks/yolov8-optimization)

examples/post_training_quantization/openvino/yolov8_quantize_with_accuracy_control/main.py

@@ -186,20 +186,17 @@ def validation_ac(
         preset=nncf.QuantizationPreset.MIXED,
         ignored_scope=nncf.IgnoredScope(
             types=["Multiply", "Subtract", "Sigmoid"],  # ignore operations
-            names=[
-                "/model.22/dfl/conv/Conv",  # in the post-processing subgraph
-                "/model.22/Add",
-                "/model.22/Add_1",
-                "/model.22/Add_2",
-                "/model.22/Add_3",
-                "/model.22/Add_4",
-                "/model.22/Add_5",
-                "/model.22/Add_6",
-                "/model.22/Add_7",
-                "/model.22/Add_8",
-                "/model.22/Add_9",
-                "/model.22/Add_10",
-                "/model.22/Add_11",
+            subgraphs=[
+                nncf.Subgraph(
+                    inputs=[
+                        "/model.22/Concat_3",
+                        "/model.22/Concat_6",
+                        "/model.22/Concat_24",
+                        "/model.22/Concat_5",
+                        "/model.22/Concat_4",
+                    ],
+                    outputs=["output0"],
+                )
             ],
         ),
     )

examples/post_training_quantization/torch/mobilenet_v2/main.py

@@ -63,7 +63,7 @@ def validate(model: ov.Model, val_loader: torch.utils.data.DataLoader) -> float:
     return accuracy_score(predictions, references)
 
 
-def run_benchmark(model_path: str, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
+def run_benchmark(model_path: Path, shape: Optional[List[int]] = None, verbose: bool = True) -> float:
     command = f"benchmark_app -m {model_path} -d CPU -api async -t 15"
     if shape is not None:
         command += f' -shape [{",".join(str(x) for x in shape)}]'
@@ -74,7 +74,7 @@ def run_benchmark(model_path: str, shape: Optional[List[int]] = None, verbose: b
     return float(match.group(1))
 
 
-def get_model_size(ir_path: str, m_type: str = "Mb", verbose: bool = True) -> float:
+def get_model_size(ir_path: Path, m_type: str = "Mb", verbose: bool = True) -> float:
     xml_size = os.path.getsize(ir_path)
     bin_size = os.path.getsize(os.path.splitext(ir_path)[0] + ".bin")
     for t in ["bytes", "Kb", "Mb"]:
@@ -97,7 +97,7 @@ def get_model_size(ir_path: str, m_type: str = "Mb", verbose: bool = True) -> fl
 
 normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
 val_dataset = datasets.ImageFolder(
-    root=f"{dataset_path}/val",
+    root=dataset_path / "val",
     transform=transforms.Compose(
         [
             transforms.Resize(256),
@@ -107,7 +107,8 @@ def get_model_size(ir_path: str, m_type: str = "Mb", verbose: bool = True) -> fl
         ]
     ),
 )
-val_data_loader = torch.utils.data.DataLoader(val_dataset)
+batch_size = 128
+val_data_loader = torch.utils.data.DataLoader(val_dataset, batch_size=batch_size)
 
 torch_model = models.mobilenet_v2(num_classes=DATASET_CLASSES)
 torch_model = load_checkpoint(torch_model)
@@ -140,8 +141,10 @@ def transform_fn(data_item: Tuple[torch.Tensor, int], device: torch.device) -> t
 # item and prepare model input data. The quantize method uses a small subset
 # (default: 300 samples) of the calibration dataset.
 
+# Recalculation default subset_size parameter based on batch_size.
+subset_size = 300 // batch_size
 calibration_dataset = nncf.Dataset(val_data_loader, partial(transform_fn, device=device))
-torch_quantized_model = nncf.quantize(torch_model, calibration_dataset)
+torch_quantized_model = nncf.quantize(torch_model, calibration_dataset, subset_size=subset_size)
 
 ###############################################################################
 # Benchmark performance, calculate compression rate and validate accuracy
@@ -150,12 +153,12 @@ def transform_fn(data_item: Tuple[torch.Tensor, int], device: torch.device) -> t
 ov_model = ov.convert_model(torch_model.cpu(), example_input=dummy_input)
 ov_quantized_model = ov.convert_model(torch_quantized_model.cpu(), example_input=dummy_input)
 
-fp32_ir_path = f"{ROOT}/mobilenet_v2_fp32.xml"
+fp32_ir_path = ROOT / "mobilenet_v2_fp32.xml"
 ov.save_model(ov_model, fp32_ir_path, compress_to_fp16=False)
 print(f"[1/7] Save FP32 model: {fp32_ir_path}")
 fp32_model_size = get_model_size(fp32_ir_path, verbose=True)
 
-int8_ir_path = f"{ROOT}/mobilenet_v2_int8.xml"
+int8_ir_path = ROOT / "mobilenet_v2_int8.xml"
 ov.save_model(ov_quantized_model, int8_ir_path, compress_to_fp16=False)
 print(f"[2/7] Save INT8 model: {int8_ir_path}")
 int8_model_size = get_model_size(int8_ir_path, verbose=True)

examples/post_training_quantization/torch/ssd300_vgg16/main.py

@@ -70,9 +70,9 @@ def run_benchmark(model_path: str, shape=None, verbose: bool = True) -> float:
 
 class COCO128Dataset(torch.utils.data.Dataset):
     category_mapping = [
-        1,2,3,4,5,6,7,8,9,10,11,13,14,15,16,17,18,19,20,21,22,23,24,25,27,28,31,32,33,
-        34,35,36,37,38,39,40,41,42,43,44,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,
-        61,62,63,64,65,67,70,72,73,74,75,76,77,78,79,80,81,82,84,85,86,87,88,89,90
+        1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 27, 28, 31, 32, 33,
+        34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
+        61, 62, 63, 64, 65, 67, 70, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90
     ]  # fmt: skip
 
     def __init__(self, data_path: str, transform: Callable):

examples/torch/common/models/classification/mobilenet_v3_tv_092.py

@@ -305,7 +305,7 @@ def _mobilenet_v3_model(
 ):
     model = MobileNetV3(inverted_residual_setting, last_channel, **kwargs)
     if pretrained:
-        if model_urls.get(arch, None) is None:
+        if model_urls.get(arch) is None:
             raise ValueError("No checkpoint is available for model type {}".format(arch))
         state_dict = load_state_dict_from_url(model_urls[arch], progress=progress)
         model.load_state_dict(state_dict)