[PT] Support custom modules in PTQ

[AlexanderDokuchaev]

Changes

• Wrap model with tracing of parameters for PTQ.
• PTModelExtractionWithFusedBiasCommand to PTModelExtractionCommand
• Removed model_analyzer.py
• Renamed PTDepthwiseConv3dSubtype to PTModuleDepthwiseConv3dSubtype metatype
• Added PTModuleDepthwiseConv3dSubtype
• Added is_subtype to OperatorMetatypeRegistry.register

Reason for changes

Support models with custom modules.

Related tickets

129581

[codecov]

Codecov Report

Attention: Patch coverage is 96.29630% with 5 lines in your changes are missing coverage. Please review.

Project coverage is 77.95%. Comparing base (ff0c070) to head (136756e).
Report is 4 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #2461       +/-   ##
============================================
- Coverage    91.16%   77.95%   -13.21%     
============================================
  Files          494      493        -1     
  Lines        45416    45448       +32     
============================================
- Hits         41403    35431     -5972     
- Misses        4013    10017     +6004     

Files	Coverage Δ
...torch/nas/bootstrapNAS/elasticity/elastic_width.py	91.72% <100.00%> (ø)
...ootstrapNAS/elasticity/multi_elasticity_handler.py	96.98% <100.00%> (ø)
...torch/nas/bootstrapNAS/elasticity/visualization.py	100.00% <100.00%> (ø)
...n/algorithms/fast_bias_correction/torch_backend.py	100.00% <100.00%> (ø)
nncf/quantization/algorithms/min_max/algorithm.py	90.54% <100.00%> (-6.38%)	⬇️
...f/quantization/algorithms/min_max/torch_backend.py	97.52% <100.00%> (+0.06%)	⬆️
nncf/torch/graph/graph_builder.py	100.00% <100.00%> (ø)
...cf/torch/graph/transformations/command_creation.py	100.00% <ø> (ø)
nncf/torch/graph/transformations/commands.py	98.86% <100.00%> (+0.01%)	⬆️
nncf/torch/model_graph_manager.py	95.56% <100.00%> (+11.01%)	⬆️
... and 9 more

... and 104 files with indirect coverage changes

Flag	Coverage Δ
COMMON	?
ONNX	?
OPENVINO	?
TENSORFLOW	30.08% <1.48%> (-0.03%)	⬇️
TORCH	65.99% <96.29%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
common	88.28% <ø> (-5.47%)	⬇️
torch	93.59% <97.54%> (+0.09%)	⬆️
tensorflow	93.74% <ø> (ø)
onnx	0.00% <ø> (-93.09%)	⬇️
openvino	25.67% <ø> (-68.50%)	⬇️
ptq	53.04% <89.28%> (-37.05%)	⬇️

[AlexanderDokuchaev]

manual/job/post_training_quantization/345 - with replace_timm_custom_modules_with_torch_native
manual/job/post_training_quantization/344/artifact/results.html - w/o replace_timm_custom_modules_with_torch_native
nightly/job/TriggerBetta/656 - (1 model failed by not related issue)

[AlexanderDokuchaev]

manual/job/post_training_quantization/351/

[alexsu52]

LGTM. Could you provide performance numbers for QAT tests?

[alexsu52]

As far as I understand, not all metatypes have a weight_port_ids attribute, then [None] should be returned, right?

[AlexanderDokuchaev]

Yes, but weighted nodes have attribute, it's filtered before using get_weight_tensor_port_ids

nncf/nncf/quantization/algorithms/min_max/torch_backend.py

Lines 353 to 356 in fa1a4ce

	def get_weight_nodes(nncf_graph: NNCFGraph) -> List[NNCFNode]:
	return [
	node for node in nncf_graph.get_all_nodes() if isinstance(node.layer_attributes, WeightedLayerAttributes)
	]

[AlexanderDokuchaev]

@alexsu52

QAT - resnet18	develop	PR	PR/dev
total	0:05:32	0:05:46	1.04
fine tune	0:04:23	0:04:35	1.05

Model	Compr. Time (develop)	Stat. collection time (develop)	Compr. Time (PR)	Stat. collection time (PR)	CT PR/dev	SC PR/dev
timm/crossvit_9_240	0:00:45	0:00:41	0:00:55	0:00:52	1.22	1.27
timm/darknet53	0:00:43	0:00:40	0:00:45	0:00:42	1.05	1.05
timm/deit3_small_patch16_224	0:00:32	0:00:29	0:00:37	0:00:34	1.16	1.17
timm/dla34	0:00:24	0:00:21	0:00:26	0:00:24	1.08	1.14
timm/dpn68	0:00:47	0:00:42	0:00:52	0:00:48	1.11	1.14
timm/efficientnet_b0	0:00:38	0:00:34	0:00:45	0:00:41	1.18	1.21
timm/efficientnet_lite0	0:00:27	0:00:24	0:00:31	0:00:29	1.15	1.21
timm/hrnet_w18	0:03:54	0:03:22	0:04:58	0:04:38	1.27	1.38
timm/inception_resnet_v2	0:02:58	0:02:37	0:03:35	0:03:20	1.21	1.27
timm/levit_128	0:01:16	0:01:10	0:01:28	0:01:23	1.16	1.19
timm/mobilenetv2_050	0:00:27	0:00:24	0:00:31	0:00:29	1.15	1.21
timm/mobilenetv3_small_050	0:00:23	0:00:21	0:00:27	0:00:24	1.17	1.14
timm/regnetx_002	0:00:27	0:00:23	0:00:29	0:00:26	1.07	1.13
timm/resnest14d	0:00:22	0:00:19	0:00:23	0:00:20	1.05	1.05
timm/resnet18	0:00:16	0:00:13	0:00:16	0:00:14	1.00	1.08
timm/swin_base_patch4_window7_224_no_sq	0:01:27	0:01:19	0:01:44	0:01:38	1.20	1.24
timm/tf_inception_v3	0:01:07	0:01:00	0:01:13	0:01:08	1.09	1.13
timm/vgg11	0:00:33	0:00:29	0:00:31	0:00:29	0.94	1.00
timm/visformer_small	0:01:04	0:01:00	0:01:08	0:01:03	1.06	1.05
timm/wide_resnet50_2	0:00:47	0:00:42	0:00:49	0:00:45	1.04	1.07