access violation when appending OpenVINO execution provider

[deischi]

Describe the issue

onnxruntime crashes in openvino_npu_plugin.dll when addingthe OpenVINO execution provider for CPU:

		std::unordered_map<std::string, std::string> options;
		options["device_type"] = "CPU";
		options["precision"] = "FP32";
		ortSessionOptions.AppendExecutionProvider("OpenVINO", options);

Callstack:

 	00007ffd287f29c0()	Unknown	No symbols loaded.
 	openvino_intel_npu_plugin.dll!00007ffd28e2ac5d()	Unknown	No symbols loaded.
 	openvino_intel_npu_plugin.dll!00007ffd28df8cfc()	Unknown	No symbols loaded.
 	openvino_intel_npu_plugin.dll!00007ffd28ccd9b8()	Unknown	No symbols loaded.
 	openvino_intel_npu_plugin.dll!00007ffd28cb5b42()	Unknown	No symbols loaded.
 	openvino_intel_npu_plugin.dll!00007ffd28ccc7a0()	Unknown	No symbols loaded.
 	openvino.dll!00007ffd377e1d18()	Unknown	No symbols loaded.
 	openvino.dll!00007ffd377e31f6()	Unknown	No symbols loaded.
 	openvino.dll!00007ffd377df3b3()	Unknown	No symbols loaded.
 	openvino.dll!00007ffd377c6511()	Unknown	No symbols loaded.
 	onnxruntime_providers_openvino.dll!onnxruntime::OVDevices::get_ov_devices() Line 22	C++	Symbols loaded.
 	onnxruntime_providers_openvino.dll!onnxruntime::OpenVINO_Provider::CreateExecutionProviderFactory(const void * void_params) Line 141	C++	Symbols loaded.
 	onnxruntime.dll!onnxruntime::OpenVINOProviderFactoryCreator::Create(const std::unordered_map<std::string,std::string,std::hash<std::string>,std::equal_to<std::string>,std::allocator<std::pair<std::string const ,std::string>>> * provider_options_map, const onnxruntime::SessionOptions * session_options) Line 1830	C++	Symbols loaded.
 	onnxruntime.dll!OrtApis::SessionOptionsAppendExecutionProvider(OrtSessionOptions * options, const char * provider_name, const char * const * provider_options_keys, const char * const * provider_options_values, unsigned __int64 num_keys) Line 111	C++	Symbols loaded.

To reproduce

Immediately crashed when adding the execution provider (see above).

onnxruntime build command:
build.bat --build --update --parallel --compile_no_warning_as_error --config RelWithDebInfo --use_openvino CPU --use_dml --build_shared_lib --skip_tests --cmake_extra_defines CMAKE_INSTALL_PREFIX=c:\dev\ort_1.20.1-install

As a workaround, I tried to remove the openvino_intel_npu_plugin.dll.
appending the execution provider is working, but creating the session later fails with this exception:

Exception during initialization: Exception from src\inference\src\cpp\core.cpp:265:
Exception from src\inference\src\dev\core_impl.cpp:567:
Device with "NPU" name is not registered in the OpenVINO Runtime

Also onnxruntime_perf_test.exe failes for OpenVINO - just without any error message.

C:\dev\onnxruntime-1.20.1\build\Windows\RelWithDebInfo\RelWithDebInfo>\dev\w_openvino_toolkit_windows_2024.6.0.17404.4c0f47d2335_x86_64\setupvars.batPython 3.12.8
[setupvars.bat] OpenVINO environment initialized

fine for CPU:

C:\dev\onnxruntime-1.20.1\build\Windows\RelWithDebInfo\RelWithDebInfo>onnxruntime_perf_test.exe -m times -r 1 -I testdata\test_resize.onnx
Session creation time cost: 0.008206 s
First inference time cost: 0 ms
Total inference time cost: 5.77e-05 s
...

fine for DML:

C:\dev\onnxruntime-1.20.1\build\Windows\RelWithDebInfo\RelWithDebInfo>onnxruntime_perf_test.exe -e dml -m times -r 1 -I testdata\test_resize.onnx
Session creation time cost: 0.33271 s
First inference time cost: 47 ms
Total inference time cost: 0.0004038 s
Total inference requests: 1
...

no output for openvino:

C:\dev\onnxruntime-1.20.1\build\Windows\RelWithDebInfo\RelWithDebInfo>onnxruntime_perf_test.exe -e openvino -m times -r 1 -I testdata\test_resize.onnx

There is a similar issue openvinotoolkit/openvino#26567, but this should have been fixed with OpenVINO 2024.5

Urgency

No response

Platform

Windows

OS Version

Windows 10, 22H2

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.20.1

ONNX Runtime API

C++

Architecture

X64

Execution Provider

OpenVINO

Execution Provider Library Version

openvino 2024.5