pypiwin32: does not check existence of pywin32_system32 in path before setting

[ghost]

on pypi, the packages have a line in the pywin32.pth file:
import os;os.environ["PATH"]+=(';'+os.path.join(sitedir,"pywin32_system32"))

however, when you have an application that reloads the modules without exiting the python process, this will cause the path to ultimately over-run the os.environ['PATH'] buffer.

The fix would be to change that line with:
import os;os.environ["PATH"]+=(';'+os.path.join(sitedir,"pywin32_system32")) if "pywin32_system32" not in os.environ["PATH"] else ''

ultimately when the buffer over-runs, all types of bad things happen to the running application.

Reported by: dharpe12

Original Ticket: pywin32/bugs/721

[mingwandroid]

Are there any plans to fix this bug?

[mhammond]

I'd take a PR

[mingwandroid]

On the Anaconda Distribution we did not go with the suggestion in the original comment (to check if pywin32_system32 is already in PATH before adding it) because modifying PATH like this breaks any attempt to use Python to query PATH which is exactly what conda does during activation and deactivation. With the code as it stands, repeated activation and deactivation leads to PATH containing multiple copies of C:\Users\builder\m64\lib\site-packages\pywin32_system32. This problem would likely apply equally to any other tool that wants to use Python to modify PATH.

So instead we put the DLLs where they'll already be found. Unfortunately this means copying them into two places. This is unfortunate but, from my perspective, a current necessity. Here is the comment and code from our build batch file detailing this:

:: Fix for https://sourceforge.net/p/pywin32/mailman/message/29498528/
:: although on that bug report Glenn Linderman claims this fix does
:: not work, it seems to work fine. I attempted a fix in the source
:: code (and the upstream developers have clearly thought about it)
:: but the fact is win32api auto-imports pywintypes??.dll as can be
:: seen from:
:: ntldd.exe /c/aroot/stage/Lib/site-packages/win32/win32api.pyd
::         pywintypes36.dll => not found
:: due to: win32/src/PyWinTypes.h:
:: define PYWINTYPES_EXPORT __declspec(dllimport)
:: .. and ..
:: pragma comment(lib,"pywintypes.lib")
::  .. and then (at least): win32/src/win32apimodule.cpp
:: PYWINTYPES_EXPORT PyObject *PyWin_NewUnicode(PyObject *self, PyObject *args);
:: therefore copying is the only recourse. At first glance, it may
:: seem that moving these DLLs would work, but _win32sysloader.cpp
:: expects to find two DLLs in site-packages/pywin32_system32
:: My attempted fix is in import-pywintypes-from-win32api.patch
:: An actual fix would be to make win32api a Python module that
:: first imports pywintypes and after that imports _win32api.
copy %PREFIX%\Lib\site-packages\pywin32_system32\*.dll %PREFIX%\Lib\site-packages\win32\

.. your guidance would be welcome. In all honesty though I do not think I will have time to look at fixing this 'properly'. The changes we do for this in Anaconda Distribution work fine for our needs (at the expense of some extra copies of some DLLs).

Hopefully this can be useful for someone motivated enough to fix this properly?

[mhammond]

:: An actual fix would be to make win32api a Python module that :: first imports pywintypes and after that imports _win32api.

Unfortunately, that "actual" fix would need to be done for every single .pyd in pywin32, as they all have that exact same issue.

copy %PREFIX%\Lib\site-packages\pywin32_system32*.dll %PREFIX%\Lib\site-packages\win32\

I suspect that means you will still have the same issue when trying to use the win32com package.

Does everything work if you "import pywintypes" before "import win32api"?

[mingwandroid]

Not sure. Our fix is good enough for us and I'm unlikely to get round to this anytime soon. I just copied all the DLLs (both) to where they'll naturally be found.

I'd remove this pywin32_system32 folder all together (it seems like some legacy decision to me?) and lay things out for Python's normal module loader so it 'just works' but I'm far from expert in this codebase and have no idea why they are where they are at present.

[mingwandroid]

I guess I'm suggesting moving the DLLs and changing where _win32sysloader.cpp looks for them, would that work?

[mhammond]

I guess I'm suggesting moving the DLLs and changing where _win32sysloader.cpp looks for them, would that work?

That change was made in build 222 (and it's actually pywintypes.py that does the searching - _win32sysloader only does the loading), so you may find it works now.

[Avasam]

Fixed by 71afa71 on Python >= 3.8
Fixed for older Python versions by #1517 by checking if it's already at the start of PATH. It might be safer for path-length if it checked whether it was anywhere in the path, removed it, then added it back to front, but #2207 could resolve that simply by removing that code path.