Deadlock when using start_blocking_portal("trio") on trio

[gschaffner]

reproducer: cd9deb1.

this is because the trio implementation of BlockingPortal._spawn_task_from_thread uses trio.from_thread.run_sync. trio.from_thread.run_sync is not allowed to be called from trio threads because it can cause deadlocks¹.

one way to fix this is by switching the trio implementation of BlockingPortal._spawn_task_from_thread from using trio.from_thread.run_sync(BlockingPortal._task_group.start_soon, ..., trio_token=...) to using TrioToken.run_sync_soon(BlockingPortal._task_group.start_soon, ...). the downside of this change is that it would change what happens if TaskGroup.start_soon fails:

• with trio.from_thread.run_sync(BlockingPortal._task_group.start_soon, ..., trio_token=...), if TaskGroup.start_soon raises it gets propagated to the anyio.from_thread.run(_sync)? call site.

• with TrioToken.run_sync_soon(BlockingPortal._task_group.start_soon, ...), if TaskGroup.start_soon raises it gets propagated up and crashes the portal's event loop.

that isn't ideal, so better options for fixing this, i think, are:

1. switch _trio.BlockingPortal._spawn_task_from_thread from using trio.from_thread.run_sync(BlockingPortal._task_group.start_soon, ..., trio_token=...) to using TrioToken.run_sync_soon(wrapper, ...), where wrapper is essentially just disable_ki_protection(AsyncioBackend.run_sync_from_thread.<locals>.wrapper).

   this is more-or-less equivalent to writing a version of TrioBackend.run_sync_from_thread that has a relaxed deadlock heuristic.

2. (in trio) relax the deadlock heuristic: see from_thread.run(_sync)? fails in any thread running Trio python-trio/trio#2534

Footnotes

1. https://github.com/python-trio/trio/blob/4286063466aade470d531ddfb42664f07ae38583/trio/_threads.py#L248-L254 ↩

[agronholm]

I'm confused. Why would you ever want to call that function from within an existing event loop?

[gschaffner]

it's useful to do for the reason described in #341. it allows one to run async code in a blocking way and from sync code. a function like

def foo():
    ...
    with anyio.from_thread.start_blocking_portal() as thread_portal:
        ...
        bar_ret = thread_portal.call(bar)
        ...
    ...

will work no matter whether foo is called by (a) fully synchronous code (code with no event loop above) or by (b) synchronous code with an event loop above it.

(asyncio | trio).run(bar) would work as an alternative in case (a), of course. for case (b), an alternative to using a separate thread is greenback, but it has its own problems¹.

Footnotes

1. https://github.com/python-trio/trio/issues/2534#issuecomment-1403290474 ↩