[NativeAOT-LLVM] support `System.Net.Http.HttpClient` on WASIp2 #2614

dicej · 2024-06-13T19:16:17Z

This adds WasiHttpHandler, a new implementation of HttpMessageHandler based on
wasi:http/outgoing-handler, plus tweaks to System.Threading to allow async Tasks to work in a single-threaded context, with ThreadPool work items dispatched from an application-provided event loop.

WASIp2 supports asynchronous I/O and timers via wasi:io/poll/pollable resource handles. One or more of those handles may be passed to wasi:io/poll/poll, which will block until at least one of them is ready. In order to make this model play nice with C#'s async/await and Task features, we need to reconcile several constraints:

WASI is currently single-threaded, and will continue to be that way for a while.
C#'s async/await and Task features require a working ThreadPool implementation capable of deferring work.
A WASI component can export an arbitrary number of functions to the host, and though they will always be called synchronously from the host, they need to be able to perform asynchronous operations before returning.
WASIp3 (currently in the design and prototype phase) will support asynchronous exports, with the top level event loop running in the host instead of the guest, and wasi:io/poll/pollable will no longer exist. Therefore, we don't want to add any temporary public APIs to the .NET runtime which will become obsolete when WASIp3 arrives.

The solution we arrived at looks something like this:

Tweak the existing ThreadPool implementation for WASI so that methods such as RequestWorkerThread don't throw PlatformNotSupportedExceptions (i.e. allow work items to be queued even though the "worker thread" is always the same one that is queuing the work)
Add two new methods to Thread:
- internal static void Dispatch: Runs an iteration of the event loop, draining the ThreadPool queue of ready work items and calling wasi:io/poll/poll with any accumulated pollable handles
- internal static Task Register(int pollableHandle): Registers the specified pollable handle to be polled during the current or next call to Dispatch
- Note that these methods are internal because they're temporary and should not be part of the public API, but they are intended to be called via UnsafeAccessor by application code (or more precisely, code generated by wit-bindgen for the application)

The upshot is that application code can use wit-bindgen (either directly or via the new componentize-dotnet package) to generate async export bindings which will provide an event loop backed by Thread.Dispatch. Additionally, wit-bindgen can transparently convert any pollable handles returned by WASI imports into Tasks via Thread.Register, allowing the component to await them, pass them to a combinator such as Task.WhenEach, etc.

Later, when WASIp3 arrives and we update the .NET runtime to target it, we'll be able to remove some of this code (and the corresponding code in wit-bindgen) without requiring significant changes to the application developer's experience.

This PR contains a few C# source files that were generated by wit-bindgen from the official WASI WIT files, plus scripts to regenerate them as desired.

dicej · 2024-06-13T19:19:56Z

TODO items:

Upstream wit-bindgen changes so we can update the code generation scripts to use an official release
~~Determine the correct way to sort response headers into HttpResponseHeaders and HttpContentHeaders~~
Run applicable existing tests and/or new tests on a WASIp2-capable runtime such as Wasmtime

dicej · 2024-06-13T20:30:28Z

Ah, that CI failure was useful; now I see how WASI tests are currently run. Looks like I'll need to make some changes to convert the test module(s) into component(s) and run them on a WASIp2-compatible runtime.

src/libraries/System.Net.Http/src/System/Net/Http/WasiHttpHandler/WasiHttpHandler.cs

src/native/external/wasi-sdk-p2.cmake

dicej · 2024-06-28T15:59:14Z

Status update on this: CI is green, and I've manually tested WasiHttpHandler by way of HttpClient to verify it works. However, I could use some guidance on how to automate this testing as part of CI.

AFAICT, CI currently only runs the library smoke tests for both the WASI and browser targets. I can see that e.g. ResponseStreamTest has methods like BrowserHttpHandler_Streaming, so it looks like there is test coverage for TARGET_BROWSER, but I'm not seeing how or where those tests are run, nor how to replicate them for WASI.

I've tried running the System.Net.Http.Functional.Tests suite, but have not had much success with that (many of the tests rely directly or indirectly on TaskAwaiter.GetResult, which leads to an infinite busy wait), and I'm not sure if debugging that is a good use of time.

So my question is: What's the best (read: most practical and efficient) way to add test coverage for WasiHttpHandler such that it can be run as part of CI (or elsewhere if there's some other automated testing infrastructure I'm missing)?

Meanwhile, I'm going to mark this "ready for review" since I believe it's ready for feedback even with the testing question unresolved.

jkotas · 2024-06-28T16:14:10Z

src/tests/nativeaot/SmokeTests/DynamicGenerics/DynamicGenerics.main.cs

@@ -165,7 +165,7 @@ public static int Main(string[] args)
        if (passed && CoreFXTestLibrary.Internal.Runner.NumPassedTests > 0)
        {
            CoreFXTestLibrary.Logger.LogInformation("All tests PASSED.");
-            return 100;


Using 100 to indicate success is a convention used by all tests under src/tests. Changing this convention in the wasm branch is going to be perpetual conflict with upstream. Is this change really necessary?

Initially, I followed @akoeplinger 's advice to print the exit code to standard out, but then @SingleAccretion advised me to change the codes instead. I'm happy to do whatever you all think is best here.

For context: the reason we need to do something special for WASIp2 here is WebAssembly/wasi-cli#11

+1 for xharness solution rather than editing 100 in too many places

I do not see how you can avoid modifying the tests that are Main-based. There is no code to be hooked there (unless you start thinking of some really involved workaround like making everything use CustomMain).

If we need to modify the tests regardless, the simplest thing is to change the exit code.

(Actually, I wanted to probe grounds for doing this upstream as well, but it may be problematic.)

could we just modify the native C main to printf exit code ?
Perhaps only when the application is build with some flags on, so that production applications don't do it.

could we just modify the native C main to printf exit code ?
Perhaps only when the application is build with some flags on, so that production applications don't so it.

I have not seen a precedent for adding such test hooks into production code.

It wouldn't work for cases that exit abnormally (Environment.Exit) - we have one such test.

Especially for NAOT the application is recompiled every time, so #ifdef seems OK to me.

For C# exit, we could have hook in PAL. But I'm not sure what to do about abort() somewhere in native code.

Especially for NAOT the application is recompiled every time, so #ifdef seems OK to me. For C# exit, we could have hook in PAL

The bootstrapper, which is where the native main is defined, is only compiled once, as part of the runtime build. I agree it is technically possible to make it work (e. g. via clever use of weak symbols, or compiler intrinsics, or something else).

My point is that it would not be to the scale of the problem, which is just running these smoke tests. If we don't convert all tests to use the zero exit code (unlikely), or don't wait enough for WASI P2.1 to add support for exit codes (possible but not terribly likely), then, when the question of how to run all of the runtime tests comes up - which will be upstream, it will be solved in some more involved manner. At that time, the same solution, if technically possible, will be adopted downstream too.

Edit: we'll be getting functional exit codes in a few months: WebAssembly/wasi-cli#44.

pavelsavara · 2024-07-01T10:26:52Z

Meanwhile, I'm going to mark this "ready for review" since I believe it's ready for feedback even with the testing question unresolved.

Yeah, enabling more tests should be independent PR.

It takes non-trivial effort in the first pass.

Also because at the same time it will test host's HTTP stack and interop, including HTTP edge cases.

I've tried running the System.Net.Http.Functional.Tests suite

That's one we want to use but we only use tests which are fully async.
Meaning tests which don't contain blocking Task.Wait and yield to the host/browser.

There is TARGET_BROWSER and PlatformDetection.IsBrowser with variations.
We will need to use the same/similar conditional tests.

[ConditionalTheory(typeof(PlatformDetection), nameof(PlatformDetection.IsBrowserDomSupportedOrNotBrowser))]

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsChromium))]

[ConditionalFact(typeof(PlatformDetection), nameof(PlatformDetection.IsBrowser))]

[ConditionalClass(typeof(PlatformDetection), nameof(PlatformDetection.IsBrowserDomSupportedOrNotBrowser))]

https://github.com/dotnet/runtime/blob/f8bcb89796d3c83264fffd913e5196d29d8d730e/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.cs#L50-L53

https://github.com/dotnet/runtime/blob/f8bcb89796d3c83264fffd913e5196d29d8d730e/src/libraries/Common/tests/TestUtilities/System/PlatformDetection.cs#L147-L151

The HTTP server role is played by NetCoreServer for browser.
https://github.com/dotnet/runtime/tree/main/src/libraries/Common/tests/System/Net/Prerequisites/NetCoreServer

https://github.com/dotnet/runtime/blob/f8bcb89796d3c83264fffd913e5196d29d8d730e/src/libraries/Common/tests/System/Net/Prerequisites/LocalEchoServer.props#L15-L19

https://github.com/dotnet/runtime/blob/main/src/libraries/Common/tests/System/Net/Configuration.Http.cs

Also note, that some of the tests assume HTTP server in the same process.
With browser that's not possible and so we offload the HTTP server role to xharness (helper which is driving the browser).
They communicate via WebSocket, which we don't have for WASI yet.
See https://github.com/dotnet/runtime/tree/main/src/libraries/Common/tests/System/Net/Prerequisites/RemoteLoopServer

Things which are broken should be marked with filter for broken scenario and a link to GH issue for it.

[ActiveIssue("https://github.com/dotnet/runtime/issues/50957", typeof(PlatformDetection), nameof(PlatformDetection.IsBrowser), nameof(PlatformDetection.IsMonoAOT))]

src/coreclr/nativeaot/Runtime/Portable/CMakeLists.txt

...libraries/System.Net.Http/src/System/Net/Http/WasiHttpHandler/WasiHttpWorld_component_type.o

src/libraries/System.Private.CoreLib/src/System/WASIp2/generate-bindings.sh

src/tests/nativeaot/SmokeTests/SharedLibrary/LibraryWorld_cabi_realloc.o

Sometimes there are two Wasm files in Jco's output; sometimes three. In any case, hard-coding the number won't fly. Signed-off-by: Joel Dice <[email protected]>

Signed-off-by: Joel Dice <[email protected]>

SingleAccretion · 2024-08-28T14:06:37Z

src/libraries/Microsoft.Extensions.FileProviders.Physical/src/PhysicalFileProvider.cs

+            //  For WASI/browser/iOS/tvOS we will proactively fallback to polling since FileSystemWatcher is not supported.
+            if (OperatingSystem.IsWasi() || OperatingSystem.IsBrowser() || (OperatingSystem.IsIOS() && !OperatingSystem.IsMacCatalyst()) || OperatingSystem.IsTvOS())


Most library changes here should be upstreamed (we will merge them here before that, but it should only be temporary).

I have many such changes in messy PR here dotnet/runtime#105838
I don't expect to merge it in current form.
The problem is that attributes are visible on the runtime API, so we better do it right on the first attempt.
But I don't know exactly what would be PNSE and what would eventually work. And that will change with next WASI preview.

I'll admit I've been using this PR as a dumping ground for miscellaneous WASI fixes as I build demo apps (this one came up when testing AspNetCore). Happy to open separate, upstream PRs as appropriate, but I figured I'd collect them all here first, at least temporarily.

@pavelsavara: yeah, I've been avoiding attributes for the time being since I also don't know which features might be supported in the future. I'm pretty sure WASI will never support signal handling, but I could imagine it might support file notification someday.

SingleAccretion · 2024-08-28T14:07:20Z

src/libraries/System.Net.Http/src/System.Net.Http.csproj

@@ -462,7 +462,7 @@
  </ItemGroup>

  <!-- TODO-LLVM: This is not upstreamable because it makes the build runtime-specific. -->


Suggested change



There is another one above (line 27) that needs to be deleted.

SingleAccretion · 2024-08-28T14:07:41Z

src/libraries/System.Net.Http/src/System/Net/Http/HttpClientHandler.cs

 // TODO-LLVM: This is not upstreamable and should be deleted when https://github.com/dotnet/runtimelab/pull/2614 is merged
-#if TARGET_WASI && !NATIVE_AOT
+#if TARGET_WASI


I presume the TODOs here should be removed?

I'm rewriting the whole file upstream to respect child resources and more.

I wasn't sure if the TODO-LLVM comments were still relevant (i.e. did they refer to just the && !NATIVE_AOT part, or the whole #if conditional?) Sounds like they're no longer relevant, so I'll remove them.

There were introduced very recently in https://github.com/dotnet/runtimelab/pull/2605/files. I assumed it was a fix for some build break.

Signed-off-by: Joel Dice <[email protected]>

See bytecodealliance/wit-bindgen#1040 Signed-off-by: Joel Dice <[email protected]>

Signed-off-by: Joel Dice <[email protected]>

Otherwise, we end up overwriting `response.Content.Headers`. Signed-off-by: Joel Dice <[email protected]>

If one or more tasks have been canceled during the call to `ThreadPoolWorkQueue.Dispatch`, one or more tasks of interest to the application may have completed, so we return control immediately without polling, allowing the app to exit if it chooses. A practical example of this is in the SharedLibrary smoke test. Without this patch, that test will take over 100 seconds to complete, whereas with this patch it completes in under a second. Signed-off-by: Joel Dice <[email protected]>

Signed-off-by: Joel Dice <[email protected]>

...and hopefully make CI happier. Signed-off-by: Joel Dice <[email protected]>

I'm still investigating whether this actually _is_ a `wasmtime-wasi-http` bug; stay tuned. Signed-off-by: Joel Dice <[email protected]>

Signed-off-by: Joel Dice <[email protected]>

pavelsavara · 2024-11-08T13:21:36Z

I rebased this PR into #2758

This change was extracted from dotnet/runtimelab#2614, which includes various fixes to enable building on non-Windows systems. We're in the process of upstreaming the parts of that PR which are not specific to NativeAOT-LLVM, and this is the latest such change. Signed-off-by: Joel Dice <[email protected]>

…9830) This change was extracted from dotnet/runtimelab#2614, which includes various fixes to enable building on non-Windows systems. We're in the process of upstreaming the parts of that PR which are not specific to NativeAOT-LLVM, and this is the latest such change.

…net#109830) This change was extracted from dotnet/runtimelab#2614, which includes various fixes to enable building on non-Windows systems. We're in the process of upstreaming the parts of that PR which are not specific to NativeAOT-LLVM, and this is the latest such change.

dicej changed the title ~~support System.Net.Http.HttpClient on WASIp2~~ [NativeAOT-LLVM] support System.Net.Http.HttpClient on WASIp2 Jun 13, 2024

dicej mentioned this pull request Jun 13, 2024

[wasi] async/await on WASIp2 dotnet/runtime#102894

Closed

jkotas added the area-NativeAOT-LLVM LLVM generation for Native AOT compilation (including Web Assembly) label Jun 13, 2024

pavelsavara mentioned this pull request Jun 20, 2024

[wasi] wasi:http + WasiHttpHandler dotnet/runtime#103752

Merged

pavelsavara reviewed Jun 20, 2024

View reviewed changes

src/libraries/System.Net.Http/src/System/Net/Http/WasiHttpHandler/WasiHttpHandler.cs Outdated Show resolved Hide resolved

pavelsavara reviewed Jun 20, 2024

View reviewed changes

src/libraries/System.Net.Http/src/System/Net/Http/WasiHttpHandler/WasiHttpHandler.cs Outdated Show resolved Hide resolved

pavelsavara reviewed Jun 20, 2024

View reviewed changes

src/native/external/wasi-sdk-p2.cmake Outdated Show resolved Hide resolved

This was referenced Jun 20, 2024

Handle Proxy Worlds bytecodealliance/componentize-dotnet#25

Closed

Remove calls to wasm-tools component new and generate wasi 0.2.0 component directly bytecodealliance/componentize-dotnet#26

Closed

dicej force-pushed the system-net-http-wasi branch 5 times, most recently from 140cece to d3cf163 Compare June 26, 2024 16:26

dicej mentioned this pull request Jun 26, 2024

translate errno=-4094 errors to "no-such-device" on Windows bytecodealliance/jco#454

Merged

dicej force-pushed the system-net-http-wasi branch 3 times, most recently from 8461ce7 to bb76b64 Compare June 26, 2024 22:27

dicej mentioned this pull request Jun 28, 2024

index out of bounds in crates/cranelift/src/debug/transform/attr.rs bytecodealliance/wasmtime#8884

Closed

dicej marked this pull request as ready for review June 28, 2024 15:59

jkotas reviewed Jun 28, 2024

View reviewed changes

pavelsavara added a commit to pavelsavara/runtime that referenced this pull request Jul 1, 2024

from dotnet/runtimelab#2614

8808137

pavelsavara reviewed Jul 1, 2024

View reviewed changes

src/coreclr/nativeaot/Runtime/Portable/CMakeLists.txt Outdated Show resolved Hide resolved

pavelsavara reviewed Jul 1, 2024

View reviewed changes

...libraries/System.Net.Http/src/System/Net/Http/WasiHttpHandler/WasiHttpWorld_component_type.o Outdated Show resolved Hide resolved

pavelsavara reviewed Jul 1, 2024

View reviewed changes

src/libraries/System.Private.CoreLib/src/System/WASIp2/generate-bindings.sh Outdated Show resolved Hide resolved

pavelsavara reviewed Jul 1, 2024

View reviewed changes

src/tests/nativeaot/SmokeTests/SharedLibrary/LibraryWorld_cabi_realloc.o Outdated Show resolved Hide resolved

dicej force-pushed the system-net-http-wasi branch from aecff25 to 28d758f Compare August 26, 2024 20:09

dicej added 3 commits August 26, 2024 16:42

fix file count confusion in SharedLibraryDriver.mjs

e32a223

Sometimes there are two Wasm files in Jco's output; sometimes three. In any case, hard-coding the number won't fly. Signed-off-by: Joel Dice <[email protected]>

fix Linux tests

e79642e

Signed-off-by: Joel Dice <[email protected]>

use TaskCompletionSource.SetCanceled instead of SetException

2f3ce60

Signed-off-by: Joel Dice <[email protected]>

SingleAccretion reviewed Aug 28, 2024

View reviewed changes

dicej and others added 10 commits August 28, 2024 09:29

use Stopwatch in HttpClient test to verify timeout works as expected

18f9f6c

Signed-off-by: Joel Dice <[email protected]>

provide more informative exception messages in WasiHttpHandler

ebdbabf

Signed-off-by: Joel Dice <[email protected]>

remove obsolete TODO-LLVM comments

9d9ed59

Signed-off-by: Joel Dice <[email protected]>

update wit-bindgen and regenerate bindings

4879365

See bytecodealliance/wit-bindgen#1040 Signed-off-by: Joel Dice <[email protected]>

to match dotnet/runtime#107096

137614e

fix disposal of pollable

b25bc9e

update wit-bindgen and regenerate bindings (again)

5f35285

Signed-off-by: Joel Dice <[email protected]>

specify zero expected exit code for WASIp2 smoke test

ddbd68a

Signed-off-by: Joel Dice <[email protected]>

convert response headers after setting content

e989a63

Otherwise, we end up overwriting `response.Content.Headers`. Signed-off-by: Joel Dice <[email protected]>

dicej force-pushed the system-net-http-wasi branch from c1fd2a7 to 98bf4dc Compare August 29, 2024 18:49

assert HTTP test in SharedLibrary completes promptly

757eeb9

Signed-off-by: Joel Dice <[email protected]>

dicej mentioned this pull request Sep 3, 2024

Additions to support gRPC WebAssembly/wasi-http#128

Open

dicej added 3 commits September 3, 2024 16:21

support reading trailers in WasiInputStream

dfdd8bc

Signed-off-by: Joel Dice <[email protected]>

increase max elapsed time in SharedLibrary test

1736bdd

...and hopefully make CI happier. Signed-off-by: Joel Dice <[email protected]>

work around possible wasmtime-wasi-http bug regarding trailing headers

dd5f8ae

I'm still investigating whether this actually _is_ a `wasmtime-wasi-http` bug; stay tuned. Signed-off-by: Joel Dice <[email protected]>

bacongobbler mentioned this pull request Sep 11, 2024

WIP: add gRPC sample dicej/spin-dotnet-sdk#21

Draft

remove temporary http-p2 smoke test

68ec226

Signed-off-by: Joel Dice <[email protected]>

pavelsavara mentioned this pull request Nov 8, 2024

[NativeAOT-LLVM] support System.Net.Http.HttpClient - leftovers #2758

Closed

pavelsavara closed this Nov 8, 2024

dicej mentioned this pull request Nov 14, 2024

fix warnings in coreclr/jit/compiler.cpp for non-Windows targets dotnet/runtime#109830

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[NativeAOT-LLVM] support `System.Net.Http.HttpClient` on WASIp2 #2614

[NativeAOT-LLVM] support `System.Net.Http.HttpClient` on WASIp2 #2614

dicej commented Jun 13, 2024 •

edited

Loading

dicej commented Jun 13, 2024 •

edited

Loading

dicej commented Jun 13, 2024

dicej commented Jun 28, 2024 •

edited

Loading

jkotas Jun 28, 2024

dicej Jun 28, 2024

pavelsavara Jul 1, 2024

SingleAccretion Jul 1, 2024

pavelsavara Jul 1, 2024 •

edited

Loading

SingleAccretion Jul 1, 2024

pavelsavara Jul 1, 2024

SingleAccretion Jul 1, 2024 •

edited

Loading

pavelsavara commented Jul 1, 2024 •

edited

Loading

SingleAccretion Aug 28, 2024

pavelsavara Aug 28, 2024

dicej Aug 28, 2024

SingleAccretion Aug 28, 2024

SingleAccretion Aug 28, 2024

pavelsavara Aug 28, 2024

dicej Aug 28, 2024

SingleAccretion Aug 28, 2024

pavelsavara commented Nov 8, 2024

		// For WASI/browser/iOS/tvOS we will proactively fallback to polling since FileSystemWatcher is not supported.
		if (OperatingSystem.IsWasi() \|\| OperatingSystem.IsBrowser() \|\| (OperatingSystem.IsIOS() && !OperatingSystem.IsMacCatalyst()) \|\| OperatingSystem.IsTvOS())

		@@ -462,7 +462,7 @@
		</ItemGroup>

		<!-- TODO-LLVM: This is not upstreamable because it makes the build runtime-specific. -->

[NativeAOT-LLVM] support System.Net.Http.HttpClient on WASIp2 #2614

[NativeAOT-LLVM] support System.Net.Http.HttpClient on WASIp2 #2614

Conversation

dicej commented Jun 13, 2024 • edited Loading

dicej commented Jun 13, 2024 • edited Loading

dicej commented Jun 13, 2024

dicej commented Jun 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pavelsavara Jul 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SingleAccretion Jul 1, 2024 • edited Loading

Choose a reason for hiding this comment

pavelsavara commented Jul 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pavelsavara commented Nov 8, 2024

[NativeAOT-LLVM] support `System.Net.Http.HttpClient` on WASIp2 #2614

[NativeAOT-LLVM] support `System.Net.Http.HttpClient` on WASIp2 #2614

dicej commented Jun 13, 2024 •

edited

Loading

dicej commented Jun 13, 2024 •

edited

Loading

dicej commented Jun 28, 2024 •

edited

Loading

pavelsavara Jul 1, 2024 •

edited

Loading

SingleAccretion Jul 1, 2024 •

edited

Loading

pavelsavara commented Jul 1, 2024 •

edited

Loading