From 559d7bc45625909f71a5c8cfb89c066141669c49 Mon Sep 17 00:00:00 2001 From: Gaukas Wang Date: Fri, 8 Mar 2024 14:02:45 -0700 Subject: [PATCH] new: add API list (#7) * new: watm-go desc and api Signed-off-by: Gaukas Wang * update: more on go runtime Signed-off-by: Gaukas Wang * new: watm v0 spec Signed-off-by: Gaukas Wang --------- Signed-off-by: Gaukas Wang --- runtime/go/go.md | 80 ++++++++++++++++++++++++++++++++-- transport-module/go/go.md | 12 +++++- transport-module/go/watm.md | 73 +++++++++++++++++++++++++++++++ transport-module/spec.md | 85 +++++++++++++++++++++++++++++++++++++ 4 files changed, 246 insertions(+), 4 deletions(-) create mode 100644 transport-module/go/watm.md create mode 100644 transport-module/spec.md diff --git a/runtime/go/go.md b/runtime/go/go.md index 4a7b14b..c12ccbc 100644 --- a/runtime/go/go.md +++ b/runtime/go/go.md @@ -7,10 +7,84 @@ has_children: true permalink: /runtime/go.html --- -# Runtime Library in Go +# Runtime Library in Go: `water` +[![Go Doc](https://pkg.go.dev/badge/github.com/refraction-networking/water.svg)](https://pkg.go.dev/github.com/refraction-networking/water) -WATER Runtime Library for Go (a.k.a. `water-go`) is built with [wazero](https://github.com/tetratelabs/wazero) providing the WebAssembly runtime with WASI support, and implements an abstracted network programming interface that roughly ressembles the standard `net` package in Go. +`water` (a.k.a. `water-go` in contrast to `water-rs`) is a WATER Runtime Library build in Go. It uses the WebAssembly runtime with WASI support from [wazero](https://github.com/tetratelabs/wazero), and implements an abstracted network programming interface that roughly ressembles the standard `net` package in Go. ## Cross-platform Support +Project WATER is designed to be cross-platform and cross-architecture. See [Cross-platform Compatibility (Go)](./go/cross-platform.html) for more details. -Project WATER is designed to be cross-platform and cross-architecture. See [Cross-platform Compatibility (Go)](./go/cross-platform.html) for more details. \ No newline at end of file +## Usage + +### Importing `water` +To use `water` in a Go project, simply import it as a module. + +By default, `water` does not recognize any transport modules as there can be many different +specifications of transport modules. To use a specific transport module, import its implementation +as well. + +```go +import ( + "github.com/refraction-networking/water" + _ "github.com/refraction-networking/water/transport/v0" // import the v0 transport module spec +) +``` + +### Working Mode: Dialer +Dialer acts like a client. It actively creates connections to a remote server (and usually is the one who sends the first message). + +```go +// ... + + // Load the WebAssembly binary into wasm as []byte. + // The rest of the code on this page assumes that wasm is already loaded. + wasm, _ := os.ReadFile("./examples/v0/plain/plain.wasm") + + config := &water.Config{ + TransportModuleBin: wasm, + } + + dialer, _ := water.NewDialerWithContext(context.Background(), config) + conn, _ := dialer.DialContext(context.Background(),"tcp", remoteAddr) + +// ... +``` + +### Working Mode: Listener +Listener acts like a server. It listens on a network address and wait for +incoming connections to accept. + +```go + lis, _ := config.ListenContext(context.Background(), "tcp", localAddr) + defer lis.Close() + log.Printf("Listening on %s", lis.Addr().String()) + + for { + conn, err := lis.Accept() + handleConn(conn) + } +``` + +### Working Mode: Relay +A relay combines the functionalities of both a dialer and a listener. It works +like a forward proxy, accepting connections from a client and forwarding them to a +remote server by dialing a connection to the remote server. + +```go + relay, _ := water.NewRelayWithContext(context.Background(), config) + + relay.ListenAndRelayTo("tcp", localAddr, "tcp", remoteAddr) // blocking +``` + +## Troubleshooting + +### Enable `wazero` debug logs + +`wazero` is the WebAssembly runtime with WASI support that `water` uses. To enable debug logs from `wazero`, pass the values below via the `context.Context` + +```go + // example of enabling FileSystem, Poll, and Sock logging scopes of wazero + ctx = context.WithValue(ctx, experimental.FunctionListenerFactoryKey{}, + logging.NewHostLoggingListenerFactory(os.Stderr, logging.LogScopeFilesystem|logging.LogScopePoll|logging.LogScopeSock)) +``` \ No newline at end of file diff --git a/transport-module/go/go.md b/transport-module/go/go.md index 4ec02d1..ab82b18 100644 --- a/transport-module/go/go.md +++ b/transport-module/go/go.md @@ -11,4 +11,14 @@ permalink: /transport-module/go.html Go is one of the most popular programming languages for building network applications, and it is possible to build (most of) Go applications into WebAssembly. -Although the official Go compiler does not fully support WebAssembly System Interface (WASI) Preview 1's export feature, TinyGo, a Go compiler designed to build Go applications in a minimal environment, does provide great support for WASI. \ No newline at end of file +Although the official Go compiler does not fully support WebAssembly System Interface (WASI) Preview 1's export feature, TinyGo, a Go compiler designed to build Go applications in a minimal environment, does provide great support for WASI. + +## Quick Start + +To see a quick example of how to build a WATM in Go, see [WATM in Go Quick Start](./go/quick-start.html). + +## WATM Builder Helper + +We provide a helper library you can use to build WATM from WebAssembly-agnostic Go code. By using this helper, only a simple intermediate layer is needed to be implemented in order to integrate existing transport written in Go with WATM. + +See [WATM Builder Helper in Go](./go/watm.html) for more details. \ No newline at end of file diff --git a/transport-module/go/watm.md b/transport-module/go/watm.md new file mode 100644 index 0000000..ae92c91 --- /dev/null +++ b/transport-module/go/watm.md @@ -0,0 +1,73 @@ +--- +layout: default +title: WATM Builder Helper in Go +grand_parent: WebAssembly Transport Module +parent: WATM in Go +nav_order: 2 +--- + +# `watm`: WATM Builder Helper + +[![Go Reference](https://pkg.go.dev/badge/github.com/refraction-networking/watm.svg)](https://pkg.go.dev/github.com/refraction-networking/watm) + +We provide `watm` ([GitHub Repository](https://github.com/refraction-networking/watm)) as a helper library one can use to build WATM from WebAssembly-agnostic Go code. It would reduce a lot of boilerplate code and make the integration process much easier when you have an existing transport written in Go and does not want to customize the WATM's behavior too much. + +Currently, `watm` is still under active development and supports only TinyGo as the compiler. + +## `tinygo/v0` package + +[![Go Reference](https://pkg.go.dev/badge/github.com/refraction-networking/watm.svg)](https://pkg.go.dev/github.com/refraction-networking/watm/tinygo/v0) + +The `tinygo/v0` package provides a set of helper functions to build WATM with spec version 0 from WebAssembly-agnostic Go code. It is designed to be used with TinyGo. + + +
+ API Reference + +```go +type RelayWrapSelection bool + +const ( + RelayWrapRemote RelayWrapSelection = false + RelayWrapSource RelayWrapSelection = true +) + +type ConfigurableTransport interface { + Configure([]byte) error +} + +type DialingTransport interface { + SetDialer(dialer func(network, address string) (v0net.Conn, error)) + Dial(network, address string) (v0net.Conn, error) +} + + +type ListeningTransport interface { + SetListener(listener v0net.Listener) + Accept() (v0net.Conn, error) +} + +type WrappingTransport interface { + Wrap(v0net.Conn) (v0net.Conn, error) +} + +func BuildDialerWithDialingTransport(DialingTransport) +func BuildDialerWithWrappingTransport(WrappingTransport) + +func BuildListenerWithListeningTransport(ListeningTransport) +func BuildListenerWithWrappingTransport(WrappingTransport) + +func BuildRelayWithListeningDialingTransport(ListeningTransport, DialingTransport) +func BuildRelayWithWrappingTransport(WrappingTransport, RelayWrapSelection) +``` + +
+ +### `tinygo/v0/net` package + +[![Go Reference](https://pkg.go.dev/badge/github.com/refraction-networking/watm.svg)](https://pkg.go.dev/github.com/refraction-networking/watm/tinygo/v0/net) + +The `tinygo/v0/net` package simulates the standard `net` package in Go, and provides a set of helper functions to build WATM with spec version 0 from WebAssembly-agnostic Go code. It is designed to be used with TinyGo. + +If you are using `tinygo/v0` properly, then most likely you don't have to interact with this package too much. However, more flexibility is provided here if you want to customize the behavior of the WATM. + diff --git a/transport-module/spec.md b/transport-module/spec.md new file mode 100644 index 0000000..f207654 --- /dev/null +++ b/transport-module/spec.md @@ -0,0 +1,85 @@ +--- +layout: default +title: WATM Specification +parent: WebAssembly Transport Module +nav_order: 3 +--- + +# Current WebAssembly Transport Module (WATM) Specification +Currently we are in `v0` of the WebAssembly Transport Module (WATM) specification. This document describes the current version of the WATM specification. + +## API +The current version of WATM utilizes the WebAssembly System Interface (WASI) Preview 1 to provide the basic functionalities to interact between the WebAssembly module and the host environment. In addition, we define the following API for the transport module. + +### Imports + +WATM imports the following functions from the host environment: + +| Function Signature | Parameters | Results | Description | +| --- | --- | --- | --- | +|`host_dial` | - | `net_fd: s32` | Ask the host to connect to a remote server. | +| `host_accept` | - | `net_fd: s32` | Ask the host to accept an incoming connection. | +| ~~`host_defer`~~ | - | - | ~~Ask the host to defer the execution of the current function.~~ Deprecated: this function is equal to `NOP`. | +| `pull_config`* | - | `conf_fd: s32` | Ask the host to provide the configuration file. | + +### Exports + +WATM exports the following functions to the host environment: + +| Function Signature | Parameters | Results | Description | +| --- | --- | --- | --- | +| `_water_init` | - | `errno: s32` | Initialize the transport module. | +| `_water_dial` | `internal_fd: s32` | `net_fd: s32` | Connect to a remote server. | +| `_water_accept` | `internal_fd: s32` | `net_fd: s32` | Accept an incoming connection. | +| `_water_associate` | - | `errno: s32` | Associate the transport module with the host environment. | +| `_water_worker` | - | `errno: s32` | Start the worker thread. | + +## API Represented in Different Programming Languages +To help developers understand the API, we provide the following representations of the API in different programming languages that could be more intuitive than the above table. + +### C representation +(planned feature) + +### Go representation +(to be added) + +### Rust representation +(to be added) + +### WIT representation +Similarly, such API can be represented in WIT. + +_Note that the following is a simplified example and may not strictly follow the WIT syntax. In other words, we do not guarantee the following code to be compilable._ + +#### Imported Functions +```wit +// imports.wit +package env; + +interface host { + host_dial: func() -> (net_fd: s32); + host_accept: func() -> (net_fd: s32); + host_defer: func(); // deprecated + pull_config: func() -> (conf_fd: s32); +} + +world host-world { + import host; +} +``` + +#### Exported Functions +```wit +// exports.wit +interface wasm { + _water_init: func() -> (errno: s32) + _water_dial: func(internal_fd: s32) -> (net_fd: s32); + _water_accept: func(internal_fd: s32) -> (net_fd: s32); + _water_associate: func() -> (errno: s32); + _water_worker: func() -> (errno: s32); +} + +world wasm-world { + export wasm; +} +``` \ No newline at end of file