Skip to content
This repository has been archived by the owner on Feb 1, 2019. It is now read-only.

Design with libp2p

Jump to bottom
James Ray edited this page Jun 6, 2018 · 11 revisions

Questions are originally from https://github.com/Drops-of-Diamond/diamond_drops/issues/34#issuecomment-388055539

Please review the following questions that I've prepared after reading through the libp2p spec at https://github.com/libp2p/specs. Try to challenge/answer the questions collectively as a team:

  1. What libp2p protocol categories/roles and associated properties, security models, and respective network size are we using on each layer of our stack, and what are scenarios where multiple protocols would run simultaneously?

Generally I would say that we try to use the modules/crates implemented in Rust already, only developing a new module as needed. For security models, this can be found here. For network size, this is still under R&D, but there could be 100 shards or up to 2^9 (512) shards, either all in one torus shaped network or each with a separate network. Each shard could have up to 2^10 proposers each of which need to run a full node for that shard, so that is up to 2^10 full nodes per shard, and 2^19 (524288) nodes in total.

  1. What collection of transport protocols will we be running libp2p protocol over?

As noted above, I expect that libp2p-tcp-transport with TCP/IP should suffice; WebSockets is also implemented and can be used for real-time data transfer between hosts.

  1. i) What port(s) will we use on the libp2p network layer to perform operations for each of the transport protocols that we’ll be using?

For TCP, 443 for HTTP over TLS (HTTPS) and 80 for HTTP. Websockets can also work over these ports.

ii) What is an example scenario where we’ll be protocol multiplexing multiple transport protocols (i.e. UTP, UDT, WebSockets, SCTP, WebRTC, TCP Remy) through a single port?

This is defined by the end use. So we can let dapp developers use what is available in rust-libp2p, and develoop other modules, as needed, by popular demand, and as aligns with other priorities. Since Ethereum can be used for virtually any economic or governance tx, I imagine that there would be many examples where multiplexing would be useful, e.g.:

  • TCP/IP for conventional web browsing (e.g. in Mist);

  • see here for 9 applications of WebSockets, similarly used for real-time applications—note that this is outdated and WebRTC is now probably a more suitable choice than WebSockets (although being able to provide support for protocols that may be considered to be superceded is important); none of the following are implemented by rust-libp2p

  • WebRTC used for real-time interactive media applications such as video conferencing and livepeer;

  • SCTP would also be useful for livepeer and other multihoming applications like videoconferencing, as well as, like UDP, performance oriented applications. Note that WebRTC uses SCTP. Useful and was initially mainly used for telecom applications. Needs to be over UDP as OSes don't support direct SCTP transmission. Also used for mediating network access in the Diameter protocol, and for reliable server pooling (less useful for P2P).

  • TCP Remy looks interesting! I also had a quick look at the Xavier paper which is more useful for general uses rather than being designed for specific topologies, and sent an email to the author asking some questions.

  • UDT; UTP

    iii) What is an example scenario where we'll be multiplexing reliable stream(s) or unreliable datagram(s) in a connection from an IPFS node to other node(s) on the transport protocol in the network layer of the communication model?

Like above, UDP is not implemented in rust-libp2p, so that still has to be done in order to be able to do this. Unreliable datagram protocols are used in many applications where performance is is more important than reliability.

UDP is suitable for purposes where error checking and correction are either not necessary or are performed in the application; UDP avoids the overhead of such processing in the protocol stack. Time-sensitive applications often use UDP because dropping packets is preferable to waiting for packets delayed due to retransmission, which may not be an option in a real-time system. — https://en.wikipedia.org/wiki/User_Datagram_Protocol

Numerous key Internet applications use UDP, including: the Domain Name System (DNS), where queries must be fast and only consist of a single request followed by a single reply packet, the Simple Network Management Protocol (SNMP), the Routing Information Protocol (RIP)[1] and the Dynamic Host Configuration Protocol (DHCP).

Voice and video traffic is generally transmitted using UDP. Real-time video and audio streaming protocols are designed to handle occasional lost packets, so only slight degradation in quality occurs, rather than large delays if lost packets were retransmitted. Because both TCP and UDP run over the same network, many businesses are finding that a recent increase in UDP traffic from these real-time applications is hindering the performance of applications using TCP, such as point of sale, accounting, and database systems. When TCP detects packet loss, it will throttle back its data rate usage. Since both real-time and business applications are important to businesses, developing quality of service solutions is seen as crucial by some.[10]

Some VPN systems such as OpenVPN may use UDP while implementing reliable connections and error checking at the application level.— https://en.wikipedia.org/wiki/User_Datagram_Protocol#Applications

UDP could be used for ENS (that said with not being an expert with ENS or DNS), as well as with the above mentioned applications.

iv) What protocol(s) would we be mounting on top of libp2p with the connection?

I feel like this may have largely been answered above, i.e. it's up to dapp developers to choose or lobby if unsupported.

v) What type of connection would we be using?

Could be similar to (plus similar to above):

This may be a single TCP or UDP port, or a single connection through WebSockets or WebRTC. In a sense, the role of the TCP/UDP network stack -- i.e. multiplexing applications and connections -- may now be forced to happen at the application level.

vi) What platforms and network setups will we need libp2p to support?

Similar to as above: use what we have, implement as needed/per priorities.

vii) How will we build a multiaddr-based network package module like go-multiaddr-net in Rust, for IPFS to delegate the transport dialing to, and for scoping the implementation of other transport protocols.

https://github.com/multiformats/rust-multiaddr

viii) What portable format encodings do we need to use for protocol messages (i.e. protobuf)?

See https://github.com/libp2p/specs/blob/master/7-properties.md#755-portable-encodings, "this has so far not been needed. For now, we use protobuf for all protocol messages exclusively." Rust doesn't have a multicodec.

ix) What are scenarios when we will use multistream, multistream-select, and Stream Multiplexing? What default multiplexor will we use with Stream Multiplexing (i.e. HTTP/2, SPDY, QUIC, or SSH)?

This is application defined. rust-libp2p only has multistream-select. I imagine HTTP2 would be the default.

x) How will we use Multicodecs for each IPFS Protocol that we use?

See above.

  1. i) How do we need to encrypt communications on libp2p for security in our implementation? Under what scenarios would users not need to operate with encrypted communications?

Depends on the dapp.

ii) Is libp2p's application of cyphers with only a minimal portion of the whole TLS standard used sufficient for our implementation?

I don't know, I guess it depends on the application, but

Since the libp2p model begins with keys, libp2p only needs to apply ciphers.

  1. i) How will users running a node behind a NAT traverse it in order to establish connections to peers in our implementation of libp2p's IPFS?

rust-libp2p doesn't seem to implempent NAT traversal (which it SHOULD), but it does implement a relay protocol (which it MUST).

Then again, there's these results:

Undecided, we can address that as needed, but I'd say that implementing gossipsub is more important.

iii) How will we check if our implementation of NAT traversal is interoperable?

Due to protobuf (using the same proto file) I think this should be fairly straightforward, but we could talk to libp2p if/when we get to implementing this. The above implementation uses protobuf (message.proto), so if has the same file as other libp2p implementations that's a good start. However it appears that a test suite for NAT traversal is missing here, I've created an issue here.

iv) How will we implement optional fallback to connection relaying communication as a transport that may be turned off by users to ensure a full connectivity graph?

rust-libp2p recently implemented a relay protocol which we'd use.

https://github.com/libp2p/rust-libp2p/tree/master/relay

  1. What network topologies will we be using for our system's routing mechanisms, peer discovery, and message propagation?

This partly depends on whether we have one torus network or a network for each shard, but it would be one of the topology types other than centralized, although the current network is structured.

Ethereum’s P2P network layer (which is usually modeled as a structured graph like Kademlia DHT)—https://hackernoon.com/eclipse-attacks-on-blockchains-peer-to-peer-network-26a62f85f11

Gossipsub also has a peer connected to a list of other peers, also indicating a structured network.

Routing: rust-libp2p uses Kademlia DHT. Note that mDNS is used for LANs, which is a lower priority for Ethereum.

peer discovery: we can rule out mDNS-discovery as a low priority;

"Random-Walk is a Discovery Protocol for DHTs" (source), thus random-walk would be appropriate for kademlia. There is no result for the term in rust-libp2p. I don't like bootstrap nodes as it requires some trust, and is more centralized and attack-prone.

message propagation: gossipsub

  1. i) What information will records (IPFS Linked Data) in our network store for discoverability? ii) How will we use libp2p to locate resources in the network and side channels? Reference: https://github.com/libp2p/specs/blob/master/3-requirements.md#37-resource-discovery

  2. What libp2p messaging protocols will we let users use after they open a new stream and register a stream handler and why?

Gossipsub, to be implemented. Again, floodsub is inefficient (floods to all peers), although it is more secure, while gossipsub connects to a list of other peers for dialing and listening, so it's more efficient.

  1. How will we adopt naming conventions in our implementation of the network?

  1. What is a scenario where different libp2p subsystem interfaces (i.e. Peer Routing, Swarm, Distributed Record Store, and Discovery) would use each other in our system implementation? Reference: https://github.com/libp2p/specs/blob/master/4-architecture.md#4-architecture

  2. What Peer Routing subsystem mechanism will we implement and why? Reference: https://github.com/libp2p/specs/blob/master/4-architecture.md#41-peer-routing

  3. What are different Swarm scenarios that we need to handle (where we are using a stream)?

  1. What Distributed Record Store system will we use and why?
  1. i) What Discovery Protocols will we use and for what scenarios? ii) Will our Discovery Protocol need encrypted discovery beacons (i.e. mDNS)? iii) Would we use a Bootstrap-List Discovery Protocol and make it use configurable?
  1. Which of the libp2p modules (transports) listed at the following reference link will we be using with the Rust implementation of libp2p https://github.com/libp2p/rust-libp2p?
  1. How will we use the Swarm Dialer?
Clone this wiki locally