Wallet sync protocol

[oleganza]

Problem

The indexing protocol #426 is designed to operate on full blockchain data, as a part of a live full node, or as a method to process archive data.

Lightweight wallets need a complementary protocol that eliminates the need to download and process the entire blockchain.

1: Wallets need to constantly update their utreexo proofs with as little bandwidth overhead as possible.

A couple of ways this could be done:
a) either by requesting an updated proof for any given utxo from a full node that indexes ALL the utxos, or at least all utxos of their clients (e.g. bank's backend).
b) or replaying utreexo transformations (Catchup maps) from any prior point to the most recent block. This is a bit more involved computationally for the server, but reduces the storage needs (although storage might be much cheaper than hashing).

2: Wallets need to detect incoming payments

Wallets need to sync up with the server on the subset of the newly created outputs matching a prefix. E.g. for K pending predicates, wallet can create 1..K unique 256-bit prefixes and request subset of block data matching those. This way, it can save bandwidth down to K/256 of tx/s rate.

Obviously, since the wallet is not fully validating, only coalesced utreexo proofs (see #338) need to be transmitted, w/o full tx program and r1cs proof.

The tricky parts:

• how large the buffer of historical data the streaming server needs to keep around?
• how to efficiently re-calculate utreexo proof updates for requested proofs? Maybe if we are doing sharding instead of a-la-carte filtering, we can precompute and store proofs?
• how to coalesce utreexo proofs for storage and transmission. The idea is to avoid repetition of the 32-byte hashes and use some graph structure with short indices instead. See utreexo: aggregated proofs #338.

This is largely the same problem as №1, since the detected payment may occur at block 1001, but the wallet syncs up only at block 1005, and at that point we need to apply all utreexo transformations in-between.

3: Wallets need to keep track of unconfirmed outgoing transactions and their unconfirmed utxos

This is a requirement to allow making chains of unconfirmed transactions w/o being blocked on waiting till tx is published. Index protocol (#426) has to solve the same problem, so we probably can have some common library solution there.