More plugins and policy ideas

[darosior]

No description provided.

[edouardparis]

Async plugin

An async plugin refers to a plugin that cannot respond instantly to the request "must the attempt be revaulted", it may require to gather information from external services or approval from human user.

using the plugin process call (PUSH)

The plugin has an internal routine that is running and reading its internal state/store.
The state/store is updated by a PUSH event which is the new_tip process call from miradord.

miradord                       async plugin
    │                            │
    │new_tip{new_attempts}       │
    ├──────────────────────────► │
    │                            │ ──────────┐  check its state
    │                            │           │  store the new attempt
    │                            │ ◄─────────┘  if not exist
    │                revault []  │
    │◄───────────────────────────┤
    │                            │
    │                            │  ──────────┐ check periodically
    │                            │            │ its state for new stored
    │                            │  ◄─────────┘ attempt
    │                            │
    │                            │                          user/external service
    │                            │ notify new attempt
    │                            ├───────────────────────────► │
    │                            │                             │
    │                            │ wait for response           │
    │                            │                             │
    │                            │ ◄───────────────────────────┤
    │                            │            revault !        │
    │                            │                             │
    │                            │                             │
    │                            ├────────┐ internal routine   │
    │                            │        │ gather info and    │
    │                            │◄───────┘ make decision about│
    │                            │          new attempts       │
    │new_tip{new_attempts}       │
    ├──────────────────────────► │
    │                            │ ──────────┐  check its state
    │                            │           │
    │                            │ ◄─────────┘
    │                revault[o]  │
    └◄───────────────────────────┘

Using an API exposed permanently by miradord (PULL)

The async plugin is running in the background and it is pulling periodically information from miradord.
In case of new_attempt pulled, it is communicating with external services or user.

miradord                    async plugin                           external services
     │                             │                                 users
     │    is there new attempts ?  │                                    │
     │ ◄───────────────────────────┤                                    │
     │the attempts                 │                                    │
     ├───────────────────────────► │                                    │
     │                             │  new attempt, give me infos !      │
     │                             ├──────────────────────────────────► │
     │                             │                                    │
     │                             │                                    │
     │                             │   info about attempt.              │
     │                             │  ◄─────────────────────────────────┤
     │                             │   info about attempt.              │
     │                             │  ◄─────────────────────────────────┤
     │                             │                                    │
     │                             ├────────┐ internal routine          │
     │                             │        │ gather info and           │
     │                             │◄───────┘ make decision about       │
     │   revault this attempt !    │          new attempts              │
     │ ◄───────────────────────────┤                                    │
     │    ACK                      │                                    │
     ├───────────────────────────► │                                    │
     │                             │                                    │
     │                             │                                    │

A third way

The plugin receive the PUSH, but will not answer instantly and will use an exposed API to answer later.

miradord                       async plugin
    │                            │
    │new_tip{new_attempts}       │
    ├──────────────────────────► │
    │                            │ ──────────┐  check its state
    │                            │           │  store the new attempt
    │                            │ ◄─────────┘  if not exist
    │                revault []  │
    │◄───────────────────────────┤
    │                            │
    │                            │  ──────────┐ check periodically
    │                            │            │ its state for new stored
    │                            │  ◄─────────┘ attempt
    │                            │
    │                            │                          user/external service
    │                            │ notify new attempt
    │                            ├───────────────────────────► │
    │                            │                             │
    │                            │ wait for response           │
    │                            │                             │
    │                            │ ◄───────────────────────────┤
    │                            │            revault !        │
    │                            │                             │
    │                            │                             │
    │                            ├────────┐ internal routine   │
    │                            │        │ gather info and    │
    │                            │◄───────┘ make decision about│
    │                            │          new attempts       │
    │   revault this attempt  API|
    │ ◄──────────────────────────|
    │   ACK                      │
    ├──────────────────────────► │
    │                            │
    │                            │              

[darosior]

I see how this may be useful to not block for the response to new_attempts. Maybe trying to have a "hooked" plugin flow was overkill and we should juste have used an RPC interface?

Is there any advantage to a mixed approach (similar to C-lightning plugins) where we both have hooks and expose an RPC interface? I don't think so for us.

Is there any drawback to have the plugins processes not be managed by the WT?

In any case that'd be a very large refactoring. I don't think we should do that soon, but at least we know the interface should be changed to support "remote manual Cancel" and maybe other things that would not be reasonable to be blocking on.
Also, blocking the main process was probably not wise: it largely increases the DoS surface of the watchtower.

[edouardparis]

A new approach based on "hooked" plugin flow.

When new_tip:

• The watchtower call each plugin in a new thread with the plugin config and the new_tip information and log the stderr response. Each thread wait the plugin to finish, but the plugins are not giving a "response to revault" to stdout only logs.
• Each plugin will do indepently their stuff and in case it may want to revault an unvaulting vault, it will call a method exposed by an rpc API.

I think it will give the advantage of:

• having every plugins errors centralised in watchtower logs (easier for admin to debug)
• A simple hook system: information is only passed to plugins and plugins are run by watchtower
• parallelism and non blocking system.
• A very simple RPC API (just the revault method for now).