Optionally use pub/sub to deliver notifications

[timothyb89]

When available, https://cloud.google.com/pubsub/docs/ should be used to deliver notifications. HipChat servers are often slow to respond and can time out. PubSub would decouple notification failures from an otherwise successful action, and help to ensure delivery since we could then retry.

Also, when we don't need to run shell commands, we can use much smaller functions. A 256 MiB function can run for 60 seconds waiting on a slow HipChat endpoint at 1/8th the cost of our main function.

[timothyb89]

This can be expanded somewhat to improve the overall architecture of the bot. An HTTP call would put a message on a queue (using pubsub or below options) to be processed later. Messages might be:

• check for updates
• apply an update
• deliver a notification

This would improve things in several areas. We'd be able to retry failed tasks (e.g. notification delivery) without repeating all steps (potentially steps with side effects, like db changes or pull requests). We'd also have a better separation of components (core logic would move to a separate message processor) with a pipeline that could process complex jobs and schedule updates without hitting timeouts.

options for self-hosted:

• kue and kue-scheduler
• bull looks nice and has scheduling and delayed jobs built in

Google's PubSub is nice enough for simple message passing, but unfortunately has no way to schedule jobs. It'd be nice to cover all the use cases here, seeing as kue and bull do so nicely. functions-cron could work, but it's a bit ugly.

Whichever self hosted solution and pubsub would go behind a plugin interface like datastore backends currently have.