Decouple Celery and Redis from backend into separate Cloud.gov services

[jtimpe]

Background

Currently, Celery workers and Redis are deployed within the same application as the backend. Decoupling Celery workers into their own application, and Redis into a separate managed service, will help optimize resource usage on the backend server. It will also allow each of these services (Celery, Redis, and the backend) to be independently scalable.

• [DRAFT] Connect to managed redis service #2590 explores how to connect Celery and Django to a managed Redis service in Cloud.gov.
• Deploy celery as a separate cloud.gov application #2592 details the steps for implementing separate Celery and Redis services in Cloud.gov (reproduced as tasks below).
• 2592/separate celery #2773 implements Celery and Redis separated from the backend, including infrastructure and deployment changes.

Acceptance Criteria

• Celery is deployed as its own application in cloud.gov
• Redis is deployed as a managed service in cloud.gov, bound to both backend and celery
  • Only one redis per environment (dev, staging, prod) is deployed. Use channel filters to separate traffic from different apps (where applicable)
• Modifications to infrastructure or the applications are deployed via circle ci
• Testing Checklist has been run and all tests pass
• README is updated, if necessary

Tasks

• Deploy redis as a managed service
  • Remove redis from application dependencies in apt.yml
  • Remove redis startup from gunicorn_start.py
  • Remove REDIS_URI env var from manifest.buildpack.yml
  • Deploy managed redis in cloud.gov, bind the service to celery and backend in deploy-backend.sh
• Deploy celery as a separate application
  • Copy manifest.buildpack.yml to manifest.celery.yml (or similar)
  • Update the command in the new manifest to start celery instead (copy/remove the celery start lines from gunicorn_start.py)
  • Copy/abstract the logic in deploy-backend.sh to also deploy the new celery manifest. The environment variable logic needs to be the same for both celery/backend, and celery must be bound to the same services as backend.
• Run Testing Checklist and confirm all tests pass