set up scheduler to create tasks for idle workers

[elfkuzco]

Rationale

Set up scheduler to create test entries for idle workers (workers who have not been seen within the configured interval). This solves #7

Changes

• update worker's last_seen_on whenever the worker submits an update to a test
• expire tests that are still pending after the configured interval
• create tests for workers that are idle
• add functionality to create a worker and save their private keys to a file. I suppose we might want to expose a CLI command to create a worker and assign them to countries using this functionality.
• allow the num_tests marker to support fixed status and/or country for tests
• add scheduler service to the dev docker compose file

Proposal

What do you think about adding a root command which takes the other commands as subcommands? Like git takes several subcommands. For example, if the root command was named mirrors-qa-backend, we could have update-mirrors, create-worker, scheduler as subcommands which take their own specific arguments. It would probably be done in a later ticket as the commands and entrypoints grow and some of them might share arguments like --verbose

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 62.90323% with 69 lines in your changes missing coverage. Please review.

Project coverage is 76.52%. Comparing base (460ac2d) to head (f195d64).

Files	Patch %	Lines
backend/src/mirrors_qa_backend/scheduler.py	0.00%	31 Missing ⚠️
backend/src/mirrors_qa_backend/db/worker.py	71.05%	10 Missing and 1 partial ⚠️
backend/src/mirrors_qa_backend/entrypoint.py	0.00%	5 Missing ⚠️
backend/src/mirrors_qa_backend/routes/auth.py	70.58%	4 Missing and 1 partial ⚠️
...kend/src/mirrors_qa_backend/routes/dependencies.py	64.28%	5 Missing ⚠️
...ckend/src/mirrors_qa_backend/settings/scheduler.py	0.00%	5 Missing ⚠️
backend/src/mirrors_qa_backend/db/__init__.py	50.00%	2 Missing ⚠️
backend/src/mirrors_qa_backend/extract.py	0.00%	2 Missing ⚠️
backend/src/mirrors_qa_backend/main.py	33.33%	2 Missing ⚠️
backend/src/mirrors_qa_backend/db/tests.py	90.90%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #20      +/-   ##
==========================================
- Coverage   80.55%   76.52%   -4.04%     
==========================================
  Files          20       25       +5     
  Lines         540      656     +116     
  Branches       47       57      +10     
==========================================
+ Hits          435      502      +67     
- Misses         92      140      +48     
- Partials       13       14       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rgaudin]

Very good.

About the create-worker thing, I think it makes little sense to handle this key stuff here (in the backend).

What I think makes sense to me is a create-worker script in backend that takes a private key as string or fpath and reads it to build pubkey/fingerprint.

Setting up a worker involves various steps that will need to be documented and including an openssl command to generate one privkey (if not supplied) is not a concern.
In your workflow, the worker is created on backend and the file has to be transferred to the worker machine…

In that sense, I think we can get rid of the key generation code in cryptography and use a static fixture for the tests

[rgaudin]

Great! thank you.

See the couple of minor suggestions to apply before merging.