Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dispatchable workflow for deleting test model runs #60

Merged
merged 24 commits into from
Nov 21, 2023
Merged

Add dispatchable workflow for deleting test model runs #60

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
a35baca
Add delete-model-runs workflow
jeancochrane Nov 16, 2023
c5c2404
Temporarily run a readonly version of the delete-model-runs workflow…
jeancochrane Nov 16, 2023
3f0b8c1
Install libgit2-dev for git2r in delete-model-runs.yaml
jeancochrane Nov 16, 2023
d70deea
Fix R styler error in delete_current_year_model_runs.R
jeancochrane Nov 16, 2023
26c61bb
Clean up docstrings and test against 2023-11-14-frosty-jacob model
jeancochrane Nov 16, 2023
3297673
Satisfy pre-commit
jeancochrane Nov 16, 2023
51843f3
Revert to workflow_dispatch event trigger for delete-model-runs.yaml
jeancochrane Nov 16, 2023
49ea1a6
Revert "Revert to workflow_dispatch event trigger for delete-model-ru…
jeancochrane Nov 16, 2023
85bfcf8
Fix typo in Delete model runs step of delete-model-runs workflow
jeancochrane Nov 16, 2023
61b9352
Test obviously bogus value for run-ids to delete-model-runs workflow
jeancochrane Nov 16, 2023
a3a28fa
Raise an error in delete_current_year_model_runs.R if no objects were…
jeancochrane Nov 16, 2023
5c9ddf6
Disable renv sandbox to speed up install/run times
jeancochrane Nov 17, 2023
a090cc1
Run delete_current_year_model_runs.R on all run IDs at once
jeancochrane Nov 17, 2023
c624b0b
Check for validity of run IDs before issuing delete operations in del…
jeancochrane Nov 17, 2023
17fcf90
Refactor run ID validity check in delete_current_year_model_runs to b…
jeancochrane Nov 17, 2023
b6e695d
Try deleting multiple invalid run IDs
jeancochrane Nov 17, 2023
0e56190
Refactor delete_current_year_model_runs.R to accept a comma-delimited…
jeancochrane Nov 17, 2023
cf4ff3c
Test a delete-model-runs workflow where one run is valid and one isn't
jeancochrane Nov 17, 2023
e23d34c
Test a delete-model-runs workflow where all run IDs are valid
jeancochrane Nov 17, 2023
6cdb449
Revert to workflow_dispatch trigger for delete-model-runs.yaml
jeancochrane Nov 17, 2023
b02de7d
Remove extraneous print statement from delete_current_year_model_runs.R
jeancochrane Nov 17, 2023
a4895c2
Clean up delete-model-runs and associated script in response to review
jeancochrane Nov 20, 2023
09cfa5b
Temporarily run delete-model-runs on pull_request event for testing
jeancochrane Nov 20, 2023
759550d
Revert "Temporarily run delete-model-runs on pull_request event for t…
jeancochrane Nov 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/delete-model-runs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ on:
inputs:
run-ids:
description: >
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: IMO it'd be really helpful here to have a default input showing an example (nonexistent) run ID.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, done in a4895c2! Note that the trade-off here is that default arguments are populated as actual values in the workflow input form, rather than placeholder text, so there's a chance of a caller misinterpreting the form and submitting it with the (fake) default values. This should just raise an error and not actually delete any data, however, so the risk is worth it in my view.

Comma-delimited list of IDs of model runs to delete (no spaces). Note
Run IDs: Space-delimited list of IDs of model runs to delete. Note
that the workflow assumes these IDs correspond to model runs for the
current upcoming assessment cycle, and if that's not the case the
deletion script will raise an error.
current assessment cycle, and if that's not the case the deletion
script will raise an error.
required: true
type: string
default: 2024-01-01-foo-bar 2024-01-02-bar-baz

jobs:
delete-model-runs:
Expand Down Expand Up @@ -52,7 +53,7 @@ jobs:
aws-region: us-east-1

- name: Delete model runs
run: Rscript ./R/delete_current_year_model_runs.R "$RUN_IDS"
run: Rscript ./R/delete_current_year_model_runs.R "${RUN_IDS// /,}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: Good thinking!

shell: bash
env:
RUN_IDS: ${{ inputs.run-ids }}
12 changes: 6 additions & 6 deletions R/delete_current_year_model_runs.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# whose artifacts should be deleted.
#
# Assumes that model runs are restricted to the current assessment cycle, where
# each assessment cycle starts in April. Raises an error if no objects matching
# each assessment cycle starts in May. Raises an error if no objects matching
# a given ID for the current year could be located in S3. This error will get
# raised before any deletion occurs, so if one or more IDs are invalid then
# no objects will be deleted.
Expand All @@ -24,9 +24,9 @@ current_year <- current_date %>% format("%Y")

# The following heuristic determines the current upcoming assessment cycle year:
#
# * From April to December (post assessment), `year` = next year
# * From January to March (during assessment), `year` = current year
year <- if (current_month < "03") {
# * From May to December (post assessment), `year` = next year
# * From January to April (during assessment), `year` = current year
year <- if (current_month < "05") {
current_year
} else {
as.character(as.numeric(current_year) + 1)
Expand All @@ -35,8 +35,8 @@ year <- if (current_month < "03") {
# Convert the comma-delimited input to a vector of run IDs. Accepting one or
# more positional arguments would be a cleaner UX, but since this script is
# intended to be called from a dispatched GitHub workflow, it's easier to parse
# one comma-delimited string than split a space-separated string passed as a
# workflow input
# one comma-delimited string than convert a space-separated string passed as a
# workflow input to an array of function arguments
raw_run_ids <- commandArgs(trailingOnly = TRUE)
run_ids <- raw_run_ids %>%
strsplit(split = ",", fixed = TRUE) %>%
Expand Down
Loading