Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add GHA release workflow #3122

Closed
wants to merge 32 commits into from
Closed

Add GHA release workflow #3122

wants to merge 32 commits into from

Conversation

zaneselvans
Copy link
Member

@zaneselvans zaneselvans commented Dec 5, 2023 •
edited
Loading

PR Overview

Add a GitHub Actions Workflow that will coordinate the production of data & software releases, whenever a date-style version tag starting with v20 is pushed. We need to create GitHub releases to trigger the Zenodo software archiving, and may or may not want to push the same release to PyPI for consistency.

Other workflows can be set to kick off on: release or with the same tagging criteria, to:

  • run a tagged build
  • publish the results to GCS, S3, and Zenodo

PR Checklist

  • Merge the most recent version of the branch you are merging into (probably dev).
  • All CI checks are passing. Run tests locally to debug failures
  • Make sure you've included good docstrings.
  • For major data coverage & analysis changes, run data validation tests
  • Include unit tests for new functions and classes.
  • Defensive data quality/sanity checks in analyses & data processing functions.
  • Update the release notes and reference reference the PR and related issues.
  • Do your own explanatory review of the PR to help the reviewer understand what's going on and identify issues preemptively.

aesharpe and others added 30 commits November 30, 2023 15:15
@aesharpe
Extract EIA923 emissions control table and add 2022 final release dat…
cad8a04 
…a for this table
@aesharpe
Add release notes for addition of the emissions_contorl table extraction
304c4dd
@zaneselvans
Merge branch 'dev' into extract-eia923-emissions-control
34c637d
@zaneselvans
Sort DBs Fly Datasette; don't distribute Datasette's metadata.yml
8c9f0cd
@zaneselvans
Merge pull request #3100 from catalyst-cooperative/extract-eia923-emi…
80b289f 
…ssions-control

Extract EIA923 emissions control table and add 2022 final release dat…
@zaneselvans
Merge branch 'dev' into better-build-outputs
f4da38d
@zaneselvans
Fix formatting in bulletized list.
f456126
@zaneselvans
Merge pull request #3106 from catalyst-cooperative/better-build-outputs
29a7443 
Sort DBs Fly Datasette; don't distribute Datasette's metadata.yml
@rousik
Add support for choosing between multiprocess and inprocess executors…
7f1fb1c 
… via cli flag (#2895)

This introduces `--dagster-workers` flag for both `pudl_etl` and `ferc_to_sqlite` scripts. Based on the value, we will either use number of multiprocess workers equal to available CPUs (default), single in-process worker (1) or number of multiprocess workers capped by arbitrary positive number.

This doesn't change default behavior, which is to use number of multiprocess dagster workers equal to number of available CPUs.
@rousik @github-actions
Update conda-lock.yml and rendered conda environment files.
5617bb0
@zaneselvans
Merge branch 'main' into dev
6f284f3
@rousik
Improve flexibility for publishing options (#2964)
85c2ea4 
Improve flexibility for publishing options

Merging this in today so we can see how it goes with tonight's build, and so I can integrate the changes with #3086.
@dependabot
Bump google-github-actions/auth from 1 to 2
8f6a693 
Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 1 to 2.
- [Release notes](https://github.com/google-github-actions/auth/releases)
- [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md)
- [Commits](google-github-actions/auth@v1...v2)

---
updated-dependencies:
- dependency-name: google-github-actions/auth
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot
Bump actions/checkout from 3 to 4
b3ba71d 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@zaneselvans
Update conda-lock.yml and rendered conda environment files.
7c30618
@zaneselvans
Merge pull request #3112 from catalyst-cooperative/dependabot/github_…
19edbee 
…actions/dev/actions/checkout-4

Bump actions/checkout from 3 to 4
@zaneselvans
Merge pull request #3109 from catalyst-cooperative/dependabot/github_…
9a94eec 
…actions/dev/google-github-actions/auth-2

Bump google-github-actions/auth from 1 to 2
@dependabot
Bump docker/login-action from 2.1.0 to 3.0.0
a790b0d 
Bumps [docker/login-action](https://github.com/docker/login-action) from 2.1.0 to 3.0.0.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](docker/login-action@v2.1.0...v3.0.0)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot
Bump docker/build-push-action from 4.0.0 to 5.1.0
b2f55b7 
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4.0.0 to 5.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v4.0.0...v5.1.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot
Bump docker/setup-buildx-action from 2.5.0 to 3.0.0
e1c8d79 
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.5.0 to 3.0.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v2.5.0...v3.0.0)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
@zaneselvans
Merge pull request #3113 from catalyst-cooperative/update-conda-lockfile
ff6d1de 
Update Lockfile
@zaneselvans @github-actions
Update conda-lock.yml and rendered conda environment files.
ecb9752
@zaneselvans
Merge pull request #3110 from catalyst-cooperative/dependabot/github_…
fc6cb73 
…actions/dev/docker/build-push-action-5.1.0

Bump docker/build-push-action from 4.0.0 to 5.1.0
@zaneselvans
Merge pull request #3108 from catalyst-cooperative/dependabot/github_…
c69db7b 
…actions/dev/docker/setup-buildx-action-3.0.0

Bump docker/setup-buildx-action from 2.5.0 to 3.0.0
@zaneselvans
Merge pull request #3111 from catalyst-cooperative/dependabot/github_…
66052a5 
…actions/dev/docker/login-action-3.0.0

Bump docker/login-action from 2.1.0 to 3.0.0
@jdangerx
Hide diffs in lock files by default. (#3103)
8393bca 
* Hide diffs in lock files by default.

* Add mypy as a test for lockfile diff visibility. pudl-bot, do your thing!

* Update conda-lock.yml and rendered conda environment files.

* Revert "Add mypy as a test for lockfile diff visibility. pudl-bot, do your thing!"

This reverts commit e7a8e82.

* Update conda-lock.yml and rendered conda environment files.

---------

Co-authored-by: jdangerx <[email protected]>
@jdangerx @github-actions
Update conda-lock.yml and rendered conda environment files.
394ce97
@zaneselvans
Add GHA release workflow
5c79ed7
@zaneselvans
Rename workflow to match filename.
51726b0
@zaneselvans
Rename jobs and replace catalystcoop.ferc-xbrl-extractor with catalys…
2c3fa58 
…tcoop.pudl
@zaneselvans zaneselvans mentioned this pull request Dec 5, 2023
Closed
@zaneselvans zaneselvans linked an issue Dec 5, 2023 that may be closed by this pull request
Automate the data release process #2756
Closed
@zaneselvans zaneselvans added the release Tasks directly related to data and software releases. label Dec 5, 2023
@zaneselvans zaneselvans self-assigned this Dec 5, 2023
@zaneselvans zaneselvans changed the base branch from dev to main December 5, 2023 16:17
@zaneselvans zaneselvans closed this Dec 5, 2023
@zaneselvans zaneselvans deleted the gha-release-workflow branch December 5, 2023 16:18
@zaneselvans zaneselvans removed a link to an issue Dec 6, 2023
Automate the data release process #2756
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@zaneselvans zaneselvans

Labels
release Tasks directly related to data and software releases.
Projects
Catalyst Megaproject
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@zaneselvans @aesharpe @rousik @jdangerx