-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Give cargo-dist a try-out #352
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,307 @@ | ||
| # Copyright 2022-2024, axodotdev | ||
| # SPDX-License-Identifier: MIT or Apache-2.0 | ||
| # | ||
| # CI that: | ||
| # | ||
| # * checks for a Git Tag that looks like a release | ||
| # * builds artifacts with cargo-dist (archives, installers, hashes) | ||
| # * uploads those artifacts to temporary workflow zip | ||
| # * on success, uploads the artifacts to a GitHub Release | ||
| # | ||
| # Note that the GitHub Release will be created with a generated | ||
| # title/body based on your changelogs. | ||
|
|
||
| name: Release | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| # This task will run whenever you push a git tag that looks like a version | ||
| # like "1.0.0", "v0.1.0-prerelease.1", "my-app/0.1.0", "releases/v1.0.0", etc. | ||
| # Various formats will be parsed into a VERSION and an optional PACKAGE_NAME, where | ||
| # PACKAGE_NAME must be the name of a Cargo package in your workspace, and VERSION | ||
| # must be a Cargo-style SemVer Version (must have at least major.minor.patch). | ||
| # | ||
| # If PACKAGE_NAME is specified, then the announcement will be for that | ||
| # package (erroring out if it doesn't have the given version or isn't cargo-dist-able). | ||
| # | ||
| # If PACKAGE_NAME isn't specified, then the announcement will be for all | ||
| # (cargo-dist-able) packages in the workspace with that version (this mode is | ||
| # intended for workspaces with only one dist-able package, or with all dist-able | ||
| # packages versioned/released in lockstep). | ||
| # | ||
| # If you push multiple tags at once, separate instances of this workflow will | ||
| # spin up, creating an independent announcement for each one. However, GitHub | ||
| # will hard limit this to 3 tags per commit, as it will assume more tags is a | ||
| # mistake. | ||
| # | ||
| # If there's a prerelease-style suffix to the version, then the release(s) | ||
| # will be marked as a prerelease. | ||
| on: | ||
| push: | ||
| tags: | ||
| - '**[0-9]+.[0-9]+.[0-9]+*' | ||
| pull_request: | ||
|
|
||
| jobs: | ||
| # Run 'cargo dist plan' (or host) to determine what tasks we need to do | ||
| plan: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| val: ${{ steps.plan.outputs.manifest }} | ||
| tag: ${{ !github.event.pull_request && github.ref_name || '' }} | ||
| tag-flag: ${{ !github.event.pull_request && format('--tag={0}', github.ref_name) || '' }} | ||
| publishing: ${{ !github.event.pull_request }} | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: recursive | ||
| - name: Install cargo-dist | ||
| # we specify bash to get pipefail; it guards against the `curl` command | ||
| # failing. otherwise `sh` won't catch that `curl` returned non-0 | ||
| shell: bash | ||
| run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.0/cargo-dist-installer.sh | sh" | ||
| # sure would be cool if github gave us proper conditionals... | ||
| # so here's a doubly-nested ternary-via-truthiness to try to provide the best possible | ||
| # functionality based on whether this is a pull_request, and whether it's from a fork. | ||
| # (PRs run on the *source* but secrets are usually on the *target* -- that's *good* | ||
| # but also really annoying to build CI around when it needs secrets to work right.) | ||
| - id: plan | ||
| run: | | ||
| cargo dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json | ||
| echo "cargo dist ran successfully" | ||
| cat plan-dist-manifest.json | ||
| echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT" | ||
| - name: "Upload dist-manifest.json" | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: artifacts-plan-dist-manifest | ||
| path: plan-dist-manifest.json | ||
|
|
||
| # Build and packages all the platform-specific things | ||
| build-local-artifacts: | ||
| name: build-local-artifacts (${{ join(matrix.targets, ', ') }}) | ||
| # Let the initial task tell us to not run (currently very blunt) | ||
| needs: | ||
| - plan | ||
| if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }} | ||
| strategy: | ||
| fail-fast: false | ||
| # Target platforms/runners are computed by cargo-dist in create-release. | ||
| # Each member of the matrix has the following arguments: | ||
| # | ||
| # - runner: the github runner | ||
| # - dist-args: cli flags to pass to cargo dist | ||
| # - install-dist: expression to run to install cargo-dist on the runner | ||
| # | ||
| # Typically there will be: | ||
| # - 1 "global" task that builds universal installers | ||
| # - N "local" tasks that build each platform's binaries and platform-specific installers | ||
| matrix: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix }} | ||
| runs-on: ${{ matrix.runner }} | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| BUILD_MANIFEST_NAME: target/distrib/${{ join(matrix.targets, '-') }}-dist-manifest.json | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: recursive | ||
| - uses: swatinem/rust-cache@v2 | ||
| - name: Install cargo-dist | ||
| run: ${{ matrix.install_dist }} | ||
| # Get the dist-manifest | ||
| - name: Fetch local artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: artifacts-* | ||
| path: target/distrib/ | ||
| merge-multiple: true | ||
| - name: Install dependencies | ||
| run: | | ||
| ${{ matrix.packages_install }} | ||
| - name: Build artifacts | ||
| run: | | ||
| # Actually do builds and make zips and whatnot | ||
| cargo dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json | ||
| echo "cargo dist ran successfully" | ||
| - id: cargo-dist | ||
| name: Post-build | ||
| # We force bash here just because github makes it really hard to get values up | ||
| # to "real" actions without writing to env-vars, and writing to env-vars has | ||
| # inconsistent syntax between shell and powershell. | ||
| shell: bash | ||
| run: | | ||
| # Parse out what we just built and upload it to scratch storage | ||
| echo "paths<<EOF" >> "$GITHUB_OUTPUT" | ||
| jq --raw-output ".upload_files[]" dist-manifest.json >> "$GITHUB_OUTPUT" | ||
| echo "EOF" >> "$GITHUB_OUTPUT" | ||
|
|
||
| cp dist-manifest.json "$BUILD_MANIFEST_NAME" | ||
| - name: "Upload artifacts" | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: artifacts-build-local-${{ join(matrix.targets, '_') }} | ||
| path: | | ||
| ${{ steps.cargo-dist.outputs.paths }} | ||
| ${{ env.BUILD_MANIFEST_NAME }} | ||
|
|
||
| # Build and package all the platform-agnostic(ish) things | ||
| build-global-artifacts: | ||
| needs: | ||
| - plan | ||
| - build-local-artifacts | ||
| runs-on: "ubuntu-20.04" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: recursive | ||
| - name: Install cargo-dist | ||
| shell: bash | ||
| run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.0/cargo-dist-installer.sh | sh" | ||
| # Get all the local artifacts for the global tasks to use (for e.g. checksums) | ||
| - name: Fetch local artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: artifacts-* | ||
| path: target/distrib/ | ||
| merge-multiple: true | ||
| - id: cargo-dist | ||
| shell: bash | ||
| run: | | ||
| cargo dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json | ||
| echo "cargo dist ran successfully" | ||
|
|
||
| # Parse out what we just built and upload it to scratch storage | ||
| echo "paths<<EOF" >> "$GITHUB_OUTPUT" | ||
| jq --raw-output ".upload_files[]" dist-manifest.json >> "$GITHUB_OUTPUT" | ||
| echo "EOF" >> "$GITHUB_OUTPUT" | ||
|
|
||
| cp dist-manifest.json "$BUILD_MANIFEST_NAME" | ||
| - name: "Upload artifacts" | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: artifacts-build-global | ||
| path: | | ||
| ${{ steps.cargo-dist.outputs.paths }} | ||
| ${{ env.BUILD_MANIFEST_NAME }} | ||
| # Determines if we should publish/announce | ||
| host: | ||
| needs: | ||
| - plan | ||
| - build-local-artifacts | ||
| - build-global-artifacts | ||
| # Only run if we're "publishing", and only if local and global didn't fail (skipped is fine) | ||
| if: ${{ always() && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }} | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| runs-on: "ubuntu-20.04" | ||
| outputs: | ||
| val: ${{ steps.host.outputs.manifest }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: recursive | ||
| - name: Install cargo-dist | ||
| run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.0/cargo-dist-installer.sh | sh" | ||
| # Fetch artifacts from scratch-storage | ||
| - name: Fetch artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: artifacts-* | ||
| path: target/distrib/ | ||
| merge-multiple: true | ||
| # This is a harmless no-op for GitHub Releases, hosting for that happens in "announce" | ||
| - id: host | ||
| shell: bash | ||
| run: | | ||
| cargo dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json | ||
| echo "artifacts uploaded and released successfully" | ||
| cat dist-manifest.json | ||
| echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT" | ||
| - name: "Upload dist-manifest.json" | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| # Overwrite the previous copy | ||
| name: artifacts-dist-manifest | ||
| path: dist-manifest.json | ||
|
|
||
| publish-homebrew-formula: | ||
| needs: | ||
| - plan | ||
| - host | ||
| runs-on: "ubuntu-20.04" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| PLAN: ${{ needs.plan.outputs.val }} | ||
| GITHUB_USER: "axo bot" | ||
| GITHUB_EMAIL: "[email protected]" | ||
| if: ${{ !fromJson(needs.plan.outputs.val).announcement_is_prerelease || fromJson(needs.plan.outputs.val).publish_prereleases }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| repository: "tremor-rs/homebrew-formulae" | ||
| token: ${{ secrets.HOMEBREW_TAP_TOKEN }} | ||
| # So we have access to the formula | ||
| - name: Fetch local artifacts | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: artifacts-* | ||
| path: Formula/ | ||
| merge-multiple: true | ||
| # This is extra complex because you can make your Formula name not match your app name | ||
| # so we need to find releases with a *.rb file, and publish with that filename. | ||
| - name: Commit formula files | ||
| run: | | ||
| git config --global user.name "${GITHUB_USER}" | ||
| git config --global user.email "${GITHUB_EMAIL}" | ||
|
|
||
| for release in $(echo "$PLAN" | jq --compact-output '.releases[] | select([.artifacts[] | endswith(".rb")] | any)'); do | ||
| filename=$(echo "$release" | jq '.artifacts[] | select(endswith(".rb"))' --raw-output) | ||
| name=$(echo "$filename" | sed "s/\.rb$//") | ||
| version=$(echo "$release" | jq .app_version --raw-output) | ||
|
|
||
| git add "Formula/${filename}" | ||
| git commit -m "${name} ${version}" | ||
| done | ||
| git push | ||
|
|
||
| # Create a GitHub Release while uploading all files to it | ||
| announce: | ||
| needs: | ||
| - plan | ||
| - host | ||
| - publish-homebrew-formula | ||
| # use "always() && ..." to allow us to wait for all publish jobs while | ||
| # still allowing individual publish jobs to skip themselves (for prereleases). | ||
| # "host" however must run to completion, no skipping allowed! | ||
| if: ${{ always() && needs.host.result == 'success' && (needs.publish-homebrew-formula.result == 'skipped' || needs.publish-homebrew-formula.result == 'success') }} | ||
| runs-on: "ubuntu-20.04" | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: recursive | ||
| - name: "Download GitHub Artifacts" | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| pattern: artifacts-* | ||
| path: artifacts | ||
| merge-multiple: true | ||
| - name: Cleanup | ||
| run: | | ||
| # Remove the granular manifests | ||
| rm -f artifacts/*-dist-manifest.json | ||
| - name: Create GitHub Release | ||
| uses: ncipollo/release-action@v1 | ||
| with: | ||
| tag: ${{ needs.plan.outputs.tag }} | ||
| name: ${{ fromJson(needs.host.outputs.val).announcement_title }} | ||
| body: ${{ fromJson(needs.host.outputs.val).announcement_github_body }} | ||
| prerelease: ${{ fromJson(needs.host.outputs.val).announcement_is_prerelease }} | ||
| artifacts: "artifacts/*" | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we want to run this on every pull request
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It skips for PRs when its configured for
planmode and doesn't generate artefacts. There is another mode that always builds release artefacts but I disabled it. The default mode isplan. If you look at our CI output for this PR you'll see plan was triggered and it skipped the release distribution process.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here's the reference in the docs:
https://opensource.axo.dev/cargo-dist/book/introduction.html#distributing
My read of that is that until we push a tag it does nothing. Our release process adds a tag so it should only fire on release for us with no other configuration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, that's complex but lets give it a shot 👍