Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: retrieve glue jobs after branch checkout #68

Merged
merged 3 commits into from
Dec 24, 2024
+7 −9
Merged

fix: retrieve glue jobs after branch checkout #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
60ffb3a
fix: retrieve glue jobs after branch checkout
patheard Dec 24, 2024
29ea41b
test: fetch main
patheard Dec 24, 2024
1b73dbe
fix: handle missing SHA
patheard Dec 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/glue_job_sync.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,6 @@ jobs:
role-session-name: TFApply
aws-region: ${{ env.AWS_REGION }}

- name: Get Glue jobs
run: ./.github/workflows/scripts/get-glue-jobs.sh

- name: Create PR if changes
env:
GH_TOKEN: ${{ github.token }}
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/scripts/sync-glue-jobs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,17 +17,18 @@ Automated sync of AWS Glue ETL jobs."

# Check if the remote branch exists
if git ls-remote --heads "$REMOTE_REPO" "$BRANCH_NAME" | grep -q "$BRANCH_NAME"; then
echo "Branch '$BRANCH_NAME' exists. Checking out and updating."
git stash
echo "Branch '$BRANCH_NAME' exists. Checking out."
git checkout "$BRANCH_NAME"
git stash apply --quiet || true
git checkout --theirs -- .
CREATE_PR="false"
else
echo "Branch '$BRANCH_NAME' does not exist. Creating new branch."
git checkout -b "$BRANCH_NAME" "$BASE_BRANCH"
git fetch "$REMOTE_REPO" "$BASE_BRANCH"
git checkout -b "$BRANCH_NAME" "$REMOTE_REPO/$BASE_BRANCH"
fi

echo "Syncing Glue jobs..."
.github/workflows/scripts/get-glue-jobs.sh

# Check for changes in the branch
if git diff-index --quiet HEAD -- "$JOB_DIR"; then
echo "No changes detected."
Expand All @@ -42,7 +43,7 @@ echo "Committing changes..."
FILES_CHANGED="$(git status --porcelain | awk '{print $2}')"
for FILE in $FILES_CHANGED; do
MESSAGE="chore: regenerate $(basename "$FILE") for $(date -u '+%Y-%m-%d')"
SHA="$(git rev-parse $BRANCH_NAME:"$FILE" || "")"
SHA="$(git rev-parse $BRANCH_NAME:"$FILE" || echo "")"
gh api --method PUT /repos/cds-snc/data-lake/contents/"$FILE" \
--field message="$MESSAGE" \
--field content="$(base64 -w 0 "$FILE")" \
Expand Down
Loading