Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Failing Dailies #1113

Merged
merged 22 commits into from
Oct 24, 2024
Merged

Fix Failing Dailies #1113

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c0af8b2
Fixed dailies.yml
pauladkisson Oct 14, 2024
27e1791
Fixed dailies.yml
pauladkisson Oct 14, 2024
eedbe91
Fixed dailies.yml
pauladkisson Oct 14, 2024
793cd56
Update CHANGELOG.md
pauladkisson Oct 15, 2024
977144e
debugging: using v0.6.4 for docker tag
pauladkisson Oct 15, 2024
6a8bc86
switched back to latest
pauladkisson Oct 21, 2024
eefe6db
updated python-versions and os-versions to use a default var
pauladkisson Oct 22, 2024
a76fc48
simplified defaults
pauladkisson Oct 22, 2024
90e083b
debugging required=true
pauladkisson Oct 22, 2024
cb74e00
debugging: manually provide python-verison and os-version
pauladkisson Oct 22, 2024
d0977f9
debugging
pauladkisson Oct 22, 2024
5c86564
debugging
pauladkisson Oct 22, 2024
c5eb431
debugging
pauladkisson Oct 22, 2024
9b0bdfd
debugging
pauladkisson Oct 22, 2024
8f063ca
removed debugging
pauladkisson Oct 22, 2024
adb885a
Merge branch 'main' into dailies
pauladkisson Oct 22, 2024
94b7e49
removed debugging
pauladkisson Oct 22, 2024
ccd4b9d
switched deploy-tests to .txt based python versions
pauladkisson Oct 23, 2024
e2808da
debug
pauladkisson Oct 23, 2024
89b7aac
added checkout
pauladkisson Oct 23, 2024
af2d720
updated dailies
pauladkisson Oct 23, 2024
5ae0f9f
updated doctests
pauladkisson Oct 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/all_os_versions.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["ubuntu-latest", "macos-latest", "windows-latest", "macos-13"]
1 change: 1 addition & 0 deletions .github/workflows/all_python_versions.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
["3.9", "3.10", "3.11", "3.12"]
23 changes: 23 additions & 0 deletions .github/workflows/dailies.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,32 +6,55 @@ on:
- cron: "0 4 * * *" # Daily at 8PM PST, 11PM EST, 5AM CET to avoid working hours

jobs:
load_python_and_os_versions:
runs-on: ubuntu-latest
outputs:
ALL_PYTHON_VERSIONS: ${{ steps.load_python_versions.outputs.python_versions }}
ALL_OS_VERSIONS: ${{ steps.load_os_versions.outputs.os_versions }}
steps:
- uses: actions/checkout@v4
- id: load_python_versions
run: echo "python_versions=$(cat ./.github/workflows/all_python_versions.txt)" >> "$GITHUB_OUTPUT"
- id: load_os_versions
run: echo "os_versions=$(cat ./.github/workflows/all_os_versions.txt)" >> "$GITHUB_OUTPUT"

build-and-upload-docker-image-dev:
uses: ./.github/workflows/build_and_upload_docker_image_dev.yml
secrets:
DOCKER_UPLOADER_USERNAME: ${{ secrets.DOCKER_UPLOADER_USERNAME }}
DOCKER_UPLOADER_PASSWORD: ${{ secrets.DOCKER_UPLOADER_PASSWORD }}

run-daily-tests:
needs: load_python_and_os_versions
uses: ./.github/workflows/testing.yml
secrets:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
S3_GIN_BUCKET: ${{ secrets.S3_GIN_BUCKET }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
python-versions: ${{ needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}
os-versions: ${{ needs.load_python_and_os_versions.outputs.ALL_OS_VERSIONS }}

run-daily-dev-tests:
needs: load_python_and_os_versions
uses: ./.github/workflows/dev-testing.yml
secrets:
DANDI_API_KEY: ${{ secrets.DANDI_API_KEY }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
S3_GIN_BUCKET: ${{ secrets.S3_GIN_BUCKET }}
with:
python-versions: ${{ needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}

run-daily-live-service-testing:
needs: load_python_and_os_versions
uses: ./.github/workflows/live-service-testing.yml
secrets:
DANDI_API_KEY: ${{ secrets.DANDI_API_KEY }}
with:
python-versions: ${{ needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}
os-versions: ${{ needs.load_python_and_os_versions.outputs.ALL_OS_VERSIONS }}

run-daily-neuroconv-docker-testing:
uses: ./.github/workflows/neuroconv_docker_testing.yml
Expand Down
29 changes: 20 additions & 9 deletions .github/workflows/deploy-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,17 @@ concurrency:
cancel-in-progress: true

jobs:
load_python_and_os_versions:
runs-on: ubuntu-latest
outputs:
ALL_PYTHON_VERSIONS: ${{ steps.load_python_versions.outputs.python_versions }}
ALL_OS_VERSIONS: ${{ steps.load_os_versions.outputs.os_versions }}
steps:
- uses: actions/checkout@v4
- id: load_python_versions
run: echo "python_versions=$(cat ./.github/workflows/all_python_versions.txt)" >> "$GITHUB_OUTPUT"
- id: load_os_versions
run: echo "os_versions=$(cat ./.github/workflows/all_os_versions.txt)" >> "$GITHUB_OUTPUT"

assess-file-changes:
uses: ./.github/workflows/assess-file-changes.yml
Expand All @@ -31,7 +42,7 @@ jobs:
0

run-tests:
needs: assess-file-changes
needs: [assess-file-changes, load_python_and_os_versions]
if: ${{ needs.assess-file-changes.outputs.SOURCE_CHANGED == 'true' }}
uses: ./.github/workflows/testing.yml
secrets:
Expand All @@ -40,28 +51,28 @@ jobs:
S3_GIN_BUCKET: ${{ secrets.S3_GIN_BUCKET }}
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with: # Ternary operator: condition && value_if_true || value_if_false
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || '["3.9", "3.10", "3.11", "3.12"]' }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || '["ubuntu-latest", "macos-latest", "macos-13", "windows-latest"]' }}
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || needs.load_python_and_os_versions.outputs.ALL_OS_VERSIONS }}

# If the conversion gallery is the only thing that changed, run doctests only
run-doctests-only:
needs: assess-file-changes
needs: [assess-file-changes, load_python_and_os_versions]
if: ${{ needs.assess-file-changes.outputs.CONVERSION_GALLERY_CHANGED == 'true' && needs.assess-file-changes.outputs.SOURCE_CHANGED != 'true' }}
uses: ./.github/workflows/doctests.yml
with: # Ternary operator: condition && value_if_true || value_if_false
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || '["3.9", "3.10", "3.11", "3.12"]' }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || '["ubuntu-latest", "macos-latest", "macos-13", "windows-latest"]' }}
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || needs.load_python_and_os_versions.outputs.ALL_OS_VERSIONS }}


run-live-service-tests:
needs: assess-file-changes
needs: [assess-file-changes, load_python_and_os_versions]
if: ${{ needs.assess-file-changes.outputs.SOURCE_CHANGED == 'true' }}
uses: ./.github/workflows/live-service-testing.yml
secrets:
DANDI_API_KEY: ${{ secrets.DANDI_API_KEY }}
with: # Ternary operator: condition && value_if_true || value_if_false
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || '["3.9", "3.10", "3.11", "3.12"]' }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || '["ubuntu-latest", "macos-latest", "macos-13", "windows-latest"]' }}
python-versions: ${{ github.event.pull_request.draft == true && '["3.9"]' || needs.load_python_and_os_versions.outputs.ALL_PYTHON_VERSIONS }}
os-versions: ${{ github.event.pull_request.draft == true && '["ubuntu-latest"]' || needs.load_python_and_os_versions.outputs.ALL_OS_VERSIONS }}


check-final-status:
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/dev-testing.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ on:
description: 'List of Python versions to use in matrix, as JSON string'
required: true
type: string
default: '["3.9", "3.10", "3.11", "3.12"]'
secrets:
DANDI_API_KEY:
required: true
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/doctests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,10 @@ on:
description: 'List of Python versions to use in matrix, as JSON string'
required: true
type: string
default: '["3.9", "3.10", "3.11", "3.12"]'
os-versions:
description: 'List of OS versions to use in matrix, as JSON string'
required: true
type: string
default: '["ubuntu-latest", "macos-latest", "windows-latest"]'


jobs:
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/live-service-testing.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,10 @@ on:
description: 'List of Python versions to use in matrix, as JSON string'
required: true
type: string
default: '["3.9", "3.10", "3.11", "3.12"]'
os-versions:
description: 'List of OS versions to use in matrix, as JSON string'
required: true
type: string
default: '["ubuntu-latest", "macos-latest", "windows-latest"]'

secrets:
DANDI_API_KEY:
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/testing.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,10 @@ on:
description: 'List of Python versions to use in matrix, as JSON string'
required: true
type: string
default: '["3.9", "3.10", "3.11", "3.12"]'
os-versions:
description: 'List of OS versions to use in matrix, as JSON string'
required: true
type: string
default: '["ubuntu-latest", "macos-latest", "windows-latest"]'


secrets:
AWS_ACCESS_KEY_ID:
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Deprecations

## Bug Fixes
* Fixed dailies [PR #1113](https://github.com/catalystneuro/neuroconv/pull/1113)

## Features
* Using in-house `GenericDataChunkIterator` [PR #1068](https://github.com/catalystneuro/neuroconv/pull/1068)
Expand Down
Loading