Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ci): save nextflow pid and tag all slurm jobs #227

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

pbelmann
Copy link
Member

@pbelmann pbelmann commented Sep 1, 2022

This PR saves the pid and a generic number of the nextflow run in file. All slurm jobs are tagged by the generic number.

Example:

[c2/2ca15f] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pFastpSplit (JobID: 11424, Sample: test2)
[d1/85ef55] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pFastpSplit (JobID: 11424, Sample: test1)
[a7/ed3ce6] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pJellyFish (2)
[7c/b3f4ba] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pJellyFish (1)
[4e/9faf88] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pNonpareil (JobID: 11424, Sample: test2)
[0e/13e83f] Cached process > wPipeline:wQualityControlList:_wFastqSplit:pNonpareil (JobID: 11424, Sample: test1)
[85/947159] Cached process > wPipeline:wAssemblyList:_wAssembly:_wCalculateMegahitResources:pPredictFlavor (JobID: 11424, Sample: test2)
[f2/ad7bde] Cached process > wPipeline:wAssemblyList:_wAssembly:_wCalculateMegahitResources:pPredictFlavor (JobID: 11424, Sample: test1)
[31/cc4784] Cached process > wPipeline:wAssemblyList:_wAssembly:pMegahit (JobID: 11424, Sample: test2)
[fa/03c8e5] Cached process > wPipeline:wAssemblyList:_wAssembly:pMegahit (JobID: 11424, Sample: test1)

Once the PR is merged a cron job on the ci machine will periodically check for running processes and delete slurm jobs if no nextflow process exists.

PR review guidelines

Thank you for submitting this PR.

Before merge:

  • The PR must be reviewed by one of the team members.

  • Please check if anything in the Readme must be adjusted, or added (development-setup, production-setup, user-guide).

  • PRs with new modules or workflow interfaces must include tests according to the developer guidelines.

  • The new code is readable, well commented and should adhere to our developer guidelines.

  • Before merging it must be checked if a squash of commits is required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant