Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to call/omit specific steps of the pipeline #1260

Closed
bhilbert4 opened this issue May 8, 2023 · 0 comments · Fixed by #1262
Closed

Add ability to call/omit specific steps of the pipeline #1260

bhilbert4 opened this issue May 8, 2023 · 0 comments · Fixed by #1262

Comments

@bhilbert4
Copy link
Collaborator

In shared_tasks/run_pipeline.py, the code assumes that all steps of calwebb_detector1 should be run on a given input file (with the exception of ramp fitting).

For the dark monitor, we want to skip the dark_current step. Further, there are _dark.fits files in the filesystem. Starting from these, only the jump and ramp-fitting steps would have to be run, which would save time compared to running the entire pipeline on the uncal version of these files.

It would be helpful to tweak the code such that lists of pipeline steps to skip/run could be provided, and the pipeline call would be updated accordingly.

Finally, at the moment the code is using the run() method of the pipeline. We should switch to using the call() method so that the pipeline will use the appropriate parameter reference files for parameters that we don't explicitly specify. This would involve switching the parameters we do specify to use a nested dictionary.

e.g.:
parameters = {'dark_current': {"skip": True}}
p = Detector1Pipeline.call(uncal_file, steps=parameters}

FYI @york-stsci

@bhilbert4 bhilbert4 linked a pull request Jun 2, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant