Added try/except for model download #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added try/except statement for the main model dl functions. Yet untested, needs testing!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hei Lukas,
I saw that the download functions runs into an error if one of the .nc files is no existent on the thredds server, which happens sometimes for whatever reason.
See for example https://thredds.met.no/thredds/catalog/aromearcticarchive/2022/10/23/arome_arctic_det_2_5km_20221023T18Z.nc (not existent on the server)
If you now want to download model data with the toolbox e.g. for the entire month of Oct. 2022 it will run into an error there and abort the entire download.
What I did now is basically adding a try/except statement to the main dl functions. It is in my opinion a cleaner way of error handling when downloading something or working with external files (that might have changed or do not exist) in general.
IMPORTANT : It is untested yet, because I was a bit afraid of destroying my unisacsi setup while still working on the oceanography report ;). So pls test it before adding it to the main branch. What I basically did with my current setup is setting save_daily_files to True and then do a for-loop with the try/except around the download_MET_model_data function, that is constantly changing the .yml file and then downloads the file just for one day. If it runs into an error, it will skip the day. It's probably not the cleanest solution for it but it works and I actually managed to download a year of data like this without errors interrupting the download, so I assume that it will work the same way inside the toolbox itself.
Regard, Jan (from the AGF-213 course)