Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Challenge #31 -ECMWF forecast data in 3D with Blender #4

Open
EsperanzaCuartero opened this issue Jan 28, 2021 · 14 comments
Open

Challenge #31 -ECMWF forecast data in 3D with Blender #4

EsperanzaCuartero opened this issue Jan 28, 2021 · 14 comments
Assignees
@sylvielamythepaut @StephanSiemen
Labels
stream-3 Stream 3 - Visualize weather, climate and atmosphere

Comments

@EsperanzaCuartero
Copy link
Contributor

EsperanzaCuartero commented Jan 28, 2021 •
edited by jwagemann
Loading

Challenge 31- ECMWF forecast data in 3D with Blender

Stream 3 - Visualize weather, climate and atmosphere

Goal

Extend existing visualisation software to support ECMWF forecast data + examples

Mentors and skills

Challenge description

For creating high quality visualisations of forecast data, Blender has proven a valuable tool. There has been various attempts to load GRIB data but they have not been sustainable and scalable. A new project started called BlenderNC to offer a Python add-on using xarray to load NetCDF data. This project should use the xarray extension cfgrib to extend BlenderNC to be able to read and visualise GRIB data. It is also expected that the documentation at https://blendernc.readthedocs.io will be updated with examples of using GRIB data. Interested candidates have a chance to work on an exciting Python project and learn how to handle weather model data and how to visualise it.


ESoWC
@EsperanzaCuartero EsperanzaCuartero added the stream-3 Stream 3 - Visualize weather, climate and atmosphere label Jan 28, 2021
@EsperanzaCuartero EsperanzaCuartero changed the title Challenge # xx -ECMWF forecast data in 3D with Blender Challenge # 31 -ECMWF forecast data in 3D with Blender Jan 28, 2021
@jwagemann jwagemann changed the title Challenge # 31 -ECMWF forecast data in 3D with Blender Challenge #31 -ECMWF forecast data in 3D with Blender Jan 29, 2021
@whatnick
Copy link

I would be interested in this challenge. I have the relevant experience in Python, Xarray and Blender. I work regularly with NetCDF and more recently with GRIB files for forcing WRF models.

@EsperanzaCuartero
Copy link
Contributor Author

Hi Tisham, thanks for your interest. If you have a question, you can start a conversation with the mentors of this challenge and tailor your proposal before submitting it. The deadline for applications is on 16 April. Best, Esperanza

@jwagemann
Copy link
Contributor

Hi,
join us for the ECMWF Summer of Weather Code Ask Me Anything session and learn all things ESoWC.

When: Wednesday, 24 March 2021 at 4 pm GMT

What: learn everything about ESoWC - how it works, the challenges this year, some tips for your proposal and listen to ESoWC experiences from previous participants

How: register here.

@whatnick
Copy link

whatnick commented Apr 8, 2021

Hi @sylvielamythepaut @StephanSiemen , @josuemtzmo and I are putting together a joint proposal for this challenge. Josue is a maintainer for the BlenderNC project and in Canberra with myself. We look forward to working on BlenderNC.

@josuemtzmo josuemtzmo mentioned this issue Apr 8, 2021
Closed
@StephanSiemen
Copy link

Hi @whatnick,
Thanks for letting us know. Sounds great! Looking forward to your proposal :-)

@whatnick
Copy link

@StephanSiemen I currently have this challenge loading the ECMWF forecast data I have using cfgrib. ecmwf/cfgrib#232 , some assistance will be appreciated.

@iainrussell
Copy link

Hi @whatnick , I'm writing a comment into the cfgrib issue, but in the meantime can you successfully use the ecCodes tool 'grib_to_netcdf' as described here if you only want to do a file conversion? https://confluence.ecmwf.int/display/ECC/grib_to_netcdf You will need to install ecCodes, perhaps through conda-forge. It might work, but mixing GRIB editions could be problematic, let's see.

@josuemtzmo
Copy link

Hello,
As far as I know, BlenderNC could support cgrib files by implementing:

 xr.load_dataset('filename.grib', engine='cfgrib')

an xarray example can be found at:
http://xarray.pydata.org/en/stable/examples/ERA5-GRIB-example.html

@iainrussell
Copy link

Another thought, if cfgrib is complaining because of a mixture of GRIB1 and GRIB2 messages, you can separate them using the filter_by_keys argument described here: https://github.com/ecmwf/cfgrib
Untested suggestion:
g1 = xr.open_dataset('mydata.grib', engine='cfgrib', backend_kwargs={'filter_by_keys': {'edition': '1'}})
g2 = xr.open_dataset('mydata.grib', engine='cfgrib', backend_kwargs={'filter_by_keys': {'edition': '2'}})
Then you can save them into separate netCDF files if that's useful.

@sylvielamythepaut
Copy link

Hi,

I just created a shared forder on Dropbox .
I put 1 set of data ( in Grib and NetCDF)

This is a 10 days forecast for 5 parameters:
2t : 2m temperature
lcc, mcc, hcc : low, Medium, High Cloud Cover
tp : Total precipitation ( I think they are accumulated .. May be we need to do a bit of processing on this one )
That should allow us to start

@whatnick
Copy link

whatnick commented May 29, 2021 •
edited
Loading

I have downloaded and verified that this data can be loaded successfully. What are the license conditions for this data. If this data can be CC licensed we could place it in our Github under LFS and use as test data in CI. I have put my exploratory code in a notebook here. https://gist.github.com/whatnick/0e2821f1abacba8d57c1ebdeea6487d7

More details on GIT LFS are available here. https://git-lfs.github.com/ . If a suitable licence can be obtained then please feel free to make a PR to blenderNC with this data in say a data folder.

@whatnick whatnick mentioned this issue May 29, 2021
Merged
2 tasks
@EsperanzaCuartero
Copy link
Contributor Author

EsperanzaCuartero commented Jun 1, 2021 via email

@sylvielamythepaut
Copy link

I do not think there is any issue with the license, but I will check and come back to you.

@josuemtzmo
Copy link

Hi Tisham, I've already sent an invite for you to create a repository under ESoWC space GitHub. I've used the GitHub handle you provided in the form https://github.com/josuemtzmo Thanks. Best Esperanza
…

Hello Esperanza, I've accepted the invitation, I've created an issue to draft the README file Issue 51.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sylvielamythepaut sylvielamythepaut

@StephanSiemen StephanSiemen

Labels
stream-3 Stream 3 - Visualize weather, climate and atmosphere
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@whatnick @jwagemann @josuemtzmo @sylvielamythepaut @StephanSiemen @iainrussell @EsperanzaCuartero