You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have been trying to use cfgrib and add their localConcepts, of which I'm still struggling a little (with the total column mass density of aerosols), to read the NOAA GEFS-Aerosol data. A sample file can be found here. How grib handles aerosol data is fairly tricky as it requires many additional tables and also many variables share the same "shortName". I would greatly appreciate any help with this as it would really help to have a tool that properly handles the air composition data put out by the NWS.
xr.open_dataset('gefs.chem.t00z.a2d_0p25.f000.grib2', engine='grib2io', filters={'typeOfFixedSurface':10})
---------------------------------------------------------------------------AttributeErrorTraceback (mostrecentcalllast)
CellIn[16], line1---->1xr.open_dataset('gefs.chem.t00z.a2d_0p25.f000.grib2', engine='grib2io', filters={'typeOfFixedSurface':10})
File~/miniforge3/envs/dev/lib/python3.12/site-packages/xarray/backends/api.py:566, inopen_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)
554decoders=_resolve_decoders_kwargs(
555decode_cf,
556open_backend_dataset_parameters=backend.open_dataset_parameters,
(...)
562decode_coords=decode_coords,
563 )
565overwrite_encoded_chunks=kwargs.pop("overwrite_encoded_chunks", None)
-->566backend_ds=backend.open_dataset(
567filename_or_obj,
568drop_variables=drop_variables,
569**decoders,
570**kwargs,
571 )
572ds=_dataset_from_backend_dataset(
573backend_ds,
574filename_or_obj,
(...)
584**kwargs,
585 )
586returndsFile~/miniforge3/envs/dev/lib/python3.12/site-packages/grib2io/xarray_backend.py:82, inGribBackendEntrypoint.open_dataset(self, filename, drop_variables, filters)
76file_index=pd.DataFrame(f._index)
78# parse grib2io _index to dataframe and acquire non-geo possible dims79# (scalar coord when not dim due to squeeze) parse_grib_index applies80# filters to index and expands metadata based on product definition81# template number--->82file_index, non_geo_dims=parse_grib_index(file_index, filters)
84# Divide up records by variable85frames, cube, extra_geo=make_variables(file_index, filename, non_geo_dims)
File~/miniforge3/envs/dev/lib/python3.12/site-packages/grib2io/xarray_backend.py:383, inparse_grib_index(index, filters)
381fork, vinfilters.items():
382ifknotinindex.columns:
-->383kwarg= {k:index.msg.apply(lambdamsg: getattr(msg, k))}
384index=index.assign(**kwarg)
385# adopt parts of xarray's sel logic so that filters behave similarly386# allowed to filter to nothing to make empty datasetFile~/miniforge3/envs/dev/lib/python3.12/site-packages/pandas/core/series.py:4924, inSeries.apply(self, func, convert_dtype, args, by_row, **kwargs)
4789defapply(
4790self,
4791func: AggFuncType,
(...)
4796**kwargs,
4797 ) ->DataFrame|Series:
4798""" 4799 Invoke function on values of Series. 4800 (...) 4915 dtype: float64 4916 """4917returnSeriesApply(
4918self,
4919func,
4920convert_dtype=convert_dtype,
4921by_row=by_row,
4922args=args,
4923kwargs=kwargs,
->4924 ).apply()
File~/miniforge3/envs/dev/lib/python3.12/site-packages/pandas/core/apply.py:1427, inSeriesApply.apply(self)
1424returnself.apply_compat()
1426# self.func is Callable->1427returnself.apply_standard()
File~/miniforge3/envs/dev/lib/python3.12/site-packages/pandas/core/apply.py:1507, inSeriesApply.apply_standard(self)
1501# row-wise access1502# apply doesn't have a `na_action` keyword and for backward compat reasons1503# we need to give `na_action="ignore"` for categorical data.1504# TODO: remove the `na_action="ignore"` when that default has been changed in1505# Categorical (GH51645).1506action="ignore"ifisinstance(obj.dtype, CategoricalDtype) elseNone->1507mapped=obj._map_values(
1508mapper=curried, na_action=action, convert=self.convert_dtype1509 )
1511iflen(mapped) andisinstance(mapped[0], ABCSeries):
1512# GH#43986 Need to do list(mapped) in order to get treated as nested1513# See also GH#25959 regarding EA support1514returnobj._constructor_expanddim(list(mapped), index=obj.index)
File~/miniforge3/envs/dev/lib/python3.12/site-packages/pandas/core/base.py:921, inIndexOpsMixin._map_values(self, mapper, na_action, convert)
918ifisinstance(arr, ExtensionArray):
919returnarr.map(mapper, na_action=na_action)
-->921returnalgorithms.map_array(arr, mapper, na_action=na_action, convert=convert)
File~/miniforge3/envs/dev/lib/python3.12/site-packages/pandas/core/algorithms.py:1743, inmap_array(arr, mapper, na_action, convert)
1741values=arr.astype(object, copy=False)
1742ifna_actionisNone:
->1743returnlib.map_infer(values, mapper, convert=convert)
1744else:
1745returnlib.map_infer_mask(
1746values, mapper, mask=isna(values).view(np.uint8), convert=convert1747 )
Filelib.pyx:2972, inpandas._libs.lib.map_infer()
File~/miniforge3/envs/dev/lib/python3.12/site-packages/grib2io/xarray_backend.py:383, inparse_grib_index.<locals>.<lambda>(msg)
381fork, vinfilters.items():
382ifknotinindex.columns:
-->383kwarg= {k:index.msg.apply(lambdamsg: getattr(msg, k))}
384index=index.assign(**kwarg)
385# adopt parts of xarray's sel logic so that filters behave similarly386# allowed to filter to nothing to make empty datasetAttributeError: 'Msg'objecthasnoattribute'typeOfFixedSurface'In [17]: xr.open_dataset('gefs.chem.t00z.a2d_0p25.f000.grib2', engine='grib2io', filters={'typeOfFirstFixedSurface':10})
---------------------------------------------------------------------------ValueErrorTraceback (mostrecentcalllast)
CellIn[17], line1---->1xr.open_dataset('gefs.chem.t00z.a2d_0p25.f000.grib2', engine='grib2io', filters={'typeOfFirstFixedSurface':10})
File~/miniforge3/envs/dev/lib/python3.12/site-packages/xarray/backends/api.py:566, inopen_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)
554decoders=_resolve_decoders_kwargs(
555decode_cf,
556open_backend_dataset_parameters=backend.open_dataset_parameters,
(...)
562decode_coords=decode_coords,
563 )
565overwrite_encoded_chunks=kwargs.pop("overwrite_encoded_chunks", None)
-->566backend_ds=backend.open_dataset(
567filename_or_obj,
568drop_variables=drop_variables,
569**decoders,
570**kwargs,
571 )
572ds=_dataset_from_backend_dataset(
573backend_ds,
574filename_or_obj,
(...)
584**kwargs,
585 )
586returndsFile~/miniforge3/envs/dev/lib/python3.12/site-packages/grib2io/xarray_backend.py:93, inGribBackendEntrypoint.open_dataset(self, filename, drop_variables, filters)
91ds=xr.Dataset()
92forvar_dfinframes:
--->93da=build_da_without_coords(var_df, cube, filename)
94ds[da.name] =da96# assign coords from the cube; the cube prevents datarrays with97# different shapesFile~/miniforge3/envs/dev/lib/python3.12/site-packages/grib2io/xarray_backend.py:526, inbuild_da_without_coords(index, cube, filename)
524data=indexing.LazilyIndexedArray(data)
525iflen(dim_names) !=len(data.shape):
-->526raiseValueError(
527"different number of dimensions on data "528f"and dims: {len(data.shape)} vs {len(dim_names)}\n"529"Grib2 messages could not be formed into a data cube; "530"It's possible extra messages exist along a non-accounted for dimension based on PDTN\n"531"It might be possible to get around this by applying a filter on the non-accounted for dimension"532 )
533da=xr.DataArray(data, dims=dim_names)
535da.encoding['original_shape'] =data.shapeValueError: differentnumberofdimensionsondataanddims: 3vs2Grib2messagescouldnotbeformedintoadatacube; It'spossibleextramessagesexistalonganon-accountedfordimensionbasedonPDTNItmightbepossibletogetaroundthisbyapplyingafilteronthenon-accountedfordimension
The text was updated successfully, but these errors were encountered:
I have been trying to use cfgrib and add their localConcepts, of which I'm still struggling a little (with the total column mass density of aerosols), to read the NOAA GEFS-Aerosol data. A sample file can be found here. How grib handles aerosol data is fairly tricky as it requires many additional tables and also many variables share the same "shortName". I would greatly appreciate any help with this as it would really help to have a tool that properly handles the air composition data put out by the NWS.
The text was updated successfully, but these errors were encountered: