Add project overview #1
Conversation
|
|
||
| ### Resources | ||
|
|
||
| - [https://github.com/great-expectations/great_expectations/issues/1942](https://github.com/great-expectations/great_expectations/issues/1942) |
There was a problem hiding this comment.
I think if you took xarray-schema and added support for types of validation that xarray could not perform (e.g. union types, functional validators) then you would end up with something that's pretty similar to calling great expectations on a pandas dataframe
There was a problem hiding this comment.
This approach would work off-the-bat for a subset of the expectations, but I don't expect it to be entirely sufficient. xarray-schema currently relies on having loaded or at least opened an xarray dataset for validation. IMO a lot of the value of this tool would be providing explanations for why datasets cannot be simply opened with Xarray (e.g., https://github.com/briannapagan/quirky-data-checker/blob/main/results/results_GES_DISC_total_quirks.png)
Co-authored-by: Alex I. Mandel <[email protected]>
|
Adding a note here that this relates to NASA Earthdata's Data Product Development Guide. For example, see Sections 6.1 and 6.2 in the "Tools for Data Product Testing" section for related data format and compliance checking tools. |
|
👋🏽 @bilts we would love to get your feedback as well! |
This PR gives a short overview of the goals and some potential milestones for building
ndquirk.I'm tagging some people who've previously expressed interest in providing feedback on this project in case you'd like to share thoughts 🙏 @abarciauskas-bgse @sharkinsspatial @jhamman @omshinde @chuckwondo @moradology @jsignell. Thanks for your consideration!