Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create an example workflow that processes data and then runs the PDG workflow (viz-staging) #38

Open
mfisher87 opened this issue May 22, 2024 · 3 comments

Comments

@mfisher87
Copy link
Member

mfisher87 commented May 22, 2024

Reference the viz-info repository for a high-level view of what we're implementing.

Take an example recipe (e.g, seal-tags) and chain its simple recipe with the PDG processing steps required to prepare the data for visualization in the QGreenland data portal.

An example that prepares the output from such a recipe can be found here: https://github.com/rushirajnenuji/dataone-gse/blob/feature-argo-helm/scripts/workflows/ice_basins_workflow.py

@trey-stafford
Copy link
Member

The ice basins dataset is currently experiencing some problems (attempts to access the data results in 404 errors).

@trey-stafford trey-stafford changed the title Create an example "medium" workflow (hydrology > ice basins) Create an example workflow that prepares data for PGD processing Dec 18, 2024
@trey-stafford trey-stafford transferred this issue from QGreenland-Net/argo-exploration Dec 18, 2024
@trey-stafford trey-stafford removed the status in QGreenland-Net Dec 18, 2024
@trey-stafford trey-stafford changed the title Create an example workflow that prepares data for PGD processing Create an example workflow that processes data and then runs the PDG workflow (viz-staging) Dec 18, 2024
@trey-stafford
Copy link
Member

Some pseudo code:

def make_viz_staging_workflow(input_artifact: Artifact | DataOneDataset) -> Workflow:
    """Take an input artifact/dataone dataset and run the viz-staging workflow on it.

    * Generate the config necessary for viz-staging
    * Run viz-staging, producing web tiles for viz
    * Trigger some event to tell cesium/web portal that a new dataset is ready for viz. (?)
    """
   ...

@trey-stafford
Copy link
Member

There's an example workflow of the PDG processing on a QGreenland dataset that we can use to start from in this branch: https://github.com/rushirajnenuji/dataone-gse/tree/feature-argo-devk8s-deploy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

4 participants