development

WCRP-CMIP · Dec 2, 2024 · 59e0222 · 59e0222
1 parent 87a08ff
commit 59e0222
Show file tree

Hide file tree

Showing 28 changed files with 193 additions and 32 deletions.
diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml
@@ -30,7 +30,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install mkdocs mkdocs-material pymdown-extensions
+        pip install mkdocs mkdocs-material pymdown-extensions mkdocs-gen-files
 
     - name: Build MkDocs
       run: mkdocs build --clean

diff --git a/docs/.DS_Store b/docs/.DS_Store
diff --git a/docs/delete/99_Acknowlegements.md → docs/.delete/99_Acknowlegements.md b/docs/delete/99_Acknowlegements.md → docs/.delete/99_Acknowlegements.md
diff --git a/...mations/creating_a_dispatch_automation.md → ...mations/creating_a_dispatch_automation.md b/...mations/creating_a_dispatch_automation.md → ...mations/creating_a_dispatch_automation.md
diff --git a/...ete/CMIP6Plus/Automations/gencv_action.md → ...ete/CMIP6Plus/Automations/gencv_action.md b/...ete/CMIP6Plus/Automations/gencv_action.md → ...ete/CMIP6Plus/Automations/gencv_action.md
diff --git a/...lete/CMIP6Plus/Automations/generate_cv.md → ...lete/CMIP6Plus/Automations/generate_cv.md b/...lete/CMIP6Plus/Automations/generate_cv.md → ...lete/CMIP6Plus/Automations/generate_cv.md
diff --git a/docs/delete/CMIP6Plus/Rules/activity_id.md → docs/.delete/CMIP6Plus/Rules/activity_id.md b/docs/delete/CMIP6Plus/Rules/activity_id.md → docs/.delete/CMIP6Plus/Rules/activity_id.md
diff --git a/docs/delete/CMIP6Plus/Rules/institution.md → docs/.delete/CMIP6Plus/Rules/institution.md b/docs/delete/CMIP6Plus/Rules/institution.md → docs/.delete/CMIP6Plus/Rules/institution.md
diff --git a/docs/delete/CMIP6Plus/index.md → docs/.delete/CMIP6Plus/index.md b/docs/delete/CMIP6Plus/index.md → docs/.delete/CMIP6Plus/index.md
diff --git a/...ools/Preprocessing/preprocess_parallel.md → ...ools/Preprocessing/preprocess_parallel.md b/...ools/Preprocessing/preprocess_parallel.md → ...ools/Preprocessing/preprocess_parallel.md
diff --git a/docs/delete/Data-Analysis-Tools/README.md → docs/.delete/Data-Analysis-Tools/README.md b/docs/delete/Data-Analysis-Tools/README.md → docs/.delete/Data-Analysis-Tools/README.md
diff --git a/...ols/Visualisation/change_in_windvector.md → ...ols/Visualisation/change_in_windvector.md b/...ols/Visualisation/change_in_windvector.md → ...ols/Visualisation/change_in_windvector.md
diff --git a/docs/delete/MIP_Tables/advanced_search.md → docs/.delete/MIP_Tables/advanced_search.md b/docs/delete/MIP_Tables/advanced_search.md → docs/.delete/MIP_Tables/advanced_search.md
diff --git a/docs/delete/MIP_Tables/index.md → docs/.delete/MIP_Tables/index.md b/docs/delete/MIP_Tables/index.md → docs/.delete/MIP_Tables/index.md
diff --git a/docs/delete/mailing_lists/whitelisting.md → docs/.delete/mailing_lists/whitelisting.md b/docs/delete/mailing_lists/whitelisting.md → docs/.delete/mailing_lists/whitelisting.md
diff --git a/docs/How_to_use_JSONLD/ _Introduction.md b/docs/How_to_use_JSONLD/ _Introduction.md
@@ -0,0 +1,46 @@
+
+
+There are several available methods to interact with the JSONLD files. 
+Instructions on how to use these are found on each respective page within the sub menu. 
+
+
+### Methods of interaction 
+
+The main categories are: 
+#### 1. Direct interaction
+Using the files locally or from the GitHub Repository. This can be done in a number of ways: 
+- local copy
+- github online interface
+- GET commands from a scripting language
+
+#### 2. CMIPLD web browsing interface. 
+This is a rudementary JSONLD static page that can access all the JSONld files. This can be used to:
+- check file contents
+- view context 
+- perform framing requests
+- explore available files
+- quickly check entries using prefixes without the need for a full URL
+
+#### 3. PyLD native python interface. 
+This is the native library for interacting with JSONLD in python. This has the benefits of: 
+- Extracting files in Python
+- Framing 
+- Performing complex checks and incorporating the CVs in a personal workflow. 
+
+#### 4. The CMIP-LD CLI
+A quick command line interface that allows dynamic JSON-LD file viewing in the terminal. 
+- performs recursive fills
+- allows the use of prefixes to easily access files
+- colapsable and coloured JSON exploration in the terminal 
+- outputs file contents or framing into a file. 
+
+
+#### 5. CMIP-LD Python Library
+A wrapper around PyLD with a number of useful tools and presets to ease the learning curve for non-technical and technical users. 
+- A list of registerd LD repositories. 
+- Can access and cross compare files both online and offline. 
+- Recursive file population (extracts all nested (linked) entries into your file)
+- Link Checker
+- LD Network generator. 
+- File Checker
+- Easy Framer and file simplifier. 
diff --git a/docs/How_to_use_JSONLD/CMIP-LD/CMIP-LD.md b/docs/How_to_use_JSONLD/CMIP-LD/CMIP-LD.md
diff --git a/docs/How_to_use_JSONLD/CMIP-LD/Downloading_all_Repos.md b/docs/How_to_use_JSONLD/CMIP-LD/Downloading_all_Repos.md
diff --git a/docs/How_to_use_JSONLD/CMIP-LD/Framing.md b/docs/How_to_use_JSONLD/CMIP-LD/Framing.md
diff --git a/docs/How_to_use_JSONLD/Command_line_interface.md b/docs/How_to_use_JSONLD/Command_line_interface.md
@@ -0,0 +1,10 @@
+Screenshots and usage to go here. 
+
+## Installation 
+
+```bash 
+pip install git+https://github.com/WCRP-CMIP/CMIP-LD.git
+```
+! warning
+Beware that this repository is currently rapidly in development, and you will have to upgrade the installed version regularly. 
+
diff --git a/docs/How_to_use_JSONLD/Direct_Interaction.md b/docs/How_to_use_JSONLD/Direct_Interaction.md
@@ -0,0 +1,49 @@
+
+## Introduction
+
+There are several available methods to interact with the JSONLD files. 
+
+
+
+
+### Direct interaction 
+
+
+
+As the files are valid JSON custom scripts can be used to interact with them directly.
+
+| **Repository**     | **GitHub Pages URL**                                         | **Repository URL**          |
+|---------------------|-------------------------------------------------------------|-----------------------------:|
+| wcrp-universe       | [https://wcrp-cmip.github.io/WCRP-UNIVERSE/](https://wcrp-cmip.github.io/WCRP-UNIVERSE/)      | [repo-link](https://github.com/wcrp-cmip/WCRP-UNIVERSE)      |
+| mip-variables       | [https://wcrp-cmip.github.io/MIP-variables/](https://wcrp-cmip.github.io/MIP-variables/)      | [repo-link](https://github.com/wcrp-cmip/MIP-variables)      |
+| cmip6plus           | [https://wcrp-cmip.github.io/CMIP6Plus_CVs/](https://wcrp-cmip.github.io/CMIP6Plus_CVs/)      | [repo-link](https://github.com/wcrp-cmip/CMIP6Plus_CVs)      |
+| cmip7               | [https://wcrp-cmip.github.io/CMIP7_CVs/](https://wcrp-cmip.github.io/CMIP7_CVs/)              | [repo-link](https://github.com/wcrp-cmip/CMIP7_CVs)          |
+| cf                  | [https://wcrp-cmip.github.io/CF/](https://wcrp-cmip.github.io/CF/)                           | [repo-link](https://github.com/wcrp-cmip/CF)                 |
+| obs4mips            | [https://wolfiex.github.io/obs4MIPs-cmor-tables-ld/](https://wolfiex.github.io/obs4MIPs-cmor-tables-ld/) | [repo-link](https://github.com/wolfiex/obs4MIPs-cmor-tables-ld) |
+
+
+
+This means that you can download a local copy with 
+```bash 
+git clone <github repo url>
+```
+View them in the browser, or use the requests / GET api to download them directly. 
+
+#### File structure
+If using the github interface, you can navigate as normal. JSONLD files are currently located under the `data_descriptors` directory in each repository. 
+
+
+If the user should so wish, they are also served using github pages, under the same filepath excluding the term `data_descriptors`. 
+
+This means that the following activity page
+
+[ https://github.com/WCRP-CMIP/WCRP-UNIVERSE/blob/main/data_descriptors/activity/cmip.json
+]( https://github.com/WCRP-CMIP/WCRP-UNIVERSE/blob/main/data_descriptors/activity/cmip.json
+ )
+
+ becomes
+
+[ https://wcrp-cmip.github.io/WCRP-UNIVERSE/activity/cmip.json]( https://wcrp-cmip.github.io/WCRP-UNIVERSE/activity/cmip.json)
+
+ where the `.json` is optional, but helps the browser open the file for viewing rather than downloading it. 
+
diff --git a/docs/How_to_use_JSONLD/PyLD.md b/docs/How_to_use_JSONLD/PyLD.md
@@ -0,0 +1,2 @@
+
+Python LD Notebooks to go in here. 
diff --git a/docs/How_to_use_JSONLD/cmipLD_web_viewer.md b/docs/How_to_use_JSONLD/cmipLD_web_viewer.md
@@ -0,0 +1,34 @@
+
+A wireframe of the JSONLD viewer can be found at : 
+
+[https://wcrp-cmip.github.io/CMIP-LD/static/viewer/](https://wcrp-cmip.github.io/CMIP-LD/static/viewer/)
+
+To test this out, we can enter the URI for a JSONLD file. E.g. for the CMIP activity we have have the ID:  
+[https://wcrp-cmip.github.io/WCRP-UNIVERSE/activity/cmip](https://wcrp-cmip.github.io/WCRP-UNIVERSE/activity/cmip)
+
+### Link simplification 
+Rather than remembering this whole link, we are also able to use the pre-defined prefix from the table before
+
+| **Repository**     | **GitHub Pages URL**                                         | **Repository URL**          |
+|---------------------|-------------------------------------------------------------|-----------------------------:|
+| wcrp-universe       | [https://wcrp-cmip.github.io/WCRP-UNIVERSE/](https://wcrp-cmip.github.io/WCRP-UNIVERSE/)      | [repo-link](https://github.com/wcrp-cmip/WCRP-UNIVERSE)      |
+| mip-variables       | [https://wcrp-cmip.github.io/MIP-variables/](https://wcrp-cmip.github.io/MIP-variables/)      | [repo-link](https://github.com/wcrp-cmip/MIP-variables)      |
+| cmip6plus           | [https://wcrp-cmip.github.io/CMIP6Plus_CVs/](https://wcrp-cmip.github.io/CMIP6Plus_CVs/)      | [repo-link](https://github.com/wcrp-cmip/CMIP6Plus_CVs)      |
+| cmip7               | [https://wcrp-cmip.github.io/CMIP7_CVs/](https://wcrp-cmip.github.io/CMIP7_CVs/)              | [repo-link](https://github.com/wcrp-cmip/CMIP7_CVs)          |
+| cf                  | [https://wcrp-cmip.github.io/CF/](https://wcrp-cmip.github.io/CF/)                           | [repo-link](https://github.com/wcrp-cmip/CF)                 |
+| obs4mips            | [https://wolfiex.github.io/obs4MIPs-cmor-tables-ld/](https://wolfiex.github.io/obs4MIPs-cmor-tables-ld/) | [repo-link](https://github.com/wolfiex/obs4MIPs-cmor-tables-ld) |
+
+
+
+This means that to get to the same page, we can now type 
+
+```
+wcrp-universe:activity/cmip
+```
+
+### Getting The result
+The web interface will not present a compacted view of the JSONLD file requested as such.
+<img style='width:800px' src='/assets/demo_images/idweb1.png'/>
+
+### Framing
+Currently this option is disabled to a CORRS same origin policy, but this will soon be fixed. 
diff --git a/docs/How_to_use_JSONLD/filestructure.md b/docs/How_to_use_JSONLD/filestructure.md
diff --git a/docs/What_is_JSONLD?/Use_and_Changes.md b/docs/What_is_JSONLD?/Use_and_Changes.md
@@ -30,10 +30,56 @@ In addition to this we apply a intuitive unique id to each item, and a context f
 
 #### Example file change to LD
 
+```json
+{
+            "description": "CMIP DECK: 1pctCO2, abrupt-4xCO2, amip, esm-piControl, esm-historical...",
+            "activity_id": "CMIP",
+            "url": "https://gmd.copernicus.org/articles/9/1937/2016/gmd-9-1937-2016.pdf"
+}
+```
+Is now updated to have an `id`, `type` and `@context`
+
+```json
+{
+            "@context": "_context_",
+            "id": "cmip",
+            "type": "activity",
+            "description": "CMIP DECK: 1pctCO2, abrupt-4xCO2, amip, esm-piControl, esm-historical...",
+            "name": "CMIP",
+            "url": "https://gmd.copernicus.org/articles/9/1937/2016/gmd-9-1937-2016.pdf"
+}
+```
 
 #### Context files   
-
-
+The context file defines how this shall be treated, where the files are located, and if there are any additional links to the fields. An example context file is given below
+```jsonld
+
+{
+    "@context": {
+                    // define the base and prefix adresses so that we do not have to specify these in the file
+                    "@base": "https://wcrp-cmip.github.io/CMIP6Plus_CVs/",
+                    "@vocab": "https://wcrp-cmip.github.io/CMIP6Plus_CVs/",
+
+                    // map id and type to @id and @type. This is done for legibility
+                    "id": "@id",
+                    "type": "@type",
+
+                    // add any external prefixes. 
+                    "cmip7": "https://wcrp-cmip.github.io/CMIP7_CVs/",
+                    "wcrp-universe": "https://wcrp-cmip.github.io/WCRP-UNIVERSE/"
+
+                    // define information for fields which may be linked. 
+                    "activity": {
+                                    "@context": "https://wcrp-cmip.github.io/WCRP-UNIVERSE/activity/_context_",
+                                    "@type": "@id",
+                            }
+    },
+
+    // framing options go here, e.g. expand all links
+    "@embed": "@always"
+}
+```
+The `_context_` files in our repositories are used both to define additional `@context` information, and to provide a basic framing context should that be required by the user. 
 
 
 ### Other changes to the CVs

diff --git a/docs/assets/.DS_Store b/docs/assets/.DS_Store
diff --git a/docs/assets/demo_images/idweb1.png b/docs/assets/demo_images/idweb1.png
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -10,6 +10,9 @@ theme:
   plugins:
     - search
     - optimize
+    # - gen-files:
+    #   scripts:
+    #     - scripts/
 
   copyright: CCBY04 2023+ Daniel Ellis for the CMIP-IPO
   palette: