Merge pull request #30 from Infectious-Disease-Modeling-Hubs/br-v1.0.0

Update docs for v1.0.0

README.md

@@ -94,9 +94,20 @@ To use Live Server:
 
 # Versioning
 
-Documentation is [versioned by using releases](https://docs.readthedocs.io/en/stable/versions.html). Releases should track releases of Hub schema versions in [`schemas` repository](https://github.com/Infectious-Disease-Modeling-Hubs/schemas). While changes to documentation text can be commited without creating a new release and will appear in th `latest` version of the documentation, **changes to documentation related to a new schema release must be accompanied by a new release in this repository**. New releases on `hubDocs` should use the same version number as the `schemas` release but without the `v` (e.g. a `v0.0.1` `schemas` version number would be released as `0.0.1` on `hubDocs`).
+Documentation is [versioned by using releases](https://docs.readthedocs.io/en/stable/versions.html). Releases should track releases of Hub schema versions in [`schemas` repository](https://github.com/Infectious-Disease-Modeling-Hubs/schemas). While changes to documentation text can be commited without creating a new release and will appear in the `latest` version of the documentation, **changes to documentation related to a new schema release must be accompanied by a new release in this repository**. New releases on `hubDocs` should use the same version number as the `schemas` release but without the `v` (e.g. a `v0.0.1` `schemas` version number would be released as `0.0.1` on `hubDocs`).
+
+When creating a new release version:
+
+1. Checkout the main branch and ensure you pull all changes from the remote repository.
+2. Create a new branch of the main branch and name it using the convention `br-<version-number>`
+3. Open `docs/source/conf.py` file and update the value of the `schema_version` variable with the version of the schema in the `schemas` repository you want the release to accompany (e.g. `v0.0.1`). This propagates the appropriate version to various substitution text elements within the docs, including the URLs pointing docson widgets to raw config schema files.
+4. If the version of the schema you are preparing the release for has not been released to `main` branch in the `schemas` repository, you can set the value of the `schema_branch` variable to the name of the branch in the `schemas` repository in which the version is being prepared (e.g. `br-v1.0.0`). This allows you to see what development versions of the schema will look like in the docson widgets while developing locally and in a pull request. If the schema has been released to `main` in the `schemas` repo, set `schema_branch` to `"main"`. The value of this variable is overriden automatically when READTHEDOCS builds the documentation on the `main` branch (or any other branch for that matter, in contrast to a pull request build) after a merge or a new release.
+5. Any any changes to the documentation needed.
+6. Commit and push changes (including changes to `conf.py`)
+7. Create pull request and merge after review.
+8. [Create a release on GitHub](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository?tool=webui#creating-a-release) labelling it with the same version number as the `schemas` release this release is associated with but without the `v` (e.g. a `v0.0.1` `schemas` version number would be released as `0.0.1` on `hubDocs`).
+
 
-Before making a new release, **ensure that URLs to schema files which power the interactive docson widget visualisation of schemas in the [`docs/source/format/hub-metadata.md` page](https://github.com/Infectious-Disease-Modeling-Hubs/hubDocs/blob/main/docs/source/format/hub-metadata.md?plain=1) are updated** to display the new versions of `admin.json` and `tasks.json` schema files.
 
 ## Contribution guidelines
 In general, contributions should be made via pull requests to the `main` branch. 

docs/source/conf.py

@@ -61,7 +61,24 @@
 # -- Options for EPUB output
 epub_show_urls = 'footnote'
 
+schema_version = "v1.0.0"
+# Use schema_branch variable to specify a branch in the schemas repository from which config schema will be source, especially for docson widgets.
+# Useful if the schema being documented hasn't been released to the `main` branch in the schemas repo yet. If version has been released already, set this to "main".
+schema_branch = "br-"+schema_version
+
+# The following statements override any custom branch assigned to schema branch if the build is being run on READTHEDOCS and is either a build for a new tag or on a branch
+# (in contrast to being run in a pull request or locally). This ensures that any production versions of the docs published on the hubDocs `main` branch always 
+# point to schema on the `main` branch of the `schemas` repository. It also allows for previewing docson widgets and links to schema in branches other than the
+# main branch in the schemas repos when developing locally or in pull requests
+import os
+build_type = os.environ.get("READTHEDOCS_VERSION_TYPE")
+if build_type is None:
+    build_type = "unknown"
+
+if build_type in ("tag", "branch"):
+    schema_branch = "main"
 
 myst_substitutions = {
-    'schema_version': "v0.0.1"
+    'schema_version': schema_version,
+    'schema_branch': schema_branch
 }

docs/source/format/hub-config.md

@@ -0,0 +1,59 @@
+(hub-config)=
+# Hub configuration files
+
+## Directory Structure
+The `hub-config` directory in a modeling hub is required to contain three files:
+   1. `admin.json` - JSON  file containing generic information about the hub as well as static configuration settings for downstream tools such as validations, visualizations, etc.
+   2. `tasks.json` - JSON file specifing modeling tasks and model output formats, which may be round-specific.
+   3. `model-metadata-schema.json` - JSON or YAML file defining format of model metadata files
+
+```{caution}
+Note:  Due to technical issues, we do not currently support json references or yaml metadata files.
+```
+
+## Purpose
+The files withing the `hub-config` directory specify general configurations for a hub as well as (possibly round-specific) details of what model outputs are requested or required. Hub configuration files are used for:
+* Validating model output submissions
+   * `tasks.json` file specifies the file format and task id, output type, value combinations (both required or optional) that submitted model output data must adhere to.
+   *  `tasks.json` file also specifies the window of submission for each round (with the time zone information in the `admin.json` file). 
+* Scoring model outputs
+   * the hub configuration files specify the scores that are used
+   * the task id variables specified in the `tasks.json` can be used to join model output data with truth data for the purpose of scoring forecasts.
+* Configuring model output visualizations
+   * Visualization tools may benefit from the ability to programmatically identify task id variables so that a separate visualization of model outputs can be generated for each combination of those variables (e.g. via facetting or menu selections). For example, it may be beneficial to produce separate visualizations for different locations or scenario ids.
+   * Visualization tools may give special treatment to the hub’s ensemble and baseline models, which are identified in the hub configuration files.
+   * The `tasks.json` file contains metadata regarding the targets including human readable description and units which can be used for visualization 
+* Report generation
+   * `admin.json` allows configuration of ensemble and baseline models to be treated specially in reports.
+
+
+## Hub administrative configuration (`admin.json` file)
+
+The administrative hub configuration file contains global administrative settings that are expected to remain fixed throughout a hub’s existence and applies to all rounds in a hub.
+
+### Hub administrative configuration (`admin.json`) Interactive Schema
+
+#### Schema Version: {{schema_version}}
+{{'[See raw schema](https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/BRANCH/SCHEMA_VERSION/admin-schema.json)'.replace('SCHEMA_VERSION', schema_version).replace('BRANCH', schema_branch)}}
+
+{{'<script src="../_static/docson/widget.js" data-schema="https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/BRANCH/SCHEMA_VERSION/admin-schema.json"></script>'.replace('SCHEMA_VERSION', schema_version).replace('BRANCH', schema_branch)}}
+
+```{note}
+   Other things we may want to consider adding here:
+* Something about truth data?
+* Something about scoring?
+* Something about report generation?
+```
+
+(tasks_metadata)=
+## Hub model task configuration (`tasks.json` file)
+The hub model task configuration file specifies the model tasks (tasks id and targets) as well as model output types. The `tasks.json` file is flexible enough to accomodate different style of hubs. Hubs can varie from a simple forecast hub (see [US Forecast Hub example](/format/intro-data-formats.md) to a more complex round related scenario hub (see [US Scenario Modeling Hub example](/format/intro-data-formats.md)).
+
+
+### Model Tasks (`tasks.json`) Interactive Schema
+
+#### Schema Version: {{schema_version}}
+{{'[See raw schema](https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/BRANCH/SCHEMA_VERSION/tasks-schema.json)'.replace('SCHEMA_VERSION', schema_version).replace('BRANCH', schema_branch)}}
+
+{{'<script src="../_static/docson/widget.js" data-schema="https://raw.githubusercontent.com/Infectious-Disease-Modeling-Hubs/schemas/BRANCH/SCHEMA_VERSION/tasks-schema.json"></script>'.replace('SCHEMA_VERSION', schema_version).replace('BRANCH', schema_branch)}}
+

docs/source/format/hub-structure.md

@@ -1,16 +1,16 @@
 (hub-structure)=
 # Structure of Hub repositories
 
-A Hub should be structured according to the following recommendations. 
+A Hub should be structured according to the following recommendations.  
 
-Generally, the Hub repository is intended primarily as a storage space for primary data. All other code and outputs related to model output validation, visualizations, reports, ensemble construction, etc., should be placed in repositories other than the primary Hub repository.
+Generally, Hub file structure is intended primarily as a storage space for primary data. All other code and outputs related to model output validation, visualizations, reports, ensemble construction, etc., should be placed in repositories other than the primary Hub location.
 
 The directory and file structure of a modeling hub should contain only the following directories and files:
 
 * Documentation files
    * Hubs should provide a documentation file (e.g., `README.md`) at the top level that describes the overall structure of the hub, as well as a documentation file within each folder that provides more detail.
 
-* `hub-metadata` directory (see {doc}`/format/hub-metadata`)
+* `hub-config ` directory (see {doc}`/format/hub-config`)
 
 * `model-output` directory (see {doc}`/format/model-output`) 
 
@@ -22,5 +22,7 @@ The directory and file structure of a modeling hub should contain only the follo
 
 * `auxiliary-data` (optional, see {doc}`/format/target-data`)
 
-* Optionally, any files necessary to define continuous integration workflows, for example for the purpose of validating submissions or updating target data. To the extent possible, only the workflow definition files should be stored within the Hub repository, with any additional scripts or functionality residing in an external repository.
+* Optionally, any files necessary to define continuous integration workflows, for example for the purpose of validating submissions or updating target data. To the extent possible, only the workflow definition files should be stored within the Hub file space, with any additional scripts or functionality residing in an external location.
+
+Although most hubs to date have been housed in GitHub repositories, the proposed structure is more general and can be adapted for use on any shared filesystem.