diff --git a/docs/Users_Guide/barplots.rst b/docs/Users_Guide/barplots.rst index 32e65f76..3c263781 100644 --- a/docs/Users_Guide/barplots.rst +++ b/docs/Users_Guide/barplots.rst @@ -1,8 +1,9 @@ +********* Bar Plots -========= +********* Description ------------ +=========== A bar plot shows comparisons among discrete categories. One axis of the chart shows the specific categories being compared, while the other @@ -22,7 +23,7 @@ Bar plots often represent counts or frequencies, however, bar plots can represent means, medians, standard deviations, or any other statistic. How-To -------- +====== Selection of options to produce the plot proceeds approximately counter-clockwise around the METviewer window. The steps to create a series @@ -92,7 +93,7 @@ plot are: There are many other options for plots, but these are the basics. Example --------- +======= The image below shows an example of the plot and set-up options for a series plot in METviewer. This example uses the database diff --git a/docs/Users_Guide/batch.rst b/docs/Users_Guide/batch.rst index 63e719cc..fdbf23dd 100644 --- a/docs/Users_Guide/batch.rst +++ b/docs/Users_Guide/batch.rst @@ -1,5 +1,6 @@ +********************* Batch Plotting Module -===================== +********************* The batch plotting system is used to generate a number of plots of data in the METviewer database. The usage statement: @@ -439,10 +440,11 @@ starting at 1 for each fcst_var. | MODE Statistics ---------------- +=============== Single Object Statistics -~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------ + MODE statistics are broken up into two categories: single and pair statistics. Single statistics are suffixed with a three letter code which indicates the group of objects over which the statistic should be calculated @@ -597,7 +599,7 @@ RATIO\_ with AREARAT\_ and OBJ with OBJA. - OBJFBIAS Pair Object Statistics -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- MODE object pair statistics are specified using the following format: STATNAME_CC where STATNAME is one of the Object Pair Statistics in the diff --git a/docs/Users_Guide/boxplot.rst b/docs/Users_Guide/boxplot.rst index a5cb6d26..e505dc15 100644 --- a/docs/Users_Guide/boxplot.rst +++ b/docs/Users_Guide/boxplot.rst @@ -1,8 +1,9 @@ +********* Box Plots -========= +********* Description ------------ +=========== Box plots, or box and whisker plots, are a way to summarize a distribution of values using Tukey's 5-number summary @@ -27,7 +28,7 @@ threshold. Here, each forecast and observation is binary, but the verification statistic is continuous. How-To ---------- +====== Selection of options to produce the plot proceeds approximately counter-clockwise around the METviewer window. The steps to create a @@ -105,7 +106,7 @@ There are many other options for plots, but these are the basics. Example --------- +======= The example below shows a boxplot of the RMSE for 2m temperature over the CONUS. Many of the standard METviewer plotting options are available diff --git a/docs/Users_Guide/common.rst b/docs/Users_Guide/common.rst index 0245046d..891cc6a7 100644 --- a/docs/Users_Guide/common.rst +++ b/docs/Users_Guide/common.rst @@ -1,5 +1,6 @@ +********************* Common XML Structures -===================== +********************* The database loading module and batch plotting module both use XML input files. Some XML structures can be used in either input file. These common structures are documented below. diff --git a/docs/Users_Guide/contourplot.rst b/docs/Users_Guide/contourplot.rst index 6dc62863..debd71eb 100644 --- a/docs/Users_Guide/contourplot.rst +++ b/docs/Users_Guide/contourplot.rst @@ -1,8 +1,9 @@ +************* Contour Plots -============= +************* Description ------------ +=========== Contour plots are a very general plot type somewhat similar to a scatter plot. The axes are specified by the user, and any will do so long as the @@ -13,7 +14,7 @@ examine the difference in statistics between two NWP models as a contour plot, as is shown in the example below. How-To -------- +====== Selection of options to produce the plot proceeds approximately counter-clockwise around the METviewer window. The steps to create a contour @@ -76,7 +77,7 @@ plot are: There are many other options for plots, but these are the basics. Example --------- +======= The example is a contour plot of temperature mean error (bias) for one model at different pressure levels over a range of initialization times. diff --git a/docs/Users_Guide/database_loading.rst b/docs/Users_Guide/database_loading.rst index 74c26e6b..39b0d761 100644 --- a/docs/Users_Guide/database_loading.rst +++ b/docs/Users_Guide/database_loading.rst @@ -1,5 +1,6 @@ +*********************** Database Loading Module -======================= +*********************** The database loading module is used to insert, update, and delete MET output data in the database. The tool is invoked using the mv_load.sh script. The @@ -134,7 +135,7 @@ change it to Example -------- +======= Here is a simple example: @@ -203,7 +204,8 @@ specified by the named folder_dates. ... Troubleshooting ---------------- +=============== + .. _test: .. list-table:: diff --git a/docs/Users_Guide/database_scrubbing.rst b/docs/Users_Guide/database_scrubbing.rst index 01f3ff05..956114b1 100644 --- a/docs/Users_Guide/database_scrubbing.rst +++ b/docs/Users_Guide/database_scrubbing.rst @@ -1,5 +1,6 @@ +************************* Database Scrubbing Module -========================= +************************* The database scrubbing utility is used to to delete data from METviewer databases that meets some user-specified selection criteria. The usage @@ -58,7 +59,7 @@ performed (**FALSE**). | Examples --------- +======== **Example 1: Prune by describing the data ()** diff --git a/docs/Users_Guide/eclvplots.rst b/docs/Users_Guide/eclvplots.rst index 1f1f94ad..2bf1bccf 100644 --- a/docs/Users_Guide/eclvplots.rst +++ b/docs/Users_Guide/eclvplots.rst @@ -1,8 +1,9 @@ +****************************** Economic Cost/Loss Value Plots -============================== +****************************** Description ------------ +=========== The Economic Cost Loss Value statistic is sometimes also called the Relative value score (:ref:`Richardson, 2000`; @@ -23,11 +24,12 @@ The ECLV score can range from -:math:`\infty` to 1. Like ROC diagrams, it gives information that can be used in decision making. Line Type ---------- +========= + ECLV requires the ECLV line type generated by either Point-Stat or Grid-Stat. How-To ------- +====== Selection of options to produce the ECLV plot proceeds approximately counter-clockwise around the METviewer window. @@ -81,7 +83,7 @@ counter-clockwise around the METviewer window. Example --------- +======= The figure below shows an ECLV plot. In this example, three different forecasting systems are used to predict precipitation at two different diff --git a/docs/Users_Guide/etb_plot.rst b/docs/Users_Guide/etb_plot.rst index 01159734..1ae97729 100644 --- a/docs/Users_Guide/etb_plot.rst +++ b/docs/Users_Guide/etb_plot.rst @@ -1,8 +1,9 @@ +******************************** Equivalence Testing Bounds Plots -================================ +******************************** Description ------------ +=========== A statistical hypothesis is an assertion about the distribution function for one or more random variables. Generally, two hypotheses are considered, a null hypothesis denoted by H0, and an alternative denoted by @@ -32,7 +33,7 @@ In this case, H0 is accepted if the CI bounds fall completely between the tolera and H0 is rejected otherwise. How-To -------- +====== Selection of options to produce the plot proceeds approximately counter-clockwise around the METviewer window. The steps to create a @@ -104,7 +105,7 @@ There are many other options for plots, but these are the basics. Example --------- +======= The image below shows an example of the plot and set-up options for a series plot in METviewer. This example uses the database diff --git a/docs/Users_Guide/index.rst b/docs/Users_Guide/index.rst index 237d3271..c7553d0e 100644 --- a/docs/Users_Guide/index.rst +++ b/docs/Users_Guide/index.rst @@ -1,6 +1,6 @@ -============ +############ User's Guide -============ +############ **Foreword: A note to METviewer users** diff --git a/docs/Users_Guide/installation.rst b/docs/Users_Guide/installation.rst index 00ac5e20..ce9eee8b 100644 --- a/docs/Users_Guide/installation.rst +++ b/docs/Users_Guide/installation.rst @@ -1,17 +1,18 @@ .. _install: +**************************** METviewer Installation Guide -============================ +**************************** Introduction ------------- +============ This chapter describes how to install METviewer. METviewer has been developed and tested on Linux operating systems. Support for additional platforms and compilers may be added in future releases. Installing METviewer --------------------- +==================== METviewer relies on the following tools. These must be installed and tested prior to installing METviewer: @@ -118,7 +119,7 @@ clone `METdataio repository `_ git clone https://github.com/dtcenter/METdataio.git Configure and build METviewer -_____________________________ +----------------------------- #. Configure the batch and loading tools: @@ -225,7 +226,8 @@ _____________________________ ln -s /d3/projects/METViewer/src_dev/apps/verif/metviewer/sql/mv_mysql.sql mv_mysql.sql Making a Database Accessible in the METviewer Web Application -------------------------------------------------------------- +============================================================= + To make a new database accessible in the METviewer Web Application click on "Reload list of databases" button in the upper right corner of the main JSP page. The list of available databases should be updated and a diff --git a/docs/Users_Guide/overview.rst b/docs/Users_Guide/overview.rst index 5b494767..10826d6a 100644 --- a/docs/Users_Guide/overview.rst +++ b/docs/Users_Guide/overview.rst @@ -1,9 +1,9 @@ +********************* Overview of METviewer -===================== +********************* - -Purpose and organization of the User's Guide --------------------------------------------- +Purpose and Organization of the User's Guide +============================================ The goal of this User's Guide is to provide basic information for users of the METviewer database and display system to enable users to create plots @@ -13,7 +13,7 @@ The METviewer User's Guide is organized as follows. The Developmental Testbed Center (DTC) --------------------------------------- +====================================== METviewer has been developed, and will be maintained and enhanced, by the Developmental Testbed Center (`DTC `_ ). The main @@ -34,8 +34,8 @@ modeling community for testing and evaluation of new model capabilities, applications in new environments, and so on. -METviewer goals and design philosophy -------------------------------------- +METviewer Goals and Design Philosophy +===================================== The METviewer tool reads MET verification statistics output from a database and creates plots using the R statistical package. The tool includes a web @@ -55,7 +55,7 @@ private sector, and operational modeling and prediction centers. METviewer Requirements ----------------------- +====================== METviewer requires installation of Java, Apache / Tomcat, MySQL, and R statistics. @@ -70,7 +70,7 @@ statistics. .. _userspec: METviewer User Specifications ------------------------------ +============================= The top menu allows the user to select a database and type for the plot. @@ -121,7 +121,7 @@ trying to compare the skill of one or more predictions. METviewer Output ----------------- +================ METviewer output all appears in the tabbed boxes at the top right quadrant of the web browser. The plot will show by default, but the user can click any @@ -160,7 +160,7 @@ recommended that the user click through at least the plot and xml windows and save this output. METviewer Plot Formatting Specifications ----------------------------------------- +======================================== METviewer plot formatting options appear in the tabbed boxes at the bottom right quadrant of the web browser. The “Titles & Labels” tab will show by @@ -193,7 +193,7 @@ reset all options to the defaults. METviewer Series Formatting Specifications ------------------------------------------- +========================================== At the bottom of the METviewer window is the series formatting box. It allows the user to control the minute details of each line on a plot, including @@ -205,7 +205,7 @@ place (so subsequent plots are identical in format). METviewer History Menu ----------------------- +====================== Along the left of the METviewer GUI, hidden by default, is the history menu. To open this menu, click the arrow at the top left side right underneath @@ -215,8 +215,8 @@ from this menu to recover plots they may have neglected to save. Clicking on the button showing loads the plot. -Future development plans ------------------------- +Future Development Plans +======================== METviewer is an evolving verification database and display software package. New capabilities are planned in controlled, successive version releases. Bug @@ -226,8 +226,8 @@ releases of METviewer. Additionally, updates to accommodate new features of the MET software are often required. -Code support ------------- +Code Support +============ METviewer support is provided through the `METplus GitHub Discussions Forum `_. diff --git a/docs/Users_Guide/perfdiag.rst b/docs/Users_Guide/perfdiag.rst index 7fae76d7..055547ec 100644 --- a/docs/Users_Guide/perfdiag.rst +++ b/docs/Users_Guide/perfdiag.rst @@ -1,8 +1,9 @@ +******************** Performance Diagrams -==================== +******************** Description ------------ +=========== Performance diagrams are used to show the relationship between categorical statistics, with axes representing detection and success (1 - false alarm) @@ -23,7 +24,8 @@ to the right side. CSI amounts are listed to the right side of the plot, with better values falling closer to the top. Line Type ---------- +========= + Performance requires categorical statistics generated by either Point-Stat or Grid-Stat. These are: @@ -32,7 +34,7 @@ or Grid-Stat. These are: * CTS How-To -------- +====== Selection of options to produce the reliability diagram proceeds approximately counter-clockwise around the METviewer window. @@ -84,7 +86,7 @@ approximately counter-clockwise around the METviewer window. There are many other options for plots, but these are the basics. Example --------- +======= The reliability diagram shown below is for 3 hour accumulated precipitation over the CONUS for 8 ensemble members at the 18 hour diff --git a/docs/Users_Guide/references.rst b/docs/Users_Guide/references.rst index c42baaf8..1eb12194 100644 --- a/docs/Users_Guide/references.rst +++ b/docs/Users_Guide/references.rst @@ -1,5 +1,6 @@ +********** References -========== +********** .. _Hoaglin: diff --git a/docs/Users_Guide/reliabilityplots.rst b/docs/Users_Guide/reliabilityplots.rst index f6ba23d6..cda0ff25 100644 --- a/docs/Users_Guide/reliabilityplots.rst +++ b/docs/Users_Guide/reliabilityplots.rst @@ -1,8 +1,10 @@ + +***************** Reliability Plots -================= +***************** Description ------------ +=========== Reliability (or attributes) diagrams are used to show the conditional bias of probabilistic forecasts. If a probability forecast is reliable, then @@ -26,13 +28,14 @@ Reliability diagrams can be used to make adjustments to the conditional bias of the forecast probabilities. Line Type ---------- +========= + Reliability diagrams require the probability statistics (PCT line type) generated by either Point-Stat or Grid-Stat. How-To -------- +====== Selection of options to produce the reliability diagram proceeds approximately counter-clockwise around the METviewer window. @@ -101,7 +104,7 @@ There are many other options for plots, but these are the basics. Example --------- +======= The reliability diagram shown below is for the probability of 24 hour precipitation with accumulations greater than or equal to 12.7 cm occurring diff --git a/docs/Users_Guide/rocplot.rst b/docs/Users_Guide/rocplot.rst index 3e942f6c..004db4c7 100644 --- a/docs/Users_Guide/rocplot.rst +++ b/docs/Users_Guide/rocplot.rst @@ -1,8 +1,9 @@ +********************************************* Receiver Operating Characteristic (ROC) Plots -============================================== +********************************************* Description ------------ +=========== MET produces a hit rate (Probability of Detection; POD) and Probability of False Detection (POFD) values for each user-specified threshold of a forecast @@ -24,7 +25,8 @@ conditioned on the forecasts. Line Type ---------- +========= + ROC requires probability statistics generated by either Point-Stat or Grid-Stat. These line types are: @@ -33,7 +35,7 @@ Grid-Stat. These line types are: * CTC How-To -------- +====== Selection of options to produce the ROC plot proceeds approximately counter-clockwise around the METviewer window. @@ -100,7 +102,7 @@ There are many other options for plots, but these are the basics. Example --------- +======= The plot shown below is for two different probability of precipitation forecasts. The dashed diagonal line represents the no-skill line. Ideal diff --git a/docs/Users_Guide/scorecard.rst b/docs/Users_Guide/scorecard.rst index 37413d85..708ef122 100644 --- a/docs/Users_Guide/scorecard.rst +++ b/docs/Users_Guide/scorecard.rst @@ -1,16 +1,17 @@ +********** Scorecards -========== +********** Description ------------ +=========== A scorecard is a graphical representation of significance and performance of a comparison, normally a comparison between two models. The scorecard can only be produced using xml code. The xml code specifications are listed in the detail below. There is no GUI interface. -EMC vs. NCAR method of determining p-values for Aggregation statistics ----------------------------------------------------------------------- +EMC vs. NCAR Method of Determining p-values for Aggregation Statistics +====================================================================== In statistics, the p-value represents the probability of a statistic having a value at least as extreme as observed, in this case a difference between two @@ -42,7 +43,8 @@ compared to a resampled distribution rather than a known, theoretical distribution. Line Type ---------- +========= + Scorecards can be run with either summary or aggregation logic. Aggregation logic is the default and generally, the better choice. The following are line types scorecards support: @@ -70,7 +72,7 @@ line types scorecards support: * RPS How-To -------- +====== The scorecard module is used to generate a scorecard PNG and HTML images using data in the METviewer database. The usage statement: @@ -316,7 +318,7 @@ system folders. image. Default value: Verification Scorecard Example -------- +======= The image below shows the output of scorecard_example.xml. diff --git a/docs/Users_Guide/seriesplots.rst b/docs/Users_Guide/seriesplots.rst index cefc74b2..c045928e 100644 --- a/docs/Users_Guide/seriesplots.rst +++ b/docs/Users_Guide/seriesplots.rst @@ -1,8 +1,9 @@ +************ Series Plots -============ +************ Description ------------ +=========== The series plot is the most general of all the METviewer plotting options. A series plot is really just a special case of a scatter plot, where the @@ -15,7 +16,7 @@ lead times or thresholds (on the X-axis). The tremendous flexibility of this plot means that the user must specify many options. How-To -------- +====== Selection of options to produce the plot proceeds approximately counter-clockwise around the METviewer window. The steps to create a @@ -93,7 +94,7 @@ There are many other options for plots, but these are the basics. Example --------- +======= The image below shows an example of the plot and set-up options for a series plot in METviewer. This example uses the database diff --git a/docs/Users_Guide/spreadskillplot.rst b/docs/Users_Guide/spreadskillplot.rst index 2ef6ad66..d4308029 100644 --- a/docs/Users_Guide/spreadskillplot.rst +++ b/docs/Users_Guide/spreadskillplot.rst @@ -1,8 +1,9 @@ +********************** Spread vs. Skill Plots -======================= +********************** Description ------------ +=========== Spread vs. Skill plots are widely used to evaluate the quality of ensemble forecasting systems. One goal of ensemble forecasting is to have the @@ -17,7 +18,7 @@ ratio is 1. How-To -------- +====== Selection of options to produce the spread vs. skill plot proceeds approximately counter-clockwise around the METviewer window. @@ -89,7 +90,7 @@ approximately counter-clockwise around the METviewer window. There are many other options for plots, but these are the basics. Example --------- +======= This example shows the 2-m temperature RMSE and spread for a series of forecast lead times for an ensemble. Plotted are the spread and the skill. diff --git a/docs/Users_Guide/taylordiag.rst b/docs/Users_Guide/taylordiag.rst index 9b1d4e67..b3ff159f 100644 --- a/docs/Users_Guide/taylordiag.rst +++ b/docs/Users_Guide/taylordiag.rst @@ -1,8 +1,9 @@ +*************** Taylor Diagrams -=============== +*************** Description ------------ +=========== Taylor diagrams are used to show a relationship between continuous statistics (:ref:`Taylor, 2001`). This type of plot is used to @@ -32,12 +33,13 @@ So, better forecasts lie closer on the Taylor diagram to observation point, while worse forecasts lie further away. Line Type ---------- +========= + Taylor requires continuous statistics, SL1L2, generated by either Point-Stat or Grid-Stat. How-To -------- +====== Selection of options to produce the Taylor diagram proceeds approximately counter-clockwise around the METviewer window. @@ -90,7 +92,7 @@ There are many other options for plots, but these are the basics. Example -------- +======= The example Taylor diagram below shows the downward longwave radiation flux (DLWRF) correlation and standard deviation for one model and seven diff --git a/docs/Users_Guide/web_service.rst b/docs/Users_Guide/web_service.rst index 1b269694..b78c0e88 100644 --- a/docs/Users_Guide/web_service.rst +++ b/docs/Users_Guide/web_service.rst @@ -1,5 +1,6 @@ +********************* METviewer Web Service -===================== +********************* The METviewer web service is a web-server driven API that parses XML requests and returns results in an XML structure. All calls are @@ -33,13 +34,13 @@ loose ordering: #. Plot request for user selected database and plot information API Request/Response Formats ----------------------------- +============================ The following sections each describe a web service API function in terms of the request/response XML structure. List Databases -~~~~~~~~~~~~~~ +-------------- List the METviewer databases that are available on the system for access through the web app. The returned list reflects the contents of the @@ -64,7 +65,7 @@ Response: List Values -~~~~~~~~~~~ +----------- The list_val request is used to conditionally list the distinct values in the stat_header table of the selected database. The stat_field element @@ -148,7 +149,7 @@ vice versa. List Statistics -~~~~~~~~~~~~~~~ +--------------- The list_stat request builds a list of all statistics available in the METviewer database for the specified fcst_var. It is primarily used to @@ -182,7 +183,7 @@ criteria is not supported in list_stat requests. Generate Plot -~~~~~~~~~~~~~ +------------- Once a user has provided all the information for a complete plot specification, the plot request handles the parsing of the plot @@ -210,8 +211,9 @@ element. NAs produced by integer overflow -Open app with predefine database -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Open App with Predefine Database +-------------------------------- + Use this url to open METviewer main page with predefine database: http://www.dtcenter.org/met/metviewer/metviewer1.jsp?db=name_of_database