Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MODE CSI calculations result in spurious results #360

Closed
2 of 23 tasks
bikegeek opened this issue Feb 13, 2024 · 2 comments · Fixed by #396
Closed
2 of 23 tasks

MODE CSI calculations result in spurious results #360

bikegeek opened this issue Feb 13, 2024 · 2 comments · Fixed by #396
Assignees
@bikegeek @CPKalb
Labels
METcalcpy: MODE Statistics requestor: METplus Team METplus Development Team required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: bug Fix something that is not working
Milestone
METcalcpy-3.0.0

Comments

@bikegeek
Copy link
Collaborator

bikegeek commented Feb 13, 2024 •
edited
Loading

Describe the Problem

There is an issue with the way the MODE CSI statistics are being calculated (inconsistent with what is documented, and results are not consistent). This was discovered and reported in the METplus Github Discussion:

This affects the R script calculations in METviewer

Expected Behavior

Environment

Describe your runtime environment:
1. Machine: (e.g. HPC name, Linux Workstation, Mac Laptop)
2. OS: (e.g. RedHat Linux, MacOS)
3. Software version number(s)

To Reproduce

Describe the steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
Post relevant sample data following these instructions:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-help-desk#ftp

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Add any new Python packages to the METplus Components Python Requirements table.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issues
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issues
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@bikegeek bikegeek added type: bug Fix something that is not working alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle requestor: METplus Team METplus Development Team required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project METcalcpy: MODE Statistics labels Feb 13, 2024
@bikegeek bikegeek added this to the METcalcpy-3.0.0 milestone Feb 13, 2024
@bikegeek bikegeek mentioned this issue Feb 14, 2024
Closed
22 tasks
@bikegeek
Copy link
Collaborator Author

bikegeek commented Jul 16, 2024 •
edited
Loading

  • the corresponding calcOBJACSI code (in METviewer/R_work/include/agg_stat_bootstrap.R) is found in the METcalcpy/util/mode_2d_arearat_statistics.py module
  • the corresponding calcOBJCSI code (n METviewer/R_work/include/agg_stat_bootstrap.R) is found in the METcalcpy/util/mode_2d_ratio_statistics.py module

@bikegeek bikegeek moved this from 📋 Backlog to 🏗 In progress in METplus-Analysis-6.0.0 Development Jul 16, 2024
bikegeek added a commit that referenced this issue Jul 16, 2024
@bikegeek
Issue #360 implement equation change to what was suggested in METplus…
da82271 
… Discussion #2496 for calculating the OBJCSI value corresponding to the calcOBJCSI function that is also implemented in R in the METviewer code.
bikegeek added a commit that referenced this issue Jul 16, 2024
@bikegeek
Issue #360 reformatting
3d1b4d4
bikegeek added a commit that referenced this issue Jul 16, 2024
@bikegeek
Issue #360 modify the equation for calculate_objacsi to suggested cha…
f04c770 
…nges in METplus Discussion #2496.
@bikegeek
Copy link
Collaborator Author

bikegeek commented Jul 23, 2024 •
edited
Loading

I removed the Y1 and Y2 Bounds values in METviewer

OBJCSI plot with updated equation:
plot_icperts_fcst_comp_both_csi-5

OBJACSI plot with updated equation (no change from plot using the original equation):
plot_icperts_fcst_comp_both_csi_a

@bikegeek bikegeek mentioned this issue Jul 24, 2024
Open
23 tasks
@bikegeek bikegeek linked a pull request Jul 26, 2024 that will close this issue
Feature 360 mode csi #396
Merged
15 tasks
bikegeek added a commit that referenced this issue Jul 30, 2024
@bikegeek
Feature 360 mode csi (#396)
2f1aeec 
* Issue #360 implement equation change to what was suggested in METplus Discussion #2496 for calculating the OBJCSI value corresponding to the calcOBJCSI function that is also implemented in R in the METviewer code.

* Issue #360 reformatting

* Issue #360 modify the equation for calculate_objacsi to suggested changes in METplus Discussion #2496.
@github-project-automation github-project-automation bot moved this from 🏗 In progress to ✅ Done in METplus-Analysis-6.0.0 Development Aug 13, 2024
@jprestop jprestop removed alert: NEED ACCOUNT KEY Need to assign an account key to this issue alert: NEED MORE DEFINITION Not yet actionable, additional definition required alert: NEED CYCLE ASSIGNMENT Need to assign to a release development cycle labels Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bikegeek bikegeek

@CPKalb CPKalb

Labels
METcalcpy: MODE Statistics requestor: METplus Team METplus Development Team required: FOR DEVELOPMENT RELEASE Required to be completed in the development release for the assigned project required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: bug Fix something that is not working
Projects
No open projects
METplus-Analysis-6.0.0 Development
Status: 🏁 Done
Milestone
METcalcpy-3.0.0
Development

Successfully merging a pull request may close this issue.

Feature 360 mode csi dtcenter/METcalcpy
4 participants
@bikegeek @jprestop @TatianaBurek @CPKalb