METplus 6 Rounding Differences #2831
-
|
Hello METplus folks, I am working on upgrading GSL verification workflows from METplus 5/MET 11 to METplus 6/MET 12. So far, 99.9% of the data is identical between the METplus 5 and METplus 6 runs. However, I have found some cases of very small differences in values that look like they are due to rounding. The workflows are run on the same machine (Hera) and on the same data. Unfortunately, I did not record whether or not I ran with METplus 6 beta-6 or METplus 6 rc1, as I used both in testing. I started with beta-6 and moved to rc-1 when it became available. These differences are so small, that we are not necessarily concerned about their impact on our verification statistics, but we thought you would want to know. Here are the differences for context. All of these numbers are from the SL1L2 line type and were found among different forecast models. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
Update, I found another rounding difference and now I know exactly which version: METplus 6.0.0-rc1/MET 12.0.0-rc1. |
Beta Was this translation helpful? Give feedback.
-
|
@lindsayblank, I certainly agree that these very small numeric differences out in the 5th decimal place are not a reason for concern. Many years ago we ran into issues with checking for equality of floating point numbers and switched to using these is_eq(...) functions to do so. They check for the differences of the numbers being within some sort of tolerance. By default, we use a "tight tolerance" of 10E-10 but in other contexts, we use a "loose tolerance" of 10E-5. I'm not that surprised or concerned about differences in the 5th decimal place of large partial sum values. Since they are sums of squares, very small differences in the raw fcst/obs input values get amplified when they're squared. I do understand that the outputs are both generated on Hera, but there might still be differences in the hardware and OS-specifics of the individual nodes. We do run nightly regression tests for MET and are currently still using R to check for differences (METplus uses Python, and we'll eventually switch to that). Here's the line of code that check for numeric diffs: It calls R's In summary, I'm glad you're paying attention to the output. I'm not concerned about this scale of diffs in partial sums, since they are at the scale of not significant floating point diffs. And if you'd like, I could point you to the Python-based differencing logic METplus uses to flag real differences. |
Beta Was this translation helpful? Give feedback.
-
|
Great! I appreciate the explanation. Since no one is concerned, I'll mark this complete. |
Beta Was this translation helpful? Give feedback.
@lindsayblank, I certainly agree that these very small numeric differences out in the 5th decimal place are not a reason for concern.
Many years ago we ran into issues with checking for equality of floating point numbers and switched to using these is_eq(...) functions to do so. They check for the differences of the numbers being within some sort of tolerance. By default, we use a "tight tolerance" of 10E-10 but in other contexts, we use a "loose tolerance" of 10E-5.
I'm not that surprised or concerned about differences in the 5th decimal place of large partial sum values. Since they are sums of squares, very small differences in the raw fcst/obs input values get amplified when they're sq…