Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mp_snprint() should preserve float values precision #14

Open
parihaaraka opened this issue Apr 16, 2021 · 2 comments
Open

mp_snprint() should preserve float values precision #14

parihaaraka opened this issue Apr 16, 2021 · 2 comments
Labels
feature A new functionality

Comments

@parihaaraka
Copy link

huge precision loss of fiber.time() values :(

@Totktonada
Copy link
Member

We can replace %g / %lg with something like %.17g / %.17lg.

However it'll not guarantee that the precision will not loss. There are two points here:

  • %.17lg (unlike %.17lf) may choose the scientific notation for a large value. So, 111<...>111 will become 1.111<...>111e+X and the precision (17) will be applied to the 'fractional part' of the normalized value, not the original one.
  • There are doubles, which becomes near to a halfway between two exact doubles, after converting to the decimal representation. Marshalling over decimal may give us back not exactly same value. See https://www.exploringbinary.com/17-digits-gets-you-there-once-youve-found-your-way/

Converting to hex would be good option for cases, when exact value should be stored.

@parihaaraka
Copy link
Author

Most users understand that binary floating point value may not preserve its original precision via decimal representation. "huge" means HUGE precision loss - out of acceptable dec<->bin transformations impact. Actually mp_snprint() must not be used for floating point numbers at all.

@kyukhin kyukhin added feature A new functionality teamC labels Sep 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new functionality
Projects
None yet
Development

No branches or pull requests

4 participants