Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

calc.c 2+0.1 = 2.99999991 #138

Open
stnKrisna opened this issue Dec 1, 2015 · 6 comments
Open

calc.c 2+0.1 = 2.99999991 #138

stnKrisna opened this issue Dec 1, 2015 · 6 comments
Labels

Comments

@stnKrisna
Copy link
Collaborator

Calculating n+0.1 returns n.99999991. Calculating n+0.2 returns n.19999982 (not that far off). Its just weird

@stnKrisna stnKrisna added the bug label Dec 1, 2015
@plankp
Copy link
Collaborator

plankp commented Dec 1, 2015

Could you also check more than one decimal placed values? Just rechecked my code and think there might be an issue for more values like 0.23 evaluated as 0.5...

@plankp plankp added this to the Calculator App milestone Dec 1, 2015
@stnKrisna
Copy link
Collaborator Author

I'll check this later. PC and laptop (VM) are on a scheduled maintenance (virus scan and malware scan) and its 12:16 AM in my country, plus I have class at 9 later. I have to sleep first.

@stnKrisna
Copy link
Collaborator Author

So, I accidentally typed 0+0.25 and the result it returns is 0.7. And 2+0.25 returns 2.70000008. So, I think it has something to do with floating point rounding (tested with VMware Fusion)

@plankp
Copy link
Collaborator

plankp commented Dec 2, 2015

No
0.23 -> 0.5 is due to the code I wrote...

@plankp
Copy link
Collaborator

plankp commented Dec 2, 2015

A pure program error... (Fixed the 0.23 -> 0.5 issue already)

@plankp
Copy link
Collaborator

plankp commented Dec 6, 2015

This issue ism most likely caused by floating point conversions. We need a double to string method...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants