handling commas and dots input in parseToken of Parser

[naronchen]

#7487

Right now I have any input consists only special characters returning a parseError or a quantity with isValid field being false.

this should be all the special characters.
/^[!@#$%^&*()_+-=[]{};':"\|,.<>/?]+$/;

Let me know if thats expected, or we wanna handle just the dots/comas occurrences

[hl662]

@mathieu-fournier I'll let you chime in on whether we should check only commas and dots, or check all special characters (and if the special characters mentioned are enough or we missing any)

[naronchen]

Just pushed a slightly different fix because Mathieu asked to handle multiple dots and comas mixed into numbers
Now I got these tests cases returning parseError.UnableToConvertParseTokensToQuantity

      "10..",
      "1.2,,3..4...7",
      ",,3..",
      "12..34",
      "1.2.3",
      "..10",
      "1...2",
      "1..",
      "1..,",
      "10,20,30..40",
      "1..e2",
      "1e..2",
      "1e2..",
      "1...2e3",
      "1e2..,3",

the ones with comas is a bit tricky

     "1,,2", // returns 12
     "1.,2", // returns 1.2

the parsing skips comas because the parsing method thinks its a thousands separator so it ignores it, will look into it more.

[rschili]

the ones with comas is a bit tricky

     "1,,2", // returns 12
     "1.,2", // returns 1.2

the parsing skips comas because the parsing method thinks its a thousands separator so it ignores it, will look into it more.

Oh, but if the algorithm detects this as thousands separator, it should not just strip it but instead consider its meaning.
I cannot find any notion that can be considered a standard, were for example "1,," would be used to represent "1,000,000"
"1,,2" probably does not represent a valid number in any standard. Neither does "1,"
I'd vote to fail here...

[naronchen]

just pushed a fix handling multiple comas case. it passes all the tests but I just dont feel very secure with regular expression solutions. Couldnt think of a better fix at the moment. Let me know what you think.

(cpp side behaves similarly with ignoring comas, I can work on that afterwards)

---

This one probably wont work with Indian numbering system? i know they group numbers in 1,00,00,000. I wonder if coma is added as a thousandSeparator if we were to use that system.

---

Just talked with Rob, gonna revert this commit and do loose checking instead

---

talked with Colin (apparently he is closest of being UX responsible for core-quantity :) ). Made the call to ignore the comas now and i just logged this proposal in backlog