7
$\begingroup$

Bug introduced in 11.0.1 or earlier and fixed in 12.0.

MMA 11.0.1.0 (Windows 10, 64-bit)

I see that "JSON" import can be buggy (154245), so my question is simple: can the following behaviour be repeated and, if so, is it a bug? (excuses if I've missed some duplicate).

So, starting with a number longer than 19 digits import (different kinds of import, including URLExecute, where I noticed that errors) fails.

1) The following works fine.

str19 = "{\"id\":19,\"number\":1234567890123456789}";
ImportString[str19, #] & /@ {"JSON", "RawJSON"}

{{"id" -> 19, "number" -> 1234567890123456789}, <|"id" -> 19, "number" -> 1234567890123456789|>}

2) Just 0 added to the number, so now it's 20-digit long.

str20 = "{\"id\":20,\"number\":12345678901234567890}";
ImportString[str20, #] & /@ {"JSON", "RawJSON"}

Import::fmterr: Cannot import data as JSON format.

Import::mnumber: The value 12345678901234567890 cannot be coerced into a machine number.

Import::jsonhintposandchar: An error occurred near character '}', at line 1:40

{$Failed, $Failed}

Improve this question
$\endgroup$
7
  • $\begingroup$ So, if I understood it well, based on that SO topic, it could work but it does not have to. Otoh we could expect more from Mathematica in this case. And if it really can't, it could at least handle it more gracefully, e.g. JS: JSON.parse("{\"id\":20,\"number\":12345678901234567890}") returns {id: 20, number: 12345678901234567000} $\endgroup$
    – Kuba ♦
    Jan 16, 2018 at 21:03
  • $\begingroup$ I think this should be reported to WRI. Since this is Mathematica one would expect it to support numbers of arbitrary size, so there is also reason to believe WRI would like to fix this. And as Kuba showed with his link, there is nothing that prevents supporting numbers of arbitrary size in the JSON specification. $\endgroup$
    – C. E.
    Jan 17, 2018 at 8:14
  • 2
    $\begingroup$ I submitted a bug report a few hours ago since this issue is still in 11.3. $\endgroup$
    – Carl Lange
    Jan 24, 2019 at 23:22
  • 2
    $\begingroup$ The response details that the issue is that the numbers no longer pass MachineNumberQ, and it will be passed to the developers. 🤷‍♂️ $\endgroup$
    – Carl Lange
    Jan 25, 2019 at 8:56
  • 1
    $\begingroup$ Using 11.3 I can't reproduce the error from the original question, somehow this issue has been fixed. Still, there is a bug remaining that only impacts negative big numbers: neither negative big integers, nor negative big reals can be imported. This bug will be fixed in the next release. I also confirm that JSON specifications do not forbid arbitrary large numeric values. $\endgroup$
    – Dorian B.
    Jan 29, 2019 at 16:47

1 Answer 1

Reset to default
1
$\begingroup$

This was a bug, fixed in version 12 (some portion of the fix may have been in later incremental updates of 11.3).

The root of the issue was that these numbers did not pass MachineNumberQ, and in some way this broke the JSON parser. I got in touch, and after a little confusion about the JSON spec, which specifies that numbers can be arbitrary precision, it was accepted as a bug. The issue appears to now have been completely fixed in version 12 for JSON, RawJSON and JavascriptExpression.

If you are running an older version, a workaround may be to use ExternalEvaluate to parse the JSON in python or node.js, or JLink or the like to parse the JSON in some other language. For small files, it may be as simple as doing a StringReplace to convert the formatting to an Association (that is, using <| and |> in place of { and } and -> instead of :, and then running ToExpression on the resulting strings (or something like TextCases["Number"]).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.