Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Trying to round-trip expressions through JSON, I'm getting unexpected errors for held expressions, and would be grateful for advice or clues. Consider, first, something that works well

Export[Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json", {1, 2, 3},"JSON"]

and read it back in

Import[Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json","JSON"]

producing the expected {1, 2, 3}. However, when I try a held expression, such as (and now you can see why I might want to do this):

Export[
  Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json",
  HoldComplete[myList = {1, 2, 3}],
  "JSON"]

and we get

Export::badval: The element Data contains invalid values. >>

I haven't been able to find anything useful on this error message. I suspect it's something to do with HoldComplete and its friends not being real expressions, but rather some kinds of special syntax in the front end or the kernel, but it's a bit surprising since one of the oft-repeated slogans in Mathematica is everything is an expression.

Btw, lest we think that the assignment to myList is the problem, the following fails with the same message:

Export[
  Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json",
  HoldComplete[{1, 2, 3}],
  "JSON"]
share|improve this question
2  
I think JSON only supports lists and rules and not arbitrary expressions. –  FJRA Mar 6 '12 at 16:21
add comment

2 Answers

up vote 6 down vote accepted

The short answer is that, as @FJRA noted in the comment, only certain types are supported. Which types? Enter the long answer.

Why the converter behaves as it does

Long answer: JSON supports only certain types, and their nested combinations, as defined e.g. here. Mathematica converter maps JSON objects to lists of rules, arrays to lists, strings to strings, plus has some special cases for True, False and Null. Once Mathematica JSON converter sees a general expression, it does not know what to do with it.

The problem with the "obvious" solution to convert to string and store as a string is that there will be no automatic way (without imposing some additional conventions) to tell which strings are really strings and which are stringified Mathematica expressions. So, IMO, the converter is doing the right thing.

Digging deeper

You can actually quite easily trace the execution the the functions of interest. If we use my debug function (from here), as

debug@Export[
   Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json", 
   HoldComplete[{1, 2, 3}], "JSON"]

It will quickly tell us to look at the function System`Convert`JSONDump`iexportJSON, which in turn points to System`Convert`JSONDump`toString. Inspecting the DownValues of the latter, you will see the procedure I described above.

Making the JSON import - export more liberal (for illustration purposes only !)

If you really want to make the JSON import - export more liberal, so that, upon seeing an unrecognized general expression, it somehow converts it to string for export, and back to expression during import, here is one way:

ClearAll[withLiberalJsonTostring];
SetAttributes[withLiberalJsonTostring, HoldAll];
withLiberalJsonTostring[code_] :=
   Block[{dv = DownValues[System`Convert`JSONDump`toString], 
         System`Convert`JSONDump`toString},
      DownValues[System`Convert`JSONDump`toString] = Most[dv];
      System`Convert`JSONDump`toString[expr_, _Integer] :=
       StringJoin[
          "StringifiedOpen",
          StringReplace[ToString[FullForm@expr],
            {"[" :> "EscapeOpen", "]" :> "EscapeClose", "," :> "EscapeComma"}],
          "StringifyClose"
       ];
      code];

and the import counterpart:

ClearAll[withLiberalJsonImport];
SetAttributes[withLiberalJsonImport, HoldAll];
withLiberalJsonImport[code_] :=
   With[{result = code},
      result /. 
        s_String :> 
          StringReplace[
             s, 
            {"EscapeOpen" :> "[", "EscapeClose" :> "]", "EscapeComma" :> ","}
          ] /. 
        s_String /; StringMatchQ[s, "StringifiedOpen" ~~ __ ~~ "StringifyClose"] :>
            ToExpression@ StringReplace[s, "StringifiedOpen" | "StringifyClose" :> ""]
   ]; 

Note that the escaping strings are arbitrary, and this will break if these particular strings are also used in different capacities in the JSON expression.

With this, we can do:

withLiberalJsonTostring[
  Export[Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json", 
  {1, 2, HoldComplete[{1, 2, 3}]}, 
  JSON"]]

and

withLiberalJsonImport@
   Import[Environment["USERPROFILE"] <> "\\AppData\\Local\\test.json", "JSON"]

(* 
  ==> {1, 2, HoldComplete[{1, 2, 3}]}
*)

Note that I don't really recommend this method as robust, just posted this code for an illustration, and to aid the understanding of the matter. This is not robust on many grounds, incuding dependence on implementation details, the escaping procedure being arbitrary and not really robust, etc.

A robust solution would be to write an alternative converter (importer / exporter) to JSON, which would do the thing you want, and use that intead. Also, please have a look at the solution by @celtschk, which is a lot cleaner and simpler.

EDIT

As @celtschk pointed out in the comments, escaping is not really necessary if we add extra string quatation marks. The mechanism to distinguish strings from stringified expressions (to be converted back to expressions during import) is still needed however.

share|improve this answer
    
Are JSON parsers really confused by brackets inside strings? –  celtschk Mar 6 '12 at 17:15
    
@celtschk You are actually right, they shouldn't. I will correct my answer. So your method is valid, however it still does not address the question of what should and shouldn't be converted back to expressions from strings. –  Leonid Shifrin Mar 6 '12 at 17:21
    
It now does address that question. However, normally this should be clear from your protocol. –  celtschk Mar 6 '12 at 17:46
add comment

The reason is that the JSON format is quite limited. It doesn't support arbitrary expressions. You can e.g. see that by trying the folllowing:

Export["test.json",someSymbol,"JSON"]

You'll get the same error.

If your goal is just to pass the expressions around (i.e. on the other side is another Mathematica instance to interpret them), the simplest solution is to package the expression in a string before sending:

Export["test.json",ToString[HoldComplete[myList={1,2,3}],InputForm],"JSON"]

and translating that string back into an expression on receiving:

ToExpression@Import["test.json","JSON"]
(*
==> HoldComplete[myList={1,2,3}]
*)

Note that I used InputForm in order to enable passing expressions which are normally represented specially. For example, Graph[{1->2,2->3}] could not be passed through otherwise.

If you want to distinguish between Mathematica expressions and other stuff, you can do the following:

Export["test.json",{"MathematicaExpression"->ToString[HoldComplete[myList={1,2,3}],InputForm]},"JSON"]

This will produce the following JSON file:

{"MathematicaExpression" : "HoldComplete[myList = {1, 2, 3}]"}

You can then read it in with

ToExpression@("MathematicaExpression"/.Import["test.json","JSON"])

You can of course first check that the Import resulted in something matching { "MathematicaExpression -> x_String }. The following will give the Mathematica expression if a Mathematica expression was explicitly passed using that mechanism, and whatever Import resulted in otherwise:

Import["test.json","JSON"]/.{{"MathematicaExpression"->x_String}:>ToExpression[x]}
share|improve this answer
    
Nice solution - +1. I added a link to yours in my answer. –  Leonid Shifrin Mar 6 '12 at 17:51
    
@LeonidShifrin: Thank you. –  celtschk Mar 6 '12 at 17:58
    
Is there a bug in your fourth code block? Shouldn't it say Export["test.json",{"MathematicaExpression"->ToString[HoldComplete[myList={1,2,‌​3}],InputForm]},"JSON"] ? –  Reb.Cabin Mar 6 '12 at 21:52
    
@Reb.Cabin: Yes, thank you. I've now fixed it. I had tested with several different expressions, and when I copy/pasted here, I replaced the latest test expression with the original held assignment. Obviously I replaced too much text. –  celtschk Mar 6 '12 at 22:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.