Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.
23

Here's a set of functions that allows to do this. The code uses many ideas found on this site and on other places on the web. It is a bit factorized already so it should be easily reusable. More on YQL and available tables here: https://developer.yahoo.com/yql https://github.com/yql Query test Edit: this API is great also and simple http://www.quandl.com/...


19

Update: Athanassios correctly pointed out that my attempt to set defaults doesn't work. The problem appears to be that the absence of the element from the first list (ad) causes the defaults list to never be checked. A condensed exhibit: ad = {"accept_rate" -> 80, "badge_counts" -> {"bronze" -> 39, "silver" -> 9}}; defaults = {"age" -> ...


17

I separated this project into two parts. The first is to compute the coordinates of the Geohash location. (*Grab the user's geographical location. The location is based on IP address, so it may not be completely accurate. It's usually good enough to get your graticute. You can replace home with with known coordinates in the form {hx, hy} if you like.*) home ...


17

Tracing the evaluation (Mathematica 11.1.1) shows that the string is passed to Developer`ReadRawJSONStream which actually produces the messages: Developer`ReadRawJSONStream[StringToStream@string, "IssueMessagesAs" -> Import] General::jsonoutofrangeunicode: Out of range unicode code point encountered. Import::jsoninvalidtoken: Invalid token ...


16

I'm Riccardo, current developer of URLRead in WL and I have some experience working with encoding in WL. I would like to inform you that this is not a bug. In modern versions of mathematica we have ByteArray, and this is a representation of bytes. But for decades strings have been both bytes and "unicode" at the same time. The problem here is that all ...


14

I ran into the same issue. Dataset and nested Associations have a structure ideally suited for JSON export, but the JSON exporter only supports lists of rules. Any other type of expression triggers the error you see. Export The workaround is to recursively convert all nested associations to lists of rules. normalAsc[expr_] := expr //. a_Association :>...


13

The bug is in the System`Convert`JSONDump`toString[] function which is defined like this: toString[num_?NumberQ, t_Integer] := If[Head[num] === Real && IntegerPart[num] == num, ToString[CForm[N[num]]] ~~ "0", ToString[CForm[N[num]]]] If my guess is correct, the purpose of the ~~ "0" part is to change 1. to 1.0. I am not very familiar with ...


13

tl;dr I think it's a memory leak (bug) and you should report it to Wolfram Support (please do!) According to my reading, you were saying that after importing JSON files many times, the kernel memory usage reported by the operating system (or some task manager program) was growing to unreasonable levels. However, the memory usage reported by the kernel ...


12

I don't believe that this is a bug. Here's why: What exactly happens? Export and ExportString will always encode "JSON" as UTF-8. This may be intentional, as UTF-8 is the default encoding for JSON files. From the documentation: Strings in the Wolfram Language are represented in JSON as UTF-8 strings, escaped as required by the JSON standard. Strings ...


11

Element as referred to in the documentation are not a JSON elements, but the kind of information you can get from the specified file format. For JSON the elements available is just Data as given by Import["out.json", "Elements"]. For a typical JPEG photo the available elements are: {"Aperture", "BitDepth", "CameraTopOrientation", "ColorMap", \ "...


11

I have failed to find a good duplicate so: assoc // Query[All, {"a" -> f}] or MapAt[f, assoc, {All, "a"}]


10

It is possible to get rid of all but one ReplaceAll. First, you need to remember that ReplaceAll tests each Rule in order, so to deal with the missing "badge_counts" condition, you can simply add that to the list, as follows {"display_name", "creation_date", "reputation", "reputation_change_week", "is_employee", "last_access_date", "user_type", "...


10

You can define your own object and tell Mathematica to interpret it as JSON object. Dummy export to load relevant contexts: In[1]:= ExportString["", "JSON"]; Tell Mathematica to interpret JSONObject symbol as possible head of JSON objects: In[2]:= ClearAll[JSONObject] System`Convert`JSONDump`$JSONObjectHead = JSONObject; Now you can use ...


10

See Szabolcs answer for the explanation. The number form there needs to be corrected because JSON needs a digit between . and e: Therefore, here the solution: System`Convert`JSONDump`toString[num_?NumberQ, t_Integer] := StringReplace[ToString[CForm[N[num]]], RegularExpression["\\.(($)|(e))"] -> ".0$1"]; For the following input: ExportString[{"a" ...


10

The result of importing a file in JSON format is a list of rules. For information about ways to use rules, see the documentation for Applying Transformation Rules. Here is an example: data = ImportString["{\"x\":1, \"y\":{\"a\":2, \"b\":[3, 4]}}", "JSON"] (* {"x" -> 1, "y" -> {"a" -> 2, "b" -> {3, 4}}} *) Retrieving a top-level property is ...


9

So I've figured out how to correctly import json. In my opinion, it's better to explicitly extract such data, field by field. Here you go: firstCase = FirstCase[xmlLarge, XMLElement["script", {"id" -> "product-data", "type" -> "template-data"}, x : ___] :> ImportString[ExportString[x, "Text"], "RawJSON"], , Infinity]; firstCase["...


8

Recent versions of Mathematica no longer use Java for parsing JSON, and work very fast. There are at least three ways to read JSON. Import[..., "JSON"] is fast an reliable. It returns nested Lists and Rules. Import[..., "RawJSON"] is faster than "JSON" and returns nested Lists and Associations. These are much easier to work with than Rules. The Developer` ...


8

So if you have: file = Import[path] then after the file is not needed: Clear[file] e.g.: In[77]:= MemoryInUse[] file = Import[StringJoin[NotebookDirectory[], "IMG_3025.jpg"]]; MemoryInUse[] Clear[file]; MemoryInUse[] Out[77]= 79593488 Out[79]= 115619864 Out[81]= 79591456


8

We can recreate this problem more easily with the following code: Export["test.json", "{\"attr1\": \"Sample with `val1`\"}"] What happens is that Export looks at the file extension .json and tries to turn the second argument into valid JSON, treating it as if it were a symbolic representation of JSON in Mathematica. Compare with this: Export["~/Desktop/...


7

The reason is that the JSON format is quite limited. It doesn't support arbitrary expressions. You can e.g. see that by trying the folllowing: Export["test.json",someSymbol,"JSON"] You'll get the same error. If your goal is just to pass the expressions around (i.e. on the other side is another Mathematica instance to interpret them), the simplest solution ...


7

With Mathematica 10 or later, I recommend converting this data structure to either an Association or a Dataset: asc = Replace[out, r : {__Rule} :> Association[r], {0, Infinity}]; ds = Dataset[asc] Then you can index it with the keys, e.g. asc[["results", 1, "geometry"]] (* <|"location" -> <|"lat" -> 48.35593, "lng" -> 10.89459|>|> ...


7

Here's a solution using your example file. Import as RawJSON because this gives us much easier to manipulate associations. json = Import[ "https://raw.githubusercontent.com/d3/d3-plugins/master/graph/data/miserables.json", "RawJSON"] After inspecting the format manually, it's easy to extract vertices: vertices = json[["nodes", All, "name"]] Links are ...


7

assoc = <|"key1" -> <|"a" -> 1, "b" -> 2|>, "key2" -> <|"a" -> 3, "b" -> 4|>|>; assoc[[All, "a"]] = Map[f, Values[assoc[[All, "a"]]]]; assoc <|"key1" -> <|"a" -> f[1], "b" -> 2|>, "key2" -> <|"a" -> f[3], "b" -> 4|>|>


6

The introduction of Assocation brings a powerful new way to handle this problem. Version 10.2 provides for import of JSON as nested associations without use of ToAssociations by using the format "RawJSON".   (* archive data *) ad = {"accept_rate" -> 75, "account_id" -> 395497, "age" -> 41, "badge_counts" -> {"bronze" -> 35, "gold" ...


6

The short answer is that, as @FJRA noted in the comment, only certain types are supported. Which types? Enter the long answer. Why the converter behaves as it does Long answer: JSON supports only certain types, and their nested combinations, as defined e.g. here. Mathematica converter maps JSON objects to lists of rules, arrays to lists, strings to strings,...


6

This seems to be a Java heap space error. I encounter this quite often when importing large Excel sheets, though I get a different error message: Import::nojmem. The help page for this error contains instructions that may be useful for you as well. Basically, you increase heap space using the following commands: Needs["JLink`"] ReinstallJava[JVMArguments -&...


6

Actually this problem is still in Mma 10.2 here is a solution ExportString["Conexión", "JSON"] (* "ConexiÃ\.b3n" *) ExportString["Conexión", "JSON", CharacterEncoding -> "UTF8"] (* "ConexiÃ\.b3n" *) FromCharacterCode[ ToCharacterCode[ExportString["Conexión", "JSON"]], "UTF8"] (* "Conexión" *) ExportString["Conexión", "RawJSON"] (* ...


6

JSON stands for JavaScript Object Notation. In JavaScript, and hence in JSON, the ordering of dictionary keys is not determined, and not meaningful. Thus Mathematica's behaviour is correct. From the JavaScript language specification, An Object is an unordered collection of properties. Each property consists of a name, a value and a set of attributes. ...


6

The example JSON string: json = ExportString[<| "Names" -> <| "Sister" -> "Nina", "Brothers" -> {<|"Older" -> "John", "Younger" -> "Jake"|>}, "somethingElse" -> "answer" |>, "DOB" -> { <|"Nina" -> 2001, "location" -> "Miami"|>, <|"John" -> 2017, "location" ->...


6

Use the RawJSON Import format. In[22]:= json = ImportString["{ \"status\": \"success\", \"data\": { \"resultType\": \"matrix\", \"result\": [ { \"metric\": { \"name\": \"scrapes\" }, \"values\": [ [ 1556901190.699, \"91158\" ], ...


Only top voted, non community-wiki answers of a minimum length are eligible