My Problem

Some people gave me a JSON file, which is exported from a camera. He asked me to parse the JSON file, and extract the value to each key. However, when I import the JSON file with:

Import[pathToTestJSON, "JSON"]

Mathematica warns: Import::jsonexpendofinput: Unexpected character found while looking for the end of input.

Potential Solution

I searched the mathematica.Stackexchange.com, and found this post: Openning JSON file, (NDJSON, new line delimited) — big file where @Jason B. explained:

Import[myfile.ndjson,"JSON"] doesn't work because the file as a whole is not valid JSON. But each line is valid JSON

But his solution is not for my situation, because my JSON record is not line-by-line, instead, its layout is a block. The following is a sample:

    "ChannelInfo" : null,
    "DevNo" : "DS-2CD2T86FWDV2-I8S20191216AACH",
    "MecNo" : "100",
    "MsgType" : 1000,
    "Timestamp" : 1606910698042

My Puzzle

(1) To take advantage of Openning JSON file, (NDJSON, new line delimited) — big file, how can I extract the JSON record block, instead of a line, and then parse the JSON with Import[jsonRecordInBlock, "JSON"]?

(2) If we do not copy the solution from Openning JSON file, (NDJSON, new line delimited) — big file, what is the solution?


JSON file for test

  • 3
    $\begingroup$ Your JSON file is invalid according to the JSON specification. You need to Import it as text and surround the string with braces { ... } then use ImportString[text, "JSON"] . You'd also need to add commas after each object delimiter. $\endgroup$
    – flinty
    Dec 8 '20 at 13:14
(* Import as text first *)
text = Import["test.json", "Text"];

(* Split out the lines *)
lines = StringSplit[text, "\n"];

(* Replace lines containing ONLY } with }, 
 * then stitch the lines back together with newlines and remove the final comma.
 * Finally, surround all objects with a JSON list [...], not braces. *)
newjson = "[\n" <> StringDrop[StringJoin[lines /. "}" -> "},", "\n"], -2] <> "]\n"

(* Import the fixed string and decode as JSON *)
result = ImportString[newjson, "JSON"]
  • $\begingroup$ By the way, this assumes all your top-level objects in the json are delimited by START OF LINE then } then NEWLINE. If you have any stray whitespace after a } then you may need to trim it. It will not work if it looks like START OF LINE whitespaces+ } as it becomes indistinguishable from the other nested delimiters. $\endgroup$
    – flinty
    Dec 8 '20 at 14:07
  • $\begingroup$ Your comment above should be part of the answer. $\endgroup$ Dec 8 '20 at 14:52
  • $\begingroup$ Thanks for your solution. Anyway, I revised part of your code for the output readability as following: mynewjson = "[\n" <> StringDrop[StringRiffle[lines /. "}" -> "},", "\n"], -1] <> "]\n"; $\endgroup$
    – PureLine
    Dec 9 '20 at 6:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.