9
$\begingroup$

I am trying to pull out a certain value from an XML tree that I have acquired from a website:

url = "http://store.nike.com/us/en_us/pd/dry-knit-mens-short-sleeve-running-top/pid-11194817/pgid-11463524";
xmlLarge = Import[url, "XMLObject"];
firstCase = Cases[xmlLarge,
  XMLElement[
   "script", {"id" -> "product-data", "type" -> "template-data"}, {___}], Infinity]

The code above returns a fragment of the XML tree that looks something like this:

{XMLElement[
  "script", {"id" -> "product-data", "type" -> "template-data"}, {"
               {\"productId\":\"11194817\",\"productGroupId\":\"\
11463524\",\"displayName\":\"Nike Dry Knit Men's Short Sleeve Running \
Top\",\"productTitle\":\"Nike Dry Knit\",\"productSubTitle\":\"Men's \
Short Sleeve Running Top\",\"pageTitle\":\"Nike Dry Knit Men's Short \
Sleeve Running Top. \
Nike.com\",\"catalogId\":\"1\",\"productSlug\":\"dry-knit-mens-short-\
sleeve-running-top\",\"url\":\"http://store.nike.com/us/en_us/pd/dry-\
knit-mens-short-sleeve-running-top/pid-11194817/pgid-11463524\",\"\
startDate\":1481875200000,\"builderUrl\":null,\"frameUtilUrl\":null,\"\
hardLaunch\":false,\"reviews\":{\"averageRating\":4.5,\"\
formattedAverageRating\":\"4.5\",\"ratingRange\":5,\"totalReviewCount\
\":4,"

I have copy and pasted only a segment of the XML tree to give a sense of its structure. The code above seems to be returning certain variables and their values ... i.e. "productTitle":"Nike Dry Knit". My goal is to extract the value given the name of the variable, specifically "totalReviewCount", which would return 4. In the Documentation, it says that : means pattern, but the set up also reminds me of Keys, so I am a bit confused. Any help and guidance would be greatly appreciated!

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Looks like json, but ImportString[...,"JSON"] fails for some reason. $\endgroup$
    – swish
    May 9, 2017 at 4:14
  • 1
    $\begingroup$ Actually what you call "XML tree" (the third argument of the extracted XMLElement) is just an unparsed string... $\endgroup$
    – Alexey Popkov
    May 9, 2017 at 5:25
  • $\begingroup$ ImportString[firstCase[[1, 3,1]], "JSON"] does not work, but if one copies it into a JSON validator then it works. And then if you copy it back it also works in Mathematica. Maybe there are hidden characters that get stripped in the copying. $\endgroup$
    – C. E.
    May 9, 2017 at 5:32
  • $\begingroup$ Whatever you do, don't try to parse it with a RegEx... $\endgroup$
    – A C
    May 9, 2017 at 6:48
  • 2
    $\begingroup$ The failure to import JSON appears to be a bug in the parser. It does not like the character that appears at position 3393 on the second line, namely the Unicode character right single quotation mark. We can reproduce the bug by evaluating ImportString["\"\:2019\"", "RawJSON"]. $\endgroup$
    – WReach
    May 9, 2017 at 14:47

2 Answers 2

Reset to default
9
$\begingroup$

So I've figured out how to correctly import json. In my opinion, it's better to explicitly extract such data, field by field. Here you go:

firstCase = 
  FirstCase[xmlLarge, 
   XMLElement["script", {"id" -> "product-data", "type" -> "template-data"}, x : ___] :> 
     ImportString[ExportString[x, "Text"], "RawJSON"], , Infinity];

firstCase["reviews", "reviewStatistics", "totalReviewCount"]
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much for this! It worked like a charm $\endgroup$
    – Vic Jongmin Youn
    May 10, 2017 at 5:24
3
$\begingroup$

As has been said, ImportString[..., "JSON"] does not work. In addition to swish's excellent answer, I discovered dreeve's JSON parser for Mathematica, here. It works:

firstCase[[1, 3, 1]] // parseJSON

{"productId" -> "11194817", "productGroupId" -> "11463524", ...

You can use GeneralUtilities`ToAssociations to turn this into an association:

assoc = GeneralUtilities`ToAssociations[firstCase[[1, 3, 1]] // parseJSON];
assoc["pageTitle"]

"Nike Dry Knit Men's Short Sleeve Running Top. Nike.com"

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.