Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have been using Mathematica to analyse some data from the StackExchange API. It is conveniently available in JSON form, which Mathematica interprets as replacement rules. However, some of the rules are nested as the right-hand side of an outer rule, as you can see in the case of badge_counts here:

Import["http://api.stackexchange.com/2.0/users?page=1&order=desc&sort=reputation&site=mathematica",
   "JSON"][[2, 2, 10]]

(* Out[1]= 
{"accept_rate" -> 75, "account_id" -> 395497, "age" -> 41, 
 "badge_counts" -> {"bronze" -> 35, "gold" -> 0, "silver" -> 11}, 
 "creation_date" -> 1326833982, "display_name" -> "Verbeia", 
 "is_employee" -> False, "last_access_date" -> 1331949804, 
 "last_modified_date" -> 1330990001, 
 "link" -> "http://mathematica.stackexchange.com/users/8/verbeia", 
 "location" -> "Sydney, Australia", 
 "profile_image" -> "http://www.gravatar.com/avatar/3df2379fc0221bb0281c0d608542bd84?d=\identicon&r=PG", 
 "reputation" -> 3571, "reputation_change_day" -> 0, 
 "reputation_change_month" -> 857, 
 "reputation_change_quarter" -> 3475, "reputation_change_week" -> 605,
 "reputation_change_year" -> 3475, "user_id" -> 8, 
 "user_type" -> "registered", 
 "website_url" -> "http://www.verbeia.com/mathematica"}
*)

Users without any badges don't have this nested rule. To handle both cases I am doing something like this to get the badge counts in a flattened list, so each row of the final list corresponds to a user of the site:

getUserData[page_Integer] := Module[{ad},
  ad = Import[
     "http://api.stackexchange.com/2.0/users?page=" <> 
      IntegerString[page] <> 
      "&order=desc&sort=reputation&site=mathematica", "JSON"][[2,2]];
  If[Length[ad] > 0,
   Join[Most[#], {"bronze", "silver", "gold"} /. 
       Last[#]] & /@ (({"display_name", "creation_date", "reputation",
          "reputation_change_week", "is_employee", "last_access_date",
          "user_type", "badge_counts"} /. ad) /. 
      "badge_counts" -> {"bronze" -> 0, "silver" -> 0, 
        "gold" -> 0}), {}]
  ]

Which I can then build up into a list of all users and their stats like this:

Monitor[
 pageM = 0;
 resultM = 
  FixedPoint[Join[#, getUserData[++pageM]] &, {}];,
  StringForm["Loading page `` ...", pageM]
 ]

The final output is list of vectors that look like this:

resultM[[10]]
(*Out[2]= {"Verbeia", 1326833982, 3571, 605, False, 1331949804, "registered", 35, 11, 0} *)

My question is: is there a more concise or elegant way to write the getUserData function, without need a Join and three separate ReplaceAll (/.) constructions?

share|improve this question
add comment

2 Answers 2

up vote 9 down vote accepted

It is possible to get rid of all but one ReplaceAll. First, you need to remember that ReplaceAll tests each Rule in order, so to deal with the missing "badge_counts" condition, you can simply add that to the list, as follows

{"display_name", "creation_date", "reputation", "reputation_change_week",
 "is_employee", "last_access_date", "user_type", 
 "badge_counts"} /. Flatten[{ record, 
   "badge_counts" -> {"bronze" -> 0, "silver" -> 0, "gold" -> 0} }]

where record is one user record. Since, "badge_counts" -> {"bronze" -> 0, "silver" -> 0, "gold" -> 0} is listed after the one in the user info, it will only be used if there is no "badge_counts" in the user info. To eliminate the "badge rules", I would remove them using ReplaceRepeated (//.) and one more replacement rule and a final Flatten:

{"display_name", "creation_date", "reputation", "reputation_change_week",
 "is_employee", "last_access_date", "user_type", 
 "badge_counts"} //. Flatten[{ record, 
   "badge_counts" -> {"bronze" -> 0, "silver" -> 0, "gold" -> 0},
   Rule[_, a_]:> a}] // Flatten

which returns for your info

{"Verbeia", 1326833982, 3571, 605, False, 1331949804, "registered", 35, 0, 11}

Edit: Originally, I used a slightly different approach instead of Rule[_, a_]:> a. I am reproducing it here, for the curious:

{"display_name", "creation_date", "reputation", "reputation_change_week",
 "is_employee", "last_access_date", "user_type", 
 "badge_counts"} /. Flatten[{ record, 
   "badge_counts" -> {"bronze" -> 0, "silver" -> 0, "gold" -> 0}}] // 
Block[{Rule=#2&}, Flatten[#]]&

which relies on the property of Block to temporarily override the behavior of Rule.

Edit 2: Here's one more alternative which should reduce it to a single pass, unlike ReplaceRepeated. This method, also, eliminates the need for the final Flatten

Clear[killRules]
killRules[a : {_Rule ..}] := Sequence @@ a[[All, 2]]

{"display_name", "creation_date", "reputation", "reputation_change_week",
 "is_employee", "last_access_date", "user_type", 
 killRules["badge_counts"]} /. Flatten[{ record, 
   "badge_counts" -> {"bronze" -> 0, "silver" -> 0, "gold" -> 0}}]
share|improve this answer
add comment

It seems to me that this is the correct way to extract values from a nested list of rules:

Data

ad = {"accept_rate" -> 75, "account_id" -> 395497, "age" -> 41, 
   "badge_counts" -> {"bronze" -> 35, "gold" -> 0, "silver" -> 11}, 
   "creation_date" -> 1326833982, "display_name" -> "Verbeia", 
   "is_employee" -> False, "last_access_date" -> 1331949804, 
   "last_modified_date" -> 1330990001, 
   "link" -> "http://mathematica.stackexchange.com/users/8/verbeia", 
   "location" -> "Sydney, Australia", 
   "profile_image" -> 
    "http://www.gravatar.com/avatar/3df2379fc0221bb0281c0d608542bd84?\
d=\ identicon&r=PG", "reputation" -> 3571, 
   "reputation_change_day" -> 0, "reputation_change_month" -> 857, 
   "reputation_change_quarter" -> 3475, 
   "reputation_change_week" -> 605, "reputation_change_year" -> 3475, 
   "user_id" -> 8, "user_type" -> "registered", 
   "website_url" -> "http://www.verbeia.com/mathematica"};

Defaults

include default values for any rules that might be missing

defaults = {"badge_counts" ->
      {"bronze" -> #, "gold" -> #, "silver" -> #}& @ Missing["NotAvailable"]};

Extraction

OptionValue[{ad, defaults}, {
   "display_name",
   "creation_date",
   "reputation",
   "reputation_change_week",
   "is_employee", "last_access_date",
   "user_type",
   "badge_counts" -> "bronze",
   "badge_counts" -> "silver",
   "badge_counts" -> "gold"
}]

{"Verbeia", 1326833982, 3571, 605, False, 1331949804, "registered", 35, 11, 0}

share|improve this answer
    
+1, I hadn't thought of OptionValue. –  rcollyer Mar 17 '12 at 13:13
    
@rcollyer is there any way to set a default value for any missing option? That would make this more elegant. Nevertheless I think this is the best way to deal with nested rules; getting a value for e.g. a -> b -> c -> d is pretty hairy otherwise. –  Mr.Wizard Mar 17 '12 at 13:19
    
The only way I'm a aware of is via SetOptions which isn't exactly what you want. –  rcollyer Mar 17 '12 at 17:02
    
@rcollyer how would you do it via SetOptions? –  Mr.Wizard Mar 17 '12 at 17:07
    
Exactly, the problem. SetOptions requires a symbol to be associated with the options. In this case, it isn't clear what symbol you'd use. –  rcollyer Mar 17 '12 at 18:05
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.