11
$\begingroup$

I often find myself working with lists of lists of data, gathering data, performing computations, and producing a variety of different sorts of output. Intuitively, it seems like this should be done with nested Map commands, but in practice that quickly becomes an unreadable mess, if I can get it to work at all. I typically end up switching the outer level(s) to a Table, which is more readable and avoids any potential problem with ambiguous Slots.

For clarity, consider the following trivial example. The following dataset describes a couple of grades at a hypothetical secondary school.

sampleStructure = {<|"grade" -> 11, 
    "students" -> {<|"name" -> "bill", "age" -> 15|>, <|
       "name" -> "susan", "age" -> 16|>}|>, <|"grade" -> 12, 
    "students" -> {<|"name" -> "manuel", "age" -> 16|>, <|
       "name" -> "morris", "age" -> 17|>, <|"name" -> "jackie", 
       "age" -> 16|>}|>};

Now we want to pass over the entire school, assigning a grade to every student. The process is designed to be perfectly objective and remove any chance of favoritism by the teacher -- if the student's age is an even number, he or she gets an A-, otherwise a B+.

I feel like there should be a very clean way to do this but in practice my code comes down to something like this:

Table[<|"grade" -> sampleStructure[[i, "grade"]], 
  "students" -> 
   Map[Append[#, <|"score" -> If[EvenQ[#[["age"]]], "A-", "B+"]|>] &, 
    sampleStructure[[i, "students"]]]|>, {i, 1, 
  Length[sampleStructure]}]

With more complex data structures, this gets ugly, and even with simple data it feels disproportionately complex. Is there a cleaner way to do this? Some kind of nested Map?

$\endgroup$
  • $\begingroup$ Have you seen MapAt? $\endgroup$ – b3m2a1 Jul 18 '17 at 20:04
7
$\begingroup$

Map at Level 3:

Map[Append[#, If[EvenQ[Last@#], "score" -> "A-", "score" -> "B+"]] &, 
  sampleStructure, {3}] // Dataset

enter image description here

$\endgroup$
9
$\begingroup$

You may use Query.

Query[
  All, 
  All, 
  All, 
  <|#, "score" -> If[EvenQ@#["age"], "A-", "B+"]|> &
 ]@sampleStructure
{<|"grade" -> 11, 
  "students" -> {<|"name" -> "bill", "age" -> 15, "score" -> "B+"|>, 
                 <|"name" -> "susan", "age" -> 16, "score" -> "A-"|>}|>, 
<|"grade" -> 12, 
  "students" -> {<|"name" -> "manuel", "age" -> 16, "score" -> "A-"|>, 
                 <|"name" -> "morris", "age" -> 17, "score" -> "B+"|>, 
                 <|"name" -> "jackie", "age" -> 16, "score" -> "A-"|>}|>}

Hope this helps.

$\endgroup$
  • 1
    $\begingroup$ Very very nice, +1 $\endgroup$ – eldo Jul 18 '17 at 22:27
  • 1
    $\begingroup$ Is it possible that I've never even seen this command before? $\endgroup$ – Michael Stern Jul 18 '17 at 22:37
  • 1
    $\begingroup$ @MichaelStern IMHO Query is the best method for manipulating Associations. $\endgroup$ – Edmund Jul 18 '17 at 22:43
  • $\begingroup$ @edmund, Using this approach is it possible to refer to higher levels in the structure. In the example above, to assign grades on the basis of each student's age and grade, rather than just his or her age? $\endgroup$ – Michael Stern Jul 19 '17 at 18:24
  • $\begingroup$ @MichaelStern Yes, this can be done you if you pickup the values on the way down. See the comments in this question (131337). $\endgroup$ – Edmund Jul 20 '17 at 12:05
5
$\begingroup$

Because it is much easier to deal with non-nested associations, I strongly recommend you consider reorganizing the data set so it is not nested. Doing so will bring two benefits: conceptually simplicity (less debugging) and better performance.

In the case of your example, write the data set as

students = {
   <|"name" -> "bill", "age" -> 15, "grade" -> 11|>,
   <|"name" -> "susan", "age" -> 16, "grade" -> 11|>,
   <|"name" -> "manuel", "age" -> 16, "grade" -> 12|>,
   <|"name" -> "morris", "age" -> 17, "grade" -> 12|>,
   <|"name" -> "jackie", "age" -> 16, "grade" -> 12|>
   };

Then your task can be implemented by writing a simple function.

assignScore[student_] :=
  Module[{pupil = student}, 
    AppendTo[pupil, "score" -> If[EvenQ[pupil["age"]], "A-", "B+"]]]

and carried out by evaluating

students = assignScore /@ students;
students // Dataset

students

Further, with this organization, all kinds of queries become simple because they are intuitive.

Group the students by age.

GatherBy[students, #age &] // Dataset

by_age

Query for those students who earned an A-.

Query[Select[#score == "A-" &]]@students // Dataset

or

Dataset[students][Select[#score == "A-" &]]

Either of the above give

high_score

In conclusion: the best way to handle nested maps of associations is to avoid them.

$\endgroup$
  • $\begingroup$ @ m_goldberg, I like your approach and especially your conclusion. In this example it works fine, since there are no missing items. What would you recommend when a few items are missing? And what when the block of data is only sparsely filled? $\endgroup$ – Romke Bontekoe Aug 7 '17 at 10:00
  • 1
    $\begingroup$ Are you familiar with Missing? A frequently used approach is to use Missing[...] as the value for keys for which values do not currently exist. $\endgroup$ – m_goldberg Aug 7 '17 at 17:21
  • $\begingroup$ @ m_goldberg, Your recommendation is to supplement the data with Missing[.]-s, such that it becomes a filled rectangular block. Thank you. $\endgroup$ – Romke Bontekoe Aug 7 '17 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.