Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

The summary box for Dataset objects shows the number of levels and elements at the bottom.

How can we get these values programmatically?

For example, "4 levels, 350 elements" in the Planets dataset:

Mathematica graphics

share|improve this question

4 Answers 4

up vote 9 down vote accepted

If we are not afraid of using undocumented functions, we can get the counts the same way that the summary box gets them:

info @ Dataset[data_, type_, _] :=
  {TypeSystem`TypeDepth[type],  TypeSystem`AtomCount[type, data]}

info @ ExampleData[{"Dataset", "Planets"}]
(* {4, 350} *)

Or, as per @Szabolcs' suggestion:

info2 @ ds_Dataset :=
  {TypeSystem`TypeDepth[#],  TypeSystem`AtomCount[##]}& @@
    Through[{Dataset`GetType, Dataset`GetData} @ ds]

info2 @ ExampleData[{"Dataset", "Planets"}]
(* {4, 350} *)
share|improve this answer
    
The old master strikes again. :D ("Old" only respectfully, and in the sense that you were the first serious Mathematica user on Stack Exchange.) –  Mr.Wizard Aug 10 at 5:07
    
I spent a few of minutes looking at the functions in TypeSystem and couldn't find them. Grrrr +1 –  Rojo Aug 10 at 5:14
    
I think the "official" way to get the type is Dataset`GetType[] (still undocumented, of course). –  Szabolcs Aug 10 at 5:18
    
@Szabolcs I added a new variation incorporating your suggestion. –  WReach Aug 10 at 13:39

This is not quite the same thing, but strongly related and very useful to know:

It appears that the undocumented option AllowedHeads, which appears in several functions in v10, can help here.

ds = ExampleData[{"Dataset", "Planets"}];

To get the depth,

ArrayDepth[
 Normal[ds],
 AllowedHeads -> {Association, List}
]

This also works:

ArrayDepth[
 Normal[ds],
 AllowedHeads -> All
]

To get the element count from this generalized array,

Times @@ Dimensions[
  Normal[ds],
  AllowedHeads -> All
]

These counts refer only to the rectangular part of the generalized array made of lists and associations. They won't give the counts 4 and 350. For that we'd need to descend deeper.


Out of curiosity, this is the complete list of System` functions having this option:

ArrayDepth, ConjugateTranspose, Depth, Dimensions, Transpose

Transpose and ConjugateTranspose are useful for taking the transpose of mixed List-Association structures.

Depth would theoretically be good for finding the number of levels in the dataset, but I don't know how to use this option with it. It seems to take only the values True and False.

share|improve this answer
    
ArrayDepth is same as Dimensions /* Length. But Dimensions itself is more informative for many data processing tasks. –  alancalvitti Oct 7 at 18:49

There's at least the brute force approach

getFromDs[what_String][ds_Dataset] := 
 FirstCase[ToBoxes@ds, 
  RowBox[{___, i_, Shortest@___, what, ___}] :> 
   With[{res = ToExpression@i}, res /; IntegerQ[res]], $Failed, 
  Infinity]

getLevels = getFromDs["levels"]; getElements = getFromDs["elements"];

Through@{getLevels, getElements}@ExampleData[{"Dataset", "Planets"}]

(* {4, 350} *)

This is an attempt at a disgusting function for the depth: the maximum number of lists or associations you can go through from the top down before bouncing in something else

We setup some tests

test[dep] ^= Inactivate@{
    dep[{1}] -> 1,
    dep[4] -> 0,
    dep[h[4, g[8]]] -> 0,
    dep[<|"a" -> 3|>] -> 1,
    dep[<|2|>] -> 0,
    dep[<|"a" -> 2|>] -> 1,
    dep[{<|"a" -> 2|>, d}] -> 2,
    dep[<|{2}|>] -> 0,
    dep[<|"a" -> {2}, "b" -> 8|>] -> 2,
    dep[<|"a" :> {<|"c" -> h[2]|>}|>] -> 3,
    dep[<|"x" -> {1}, "x" -> 3|>] -> 1,
    dep[<|"x" :> <|"x" -> {1}, "x" -> 3|>|>] -> 2
    };

runTest[sym_] := 
 Inactive[TestReport][
   test[sym] /. (x_ -> y_) :> Inactive[VerificationTest][x, y]] // 
  Activate

The code

dep[ds_Dataset] := dep[Normal@ds];
dep[stuff_] := idep[0, stuff];

SetAttributes[idep, HoldAllComplete];
idep[i_, l_List | 
    l_Association?(Function[i, AssociationQ@Unevaluated@i, 
       HoldAllComplete])] := 
  1 + Max[Function[x, idep[i, x], HoldAllComplete] /@ 
      Unevaluated@l /. {a_Association :> Values@a}];
idep[i_, a_Association /; 
    MatchQ[Unevaluated@a, 
     HoldPattern[
      Association[(_Rule | _RuleDelayed) ... | {(_Rule | \
_RuleDelayed) ...}]]]] :=

  Module[{v = Block[{Association = HoldComplete}, a]},
   With[{rhs = v[[All, 2]]}, 
    1 + Max[Function[x, idep[i, x], HoldAllComplete] /@ 
        Unevaluated@rhs // ReleaseHold]]];
idep[i_, _] := i;

When we run the tests we see that the last one failed runTest[dep]["TestsFailed"]. That was expected. I don't know exactly how it makes sense to design this. The fact that associations don't become associations until they are evaluated, plus the fact that one has the possibility of holding any of its arguments through using :> instead of -> makes it confusing. Should one count as valid associations that aren't yet associations but are lexically well written? This does that, but this assumes that all valid unevaluated associations are Associations with a list or sequence of rules inside, and doesn't check if those rules may have repeated keys which might end up deleting an element, and you can't really check without evaluating them and leaking.

share|improve this answer
    
One could probably get the number of elements similary, taking the Total instead of 1 + Max –  Rojo Aug 9 at 4:09

Here is another way of calculating the number of levels and elements. It first extracts the Values of each Association and then uses the resulting List to calculate the numbers. It would be nice to know if there is a simpler way of calculating the number of levels from the List.

levelsAndElements[data_Dataset] := Module[{lists, elements, levels},
  lists = Normal@data //. assoc_Association :> Values[assoc];
  elements = Length@Flatten[lists];
  levels = Depth[DeleteCases[lists, Except[_List], Infinity]] - 1;
  {levels, elements}]

levelsAndElements[ExampleData[{"Dataset", "Planets"}]]
{4, 350}
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.