Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Try this code in Mathematica 10:

Dataset@Table[Association[ToString[#] -> # & /@ Range[4]], {2}]

I got a nicely formatted table:

enter image description here

But when I do the same thing for 5 columns, it won't get formatted at all:

Dataset@Table[Association[ToString[#] -> # & /@ Range[5]], {2}]

gives:

enter image description here

Is that what supposed to be? Don't you think 4 column is a pretty stingy limitation?

share|improve this question
    
What you see is similar to the output of the model fit functions and TimeSeries for instance. You get a summary object that symbolizes the data structure but is not meant to fully represent it visually. For formatting tables we have things like Grid or TableForm. –  Sjoerd C. de Vries Jul 12 at 1:12
    
Is there a method similar to FullForm or options for Dataset that attempts to display more of the structure? It also crops rows if there are many. –  alancalvitti Jul 12 at 2:17
    
@SjoerdC.deVries Are you aware of any new functions that are specifically meant for formatting Dataset and won't require extracting both the tabular data and the headers separately? –  Szabolcs Aug 8 at 20:25
    
@Szabolcs I'm afraid not. –  Sjoerd C. de Vries Aug 10 at 21:22

4 Answers 4

up vote 23 down vote accepted

Theoretically, Dataset supports any number of columns.

The behavior you are seeing is actually because the type deduction that Dataset is doing behind the scenes isn't perfect (and indeed in some sense cannot be perfect). Your synthetic example is such that your second list of associations is "most consistent" with a particular type that doesn't typeset as a table.

You can see what type Dataset deduced in a given case by using Dataset`GetType. First get TypeSystem onto your context path, so that the types aren't fully qualified and are easier to read:

Needs["TypeSystem`"];

Then use GetType:

In[2]:= Dataset`GetType @ Dataset @ Table[Association[ToString[#] -> # & /@ Range[4]], {2}]
Out[2]= Vector[Struct[{"1", "2", "3", "4"}, 
 {Atom[Integer], Atom[Integer], Atom[Integer], Atom[Integer]}], 2]

Notice that the type of your data has been deduced to be a Vector (homogenous list) of Structss (heterogenous associations), or in other words a row-oriented table.

But now do:

In[3]:= Dataset`GetType @ Dataset @ Table[Association[ToString[#] -> # & /@ Range[5]], {2}]
Out[3]= Vector[Assoc[Atom[String], Atom[Integer], 5], 2]

Here, your data has been deduced as a Vector of Assocs (homogenous associations). Assocs are a type that doesn't care what keys are present, just that they all have the same type, and also that the values have the same type.

That happened because according to the internal heuristics, an Assoc is considered to be a more parsimonious type as soon as we cross the threshold of 4 fields. But this would not be true if we looked at an association whose values were different types, instead of all being integers:

In[2]:= DeduceType @ Table[<|"A" -> 1, "B" -> 2, "C" -> 3, "D" -> 4, "E" -> "bar"|>, {5}]
Out[2]= Vector[Struct[{"A", "B", "C", "D", "E"}, 
  {Atom[Integer], Atom[Integer], Atom[Integer], Atom[Integer], Atom[String]}], 5]

The only consistent type here is a Vector of Structs (notice I'm using DeduceType directly, which is what Dataset uses upon construction). And indeed, this more complex Dataset typesets as a table, owing to the inner Struct type:

Dataset

Although it isn't documented and is therefore of course subject to change, you can force a specific type to be used by supplying a second argument to Dataset:

Dataset[
  Table[Association[ToString[#] -> # & /@ Range[5]], {5}], 
  Vector[Struct[{"1", "2", "3", "4", "5"}, 
    {Atom[Integer], Atom[Integer], Atom[Integer], Atom[Integer], Atom[Integer]}]]]

This will typeset as a table, as you desire:

dataset-row-oriented

share|improve this answer
    
Re "Assoc is considered to be a more parsimonious type as soon as we cross the threshold of 4 fields" - where did 4 come from? This book says 42: en.wikipedia.org/wiki/The_Hitchhiker%27s_Guide_to_the_Galaxy –  alancalvitti Jul 14 at 20:19
    
Does the internal Dataset structure only affect its Format or other operations as well? –  alancalvitti Jul 15 at 17:52
    
@alancalvitti 4 is really just a codification of my prior belief in the likelihood of different forms of data. The Dataset type ('schema' might be a better word) is used for formatting, and also validation. If you try an invalid query it'll stop you shooting yourself in the foot, with a (hopefully) helpful error message telling you why. In the future it'll also be used for autocomplete prediction and NLP queries against data. –  Taliesin Beynon Jul 15 at 19:33
    
I think we all appreciate the new functionality. However, working w real datasets, I resort to // Normal and subsequent operations. The built in format is too limiting. At least make it interactive so users can choose thresholds in Level and other dimensions. –  alancalvitti Jul 15 at 19:36
1  
The definitions associated with TypeSystem`AnyLength are a bit weird. Plus[___, a, ___] is in fact equivalent to Plus[a, ___] because of the attributes of Plus. The same is true for Times. Minus is a single-argument function, so that definition was probably meant to use Subtract. Yet Subtract is not needed because x - y parses directly to Plus[x, Times[-1, y]] and never gets interpreted as Subtract. –  Szabolcs Aug 10 at 15:59

If you look at InputForms of produced Datasets you will see significant difference:

Dataset@Table[Association[ToString[#] -> # & /@ Range[4]], {2}] // InputForm
Dataset[{<|"1" -> 1, "2" -> 2, "3" -> 3, "4" -> 4|>, 
         <|"1" -> 1, "2" -> 2, "3" -> 3, "4" -> 4|>}, 
 TypeSystem`Vector[TypeSystem`Struct[{"1", "2", "3", "4"}, 
   {TypeSystem`Atom[Integer], TypeSystem`Atom[Integer], TypeSystem`Atom[Integer], 
    TypeSystem`Atom[Integer]}], 2], <|"ID" -> 34613158063874|>]
Dataset@Table[Association[ToString[#] -> # & /@ Range[5]], {2}] // InputForm
Dataset[{<|"1" -> 1, "2" -> 2, "3" -> 3, "4" -> 4, "5" -> 5|>, 
         <|"1" -> 1, "2" -> 2, "3" -> 3, "4" -> 4, "5" -> 5|>}, 
 TypeSystem`Vector[TypeSystem`Assoc[TypeSystem`Atom[String], 
   TypeSystem`Atom[Integer], 5], 2], <|"ID" -> 93437030149901|>]

Let us try to reproduce the first structure with 5 columns:

Dataset[Table[Association[ToString[#] -> # & /@ Range[5]], {2}], 
 TypeSystem`Vector[
  TypeSystem`Struct[ToString /@ Range[5], 
   Table[TypeSystem`Atom[Integer], {5}]], 2]]

screenshot

Voilà! I have checked this method with 200 columns and it works!

share|improve this answer
    
Is there a way to override the built in Format for Dataset? Anything is better than this, eg // Normal // TableForm –  alancalvitti Jul 14 at 20:21
    
AFAIK currently there is no simple way to change the style of Dataset in StandardForm other than re-implementing Format[Dataset]. Such questions of course should be asked to the developer, Taliesin Beynon, not me. –  Alexey Popkov Jul 15 at 6:54
6  
@alancalvitti I don't see why we shouldn't let you set a customer formatter. I'll add it to my list. –  Taliesin Beynon Jul 15 at 19:34
    
@TaliesinBeynon, thanks, look forward. Despite its limits, Dataset is a great product that is already making query dev faster, more compact and flexible. –  alancalvitti Jul 30 at 17:48
    
@AlexeyPopkov, for some data shapes like 2-dim relational tables I use // Normalize // TableForm with headers based on the Keys. So it's simple. But why doesn't Unprocting Dataset and then UpSet Format, work? –  alancalvitti Jul 30 at 18:00

I think that ideally there would be a separate function for displaying a dataset in the preferred way. Automatic formatting should only produce a summary, i.e. truncate the data if it is too large. But we should have the option to display the complete dataset and tweak the way it is displayed, if necessary.


This is an attempt at writing a function that will take a dataset having the form of a 2D table and will try to format it in a reasonable way. It doesn't aim to be general and will probably fail on more complicated datasets. The way I'm trying to determine the type (schema) is probably clumsy, and I'll take any suggestions on how to do this better (I'm sure there are undocumented functions to do this properly).

Basic usage:

ds = Dataset[Association["a" -> Association["a" -> 1, "b" -> 2], "b" -> Association["a" -> 3, "b" -> 4]]];

formatDS[ds]

Mathematica graphics

This is extended from something for my own use, hence the small font size and the RotateLabel option. People might want to tweak at least the font size when using this.


ClearAll[formatDS]
Options[formatDS] = {RotateLabel -> False};
formatDS[ds_Dataset, opt : OptionsPattern[]] :=
 Module[
  {$headerColour, $frameColour,
   hasRowKeys, hasColKeys,
   colKeys, rowKeys, content,
   rot, type, form}
  ,
  $headerColour = GrayLevel[0.9];
      $frameColour = GrayLevel[0.8];
  type = Dataset`GetType[ds];
  hasRowKeys = MatchQ[type, HoldPattern[Assoc[_Atom, __] | _Struct]];
  hasColKeys = 
   MatchQ[type, 
    HoldPattern[
     Assoc[_Atom, _Assoc | _Struct, ___] | 
      Vector[_Assoc | _Struct, ___]]];
  If[hasRowKeys, rowKeys = Normal@ds[Keys]];
  If[hasColKeys, colKeys = Normal@ds[1, Keys]];
  rot = If[TrueQ@OptionValue[RotateLabel], Rotate[#, Pi/2] &, 
    Identity];
  Which[
   hasRowKeys && hasColKeys,
   content = Normal@ds[Values, Values];
   form = Sequence[ArrayFlatten[( {
        {Null, {rot /@ colKeys}},
        {List /@ rowKeys, content}
       } )], Background -> {{$headerColour}, {$headerColour}}];
       ,
       hasRowKeys,
       content = Normal@ds[Values];
       form = 
        Sequence[ArrayFlatten[{{List /@ rowKeys, content}}], 
         Background -> {{$headerColour}, None}];
       ,
       hasColKeys,
       content = Normal@ds[All, Values];
       form = Sequence[ArrayFlatten[( {
            {{rot /@ colKeys}},
            {content}
           } )], Background -> {None, {$headerColour}}];
       ,
       True, form = content = Normal[ds]
       ];
      Style[
       Grid[
        form,
        Frame -> All,
        FrameStyle -> $frameColour,
    Spacings -> {1, 1},
    ItemSize -> Full
    ],
   "Text", FontSize -> 10
   ]
  ]
share|improve this answer

Here is a cheap and dirty function that takes a Vector of Associations variant of a Dataset and converts it into a Vector of Structs. No guarantees this actually works on a variety of Datasets, but it worked for me. If the keys in each element of the Dataset are not the same, this function will not work properly.

 structDisplay[d_Dataset] := Module[{struct1, types},
 Needs["TypeSystem`"];
 struct1 = Normal@Keys[d[1]];
 types = 
 Replace[DeleteDuplicates /@ 
 Transpose[Map[DeduceType, Normal[Values /@ d], {2}]], {{t_} :> 
  t, {t_, u__} :> AnyType}, 1];
 Dataset[Normal@d, Vector[Struct[struct1, types], Length[d]]]
 ]
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.