Creating a R DataFrame like construct in Mathematica

Question

I kept getting my data mixed up so I decided to see if I could construct a package adding a R DataFrame like construct to Mathematica. I managed to cobble something together that sort of works, but I've run into some snags. I decided to use replacement rules as the basic data structure for the dataframe. I was really the only solution I could think of that wouldn't be too convoluted to construct.

What I have thus far package-wise is this:

BeginPackage["DataFrame`"]

CreateDataFrame::usage="CreateDataFrame[data] creates a data frame out of the data table. It assumes the first row is the header."

FrameNames::usage="Names[dataframe] gives the header names of the dataframe."

Begin["`Private`"]

CreateDataFrame[data_]:=
Dispatch@Table[
    Rule[i,Rest@First@Select[Transpose@data,First@#==i&]],
    {i,First@data}
]

FrameNames[dataFrame_Dispatch]:=Table[dataFrame[[1,i,1]],{i,Length@dataFrame[[1]]}]

End[]

EndPackage[]

I also tried to created some custom notation to access parts of the dataframe but I haven't yet figured out how to get it to work when using a package:

Needs["Notation`"]
Notation[ParsedBoxWrapper[
RowBox[{"x_Dispatch", " ", "$", " ", 
     "y_String"}]] \[DoubleLongRightArrow] ParsedBoxWrapper[
RowBox[{"ReplaceAll", "[", 
RowBox[{"y_String", ",", "x_Dispatch"}], "]"}]]]

So I have a number of problems I was hoping someone here could help me with:

How would I go about defining the custom notation from within the package. Currently I have to define it in the notebook to get it to work?

Could I somehow give the dataframe a custom head of DataFrame, preferably without breaking anything?

Is there any way to make the formatting of the output of CreateDataFrame nicer?

While the FrameNames function works as it is I feel it would be more Mathematica-like if it were possible to access it like this instead: dataframe["Names"], but I'm not sure how to make it so.

The custom notation I defined with $ sort of works it is still a bit awkward. To use it I have to make sure I manually include the space between that dataframe name and the $, as well use quotation marks for the variable name. Any ideas for streamlining it more would be wonderful.

Then there is the performance side. The performance isn't terrible, but it is not great either, especially not when creating the dataframe. Any possible recommendations for improvements here would always be appreciated.

P.S. I realise I am piling on a bunch of questions here but I suspect there is a lot of overlap in the answers to them and so splitting the questions up into separate posts would not be helpful.

Answer 1

I won't say anything about the performance, since I think there are not enough information what exactly your data is. Furthermore, I won't say anything about the Notation Package because I don't think it's necessary.

Could I somehow give the dataframe a custom head of DataFrame, preferably without breaking anything?

Yes, I think that's and easy and nice way to go, because it solves most of your issues. Let's assume your data are simple matrices, which you can test with MatrixQ than you don't have to create a specific constructor function. You call your constructor simply DataFrame and define it like

DataFrame[data_?MatrixQ] := DataFrame[
  Dispatch@Table[
    Rule[i, Rest@First@Select[Transpose@data, First@# == i &]], {i, 
     First@data}]
]

Note, that it returns again a DataFrame but since now there is a dispatched set of rules in it, it just stays unevaluated. Now, you mentioned that it would feel natural to call dataframe["Names"] and indeed, this is possible by a simple definition

DataFrame[data_Dispatch]["Names"] := Table[data[[1, i, 1]], {i, Length@data[[1]]}]

Another thing you mentioned is the formatting. Here you can use Format to tell, how you want your DataFrame look like. This is the same approach as it is used when you interpolate a function and you get back an InterpolatingFunction with some information but not with the whole internal data.

Let's say you want to know how many names your DataFrame has when is displayed, then you define for instance

Format[DataFrame[data_Dispatch]] := "DataFrame"[Length[data[[1]]]]

Let's try what we can do with all this:

df=DataFrame[RandomInteger[10,{5,5}]]

(* Out[27]= DataFrame[5] *)

df["Names"]

(* Out[28]= {3,7,5,5,0} *)

In that way you can define most functions. There are other approaches and one might come handy when you want to access the values of your DataFrame. I suppose one very natural access would be when you could call df[[name]]. The problem here is, that you don't really want to extract the Part of your structure. Rather you like to call name/.data where data is the dispatched table.

On the other hand, you don't want to redefine Part, but what you can do is using TagSet

DataFrame /: Part[d_DataFrame, i_] := i /. First[d]

This binds the definition to DataFrame and states something like when DataFrame appears in an expression like Part[d_DataFrame, i_], then replace it with the right hand side.

Let's try it and use strings as names

df = DataFrame[
  Prepend[RandomInteger[10, {4, 4}], {"blub", "foo", "boing", "bar"}]];

df["Names"]
df[["boing"]]

(*
Out[37]= {"blub", "foo", "boing", "bar"}

Out[38]= {5, 5, 10, 0}
*)