For neural networks mathematica likes 2 ways to structure training sets:

listFormatData = {
<|"Input" -> input1, "OutputKey1" -> OutputKey1Input1Output, ...|>,
<|"Input" -> input2, "OutputKey1" -> OutputKey1Input2Output, ...|>,
...
}

and

assocFormatData = <|
"Input" -> {input1, input2, ...}, 
"OutputKey1" -> {OutputKey1Input1Output, OutputKey1Input2Output, ...}
, ...
|>

Is the a stock way to convert these 2 formats properly?

Concrete examples:

listData = {
<|"Input" -> 1, "double" -> 2, "squared" -> 1|>,
<|"Input" -> 2, "double" -> 4, "squared" -> 4|>,
<|"Input" -> 3, "double" -> 6, "squared" -> 9|>
}

and

assocData = <|
"Input" -> {1, 2, 3}, 
"double" -> {2, 4, 6}, 
"squared" -> {1, 4, 9}
|>
  • 1
    What does Normal[Transpose[Dataset[listData]]] return for you? (Also if you replace listData with assocData?) – J. M. is computer-less Oct 19 at 14:43
up vote 10 down vote accepted

You can use the undocumented AllowedHeads option of Transpose to do this:

tr = Transpose[listData, AllowedHeads -> {Association, List}]

<|"Input" -> {1, 2, 3}, "double" -> {2, 4, 6}, "squared" -> {1, 4, 9}|>

and back:

Transpose[tr, AllowedHeads -> {Association, List}]

{<|"Input" -> 1, "double" -> 2, "squared" -> 1|>, <|"Input" -> 2, "double" -> 4, "squared" -> 4|>, <|"Input" -> 3, "double" -> 6, "squared" -> 9|>}

Merge[listData, Identity]

<|"Input" -> {1, 2, 3}, "double" -> {2, 4, 6}, "squared" -> {1, 4, 9}|>

listFormatData = {<|"Input" -> input1,  "OutputKey1" -> OutputKey1Input1Output|>,
   <|"Input" -> input2, "OutputKey1" -> OutputKey1Input2Output|>};
Merge[listFormatData , Identity]

<|"Input" -> {input1, input2}, "OutputKey1" -> {OutputKey1Input1Output, OutputKey1Input2Output}|>

A function that is of great use here, is AssociationTranspose in the GeneralUtilities package. It does the same thing as regular transpose, but works with List and Associations alike (and mixtures of the two):

listData = {
  <|"Input" -> 1, "double" -> 2, "squared" -> 1|>, 
  <|"Input" -> 2, "double" -> 4, "squared" -> 4|>,
  <|"Input" -> 3, "double" -> 6, "squared" -> 9|>
};
assocData = <|"Input" -> {1, 2, 3}, "double" -> {2, 4, 6}, "squared" -> {1, 4, 9}|>;
GeneralUtilities`AssociationTranspose[listData]  === assocData
GeneralUtilities`AssociationTranspose[assocData] === listData

Out[11]= True

Out[12]= True

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.