How to merge Datasets representing matrices with named rows/columns?

Question

I am working with square (non-symmetric) matrices with named rows and columns. The data I receive is sometimes messy and rows/columns may come in various permutations. I usually need to semi-manually put every matrix into a canonical permutation before I can work with it. I'd rather never have to think about permutations and not even define a canonical one, just index the matrices using the names of their rows/columns.

I was hoping that Dataset would be helpful here, and I made Dataset objects like these:

ds1 = Dataset[<|"a" -> <|"a" -> 1, "b" -> 2|>, "b" -> <|"a" -> 3, "b" -> 0|>|>]

ds2 = Dataset[<|"a" -> <|"b" -> 1, "a" -> 5|>, "b" -> <|"b" -> 1, "a" -> 2|>|>] (* I used a different permutation on purpose *)

What is the simplest way to merge these together to obtain the following?

Dataset[<|"a" -> <|"a" -> {1, 5}, "b" -> {2, 1}|>, "b" -> <|"a" -> {3, 2}, "b" -> {0, 1}|>|>]

Instead of merging with List I will usually need to merge with other operations, e.g. Plus. Once I've built a dataset with List it's easy to do that though.

Answer 1

Since the question stems from working with matrices with named rows and columns I think the answer below is relevant. My answer does not use or deal with Dataset objects, but with sparse matrices with named rows and columns.

In the last few years I have used a lot the R base library Matrix that has implementation of sparse matrix objects and efficient computations. To the sparse matrices from the Matrix R library one can assign and retrieve row and column names with the functions colnames and rownames. Sometimes I miss this in Mathematica so I started a Mathematica package that implements similar functionalities. The package does not use RLink, it has purely Mathematica language implementations.

The RSparseMatrix objects of this package can deal with the permutations of row and column names. (Note that the package is not finished and in some functions the row and column names are ignored.)

This document has demos/tests. (Arithmetic operations, matrix products, joining, profiling, etc.)

This blog post has a neat application example.

Load

This loads the package from GitHub:

Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/Misc/RSparseMatrix.m"]

Creation

Create a RSparseMatrix object with array rules (like for SparseArray) and names of rows, columns, and dimensions:

rmat = MakeRSparseMatrix[{{1, 1} -> 1, {2, 2} -> 2, {4, 3} -> 3, {1, 4} -> 4},
    "ColumnNames" -> {"a", "b", "c", "d"}, "RowNames" -> {"A", "B", "C", "D"},
    "DimensionNames" -> {"U", "V"}];

Show the created matrix:

rmat // MatrixForm

The creation can be done with a SparseArray object:

rmat = ToRSparseMatrix[SparseArray[rmat], 
   "ColumnNames" -> {"a", "b", "c", "d"}, "RowNames" -> {"A", "B", "C", "D"}, 
   "DimensionNames" -> {"U", "V"}];

rmat // MatrixForm

Permutations

Here we permute the matrices:

MatrixForm[rmat[[RotateLeft[RowNames[rmat], 2]]]]

MatrixForm[rmat[[All, RotateLeft[ColumnNames[rmat], 2]]]]

MatrixForm[
 rmat[[RotateLeft[RowNames[rmat], 2], RotateLeft[ColumnNames[rmat], 2]]]]

Converting to SparseArray

At some point it is better to convert to an inherent data structure of Mathematica:

SparseArray[rmat[[All, RotateLeft@ColumnNames[rmat]]]] // MatrixForm

Answer 2

One way to do this would be to use MapIndexed. For example:

MapIndexed[{#, ds2 @@ #2} &, ds1, {2}]

or, to add matching entries:

MapIndexed[# + ds2 @@ #2 &, ds1, {2}]