# Converting from "matrix" data into "coordinate" data

Say I have data which looks like data1 - this is "matrix" like data (is there a better descriptor?). The data looks like a matrix, and at each point in the matrix, it has a value. I can plot these in ListContourPlot and the like. e.g.

datafunction = MultinormalDistribution[{0, 0}, {{2, 1/2}, {1/2, 1}}];

(* matrix like data *)

data1 = N[ Table[PDF[datafunction, {x, y}] /. {x -> xinsert, y -> yinsert}, {xinsert, -4, 4, 1}, {yinsert, -2, 2, 1}]];
ListContourPlot[data1]


However, I can also create the same effect by making "coordinate" like data, where the data is a list of coordinates.

(* coordinate like data *)

data2 = RandomVariate[MultinormalDistribution[{0, 0}, {{2, 1/2}, {1/2, 1}}], 1000];
ListPlot[data2]


How would I convert data1 into data2? How do I convert matrix-like into coordinate-like?

I need to do some PCA analysis, I require the data to be in the form of individual points.

• How do you think one could infer the individual counts from a total count? Once we have totaled data and thrown away the parts there is no way to reconstruct them. The mapping between sums and their constituents is not bijective. Mar 9, 2019 at 16:36

The reshaping can be done in several ways. Below are given three using SparseArray, "SSparseMatrix.m", and "DataReshape.m". (The first one is/was the accepted answer.)

I do these kind of transformations often while using sparse matrices (with named rows and columns) and Dataset objects. So, this answer is a good place to mention the related packages.

## Data generation

First generating the data (simpler than in the question):

datafunction = MultinormalDistribution[{0, 0}, {{2, 1/2}, {1/2, 1}}];

data1 = N[Table[PDF[datafunction][{x, y}], {x, -4, 4, 1}, {y, -2, 2, 1}]];

MatrixForm[data1]


Make index-to-value associations corresponding to the ranges used to make data1:

aX = AssociationThread[Range[Length[#]], #] &@Range[-4, 4, 1];
aY = AssociationThread[Range[Length[#]], #] &@Range[-2, 2, 1];


Convert to a sparse array, take the corresponding rules, and map the {x,y} indexes to the actual x's and y's.

arules = Most[ArrayRules[SparseArray[data1]]];
data2 = Map[{aX[#[[1, 1]]], aY[#[[1, 2]]], #[[2]]} &, arules]


Plot (note the axes ticks):

ListContourPlot[data2]


This answer is just a package-based version of the previous one.

Import["https://raw.githubusercontent.com/antononcube/\
MathematicaForPrediction/master/SSparseMatrix.m"]

smat1 = ToSSparseMatrix[SparseArray[data1],
"RowNames" -> Map[ToString, Range[-4, 4, 1]],
"ColumnNames" -> Map[ToString, Range[-2, 2, 1]]]

data2a = SSparseMatrixToTriplets[smat1];
data2a = data2a /. x_String :> ToExpression[x];

data2a == data2
(* True *)


The package "DataReshape.m" was made because I have to often convert tabular data (Dataset objects) from "wide form" to "long form" and vice versa. (For R there are at least two dedicated packages from RStudio for doing these kind of transformations.)

Import["https://raw.githubusercontent.com/antononcube/\
MathematicaForPrediction/master/DataReshape.m"]

ds1 = Dataset[
AssociationThread[Range[-2, 2, 1], #] & /@ data1]];

?ToLongForm

(* ToLongForm[ds_Dataset, idColumns_, valueColumns_] converts the \
dataset ds into long form. The resulting dataset has the columns \
idColumns and the columns "Variable" and "Value" derived from \
valueColumns. *)

data2b = Normal[ToLongForm[ds1][Values]];

data2b == data2
(* True *)


An alternative approach based on Rescaleing the "NonzeroPositions" of SparseArray[data1]:

xrange = {-4, 4};
yrange = {-2, 2};
sa = SparseArray[data1];
nzp = sa["NonzeroPositions"];
nzv = sa["NonzeroValues"];
data2b = Join[Transpose[Rescale[#, MinMax@#, #2] & @@@
Thread[ {Transpose@nzp, {xrange, yrange}}]], List /@ nzv, 2];

data2b == data2 (* from Anton's answer *)


True