0
$\begingroup$

I'm trying to write my first neural network in Mathematica: a simple autoencoder. The network takes as input an array of size (2, 100) and reconstructs a vector of length 100 (see below). I'm trying to replicate the code on Wolfram's site, but am having trouble setting up the loss. I don't think I'm understanding the error message below. At In[856] it seems to me that I generate a network whose output is a vector of 100, so why is it upset at the dimension for layer loss (which I don't even specify?). Any help getting this fixed would be much appreciated.

Edit: Thanks to the great answer below I've made some progress on this, and think I'm understanding the NetGraph component a bit better. I've changed the configuration so that the mean loss layer takes my target output as an input (which I've named gyroscope).

Overall, the training data is "input", a (19, 2, 100) array, and "gyroscope" a (19, 100) array. However, when I try to train the network, using:

results = NetTrain[net, <|
   "Input" -> input,  
   "Gyroscope" -> gyroscope
   |>]
trained = results["TrainedNet"]

I'm told that I was supposed to supply the network with two inputs, but only gave it one. As far as I can tell, I'm giving it two. What am I missing?

For reference, here's my new NetGraph specification and general network specification below that.

net = NetGraph[
  <|
   "autoencoder" -> autoencoder,
   "loss" -> MeanAbsoluteLossLayer[]
   |>,
  {
   NetPort["Input"] -> "autoencoder" -> "loss",
   NetPort["Gyroscope"] -> NetPort["loss", "Target"]
   }
  ]

autoencoder = 
 NetChain[{FlattenLayer[], LinearLayer[64], BatchNormalizationLayer[],
    ElementwiseLayer["ReLU"], 32, ElementwiseLayer["ReLU"], 16, 
   BatchNormalizationLayer[], ElementwiseLayer["ReLU"], 32, 
   BatchNormalizationLayer[], ElementwiseLayer["ReLU"], 64, 
   BatchNormalizationLayer[], ElementwiseLayer["ReLU"], 100}, 
  "Input" -> {2, 100}]
```
$\endgroup$
  • 1
    $\begingroup$ Welcome to Mathematica.StackExchange! In future questions, add code as plain text instead of images. Your problem is here: "...a simple autoencoder. The network takes as input an array of size (2, 100) and reconstructs a vector of length 100". Replace last layer in autoencoder with LinearLayer[{2, 100}]. $\endgroup$ – Alexey Golyshev May 24 at 4:12
  • $\begingroup$ Edit: To be clear, I am aware that autoencoders are typically used to reconstruct the input, but I'm using a variant here. Let's simply call it a net which is forced to constrain its feature representation to be very low dimension before then predicting the output vector. To reiterate, the network takes in a (2 x 100) array and predicts a (100)-vector. I currently have these in the form of variables "inputdata" which is a (19, 2, 100) tensor, and "outputdata" which is a (19, 100) tensor. For the answer given below, I think I would need to change the NetGraph specification, right? Thanks! $\endgroup$ – user65768 May 24 at 14:35
2
$\begingroup$

Autoencoders are a special type of neural network architectures in which the output is same as the input. Autoencoders are trained in an unsupervised manner in order to learn the exteremely low level repersentations of the input data. These low level features are then deformed back to project the actual data.

A typical autoencoder architecture comprises of three main components:

  • Encoding Architecture : The encoder architecture comprises of series of layers with decreasing number of nodes and ultimately reduces to a latent view repersentation.
  • Latent View Repersentation : Latent view repersents the lowest level space in which the inputs are reduced and information is preserved.
  • Decoding Architecture : The decoding architecture is the mirro image of the encoding architecture but in which number of nodes in every layer increases and ultimately outputs the similar (almost) input.

enter image description here

autoencoder = NetChain[{LinearLayer[64], LinearLayer[{2, 100}]}, "Input" -> {2, 100}]

enter image description here

net = NetGraph[
  <|
   "autoencoder" -> autoencoder,
   "loss" -> MeanAbsoluteLossLayer[]
   |>,
  {
   NetPort["Input"] -> "autoencoder" -> "loss",
   NetPort["Input"] -> NetPort["loss", "Target"]
   }
  ]

enter image description here

n = 50;
X = RandomReal[{-1, 1}, {n, 2, 100}];

netT = NetTrain[net, <|"Input" -> X|>]

autoencoder = NetTake[netT, "autoencoder"]

enter image description here

ListPlot[Flatten[X[[1]]], Filling -> Axis]

enter image description here

ListPlot[Flatten[autoencoder@X[[1]] - X[[1]]], Filling -> Axis]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.