5
$\begingroup$

In this paper,they design a network like this.

enter image description here

The left(green) is a normal network that mapping one feature to another feature,the right(red) side is an autoencoder(using RNN).But a switch is inserted.

During training the switch is set randomly for each network.

So the embedding vector can learn multiform information.

Using MNIST to make a example. In order to learn image features,I define this network(for simplify,only use DNN).

resource = ResourceObject["MNIST"];
trainingData = ResourceData[resource, "TestData"];
trainingData = {List @@ # -> First@#} & /@ trainingData;

enter image description here

NetGraph[{FlattenLayer[], 8, 28*28, ReshapeLayer[{1, 28, 28}], 
    UnitVectorLayer[10, "Input" -> "Integer"]}, {1 -> 2 -> 3 -> 4,NetPort["digit"] -> 5}, 
    "Input" -> NetEncoder[{"Image", {28, 28}, ColorSpace -> "Grayscale"}], 
    "Output1" -> NetDecoder[{"Image", ColorSpace -> "Grayscale"}]]

enter image description here

But I want the net like this: enter image description here

So the image features not only can learn information from image pixels,but only learn information about digits.

How to do it?

$\endgroup$
7
$\begingroup$

If I understood correctly, you want that the node 3 either sees the output of node 2 or of node 5 (that both have the same dimensions) depending on a randomly chosen "switch" value.

You could add the "switch" variable as an extra input (with the same dimensions as the output of 2/5), that you set randomly to 0 or 1 for every sample (e.g. sample generator function), and then use ThreadingLayer[#1*#2&] and ThreadingLayer[#1*(1-#2)&] to set either node 2 or 5 to 0, then add the results, like this (untested):

NetGraph[{net1, net2, ThreadingLayer[#1*#2 &], 
  ThreadingLayer[#1*(1 - #2) &], ThreadingLayer[Plus], net3},
 {
  NetPort["Input"] -> 1, {1, NetPort["Switch"]} -> 3,
  NetPort["Input"] -> 2, {2, NetPort["Switch"]} -> 4,
  {3, 4} -> 5 -> 6 -> NetPort["Output"]
  }]

enter image description here

$\endgroup$
  • $\begingroup$ I don‘t know how ThreadingLayer[#1*(1 - #2) &] determine which is #1,which is #2.If #1 means switch,it will leads problem $\endgroup$ – partida Sep 1 '17 at 11:11
  • $\begingroup$ I think the function arguments are ordered like the inputs in the graph, i.e. for{1, 2} -> 3 #1 will be the output of node 1 and #2 will be the output of node 2 $\endgroup$ – Niki Estner Sep 1 '17 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.