4
$\begingroup$

I'd like to implement the following function $f$ in a neural net:

$$ f(i,j)=s_i-s_j $$

where $1\leq i,j\leq N$ are integers and the $s_i\in\mathbb{R}$ are learnable parameters.

What's the best way to do this?

EmbeddingLayer[1, n, "Input" -> 2] appears promising; it maps integers pair of integers $(i,j)$ to learnable real numbers $(s_i,s_j)$. However, I haven't figured out how to subtract them afterwards.

Attempt 1

I tried this:

NetChain[{
  EmbeddingLayer[1, n, "Input" -> 2],
  ThreadingLayer[Subtract]
  }]

which fails with

NetChain::ninctyp2: Incompatible types for output of layer 1, EmbeddingLayer[1,\[Ellipsis]], and input to layer 2, ThreadingLayer[Subtract,\[Ellipsis]]; a 2*1 matrix is not compatible with a pair of tensors, respectively.

It fails because the ThreadingLayer expects two scalar inputs, not one input that's a 2-vector, but am not sure how to massage the data into the right form.

Attempt 2

This appears to work, but seems hokey:

NetGraph[{
  EmbeddingLayer[1, n, "Input" -> 2],
  PartLayer[1],
  PartLayer[2],
  ThreadingLayer[Subtract]
  },
 {1 -> 2, 1 -> 3, {2, 3} -> 4}]

enter image description here

It sends the first layer output in two directions, pulls out the first and second elements, and recombines them later. I feel like there's got to be a simpler way than this.

$\endgroup$
2
$\begingroup$

Since $s_i-s_j$ is essentially a dot product with the vector {1,-1}, you could use:

net = NetGraph[
  {
    ConstantArrayLayer["Array" -> {{1, -1}}],
    EmbeddingLayer[1, n, "Input" -> 2],
    DotLayer[]
  }, {1 -> 3, 2 -> 3}]

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.