I'm doing some work exploring modelling neural networks in which each neuron could represent a different type of physical system. What I want to do is, I think, quite different to what an ordinary neural network does. So far I had been building up all of the functions and capabilities for this type of network from scratch - but now that I'm at the stage where I'd like to start implementing training algorithms, and I think it could be very complicated.
If it's possible, I'd very much like to use the existing neural network architecture and capabilities. But in order to do so, I'd need to make sure that it can incorporate the type of computation I'm trying to model. My question is: is it possible to define new layer types in Mathematica? Specifically:
- Can I create layers in which each neuron's input and output is a vector, not a scalar? For example, the the neuron might take the curl of the vector -- whereas in usual neural networks the inputs are summed to yield a scalar, which is then fed into a function such as sigmoid.
- Can I create layers in which each neuron calculates its output stochastically?
The idea is that this type of neural network would re-create the behaviour of ordinary neural networks in the limit that the stochastic function is replaced with a deterministic one, and when the length of the vector at each neuron drops to 1.
