11
$\begingroup$

I would like to perform a Quantile Regression within Mathematica's neural network framework.

Since there exist a couple of tutorials for this exact problem in Tensorflow, i.e. Deep Quantile Regression, I know this can be done by defining the right loss function for this particular problem, i.e.

$$\rho_\tau(u)= \max[ u\tau, u(\tau-1)].$$

Then a specific quantile $\tau$ can be found by minimizing the expected loss of $y-f(X)$ with respect to $f(X)$, where $f(X)$ is the predicted (quantile) model and $y$ is the observed value for the corresponding input $X$.

My question then, is it possible to define a custom loss function for NetTrain[], and if so, how can this be done?


An example of the data can be loaded by,

data = Flatten[Uncompress[Import["https://pastebin.com/raw/G2kdCu4i"]]]

Note that in this particular setting the observed values are 3-tuples.


Disclaimer, this is my first time using Mathematica's neural network framework.

$\endgroup$
  • $\begingroup$ Yes, you can, check the documentation for LossFunction. It can be a little limited however. It might be helpful if you give some example of the output data and the target data. $\endgroup$ – Carl Lange Oct 12 '18 at 12:41
  • $\begingroup$ @CarlLange I did came across the documentation for LossFunction but it only used built-in layers which threw me off. I also added an example of my data. $\endgroup$ – user19218 Oct 12 '18 at 13:14
  • $\begingroup$ It looks like your QuantileLossLayer answers your question - you could post it as an answer and close the question! $\endgroup$ – Carl Lange Oct 12 '18 at 18:08
8
$\begingroup$

Going through the documentation of LossFunction it dawned on me that I needed to define a custom Layer via NetGraph, hence

QuantileLossLayer[τ_] := NetGraph[<|
   "thread" -> ThreadingLayer[(#1 - #2) &],
   "loss" -> ElementwiseLayer[(Max[# τ, # (τ - 1)]) &],
   "sum" -> SummationLayer[]|>, 
   {{NetPort["Target"], NetPort["Input"]} -> "thread" -> "loss" -> "sum"}]

It can then be used for the training on the example data, e.g.

net = NetChain[{8, Tanh, 16, Tanh, 3}];
trained = NetTrain[net, data, LossFunction -> QuantileLossLayer[.2]]
$\endgroup$
  • $\begingroup$ What is the net, net? $\endgroup$ – Anton Antonov Oct 17 '18 at 12:24
  • 1
    $\begingroup$ @AntonAntonov net is any neural network you deem fit, but I get your point and added an working example. $\endgroup$ – user19218 Oct 17 '18 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.