Quantile Regression using Neural Networks (Custom Loss function)

Question

I would like to perform a Quantile Regression within Mathematica's neural network framework.

Since there exist a couple of tutorials for this exact problem in Tensorflow, i.e. Deep Quantile Regression, I know this can be done by defining the right loss function for this particular problem, i.e.

$$\rho_\tau(u)= \max[ u\tau, u(\tau-1)].$$

Then a specific quantile $\tau$ can be found by minimizing the expected loss of $y-f(X)$ with respect to $f(X)$, where $f(X)$ is the predicted (quantile) model and $y$ is the observed value for the corresponding input $X$.

My question then, is it possible to define a custom loss function for NetTrain[], and if so, how can this be done?

---

An example of the data can be loaded by,

data = Flatten[Uncompress[Import["https://pastebin.com/raw/G2kdCu4i"]]]

Note that in this particular setting the observed values are 3-tuples.

---

Disclaimer, this is my first time using Mathematica's neural network framework.

Answer 1

Going through the documentation of LossFunction it dawned on me that I needed to define a custom Layer via NetGraph, hence

QuantileLossLayer[τ_] := NetGraph[<|
   "thread" -> ThreadingLayer[(#1 - #2) &],
   "loss" -> ElementwiseLayer[(Max[# τ, # (τ - 1)]) &],
   "sum" -> SummationLayer[]|>, 
   {{NetPort["Target"], NetPort["Input"]} -> "thread" -> "loss" -> "sum"}]

It can then be used for the training on the example data, e.g.

net = NetChain[{8, Tanh, 16, Tanh, 3}];
trained = NetTrain[net, data, LossFunction -> QuantileLossLayer[.2]]