I'm struggling with a small data set and a slow calculation. I have hundreds of small 2D data arrays and need to integrate across several lines parallel to the y-axis.

Let me start with the data array:


After rescaling and truncating the distribution, it looks like this:


plot of distribution

I am interested in the profile across a line parallel to the y-axis (say e.g. $x=0.5$) and do handle this profile as a distribution for this specific x-value. I need the 95% quantile for this specific distribution at the specific x-value.

Finally, I am interested in all the quantile-values for all x-values between 0 and 1 (in the rescaled domain) and do rescale them back afterwards.

I have written the following function, which works but need approx. 10 seconds (even without the line plot and sampling at 20 data points between 0 and 1). I have to evaluate hundreds of data sets with this function, so I need to speed it up.

myquantile[data, 0.95] // Timing

The final InterpolatingFunction looks like this (in the regular domain):

plot of interpolant

Can anybody point me in the right direction? Is it possible to get the line distribution in a different and faster way? A MarginalDistribution is pretty much what I want, but it is the distribution for all x-values and it gives me no information on individual x-values.


  • 1
    $\begingroup$ @0x4A4D: how have you changed the [Psi] etc. into the "real" symbols? It was showing up differently, when I posted this question? $\endgroup$ – 32u-nd Jun 20 '13 at 13:21
  • $\begingroup$ Well, magic... okay, no. I knew the Unicode for these, so I put 'em in. $\endgroup$ – J. M.'s technical difficulties Jun 20 '13 at 13:24

You can setup a partial differential equation to integrate along $y$. It costs less than 0.1 second on my old machine.

sol = NDSolve[{
               D[cdf[x, y], y] == pdf[x, y],
               cdf[x, 0] == 0
              cdf, {x, 0, 1}, {y, 0, 1}]

quantileLine = ContourPlot[Evaluate[
                      cdf[x, y]/cdf[x, 1] == .9 /. sol[[1]]],
                      {x, 0, 1}, {y, 0, 1}]

enter image description here

Or if you want an explicit InterpolatingFunction:

quantileLineFunc = Interpolation@
        GraphicsComplex[pts_, others__] :>
             Cases[others, Line[idx_] :> idx, ∞][[1]]
| improve this answer | |
  • $\begingroup$ @ Silvia: thanks a lot; your answer is very obvious, since you just use the dependency between the cdf and pdf. I was completely blocked in mind and haven't thought about other directions to solve this. Again, thanks a lot! $\endgroup$ – 32u-nd Jun 20 '13 at 15:31
  • $\begingroup$ @akm You're welcome! $\endgroup$ – Silvia Jun 20 '13 at 15:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.