Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I have a data set of x,y,z values and I fit a function of x,y to the data. This works, but I can't come up with a nice way to visualize the data. 3D plots are not very clear on paper and a contour plot of two data sets doesn't work either. What would be a clear and simple way to show the data and the fit?

data = Import["https://dl.dropbox.com/u/3538352/data.dat"];
fit = NonlinearModelFit[
       data, 
       A*Exp[-(y - y0 - y1 Cos[2 (x/180*Pi)])^2/(w0 + w1 Cos[2 x/180*Pi])^2],
       {A, {y0, 0.04}, {y1, 0.00}, {w0, 0.03}, {w1, 0.01}},
       {x, y}
      ];
Show[
     ListPointPlot3D[data],
     Plot3D[fit["BestFit"], {x, 0, 180}, {y, 0, 0.1}]
    ]

EDIT:

What I decided to do for now is using a DensityPlot with the data with the ContourPlot of the fit (similar to Rahul Narain's answer). This does not really show the quality of the model, so I will add other plots, candidates are

  • residuals vs. predicted values (similar to chris's answer)
  • distribution of residuals (chris's answer)
  • Q-Q plot using QuantilePlot[fit["FitResiduals"]]
  • the plot Rahul Narain suggested

enter image description here

share|improve this question
    
you might want to try variations of PlotStyle -> Directive[Yellow, Specularity[White, 20], Opacity[0.3]] and possibly BoxRatios -> {1, 1, 1} –  chris Dec 28 '12 at 16:58
7  
A standard--and very effective--way to show the fit is to plot the residuals (equal to actual - estimated) against $x$ and $y$. For a good fit they should cluster evenly and randomly around the $xy$ plane. Because there's no question about where that plane lies, you don't even need the third dimension. For instance, many people map out the residuals in 2D using scaled and/or colored point symbols to represent their sizes and signs. –  whuber Dec 28 '12 at 17:24
    
@whuber: I think this is what I was looking for. I was playing with the residuals, but plotting them vs. the actual values (chris's answer) gives some interesting insights. Thanks a lot –  jenson Dec 28 '12 at 20:32

4 Answers 4

up vote 27 down vote accepted

If what you want to visualize is how good the fit is, then you should do as @whuber suggests and plot the residuals, that is, the difference between the data and the fitted function. Below, each data point is drawn as a point with area proportional to the magnitude of the residual. Red means that the data value is higher than the fit; blue means the data is lower. For context, the contours of the fit are plotted in the background.

residual[{x_, y_, z_}] := Evaluate[z - fit["BestFit"]]
rmax = Max[(Abs@residual@#) & /@ data];
residualPoint[p : {px_, py_, pz_}] := Module[{r},
  r = residual[p]/rmax;
  {AbsolutePointSize[10 Sqrt@Abs@r], 
   ColorData["ThermometerColors"][(r + 1)/2], Point[{px, py}]}]
Show[ContourPlot[fit["BestFit"], {x, 0, 180}, {y, 0, 0.1}, 
  AspectRatio -> 1, ContourShading -> None], 
 Graphics[Flatten[residualPoint /@ data]]]

There does seem to be a little bit of systematic bias in the residuals. If I had to guess, I'd say the fitted function tends to overestimate the data at the left and right ends of the "ridge", and underestimates the data at the lower corners. And perhaps the ridge ought to form a narrower "V". However, I am not a statistician.

Anyway, if you want to both visualize the shape of the fit and also indicate how far it is from the data simultaneously, you could lay these residual markers right on top of your 3D plot.

residualPoint3D[p : {x_, y_, z_}] := 
 Evaluate@Module[{r}, r = residual[p]/rmax;
   {AbsolutePointSize[2 + 8 Sqrt@Abs@r], 
    ColorData["ThermometerColors"][(r + 1)/2], 
    Point[{x, y, fit["BestFit"] + 1*^-5}]}]
Show[Plot3D[Evaluate@fit["BestFit"], {x, 0, 180}, {y, 0, 0.1}, 
  Mesh -> 5, MeshFunctions -> {#3 &}, MeshStyle -> Gray, 
  Lighting -> {{"Ambient", White}}], 
 Graphics3D[Flatten[residualPoint3D /@ data]]]

share|improve this answer
    
+1. It is useful at this stage to plot the residuals against the fit: that will make your comments more precise. BTW, fit["FitResiduals"] is a built-in way to obtain the residuals. –  whuber Dec 28 '12 at 18:05
    
I think I misunderstood your comment. @chris seems to have taken care of it though. :) –  Rahul Narain Dec 28 '12 at 20:03
    
@jenson: Please have a look at my edited answer. –  Rahul Narain Dec 28 '12 at 22:51
    
I really like the simplicity of the first plot compared to the 3d plots. Even though I think whuber's comment and Chris' answer are worth reading, you stand out with the simplicity of your visualization. One plot that shows the fit and the residuals without a lot of colors, lines or perspectives. –  jenson Dec 29 '12 at 0:27
    
@jenson: I think that's what exactly what whuber meant by plotting in "the $xy$ plane... You don't even need the third dimension." 2D is nice because you don't have to deal with things like occlusions and foreshortening that could obscure the data. I'm reminded of whuber's remarkably simple but illuminating graphic showing the vote counts of the moderator election. –  Rahul Narain Dec 29 '12 at 15:39

I would try

  plt=Show[ListPointPlot3D[data, ColorFunction -> "Rainbow"], 
  Plot3D[fit["BestFit"], {x, 0, 180}, {y, 0, 0.1}, 
  PlotStyle -> 
  Directive[Yellow, Specularity[White, 20], Opacity[0.3]]], 
  BoxRatios -> {1, 1, 1}]

Mathematica graphics

Then you can change perspective?

  GraphicsArray[{{plt, 
  Show[plt, ViewPoint -> Front]}, {Show[plt, ViewPoint -> Top], 
  Show[plt, ViewPoint -> {1, 1, 1}]}}, ImageSize -> 600]

Mathematica graphics

EDIT

Following @whube's suggestion and using @ArgentoSapiens's code we see that the residual have a reasonable distribution:

 (Last /@ actualpredicted[[1]]) - (Last /@ actualpredicted[[2]]) // Histogram

Mathematica graphics

With a relative error of 25 %

 StandardDeviation[(Last /@ actualpredicted[[1]]) - (Last /@ 
 actualpredicted[[2]])]/Mean[(Last /@ actualpredicted[[1]])]


  (* 0.24 *)

If we look at the distribution of residual versus the actual value:

  {(Last /@ 
   actualpredicted[[1]]), (Last /@ actualpredicted[[1]]) - (Last /@ 
   actualpredicted[[2]])} // Transpose // ListPlot

Mathematica graphics

We do find a bias for higher values.

And since I am stealing shamelessly from the other answers, a mixture of them all yields

  plt2 = With[{actualpredicted = {data, 
  Transpose[
  Append[Transpose[fit["Data"][[All, {1, 2}]]], 
   fit["PredictedResponse"]]]}}, 
  Show[ListPointPlot3D[actualpredicted, ColorFunction -> "Rainbow"], 
  plt, Graphics3D[{Opacity[0.25], 
  Line /@ Transpose[actualpredicted]}], BoxRatios -> {1, 1, 1}]]

  GraphicsGrid[{{plt2, 
  Show[plt2, ViewPoint -> Front]}, {Show[plt2, ViewPoint -> Top], 
  Show[plt2, ViewPoint -> {1, 1, 1}]}}, ImageSize -> 600] 

Mathematica graphics

share|improve this answer
    
Thanks a lot. With the plot where you see the bias you might be on to something. The data is actually a histogram and the bias might be binning related. –  jenson Dec 28 '12 at 20:39
    
I checked the data again. The "line" in your "distribution of residual versus the actual value" plot appears cause you plot the actual data point. If the data point is very large, the deviation from the fitted function is also large. That seems to be the reason for this "line" in the plot. Using the predicted value instead, leads to a rather symmetric distribution of the values –  jenson Dec 29 '12 at 0:16

I like to draw the predicted and actual responses and connect them with a little line. That shows where the fit is good and where it isn't.

With[{ actualpredicted={ data, Transpose[ Append[ Transpose[ fit["Data"][[All,{1,2}]]],
                                                  fit["PredictedResponse"] ] ] } },
     Show[ ListPointPlot3D[actualpredicted, PlotStyle -> {Red, Gray}],
           Graphics3D[{Opacity[0.25], Line /@ Transpose[actualpredicted]}]
         ]
    ]

enter image description here

share|improve this answer
1  
+1 We should merge our answers for best effect ;-) –  chris Dec 28 '12 at 17:29

Edit: actually this is very redundant with chris' answer. I keep it for the moment though.

I like to style the data points according to offset from the fit:

Show[{
 Plot3D[fit[x, y], {x, 0, 175}, {y, 0., 0.1},     
        ColorFunction -> "DarkRainbow", 
        PlotStyle -> None, 
        Mesh -> 10],   
 ListPointPlot3D[Partition[data, 1], 
                 Ticks -> Automatic, 
                 Axes -> True,  
                 FillingStyle -> Thin,     
                 Filling ->Thread[Range[1, Length[data]] -> 
                      Table[fit[Sequence @@ data[[i, {1, 2}]]], 
                            {i, 1, Length[data]}]],
                 ColorFunction -> Function[{x, y, z}, 
                 Blend[{{-.0004, Darker@Red}, {-.0002, Red}, {0, Green}, {.0002, 
                 Blue}, {.0004, Lighter@Blue}}, z - fit[x, y]]],
                 ColorFunctionScaling -> False, 
                 BoxRatios -> Automatic,     
                 FillingStyle -> Directive[LightGreen, Thick]]
      }]

Mathematica graphics

share|improve this answer
    
this is not redundant and quite useful as it shows systematic bias in 2D. –  chris Dec 28 '12 at 21:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.