1
$\begingroup$

I have a list of points {{t1-0,t2-t1}....} which are differences of time of a repeated experiment which gives t1's and t2's times. I have plotted the data to see if they have some correlation and gives me the following plot:

enter image description here

It seems they are correlated. I would like to plot these data colored using a color indicating the concentration of points. For example, from more concentration to less concentration, dark-red to dark-blue (similar to "Rainbow" color style, and a bar legend at the right side. I have been testing several options and web pages, but I can not figure out how to make it. I would rather like to ask, if possible, how to remove the frame and the ticks but leave the axes numbers.

Thank you very much for your help

|improve this question|||||
$\endgroup$
  • $\begingroup$ Have you seen SmoothDensityHistogram[]? $\endgroup$ – J. M.'s technical difficulties♦ Aug 6 '17 at 18:33
  • $\begingroup$ It's not obvious to me that there is a non-zero correlation. Consider the plot of two independent samples (where by definition there is no correlation): x1 = RandomVariate[LogNormalDistribution[3.5, 1], 50000]; x2 = RandomVariate[LogNormalDistribution[3.5, 1], 50000]; ListPlot[Transpose[{x1, x2}], PlotRange -> {{0, 250}, {0, 250}}, AspectRatio -> 1, AxesStyle -> White, TicksStyle -> Black]. $\endgroup$ – JimB Aug 6 '17 at 18:40
  • $\begingroup$ The plot described in my earlier comment will look a lot like yours. As far as exploring whether there is a non-zero correlation you might want to try a plot of the square-roots (or some other fractional power close to zero) on both variables to get the majority of the points in the middle of the figure. Alternatively, you could take slices of points say between 15 and 20 for x1 and between 100 and 105 for x1 and compare the resulting histograms for x2. If they differ much in any observed way, then further investigation of the correlation structure is warranted. $\endgroup$ – JimB Aug 6 '17 at 18:47
  • $\begingroup$ @J.M. Thank you very much SmoothDensityHistogram[] function is what I was looking for. $\endgroup$ – user1993416 Aug 6 '17 at 19:56
  • $\begingroup$ @JimBaldwin Thank you very much for your help. I would ask you to add something else to your second comment. It seems it would be better to manipulate the data before the representation. Would you explain something else about the suggested slices or give an example? Thank you very much $\endgroup$ – user1993416 Aug 6 '17 at 20:02
4
$\begingroup$

An extended comment...

The word "correlated" is many times used to suggest some sort of statistical dependence of one variable on another (especially outside the field of Statistics). But there are many forms of "statistical dependence" so no summary statistic like a correlation coefficient always gives an adequate picture of what's going on. Hence, one looks at a variety of forms of dependence. Fortunately you seem to have lots of data which gives you a lot of leeway (but also the potential for more complication).

One aspect of "statistical dependence" is that if there is some sort of dependence, then the distribution of values for one variable will depend on the values of another variable. So with lots of data we could look at a few histograms where the range of one of the variables is restricted. Here are two such slices:

(* Generate some data *)
data = Transpose[{RandomVariate[LogNormalDistribution[3.5, 1], 50000],
    RandomVariate[LogNormalDistribution[3.5, 1], 50000]}];

(* Slice 1 *)
data1 = Select[data, 15 < #[[1]] < 20 &][[All, 2]];
Histogram[data1, {10}, "PDF", PlotRange -> {{0, 250}, {0, 0.02}}]

Histogram 1

(* Slice 2 *)
data2 = Select[data, 100 < #[[1]] < 105 &][[All, 2]];
Histogram[data2, {10}, "PDF", PlotRange -> {{0, 250}, {0, 0.02}}]

Histogram 2

The first slice is centered around the horizontal variable being around 17.5 and the second slice is centered around 102.5. I don't see very much difference between the two histograms (displayed as PDF's). (Of course, the two variables generated were by definition independent so the histograms ought to look very similar.)

An alternative is to use some sort of Generalized Additive Model (gam) (which is a nonparametric regression approach). While some folks have created their own gam packages in Mathematica, there is no built-in Mathematica function to do so (yet?). A very, very crude approximation is to take slices across one of the variables and calculate the means of the other variable for each slice. Then plot the means with the data in the background.

bw = 10  (* bandwidth *)
means = Table[{x, 
    Mean[Select[data, x - bw/2 < #[[1]] < x + bw/2 &][[All, 2]]]},
 {x, bw/2, 250, bw}];
Show[ListPlot[data],
     ListPlot[means, Joined -> True, PlotStyle -> Red]]

Moving windows of means

Here, too, there does not seem to be evidence of the vertical variable statistically dependent on the horizontal variable.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thank you very much for your explanation. I can see how with the data generated are independent. If they were correlated the means should plot a line with some slope. I will test this computation with my data. Thank you! $\endgroup$ – user1993416 Aug 6 '17 at 22:39
  • $\begingroup$ My data is independent too. Thanks $\endgroup$ – user1993416 Aug 6 '17 at 22:53
  • 1
    $\begingroup$ If you haven't done so already, I would run the above displays on the "transformed" data (square root, 1/10th power, etc.) and to have something more solid I'd run the gam function in the mgcv package in R. The above displays are just to get a "feel" for what the dependence (if any) might be. $\endgroup$ – JimB Aug 6 '17 at 23:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.