# Data extraction from a picture of a graph

How can I extract data from this picture of a graph?

i = Import["https://i.sstatic.net/Ac8m0.png"];


The caption of the picture reads: " Two-dimensional histogram values measured . Tick labels on the color bar are bin counts (17 force bins, 32 lifetime bins, and n = 803 observations)"

Plan of Attack

I divided the image into two parts: 1st the main graph and 2nd the legend. I am trying to extract average pixel in the each block but I am not able to do so becuase the edges between the boxes are not well defined... so, I am trying to define the edges, which is kinda hard to do accurately.

I am trying to use ComponentMeasurements and MorphologicalComponents. Before using those two tools, I need to have well-defined boxes.

Do you guys have better algorithm to extract data?

UPDATE

@SimonWoods did a great job of extracting all these data. Now, I am trying to make sense out of this data. Using the data from legend, I made a list:

m = {{0, keycols[[1]]}, {1, keycols[[2]]}, {2, keycols[[3]]}, {3, keycols[[4]]}, {4, keycols[[5]]}, {5, keycols[[6]]}, {6, keycols[[7]]}, {7, keycols[[8]]}, {8, keycols[[9]]}, {9, keycols[[10]]}, {10, keycols[[11]]}, {11, keycols[[12]]}, {12, keycols[[13]]}, {13, keycols[[14]]}, {14, keycols[[15]]}, {35, keycols[[19]]}, {96, keycols[[24]]}}

Then, I tried to interpolate:

ip = Interpolation[m, Method -> "Spline", InterpolationOrder -> 1] (I don't know if assuming it's linear is correct or not but if the total count is correct then I can be sure if not then I can try different interpolation order)

and I got this:

However, what I want is a function that takes in the RGB and gives count. So, I want inverse of this interpolation, which I couldn't get.

• Why do you need to go to the edges of the blocks? The sampling is uniform - can't you only consider pixels in the middle of the blocks? Commented Jun 29, 2016 at 21:00
• @SimonWoods I think I can do that but since I am new to this and doing image processing first time, I was trying to follow: ComponentMeasurements, where you can see that well-defined edges helps a lot.
– sra
Commented Jun 29, 2016 at 21:03
• This looks like a MATLAB plot, shown with the (old) default colormap. If so, I think that the scale may be linear in Hue. Commented Jun 29, 2016 at 21:23
• @mikado I do not know how they made this graph. I would also guess it as a colormap. I grabbed it from a Science paper and I want to extract data from this graph to do some analysis. I really do not understand their scaling. They do not do a good job describing it in the paper
– sra
Commented Jun 29, 2016 at 21:27
• Have you considered asking the authors for their data? Commented Jun 30, 2016 at 6:01

This is not a complete solution, but might get you on the way.

With a little bit of trial and error you can identify the coordinates of the blocks and the key in the image:

img = Import["https://i.sstatic.net/Ac8m0.png"];

pts = Table[{x, y}, {x, 65, 598, 33}, {y, 64, 810, 24}];
key = Table[{697, y}, {y, 60, 810, 30}];

HighlightImage[img, {Flatten[pts, 1], ImageMarker[key, "Circle"]}]


Sample the image at the "key" coordinates to get the key colours:

keycols = ImageValue[img, key];

Row[RGBColor /@ keycols]


You can now sample the colours of the blocks and map the results into the key index using Nearest:

nf = Nearest[keycols -> Automatic];

data = Map[First[nf@ImageValue[img, #]] &, pts, {2}];


data is an array of values from 1 (white) to 26 (dark red). To check it, we can reconstruct the original blocks from the key colours:

Graphics @ Raster[Map[keycols[[#]] &, Transpose[data], {2}]]


What remains is to convert the key colour indices (1 to 26) into actual values (0 to 100?) taking into account the non-linear scale.

• For reference: here is a Mathematica implementation of the jet() colormap. Commented Jun 30, 2016 at 0:37
• According to the OP the figure caption reads " ... n = 803 observations". Since the figure is a DensityHistogram, this total number of observations can be used to calibrate the non-linear color scaling. Commented Jul 2, 2016 at 6:46

For the legend at the bottom, the traditional method of identifying color boundaries is the Fast Fourier Transform (FFT): the resulting frequencies will be large where the color is changing quickly -- the boundaries. Obviously, FFT will not work on the far right-hand side, but then there are no observations there either. After applying the FFT, choose a frequency cutoff value that selects the number of boundaries most closely approximating the real number.