# Understanding Bilinear ImageResize

In trying to understand basic image transformations, I have attempted to code my own image resize function that uses bilinear interpolation. I checked my work against Mma's ImageResize[...,Resampling->"Bilinear"] command and the results are similar, but not the same.

The following minimal example demonstrates how I do not fully understand Mma's algorithm. Let im={{1,.5,0}}, a 3x1 picture of grayscale values. "Common sense" and knowledge of bilinear interpolation tell me that resizing this image to 5x1 should give {{1,.75,.5,.25,0}}. However, ImageData[ImageResize[Image[{{1, .5, 0}}], {5, 1}, Resampling -> "Bilinear"]] returns {{1,.8,.5,.2,0}}.

Resizing to a 6x1 gives {{1., 0.875, 0.625, 0.375, 0.125, 0.}}, where the middle 4 terms are equally spaced at .25 apart, but the first and last two elements are spaced 0.125 apart.

Resizing to a 9x1 gives {{1., 1., 0.833333, 0.666667, 0.5, 0.333333, 0.166667, 0., 0.}}, which further confuses me as the first and last two entries are repeated.

These results are similar to what MATLAB produces, though Octave (I am told) produces what I think should be right - where all entries are equally spaced apart from 1 down to 0. I checked with two image editing programs, and one gave results similar to Mma and one gave results similar, but not exactly the same, to my "common sense."

Can anyone explain how Mathematica is implementing bilinear interpolation?

-

In your "common sense" interpretation, you are thinking of the $3$-pixel image {1, 0.5, 0} as a function $$1\mapsto1, \quad 2\mapsto0.5, \quad 3\mapsto0$$ on the domain $[1,3]$. When you resize to a $5$-pixel image, you map $[1,5]$ to this domain and sample the interpolated function.

Hypothesis: You should instead consider the domain to be $[0,3]$, divided into $3$ unit-size cells, with pixel values placed at the midpoints: $$\tfrac12\mapsto1, \quad 1{\tfrac12}\mapsto0.5, \quad 2{\tfrac12}\mapsto0.$$

This has the advantage that the domain is actually $3$ units long, not $2$. Which means, among other things, that if you were to resize to a $6$-pixel image, image features would become exactly twice as big in terms of pixels.

Anyway, let's try interpolating this to $5$ pixels:

f = Interpolation[{{0, 1}, {0.5, 1}, {1.5, 0.5}, {2.5, 0}, {3, 0}},
InterpolationOrder -> 1] (* extra values to avoid extrapolation *)
x = {0.5, 1.5, 2.5, 3.5, 4.5}
f[3 #/5] & /@ x
(* Out= {1., 0.8, 0.5, 0.2, 0.} *)


Seems to be consistent with what Mathematica gives.

-
That's it! I checked it against various resize values and it works. The extra values at the end to avoid extrapolation are also key, and help explain why the endpoints were treated "differently." Thanks. –  GregH Feb 24 '14 at 19:12
+1 nice, wish I saw this before working it out the hard way! –  george2079 Feb 24 '14 at 21:36
+1, nice concise answer. –  ciao Feb 24 '14 at 22:37

Well,you can find out the coeffs that Mathematica is using. I haven't tried to find a closed formula for general dimensions, but it doesn't seem too hard.

fromdim = 5;
todim = 10;
s = {#,  Chop@ImageData[ ImageResize[Image[{#}], {todim, 1},
Resampling -> "Bilinear"]]} & /@ Tuples[{0., 1.}, fromdim];
coeff = Solve[And @@@ Table[s[[k, 1]].Table[x[i, j], {i, fromdim}] == s[[k, 2, 1, j]],
{j, todim}, {k, Length@s}]];
m = Transpose@ SparseArray[{i_, j_} :> (x[i, j] /. coeff[[1]]), {fromdim, todim}];
m // MatrixForm


i = {1, 0, 1, 0, 1};
Column@{ImageResize[Image[{i}], {todim, 1}, Resampling -> "Bilinear"], Image[{m.i}]}


-
This is interesting. I don't yet see how to get a closed formula. I played around with various values of fromdim and todim and it is clear that the matrix m has a block pattern to it. It isn't clear to me what defines that pattern. –  GregH Feb 24 '14 at 17:27

If you look at a somewhat larger "image" you can see that the end points are being treated specially:

 n = 4
Show[{ Plot[  36(x - 2)/5 + 1/12 , {x, 1, 9}],
ListPlot[
ImageData[
ImageResize[Image[{Range[0, 1, 1/n]}], {2 n + 1, 1},
Resampling -> "Bilinear"]][[1]],
PlotStyle -> {PointSize -> .02}]}, PlotRange -> All]


 n = 2
Show[{ Plot[  3 (x - 2)/10 + 1/5 , {x, 1, 6}],
ListPlot[
ImageData[
ImageResize[Image[{Range[0, 1, 1/n]}], {2 n + 1, 1},
Resampling -> "Bilinear"]][[1]],
PlotStyle -> {PointSize -> .02}]}, PlotRange -> All]


I manually sorted out those formulas and already spend too much time trying to back out the general n expression.

..Since I worked it out here is an explicit formula:

 With[ {n1 = 5, n2 = 11},
{ ImageData[
ImageResize[Image[{#} &@Range[0, 1, 1/(n1 - 1)]], {n2, 1},
Resampling -> "Bilinear"]][[1]],
Max[0,  Min[1, #]] & /@
((2 n1 Range[ 0  , n2 - 1  ] -  n2 +   n1 ) /(2 (n1 - 1) n2)) } ] // N


0., 0.0454545, 0.159091, 0.272727, 0.386364, 0.5, 0.613636, 0.727273, 0.840909, 0.954545, 1.

0., 0.0454545, 0.159091, 0.272727, 0.386364, 0.5, 0.613636, 0.727273, 0.840909, 0.954545, 1.

 With[ {n1 = 3, n2 = 5},
{ ImageData[
ImageResize[Image[{#} &@Range[0, 1, 1/(n1 - 1)]], {n2, 1},
Resampling -> "Bilinear"]][[1]],
Max[0,  Min[1, #]] & /@
((2 n1 Range[ 0  , n2 - 1  ] -  n2 +   n1 ) /(2 (n1 - 1) n2)) } ] // N


0., 0.2, 0.5, 0.8, 1.

0., 0.2, 0.5, 0.8, 1.

-
I agree that the endpoints are being treated differently. I think that is the part of bilinear interpolation that I don't fully understand. I thought I would come up with a formula, then realized the cases where the endpoints are repeated. That throws me off. –  GregH Feb 24 '14 at 17:23
One way to look at this, the image gets scaled slightly larger than you would expect (by a fraction of a pixel), then the end values are held fixed. If you could work out that scale factor the rest of the calulaiton is straightforward. –  george2079 Feb 24 '14 at 17:32