# How to remap a fisheye image?

I want to flatten a series of fisheye images by remapping them to a rectinlinear projection.

To achieve this, I need to be able to remap the pixels of the image using fisheye correction formulas for the x-, and y-coordinates. How can I achieve this? I have found this question and this question but I wonder how to use this for fisheye correction.

So far I've tried to use ImageTransformation for this, but I can't get the function to work properly.

  f[pt_] := With[{s = {.5, .5}},
Module[{rd, polarcoor, ru, newcoor},
rd = Norm[pt - s]^2/Norm[s];
polarcoor =
CoordinateTransform[{"Cartesian" -> "Polar", 2}, (pt - s)];
ru = 1*Tan[2*ArcSin[((polarcoor[[1]])/(2*1))]];
CoordinateTransform["Polar" -> "Cartesian", {ru, polarcoor[[2]]}]
]
]
ImageTransformation[image,f]


This should first translate the image coordinates to polar coordinates, then calculate the new r (ru=r undistorted), and than translate these back to cartesian coordinates. The ru = 1*Tan[2*ArcSin[((polarcoor[[1]])/(2*1))]] is based on the above links, with a random value chosen as f. It fi

I get an error message, saying the function doesn't map. Anyone have an idea how to fix this and any further suggestions on how to improve the code?

Update 2 22-10-2013 Changed the code to:

image = Import["http://i.stack.imgur.com/JDX9f.jpg"];
r[pt_] := Module[{rd, ru, polarcoor, a},
rd = Norm[pt];
ru = *Transformation formula*
a = ArcTan @@ (pt);
ru {Cos[a], Sin[a]}
]


Using ru = Sqrt[rd]; ImageTransformation[image, r, DataRange -> {{-1, 1}, {-1, 1}}] gives

Using ru = ArcTan[rd]; ImageTransformation[image, r, DataRange -> {{-2, 2}, {-2, 2}}] gives

Both look like a step in the right direction, with straightened lines, but I got them by trial-and-error so I don't know the correctness.

-
your function must always return a position. Your CoordinateTransform[]s are failing for some values ("incompatible with the coordinate assumptions of the specified coordinate chart") – cormullion Oct 21 '13 at 15:39
Seems to work if you evaluate f symbolically, e.g. g[{x_, y_}] = f[{x, y}]; ImageTransformation[image, g] – Simon Woods Oct 21 '13 at 17:32
re your latest update - try restarting your kernel, it doesn't misbehave here. – cormullion Oct 22 '13 at 10:03
@cormullion: Still not working here. Strange. – ErikP Oct 22 '13 at 10:12

It seems to me that your radial remapping is working okay, but the original image appears to have an asymmetric distortion, in that the vertical lines of the buildings converge towards a point which is slightly up from the centre of the image. I think it will be necessary to correct for that before doing the radial remapping. Here's a simple attempt to do the correction.

image = Import["http://i.stack.imgur.com/JDX9f.jpg"] ~ImagePad~ -7;

f[c_][{x_, y_}] := Module[{θ, ϕ},
ϕ = ArcTan[x, y];
θ = π/2 Sqrt[x^2 + y^2];
c (1 - Sin[θ]) + 2 θ/π {Cos[ϕ], Sin[ϕ]}]

ImageTransformation[image, f[{0, 0.2}],
DataRange -> {{-1, 1}, {-1, 1}}, PlotRange -> {{-1, 1}, {-1, 1}}]


I think this is an improvement, though far from perfect. Of course it all comes down to knowing the form of the distortion.

I misunderstood the aim of the question and provided a transformation which unrolls the fisheye image into a panorama. I will leave it here as it attracted a good number of upvotes so I assume it is of interest.

f[x0_, y0_][{th_, r_}] := (1 - 2 r/Pi) ({Sin[th], Cos[th]} - {x0, y0}) + {x0, y0}

ImageTransformation[image, f[0, 0.2],
DataRange -> {{-1, 1}, {-1, 1}}, PlotRange -> {{0, 2 Pi}, {0, Pi/2}}]


-
Different way than I've been thinking (see update): For fish-eye undistortion, the radius from image centre to a pixel must be corrected. So I've tried to find that radius (by getting polar coordinates), apply a function to it and than translate the new polar coordinates back to Cartesian. Not getting it to work so far though. – ErikP Oct 22 '13 at 9:32
I am not looking for a way to make panorama images, which is basically what you have achieved. Looks nice though! – ErikP Oct 22 '13 at 14:10
@ErikP, sorry - I misunderstood the aim of the remapping. This must be the highest number of upvotes I've ever had for an answer which utterly fails to answer the question! – Simon Woods Oct 22 '13 at 19:31
no problem, but do you have any ideas on how to solve the question, or give my current solution a nudge in the right direction? – ErikP Oct 23 '13 at 8:14
@ErikP, I'll have another look. The asymmetry of the distortion makes it more complicated than it otherwise would be... – Simon Woods Oct 23 '13 at 9:06

"Fisheye projection" can mean multiple different things. The most common projection used by lenses is the equal-area one, for which you were using a formula:

$$r = 2f\sin(\theta/2)$$

Here $\theta$ is the angle under which a point is visible and $r$ is the distance of that point's projection from the optical axis on the image plane. Please see the link above for a sketch of the coordinate system. The parameter $f$ is the focal length of the lens, but for our purposes it just controls the radius of the 180-degree circle on the image plane.

The image you posted doesn't look like a true fisheye photo. It looks like it is a stitched panorama. You can see the stitching artefacts. Thus there is a chance that it doesn't use equal area projection. For simplicity I'm going to assume that it uses an equi-angular projection, i.e. $r = f\theta$.

To remap the image to a rectilinear projection, let us first choose a coordinate system as follows:

• put the origin in the middle
• let the half-width of the image be 1

We achieve this with DataRange -> {{-1, 1}, {-1, 1}} in ImageTransformation.

Choose a transformation function:

trafo = Function[{theta}, 2 Sin[theta/2]]; (* equal area *)

trafo = Function[{theta}, theta]; (* equi-angular *)


Let us set f now. If the 180-degree image circle filled out the frame, it would be

f = 1/trafo[Pi/2];


But it's clear from looking at the image that the field of view is larger than 180 degrees. We can see the ground in all directions. (BTW this is also a strong indicator that this is not a real fisheye image, as greater than 180-degree FoV lenses are rare.) So the radius of a 180-degree circle is less than 1. With some experimentation I found that 0.75 gives good results.

f = 0.75/trafo[Pi/2];


fishImg = Import["http://i.stack.imgur.com/JDX9f.jpg"]


f = 0.75/trafo[Pi/2];

Transform:

ImageTransformation[
fishImg,
Function[{px},
With[{r = Norm[px]},
trafo[ArcTan[r/f]] f/r px
]
],
DataRange -> {{-1, 1}, {-1, 1}},
PlotRange -> 2 {{-1, 1}, {-1, 1}}
]


The crop of the resulting image is set using PlotRange.

You can experiment with other projection functions too and see if you'll get better results. There won't be a large difference though.

-