How to remap a fisheye image?

Question

I want to flatten a series of fisheye images by remapping them to a rectinlinear projection.

To achieve this, I need to be able to remap the pixels of the image using fisheye correction formulas for the x-, and y-coordinates. How can I achieve this? I have found this question and this question but I wonder how to use this for fisheye correction.

So far I've tried to use ImageTransformation for this, but I can't get the function to work properly.

  f[pt_] := With[{s = {.5, .5}},
  Module[{rd, polarcoor, ru, newcoor},
   rd = Norm[pt - s]^2/Norm[s];
   polarcoor = 
    CoordinateTransform[{"Cartesian" -> "Polar", 2}, (pt - s)];
   ru = 1*Tan[2*ArcSin[((polarcoor[[1]])/(2*1))]];
   CoordinateTransform["Polar" -> "Cartesian", {ru, polarcoor[[2]]}]
   ]
  ]
ImageTransformation[image,f]

This should first translate the image coordinates to polar coordinates, then calculate the new r (ru=r undistorted), and than translate these back to cartesian coordinates. The ru = 1*Tan[2*ArcSin[((polarcoor[[1]])/(2*1))]] is based on the above links, with a random value chosen as f. It fi

I get an error message, saying the function doesn't map. Anyone have an idea how to fix this and any further suggestions on how to improve the code?

Update 2 22-10-2013 Changed the code to:

image = Import["https://i.stack.imgur.com/JDX9f.jpg"];
r[pt_] := Module[{rd, ru, polarcoor, a},
  rd = Norm[pt];
  ru = *Transformation formula* 
  a = ArcTan @@ (pt);
  ru {Cos[a], Sin[a]}
  ]

Using ru = Sqrt[rd]; ImageTransformation[image, r, DataRange -> {{-1, 1}, {-1, 1}}] gives

Using ru = ArcTan[rd]; ImageTransformation[image, r, DataRange -> {{-2, 2}, {-2, 2}}] gives

Both look like a step in the right direction, with straightened lines, but I got them by trial-and-error so I don't know the correctness.

Answer 1

It seems to me that your radial remapping is working okay, but the original image appears to have an asymmetric distortion, in that the vertical lines of the buildings converge towards a point which is slightly up from the centre of the image. I think it will be necessary to correct for that before doing the radial remapping. Here's a simple attempt to do the correction.

image = Import["https://i.stack.imgur.com/JDX9f.jpg"] ~ImagePad~ -7;

f[c_][{x_, y_}] := Module[{θ, ϕ},
  ϕ = ArcTan[x, y];
  θ = π/2 Sqrt[x^2 + y^2];
  c (1 - Sin[θ]) + 2 θ/π {Cos[ϕ], Sin[ϕ]}]

ImageTransformation[image, f[{0, 0.2}], 
 DataRange -> {{-1, 1}, {-1, 1}}, PlotRange -> {{-1, 1}, {-1, 1}}]

I think this is an improvement, though far from perfect. Of course it all comes down to knowing the form of the distortion.

Original answer

I misunderstood the aim of the question and provided a transformation which unrolls the fisheye image into a panorama. I will leave it here as it attracted a good number of upvotes so I assume it is of interest.

f[x0_, y0_][{th_, r_}] := (1 - 2 r/Pi) ({Sin[th], Cos[th]} - {x0, y0}) + {x0, y0}

ImageTransformation[image, f[0, 0.2], 
 DataRange -> {{-1, 1}, {-1, 1}}, PlotRange -> {{0, 2 Pi}, {0, Pi/2}}]

Answer 2

"Fisheye projection" can mean multiple different things. The most common projection used by lenses is the equal-area one, for which you were using a formula:

$$ r = 2f\sin(\theta/2) $$

Here $\theta$ is the angle under which a point is visible and $r$ is the distance of that point's projection from the optical axis on the image plane. Please see the link above for a sketch of the coordinate system. The parameter $f$ is the focal length of the lens, but for our purposes it just controls the radius of the 180-degree circle on the image plane.

The image you posted doesn't look like a true fisheye photo. It looks like it is a stitched panorama. You can see the stitching artefacts. Thus there is a chance that it doesn't use equal area projection. For simplicity I'm going to assume that it uses an equi-angular projection, i.e. $r = f\theta$.

To remap the image to a rectilinear projection, let us first choose a coordinate system as follows:

• put the origin in the middle
• let the half-width of the image be 1

We achieve this with DataRange -> {{-1, 1}, {-1, 1}} in ImageTransformation.

Choose a transformation function:

trafo = Function[{theta}, 2 Sin[theta/2]]; (* equal area *)

trafo = Function[{theta}, theta]; (* equi-angular *)

Let us set f now. If the 180-degree image circle filled out the frame, it would be

f = 1/trafo[Pi/2];

But it's clear from looking at the image that the field of view is larger than 180 degrees. We can see the ground in all directions. (BTW this is also a strong indicator that this is not a real fisheye image, as greater than 180-degree FoV lenses are rare.) So the radius of a 180-degree circle is less than 1. With some experimentation I found that 0.75 gives good results.

f = 0.75/trafo[Pi/2];

Load the image:

fishImg = Import["https://i.stack.imgur.com/JDX9f.jpg"]

f = 0.75/trafo[Pi/2];

Transform:

ImageTransformation[
 fishImg,
 Function[{px},
  With[{r = Norm[px]},
   trafo[ArcTan[r/f]] f/r px
   ]
  ],
 DataRange -> {{-1, 1}, {-1, 1}},
 PlotRange -> 2 {{-1, 1}, {-1, 1}}
 ]

The crop of the resulting image is set using PlotRange.

You can experiment with other projection functions too and see if you'll get better results. There won't be a large difference though.