Distorting image to ensure consistent vanishing point

Question

Here's Giorgio de Chirico's painting, Ariadne:

The Surrealist artist deliberately broke the rules of geometric perspective, giving the work his characteristic sense of mystery and unease.

What would the work look like if the perspective were instead consistent? Specifically, here are potential perspective lines (in red):

Here are the coordinates of the lines:

Show[ariadne, 
 Graphics[{Red, Thickness[0.01],
   Line[{{280, 410}, {350, 450}}],
   Line[{{530, 450}, {310, 400}}],
   Line[{{280, 300}, {600, 20}}],
   Line[{{300, 1}, {140, 280}}],
   Line[{{130, 1}, {60, 250}}],
   Line[{{250, 1}, {140, 250}}]
   }]]

What I'm seeking is a global distortion of the painting such that the red perspective lines (extended) remain straight yet meet at a single vanishing point. (The ideal is that the user can specify the vanishing point's two-dimensional location, such as somewhere between the train and the white tower.)

While GeometricTransformations (such as a homography) can distort a single image patch, I've found it rather tricky to automatically distort the entire image to ensure a single vanishing point because the distortion is different in different locations.

I suspect the best approach is to split the image into wedges, between successive (red) perspective lines, then within each wedge transform the image to force the straight boundaries (defined by the perspective lines) to meet at the specified vanishing point. Thus the transformation will be different within each such wedge, yet the perspective lines will remain straight (as required).

(It would be very cool to create a Manipulate which allowed the user to drag the central vanishing point around the plane, but this is not part of this question/request.)

Any suggestions?

Here's one approach that might work. Create consistent perspective lines on the target painting, all meeting at a single vanishing point. (Doing this by hand would be acceptable.) Then create a list of source points (on the original painting) with the corresponding points on the target painting, of the form $(x_i, y_i) \to (x_i^\prime , y_i^\prime )$. So you could make 10 points on each line with 10 points on each target line.

Then, learn a two-dimensional interpolation function, $f(x,y) = (x^\prime, y^\prime)$. Implement this with ImageTransformation. Finally, apply this interpolation function to all points in the original painting.

It would be important to find the simplest function $f$ that maps the data points (or increase the total number of such points selected from lines, or create more perspective lines) in order to ensure that the mapping is one-to-one (i.e., the target images doesn't "fold over" upon itself).

Maybe this will work....

Answer 1

Not sure this gives what you need, but it might be a starting point.

We pick two of the lines in your list and two reference points on each line (the intersections of the lines with two vertical infinite lines). We create two additional points using the intersections of two infinite lines (formed by a locator and the starting points on the reference lines) with one of the vertical lines. We use the two sets of 4 points to construct a transformation function and use it with ImagePerspectiveTransformation:

l1 = Line[{{530, 450}, {310, 400}}];
l2 = Line[{{600, 20}, {280, 300}}];
infl1 = InfiniteLine[{{530, 1}, {530, 450}}];
infl2 = InfiniteLine[{{500, 1}, {500, 450}}];
fourcoords = RegionIntersection[##][[1, 1]] & @@@ Tuples[{{infl1, infl2}, {l1, l2}}];

start = {l1[[1, 1]], RegionIntersection[infl1, l2][[1, 1]]};

tF[pt_] := FindGeometricTransform[N @ Join[start, 
      RegionIntersection[infl2, Line[{#, pt}]][[1, 1]] & /@ start], 
    N @ fourcoords, "Transformation" -> "Perspective"][[2]];


DynamicModule[{pt = ImageDimensions[ariadne] {1/3, 1}}, 
 Panel@Row[{LocatorPane[Dynamic[pt], 
     Dynamic @ Show[ariadne, 
       Graphics[{Green, PointSize @ Large, Point @ fourcoords, 
          Thick, Red, l1, l2, Orange, 
         RegionIntersection[infl2, Line[{start[[1]], pt}]], 
         RegionIntersection[infl2, Line[{start[[2]], pt}]], 
         Blue, Line /@ ({#, pt} & /@ start)}], 
       ImageSize -> 500, 
       PlotRange -> {{1, ImageDimensions[ariadne][[1]]}, 
           {1, 3/2 ImageDimensions[ariadne][[2]]}}]], 
    Dynamic @ Pane[ImagePerspectiveTransformation[ariadne, tF[pt], 
       DataRange -> Full, PlotRange -> All], ImageSize -> 500]}, 
   Spacer[10]]]

Answer 2

Here's how I (sort of) solved this problem. I took each source line segment (specified by the user) and computationally tipped it around its midpoint so that the line (extended) passed through a user-specified vanishing point. Then, I created $n$ surrogate or interpolated points along each source line and each corresponding (tipped) target line. So if I have (say) five source lines, I have 50 source points and 50 corresponding target points. (The only points that are identical in these two sets are the mid-points of the corresponding source and target lines, because I pivoted the lines around their midpoint.)

Then I used a slight modification of Markus van Almsick's radial basis function interpolation routines (described here), used in the 2D-to-2D mapping case. I mapped each source point to its corresponding target point. The rest of the image is interpolated.

Here's what I got using seven lines in the de Chirico painting:

Note that the mapped red lines in the original problem indeed, when extended, all meet at a (single) vanishing point I specified by hand, as desired.

The curved warping of the archway at the rear is due to the fact that I used one long red source line alone the base and a short red source line along the roof line. When the roof line is tipped (about its center), there is a resulting distortion.

I'm pretty sure I can fix that by tipping each line about a point proportional to the distance to the vanishing point (rather than its center), or extending short lines "outside" the painting to make them as long as the companion lines.

There are also several parameters/methods that can be adjusted in the interpolation routine, which I'll try.

I think I can also map other straight lines to straight lines, even if they are not perspective lines, such as on the white tower.

Anyway... this works well enough as a start!