# Distorting image to ensure consistent vanishing point

Here's Giorgio de Chirico's painting, Ariadne:

The Surrealist artist deliberately broke the rules of geometric perspective, giving the work his characteristic sense of mystery and unease.

What would the work look like if the perspective were instead consistent? Specifically, here are potential perspective lines (in red):

Here are the coordinates of the lines:

Show[ariadne,
Graphics[{Red, Thickness[0.01],
Line[{{280, 410}, {350, 450}}],
Line[{{530, 450}, {310, 400}}],
Line[{{280, 300}, {600, 20}}],
Line[{{300, 1}, {140, 280}}],
Line[{{130, 1}, {60, 250}}],
Line[{{250, 1}, {140, 250}}]
}]]


What I'm seeking is a global distortion of the painting such that the red perspective lines (extended) remain straight yet meet at a single vanishing point. (The ideal is that the user can specify the vanishing point's two-dimensional location, such as somewhere between the train and the white tower.)

While GeometricTransformations (such as a homography) can distort a single image patch, I've found it rather tricky to automatically distort the entire image to ensure a single vanishing point because the distortion is different in different locations.

I suspect the best approach is to split the image into wedges, between successive (red) perspective lines, then within each wedge transform the image to force the straight boundaries (defined by the perspective lines) to meet at the specified vanishing point. Thus the transformation will be different within each such wedge, yet the perspective lines will remain straight (as required).

(It would be very cool to create a Manipulate which allowed the user to drag the central vanishing point around the plane, but this is not part of this question/request.)

Any suggestions?

Here's one approach that might work. Create consistent perspective lines on the target painting, all meeting at a single vanishing point. (Doing this by hand would be acceptable.) Then create a list of source points (on the original painting) with the corresponding points on the target painting, of the form $$(x_i, y_i) \to (x_i^\prime , y_i^\prime )$$. So you could make 10 points on each line with 10 points on each target line.

Then, learn a two-dimensional interpolation function, $$f(x,y) = (x^\prime, y^\prime)$$. Implement this with ImageTransformation. Finally, apply this interpolation function to all points in the original painting.

It would be important to find the simplest function $$f$$ that maps the data points (or increase the total number of such points selected from lines, or create more perspective lines) in order to ensure that the mapping is one-to-one (i.e., the target images doesn't "fold over" upon itself).

Maybe this will work....

• This is a fascinating idea. Unfortunately I don’t have the first clue how to go about it, but I very much look forward to the result. – MarcoB Jan 25 at 3:16
• There's a great community post about using seam carving to rescale images, I wonder if you could use a similar method here... community.wolfram.com/groups/-/m/t/960843 – Carl Lange Jan 25 at 10:13
• @CarlLange: Thanks for the link to scene carving. (Two decades ago my research lab patented just such a procedure that carved in two dimensions.). But alas this is of little help... it is the distortion that is of relevance, and I don't want to "lose" any portion of the image. – David G. Stork Jan 25 at 19:24

Not sure this gives what you need, but it might be a starting point.

We pick two of the lines in your list and two reference points on each line (the intersections of the lines with two vertical infinite lines). We create two additional points using the intersections of two infinite lines (formed by a locator and the starting points on the reference lines) with one of the vertical lines. We use the two sets of 4 points to construct a transformation function and use it with ImagePerspectiveTransformation:

l1 = Line[{{530, 450}, {310, 400}}];
l2 = Line[{{600, 20}, {280, 300}}];
infl1 = InfiniteLine[{{530, 1}, {530, 450}}];
infl2 = InfiniteLine[{{500, 1}, {500, 450}}];
fourcoords = RegionIntersection[##][[1, 1]] & @@@ Tuples[{{infl1, infl2}, {l1, l2}}];

start = {l1[[1, 1]], RegionIntersection[infl1, l2][[1, 1]]};

tF[pt_] := FindGeometricTransform[N @ Join[start,
RegionIntersection[infl2, Line[{#, pt}]][[1, 1]] & /@ start],
N @ fourcoords, "Transformation" -> "Perspective"][[2]];

Panel@Row[{LocatorPane[Dynamic[pt],
Graphics[{Green, PointSize @ Large, Point @ fourcoords,
Thick, Red, l1, l2, Orange,
RegionIntersection[infl2, Line[{start[[1]], pt}]],
RegionIntersection[infl2, Line[{start[[2]], pt}]],
Blue, Line /@ ({#, pt} & /@ start)}],
ImageSize -> 500,
DataRange -> Full, PlotRange -> All], ImageSize -> 500]},
Spacer[10]]]


• A start... but I had accomplished this (in essence). The real challenge is that there are several set of perspective lines, each defining a different transform. I'll keep working on this. – David G. Stork Jan 25 at 4:19
• Here are some ideas about your problem: Your transformation should map lines onto lines, what restricts possible transformations a lot. As far as I know, only linear and the more general bilinear transformations map lines onto lines. However, both families of transformation to not map more than one point onto a point. Bute the original image has perspective lines that meet at different points. Therefore there is NO transformation that preserves lines and maps these points into one. – Daniel Huber Jan 31 at 17:59
• @DanielHuber: Affine transformations map lines to lines. Of course no single, linear transform, applied to the entire painting, can work. That's why the mapping must be piece-by-piece. Imagine the painting were on stretchable rubber. We could surely place a tack through a desired vanishing point, and then put tacks along each red line such that the red lines (extended) pass through the vanishing point. True, not all straight lines in the painting would (after warping) remain straight, but that's fine. I need only the selected red ones to remain straight. – David G. Stork Jan 31 at 19:12
• Möbius or bilinear transformation constitute a group. That is if you apply one after the other, the double transformation is still bilinear, that is, it can not map several crossing points to one. – Daniel Huber Jan 31 at 20:00
• You are still caught up in thinking that there is a single mapping applied to the entire painting, rather than piecemeal, or of fundamentally different forms in different regions. I can certainly apply tow DIFFERENT affine transforms to two different patches and force their separate vanishing points to overlap. Agree? – David G. Stork Feb 1 at 6:22

Here's how I (sort of) solved this problem. I took each source line segment (specified by the user) and computationally tipped it around its midpoint so that the line (extended) passed through a user-specified vanishing point. Then, I created $$n$$ surrogate or interpolated points along each source line and each corresponding (tipped) target line. So if I have (say) five source lines, I have 50 source points and 50 corresponding target points. (The only points that are identical in these two sets are the mid-points of the corresponding source and target lines, because I pivoted the lines around their midpoint.)

Then I used a slight modification of Markus van Almsick's radial basis function interpolation routines (described here), used in the 2D-to-2D mapping case. I mapped each source point to its corresponding target point. The rest of the image is interpolated.

Here's what I got using seven lines in the de Chirico painting:

Note that the mapped red lines in the original problem indeed, when extended, all meet at a (single) vanishing point I specified by hand, as desired.

The curved warping of the archway at the rear is due to the fact that I used one long red source line alone the base and a short red source line along the roof line. When the roof line is tipped (about its center), there is a resulting distortion.

I'm pretty sure I can fix that by tipping each line about a point proportional to the distance to the vanishing point (rather than its center), or extending short lines "outside" the painting to make them as long as the companion lines.

There are also several parameters/methods that can be adjusted in the interpolation routine, which I'll try.

I think I can also map other straight lines to straight lines, even if they are not perspective lines, such as on the white tower.

Anyway... this works well enough as a start!

• Do you think you could include your code? – Carl Lange Feb 2 at 21:33
• @CarlLange: Sorry, it involves perhaps $200$ lines of van Almsick's interpolation code, perhaps $100$ lines of my line tipping and subdividing code, and most certainly is not yet ready for distribution. I may work on the code for a bit and write up something for art scholars (my target audience), in which case I'd make the code available (assuming van Almsick's permission). – David G. Stork Feb 2 at 21:36
• No worries. Perhaps in that case you could link source material for the referenced works. I would be interested to learn some more of the mechanics. – Carl Lange Feb 2 at 21:51
• Link to van Almsick's talk now included in the solution text. – David G. Stork Feb 2 at 21:53
• Thank you very much! – Carl Lange Feb 2 at 22:14