The $over$ operator correctly implemented
According to the Wikipedia, when composing $A$ $over$ $B$, the output alpha channel value $\alpha_O$ and the output color channel value $C_O$ are calculated as follows:
$\begin{cases}\alpha_O = 1 - (1 - \alpha_A) (1 - \alpha_B)
\\C_O = \frac{\alpha_A C_A + (1 - \alpha_A)\alpha_B C_B}{\alpha_O}, \text{if $\alpha_O \neq 0$}
\\C_O = 0, \text{if $\alpha_O = 0$}
\end{cases}$
where $\alpha_A$ and $\alpha_B$ are alpha channel values of $A$ and $B$, and $C_A$ and $C_B$ – color channel values of $A$ and $B$ correspondingly.
This can be directly implemented in Mathematica 11.1 or above as follows:
imageCompose[b_Image, a_Image] :=
Module[{alphaA = AlphaChannel@a, alphaB = AlphaChannel@b, alphaO,
cA = RemoveAlphaChannel@a, cB = RemoveAlphaChannel@b},
alphaO = 1 - (1 - alphaA) (1 - alphaB);
SetAlphaChannel[(alphaA*cA + (1 - alphaA) alphaB*cB)/alphaO, alphaO]]
Let us check the associative property:
{{i0, i1, i2}} = ImagePartition[
Import["https://i.sstatic.net/r13gh.png"], {Scaled[1/3], Scaled[1]}]
{i0~imageCompose~(i1~imageCompose~i2), (i0~imageCompose~i1)~imageCompose~i2}
Equal @@ %
ColorSeparate /@ %%
True
It holds! So what is the problem with ImageCompose of version 10 and later?
Current implementation of ImageCompose: the diagnosis
When writing the above implementation for the first time I unintentionally made a simple mistake: I forgot to divide the output value for the color channel by $\alpha_O$. Here is what happened:
imageComposeWrong[b_Image, a_Image] :=
Module[{alphaA = AlphaChannel@a, alphaB = AlphaChannel@b, alphaO,
cA = RemoveAlphaChannel@a, cB = RemoveAlphaChannel@b},
alphaO = 1 - (1 - alphaA) (1 - alphaB);
SetAlphaChannel[alphaA*cA + (1 - alphaA) alphaB*cB, alphaO]]
{i0~imageComposeWrong~(i1~imageComposeWrong~i2),
(i0~imageComposeWrong~i1)~imageComposeWrong~i2}
Equal @@ %
ColorSeparate /@ %%
False
The output looks exactly the same as for the current ImageCompose:
{i0~ImageCompose~(i1~ImageCompose~i2), (i0~ImageCompose~i1)~ImageCompose~i2}
Equal @@ %
ColorSeparate /@ %%
False
Numerical comparison reveals tiny differences due to rounding off errors. But the final diagnosis is clear: the developer just forgot to divide the color channel by the alpha channel!
It is a great shame that during more than three years after the release of version 10.0.0 nobody noticed this in the company! Do they themselves use this functionality – or not?!
Please, do not be lazy to report this to the technical support, so that this shameful bug will be fixed as soon as possible! A high priority is given to bugs, which many users write about...
The remedy
From the above considerations the remedy is obvious: we must just divide the color of the ImageCompose output by the alpha channel:
icFix[img_Image] := img/AlphaChannel[img];
{i0~icFix@*ImageCompose~(i1~icFix@*ImageCompose~i2), (i0~icFix@*ImageCompose~i1)~icFix@*ImageCompose~i2}
Subtract @@ % // MinMax
ColorSeparate /@ %%
{-3.10689*10^-6, 3.08454*10^-6}
As one can see, there are still tiny differences due to rounding-off errors, but the associative property in fact is restored and the output is correct!
Original answer
Citing a comment by Rahul:
Well, that's certainly undesirable! Alpha compositing is supposed to be associative (i0~ImageCompose~(i1~ImageCompose~i2) should equal (i0~ImageCompose~i1)~ImageCompose~i2) and this doesn't do that. One could implement correct alpha compositing manually using ImageApply, but let's see if someone has a better way.
Indeed, in versions 8.0.4 and 9.0.1 ImageCompose is associative:
$Version
{{i0, i1, i2}} = ImagePartition[
Import["https://i.sstatic.net/r13gh.png"], {Scaled[1/3], Scaled[1]}]
{i0~ImageCompose~(i1~ImageCompose~i2), (i0~ImageCompose~i1)~ImageCompose~i2}
Equal @@ %
ColorSeparate /@ %%
"9.0 for Microsoft Windows (64-bit) (January 25, 2013)"
True
... while starting from version 10.0 it is not:
$Version
{{i0, i1, i2}} = ImagePartition[
Import["https://i.sstatic.net/r13gh.png"], {Scaled[1/3], Scaled[1]}]
{i0~ImageCompose~(i1~ImageCompose~i2), (i0~ImageCompose~i1)~ImageCompose~i2}
Equal @@ %
ColorSeparate /@ %%
"10.0 for Microsoft Windows (64-bit) (September 9, 2014)"
False
It is also worth to note that despite that Overlay[{i0, i1}] and Show[i0, i1] look the same as the old ImageCompose[i0, i1], they are not the same. But they do (approximately) equal to each other and do approximately hold the associative property:
$Version
overlayCompose[i0_, i1_] := Rasterize[Overlay[{i0, i1}], "Image", Background -> None];
showCompose[i0_, i1_] := Rasterize[Show[i0, i1], "Image", Background -> None];
{overlayCompose[i0, i1], showCompose[i0, i1], i0~ImageCompose~i1}
ColorSeparate /@ %
{i0~overlayCompose~(i1~overlayCompose~i2), (i0~overlayCompose~i1)~ overlayCompose~i2}
ColorSeparate /@ %
{i0~showCompose~(i1~showCompose~i2), (i0~showCompose~i1)~showCompose~ i2}
ColorSeparate /@ %
"9.0 for Microsoft Windows (64-bit) (January 25, 2013)"
As one can see from the above, Overlay introduces artifact at the top, while Show does not.
i0~ImageCompose~(i1~ImageCompose~i2)should equal(i0~ImageCompose~i1)~ImageCompose~i2) and this doesn't do that. One could implement correct alpha compositing manually usingImageApply, but let's see if someone has a better way. $\endgroup$