10
$\begingroup$

I like Moku hanga and like changing images into Moku hanga style. (Using this kind of apps.)

I experimented with transforming images into Moku hanga style using the neural networks procedure that is explained here (Community), here (MSE), and the documentation.

I can get good results but very slowly. Here is an example of such effort.

Example

Source

"plain"

Style

"bonzai"

Result

"res"

Result with a dedicated app

jixpixres

Question

What would be a faster way to to do this in Mathematica?

I assume:

  • one way is to use hand made image filters;

  • another is to use algorithms like the one in the project "Image analogies".

$\endgroup$
  • $\begingroup$ On the EWTC 2017 Markus v. Almsick (wolfram.com/events/technology-conference-eu/2017/speakers.html) has presented a new functionality/methodology to copy the "image style" from one image to another. This should be part of the next release of Mathematica. You should also watch out for the conference notebooks in the next weeks. $\endgroup$ – akm Jul 3 '17 at 7:35
16
+500
$\begingroup$

Teaser

Mathematica graphics

I can identify at least 4 steps the dedicated app is performing:

  1. reduction of the number of colours
  2. reduction of the color-saturation
  3. enhancing and blackening of image edges
  4. overlaying a canvas structure

If you read how such Moku Hanga are made, you find that there are several layers carved into wood that are coloured and printed over another. In a final step, the black outline is printed. It seems not uncommon that they used rather vivid colours, so step 2. should be replaced by "artistic recolouring", but for the sake of your beer cans, let's try to imitate what the app did.

Recolouring and colour reduction

Here, we only care about the colours in the image and try to transform them into what looks pleasing by simultaneously reducing their number. There are several ways of doing this and you can look at ColorQuantize or ImageEffect. Let me self-promote my answer here and use the function therein.

What it does is that it replaces colours in an image with a set of predefined colours and chooses the ones that are close together. I mention this function since it can easily be used if you want to artistically recolour the image. Here, I'm going to use a set of continuous colours, but I'm reducing their possible color-range {v, 0.07, .6, .1} (this is the hue of "HSB").

With[{cols = 
   List @@@ 
    Flatten[Table[
      ColorConvert[Hue[v, s, b], "RGB"], {v, 0.07, .6, .1}, {s, 0, 
       1, .1}, {b, 0, 1, .1}]]}, 
 fC = Compile[{{rgb, _Real, 1}}, 
   Module[{min = 100.0, currMin = 100.0, minCol = cols[[1]]}, 
    Do[If[(currMin = Norm[rgb - c]) < min, minCol = c;
      min = currMin], {c, cols}];
    minCol], CompilationTarget -> "C", Parallelization -> True, 
   RuntimeAttributes -> {Listable}]]

I'm going to use this with your original image while slightly adjusting the image

img = Import["https://i.imgur.com/UTSrS25l.jpg"];
{h, s, b} = ColorSeparate[img, "HSB"];

col = Image[fC@ImageData[ImageAdjust[img, {-.2, .3}]]]

Mathematica graphics

That looks not bad, but there are some ugly hard colour edges in the brighter parts. We take care of that soon. Basically, what you see here is step 1. and 2. combined.

Step 3, enhancing image edges

There are million ways to do this, but I noticed a wiggly structure in top part of the apps output image. This suggests that they used a filter similar to curvature flow for enhancing edges. Therefore, I'm going to use a CurvatureFlowFilter to smear the image along the edges before I'm going to extract the edges itself with a GradientFilter.

Since the output of the gradient filter is not really prominent, we will use a gamma correction to push the edges up. Ah, yes, and of course we want black edges, so a ColorNegate seems like a good idea:

grad = ImageAdjust[
  ColorNegate[GradientFilter[CurvatureFlowFilter[b, 15, .7], 1]], {0, 0, 10}]

Mathematica graphics

Now, we can combine the two images by using the colour and saturation from are recoloured image and combine its brightness channel with the found edges. Note that we now take care of the hard colour borders by smearing the brighness channel before combining it with our edge image. This won't hurt because the cool edges are in our grad image.

In addition to that, we can take care of step 4 here as well. For this, you need a very bright structured image of a canvas. Just download any you like, take the brightness part of the image and adjust it, so that it is almost completely white with the structure barely visible like this here

Mathematica graphics

This canvas is going to be included in the brightness part as well:

canvas = ColorConvert[
  ImageResize[Import["http://i.stack.imgur.com/gMmTZ.png"], 
ImageDimensions[img]], "Grayscale"];
{ch, cs, cb} = ColorSeparate[col, "HSB"];
ColorCombine[{ch, cs, ImageMultiply[canvas, 
  ImageMultiply[GaussianFilter[cb, 3], grad]]}, "HSB"]

Mathematica graphics

That's basically it. Now you can try your own versions of colours or you can enhance edges to a greater degree. Why not for instance use a completely different coloring?

Mathematica graphics

$\endgroup$
  • $\begingroup$ This is very helpful, thanks a lot ! $\endgroup$ – Anton Antonov Jul 4 '17 at 16:20
  • $\begingroup$ Last week, I used this post to give a clarifying example in a Machine Learning for Digital Humanities class in Oxford. It was in a comparison of Neural Network algorithms with dedicated algorithms -- the latter generally have better performance and more interpretable intermediate steps. $\endgroup$ – Anton Antonov Jul 10 '17 at 15:10
  • 2
    $\begingroup$ @AntonAntonov I really would like to see Deep Learning flourish but you need to be completely dedicated to it, or you won't understand why it works or why it doesn't. Analyzing noise, frequencies or colorspaces is indeed the more rewarding approach, plus, in my part of research, we usually don't have large training sets for our data. Finally, a 500 bounty and no one here that suggests a DL solution speaks for itself. But if you are part of a DL research group, that's gonna be awesome, I'm sure. $\endgroup$ – halirutan Jul 10 '17 at 15:23
  • 1
    $\begingroup$ I have to say I am much more interested of the algorithm described in the project "Image analogies" than using Deep Learning or Neural Networks. More or less for the same reasons you mentioned: small "training" sets and the need of interpretation and/or enlightenment . $\endgroup$ – Anton Antonov Jul 10 '17 at 15:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.