# Efficient multicolor search

I'm hoping someone can help me improve the following fun code-snippet, which takes a target color and searches through a set of images to find the ones that most closely match:

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
cols = DominantColors[#, 10, {"Color", "Coverage"}] & /@ data;
ColorSlider[Dynamic[targetColor]]
Dynamic[distances = Table[Min[(Function[c, ColorDistance[c, targetColor]] /@ cc[[All, 1]])*(1/cc[[All, 2]])], {cc, cols}];
Magnify@Row@data[[Ordering[distances, 5]]]
]


Ok, so it basically works. But here are the two things missing:

Problem 1. I’m hoping someone can help make it more efficient so it can scale a little better, the example runs on only 100 images. I would like to replace the brute-force Table with a call to Nearest - but I'm not quite sure how.

Problem 2. I'd also like to search based on multiple colors (2 or more weighted colors), but have not had much success. Here's the naive approach of adding the distances:

Column@{ColorSlider[Dynamic[targetColor1]], ColorSlider[Dynamic[targetColor2]]}
Dynamic[distances = Table[Min[(Function[c, ColorDistance[c, targetColor1]] /@ cc[[All, 1]])*(1/cc[[All, 2]]) + (Function[c, ColorDistance[c, targetColor2]] /@ cc[[All, 1]])*(1/cc[[All, 2]])], {cc, cols}]; Magnify@Row@data[[Ordering[distances, 10]]]]


This doesn't work very well, I'm thinking of perhaps tweaking the distance function and using the LAB colorspace.

• Some sort of dimension reduction sounds like it could be useful here. – ktm Feb 5 '19 at 4:52
• A list of color coordinates with weights doesn’t seem super high dimensional though right? – M.R. Feb 5 '19 at 4:58
• Your example with two query colors actually isn’t that bad – user5601 Feb 5 '19 at 5:29
• So it's not super terrible, the but colors sort of average out and most importantly - the lookup is totally inefficient! – M.R. Feb 5 '19 at 16:26

An alternative approach might be to use the built-in ImageDistance function.

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
ColorSlider[Dynamic[targetColor]]
Dynamic[dist = ImageDistance[Image[targetColor], #] & /@ data;
Magnify@Row@data[[Ordering[dist, 5]]]]


Of course you can choose any of the possible DistanceFunctions. This can be sped up significantly by using a NearestFunction to calculate the ImageDistance between the target and the collection of images. For example:

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
nf = Nearest[data];
ColorSlider[Dynamic[targetColor]]
Dynamic[nf[Image[targetColor], 5]]


To address the issue of a multiple color selector, this can be incorporated smoothly by setting the "targetColor" image to have two different colors. Now the ImageDistance will react to both the chosen colors.

data = ResourceData["CIFAR-10", "TestData"][[;; ;; 100, 1]];
nf = Nearest[data];
{ColorSlider[Dynamic[targetColor1]], ColorSlider[Dynamic[targetColor2]]}
Dynamic[targetColorAll =
ImageAssemble[{Image[targetColor1], Image[targetColor2]}]]
Dynamic[nf[Image[targetColorAll], 5]]

• Earth mover is notoriously slow and won’t scale – M.R. Feb 7 '19 at 2:10
• None of them are fast – M.R. Feb 7 '19 at 2:28
• The only way to scale it is to compute color features and index them into a approximate nearest neighbors forrest – M.R. Feb 7 '19 at 2:48
• Also this doesn’t address multicolor search – M.R. Feb 7 '19 at 4:26
• See updates, which address both your issues, still within the framework of using ImageDistance. – bill s Feb 7 '19 at 16:26

Here's a NearestFunction that will do what you did with Table:

nf = Nearest[
Hold /@ cols -> "Index", (* Hold seems to be necessary to make sure Nearest doesn't get confused by the inner lists *)
DistanceFunction -> Function[
Min @ Divide[
Map[
Function[col,
ColorDistance[#1, col]
],
#2[[1, All, 1]]
],
#2[[1, All, 2]]
]
]
];


I also recommend that you use the TrackedSymbols -> {targetColor} option in your Dynamic to make sure it only updates whenever targetColor changes.