Conforming more than hundred thousand images

Question

I have a large dataset of images with different dimensions, perhaps even colorspaces, etc. What I want to do is to conform all of these images;

Rather than importing the whole bunch (which is impossible without infinite RAM), I wanted to first conform two images, later use them as basis to import the next small batch of images and conform them with the initially conformed two images. My tests indicate that such an approach is generally ok, meaning that in the end the next batch of images will be conformed the same way as the first two, however I cannot be sure.

The question is whether my tests are correct or not, i.e. in a code like this:

referenceImgs = Import[filenames[[1;;2]]];
conformed = ConformImages[referenceImgs];
newImg = Import[filenames[3]];
ConformImages[{referenceImgs[[1]], newImg}]; 

will be equivalent to:

imgs = Import[filenames[[1;;3]]]; 
ConformImages[filenames];

or no?

Or even rather than "whether it will be equivalent" will the final 3 images be conformed similar to how they would have been conformed in case 2. Albeit maybe in a different way?

Question 2: Is there a better way to achieve this result?

Answer 1

For Q1 I don't think it can be guaranteed that this will work as expected. e.g. the docs say For multichannel images with a different number of channels, the maximum number of channels is used so after you've conformed images 1,2, if image 3 is then added to the conformation, and image 3 has a larger number of channels image 1 and 2 will be changed to conform to image 3.

Q2: It looks like the best way to achieve what I think you're asking is to use the second spec argument. You'll want to conform a few images first and check the result:

refConform = ConformImages@(Import/@filenames[[1;;2]]);
(*Now check that refConform look good*)

And then proceed in batches:

nextBatch = ConformImages[#,First@refConform]&@(Import/@filenames[[3;;3+batchsize]]);
(*Now export this batch and continue onto the next.*)

Where batchsize is however many images you think you can handle in one go. I assume you know how you'll handle the iterative portion of this. If not, that should probably be asked as a separate question.