2
$\begingroup$

I have a large set (>10000) of uncompressed 8 bit images. Their size is 1600*1200 pixels.

The first 24 pixels in the upper most line starting from left contain some important information (time stamp etc.) about the image.

I want to read out all byte values of these 24 pixels.

Finally I want to save all byte data as right aligned numbers into a text file, a single line with 24 numbers for each image.

I CANNOT READ IN ALL IMAGES INTO THE MEMORY AT ONCE BECAUSE OF MEMORY PROBLEMS.

For testing I have uploaded a set of 1000 cropped images, each having a dimension of 50*50 pixels. They can be downloaded from here: https://drive.google.com/open?id=0B9wKP6yNcpyfUUlsQU1iVWJ0b2M

How can I improve (speed up) my code?

ChoiceDialog[{FileNameSetter[Dynamic[imageDir], "Directory"], Dynamic[imageDir]}];
SetDirectory[imageDir];

fNames = FileNames["*.png"];
n = Length[fNames];

timeStampValues = Array[0 &, {n, 24}];
SetSharedVariable[timeStampValues];

ParallelDo[

   byteValues = Import[fNames[[i]], "Data"];
   timeStampValues[[i, All]] = Flatten@byteValues[[1, 1 ;; 24]];

   , {i, 1, n}
]; // AbsoluteTiming

{17.7253, Null}
$\endgroup$
6
$\begingroup$
  • Start by replacing Import with

    ImageData@First@Image`ImportExportDump`ImageReadPNG[fNames[[‌​i]]]
    

    This is the low level function called by Import on PNG files; calling it directly bypasses A TON of behind-the-scenes checking. On a 350-picture subset, the ParallelDo[Import...] alone was taking 70s on my laptop; the ImageReadPNG version took 4s.

  • The Flatten call within the ParallelDo loop also doesn't seem necessary, since byteValues[[1, 1;;24]] is already a flat vector. That won't probably affect timing quite as much though.

In short, I propose to use:

ParallelDo[
  byteValues = ImageData@First@Image`ImportExportDump`ImageReadPNG[fNames[[i]]];
  timeStampValues[[i]] = byteValues[[1, 1 ;; 24]],
  {i, 1, n}]

On your sample data set, I get the following timings:

  • your version: 61 seconds
  • my version: 9.2 seconds
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.