8
$\begingroup$

I will get large RGB AVI files (35 Hz, 1600$\times$1200 pixels) of up to 100 GB size. All three channels contain the same grayscale image. I want to extract each AVI file into one channel png images. To do that I wanted to use the following code snippet:

nImages = Length@Import[aviFileName, "Frames"];

ParallelDo
[
  image = ColorConvert[Import[aviFileName, {"Frames", i}], "Grayscale"]; 
  imageName = ToString@PaddedForm[i, 6, NumberPadding -> {"0", ""}];
  Image`ImportExportDump`ImageWritePNG[StringJoin[imageName, ".png"], image];
  , {i, 1, nImages}
];

Questions:

  1. Can such large files be read this way?

  2. How can the performance be improved?

$\endgroup$
9
  • 4
    $\begingroup$ This is not a very backed up answer, but from my experience it can be tough to work with files that big in Mathematica, because Mathematica usually tries to load everything into memory when Importing. Because of that i try to avoid loading files bigger than ˜1GB into Mathematica. For your specific problem i would try a specialised tool like VirtualDub or Adobe Premiere or FinalCutPro/Compressor if you're on a mac. But i would very much like if someone with more Mathematica knowledge proves me wrong :) $\endgroup$ Mar 11, 2017 at 15:53
  • 1
    $\begingroup$ Have you tried this on your avi file? I'm sure that importing all frames of 100GB avi files, which you do in your first line, won't work because it will load them into your RAM. If you have more than 100GB RAM indeed and this works, then one way to try would be to turn it into a list {{1, frame1},{2, frame2},...} and process this in parallel. Still, it is not sure that this will work. $\endgroup$
    – halirutan
    Mar 11, 2017 at 16:04
  • 1
    $\begingroup$ @mrz: ImageMagick also seems worth a look. It seems it could do both steps you want (avi->png and rgb->grayscale). $\endgroup$ Mar 11, 2017 at 16:07
  • 1
    $\begingroup$ If you are on Windows you can use MathMF from here. It reads frames one at a time so there should be no problem working with a very large file. $\endgroup$ Mar 11, 2017 at 17:31
  • 1
    $\begingroup$ ffmpeg would be my choice. $\endgroup$
    – Felix
    Mar 12, 2017 at 21:12

1 Answer 1

12
$\begingroup$

Here is an update where I compared the image extraction (8bit, 1 channel) from two AVI files:

enter image description here

My computer (Mathematica 11.0.1, Win 10 Pro, Intel i7 7490-MX, 3.1GHz, 4 cores) has 16 GB RAM and it never used in total more than 5 GB RAM, all running processes included.

The code can be used for multiple AVIs in different subdirectories in a certain base directory:

(* Read in folder which is base dir to search for avi files in all subdirs *)

ChoiceDialog[{FileNameSetter[Dynamic[baseDir], "Directory"], Dynamic[baseDir]}];

SetDirectory[baseDir];

aviNames = FileNames["*.avi", baseDir, Infinity];

numberAVIFiles = Length[aviNames];

aviDirs = DirectoryName[#] & /@ aviNames;

pngBaseDirs = FileBaseName[#] & /@ aviNames;

pngFullDirs = Transpose@{aviDirs, pngBaseDirs} /. {a_, p_} :> StringJoin[a, p];


(* Create a new directory for pngs if it does not exist otherwise delete it and create a new one *)
(If[DirectoryQ[#], DeleteDirectory[#, DeleteContents -> True]; CreateDirectory[#], CreateDirectory[#]]) & /@ 
  pngFullDirs;

Do[
   numberImages = Length@Import[aviNames[[j]], "Frames"];

   ParallelDo[

    image = ColorConvert[Import[StringJoin[aviNames[[j]]], {"Frames", i}], "Grayscale"];

    fileNameCounter = ToString@PaddedForm[i, 6, NumberPadding -> {"0", ""}];

    Image`ImportExportDump`ImageWritePNG[
     StringJoin[pngFullDirs[[j]], "\\", pngBaseDirs[[j]], "_", fileNameCounter, ".png"], image];

    fileName = StringJoin[pngBaseDirs[[j]], "_", fileNameCounter, ".png"];


    , {i, 1, numberImages}
    ];

   , {j, 1, numberAVIFiles}
   ];
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.