Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I need to do frame-by-frame analysis of videos. I can import the video files I need, but each file is too large to import at once. I can use the Import[fn, "Frames",{n}] command to import images one-by-one, but this takes a long time once n gets large. For example, the first frame it takes 0.14 sec, but the 200th frame takes 2.2 sec. Mathematica seems to be reading from the start of the file each time I do this. Obviously, this will not work. Is there some way to speed this up. E.g. using streams? Or tricking Mathematica to use ImageCapture to get images from a video file instead of a webcam? Thanks

share|improve this question
    
Welcome to Mathematica.SE, Vilis! I have added a little code formatting to make your question easier to read. If you would like to know more about formatting posts, see this short guide. Don't forget to mark the answer that helped you most as "accepted" by clicking the checkmark on the left-hand side of the post. –  Verbeia Jan 24 '13 at 2:12
5  
You can use ffmpeg or similar to export each frame as an image named like frame-%d.png and then you can easily import one chunk of frames at a time by reading appropriate file. –  ssch Jan 24 '13 at 2:13
    
Streams does have a certain appeal. It is to bad I don't have time to put something together for it. –  rcollyer Jan 24 '13 at 3:07
3  
What format is your video file? I tried a quick test with an AVI file and found that all frames took about the same amount of time to import. –  Todd Gayley Jan 24 '13 at 5:43
1  
You can also import a list of frames: Import[movie, {"ImageList", Range[...]}] –  cormullion Jan 24 '13 at 7:56
add comment

2 Answers

If you only want to read it linearly you can tell ffmpeg to dump the video to stdout and then read width*height*bytes-per-pixel bytes at a time to get the video frame-by-frame:

openVideo[fname_, w_, h_] := 
 Module[{video}, 
  video["stream"] = 
   OpenRead[
    "!ffmpeg -i " ~~ fname ~~ " -loglevel quiet -f rawvideo -pix_fmt rgb24 -", 
    BinaryFormat -> True];
  video["SkipFrame", n_Integer: 1] := Skip[video["stream"], Byte, n*3*w*h];
  video["NextFrame"] := Partition[Partition[
     BinaryReadList[video["stream"], "Byte", 3*w*h]
   , 3], w];
  video["NextFrame", n_Integer] := Table[video["NextFrame"], {n}];
  video["NextImage"] := Image[video["NextFrame"], "Byte"];
  video["NextImage", n_Integer] := Table[video["NextImage"], {n}];
  video]

Here's an example:

(* Create a test movie *)
file = $TemporaryPrefix <> "testvid.avi";
Export[file,Table[Rasterize[i, ImageSize -> {352, 200}], {i, 1, 50}]];

video = openVideo[file,352,200]
video["NextImage",2]
video["SkipFrame",10]
video["NextImage"]

first

second

For some reason Mathematica seems to read through the entire video when doing Close[video["stream"]] and it might be worth to kill -15 the ffmpeg process manually to speed it up for large files.

For a 720x404 h264 mp4 video on a modest laptop it takes 13s to skip 200 frames and 0.07s to read a frame. The skipping could possibly be sped up by starting a new ffmpeg process that begins at desired frame. I would compare speeds to Import but Mathematica can't read it (for the test video Import takes .2s for first frame compared to .02s ). So as a bonus with this way you can work with many many more video formats :)

share|improve this answer
add comment

The performance can be improved by loading more than 100 frames at a time. To test this we can run:

LoadNFrames[n_] := (Import[video , {"Frames" , Range[1000, 1000 + n]} ]; n);

times = Table[ AbsoluteTiming[ LoadNFrames[n] ], 
         {n, {1, 2, 5, 10, 20, 30, 50, 100, 150, 200, 300, 400, 500, 600, 700, 800}}]

time it takes to load a frame VS frames loaded

One can see that in Mathematica there is a penalty for loading only few frames. We can improve the performance by preloading the video and storing it in a buffer. For instance:

buffer = Import[video, "ImageList"]; 

GetFrame[n_] := buffer[[n]]

The method uses a lot of memory, therefore, for large videos you will need to load parts of the video. I also typically cut the region of interest before putting it o the buffer.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.