# How to efficiently import videos

I need to do frame-by-frame analysis of videos. I can import the video files I need, but each file is too large to import at once. I can use the Import[fn, "Frames",{n}] command to import images one-by-one, but this takes a long time once n gets large. For example, the first frame it takes 0.14 sec, but the 200th frame takes 2.2 sec. Mathematica seems to be reading from the start of the file each time I do this. Obviously, this will not work. Is there some way to speed this up. E.g. using streams? Or tricking Mathematica to use ImageCapture to get images from a video file instead of a webcam? Thanks

-
Welcome to Mathematica.SE, Vilis! I have added a little code formatting to make your question easier to read. If you would like to know more about formatting posts, see this short guide. Don't forget to mark the answer that helped you most as "accepted" by clicking the checkmark on the left-hand side of the post. –  Verbeia Jan 24 '13 at 2:12
You can use ffmpeg or similar to export each frame as an image named like frame-%d.png and then you can easily import one chunk of frames at a time by reading appropriate file. –  ssch Jan 24 '13 at 2:13
Streams does have a certain appeal. It is to bad I don't have time to put something together for it. –  rcollyer Jan 24 '13 at 3:07
What format is your video file? I tried a quick test with an AVI file and found that all frames took about the same amount of time to import. –  Todd Gayley Jan 24 '13 at 5:43
You can also import a list of frames: Import[movie, {"ImageList", Range[...]}] –  cormullion Jan 24 '13 at 7:56

If you only want to read it linearly you can tell ffmpeg to dump the video to stdout and then read width*height*bytes-per-pixel bytes at a time to get the video frame-by-frame:

openVideo[fname_, w_, h_] :=
Module[{video},
video["stream"] =
"!ffmpeg -i " ~~ fname ~~ " -loglevel quiet -f rawvideo -pix_fmt rgb24 -",
BinaryFormat -> True];
video["SkipFrame", n_Integer: 1] := Skip[video["stream"], Byte, n*3*w*h];
video["NextFrame"] := Partition[Partition[
, 3], w];
video["NextFrame", n_Integer] := Table[video["NextFrame"], {n}];
video["NextImage"] := Image[video["NextFrame"], "Byte"];
video["NextImage", n_Integer] := Table[video["NextImage"], {n}];
video]


Here's an example:

(* Create a test movie *)
file = \$TemporaryPrefix <> "testvid.avi";
Export[file,Table[Rasterize[i, ImageSize -> {352, 200}], {i, 1, 50}]];

video = openVideo[file,352,200]
video["NextImage",2]
video["SkipFrame",10]
video["NextImage"]


For some reason Mathematica seems to read through the entire video when doing Close[video["stream"]] and it might be worth to kill -15 the ffmpeg process manually to speed it up for large files.

For a 720x404 h264 mp4 video on a modest laptop it takes 13s to skip 200 frames and 0.07s to read a frame. The skipping could possibly be sped up by starting a new ffmpeg process that begins at desired frame. I would compare speeds to Import but Mathematica can't read it (for the test video Import takes .2s for first frame compared to .02s ). So as a bonus with this way you can work with many many more video formats :)

-