# Importing videos in Mathematica

Mathematica's Import command is purportedly able to import .AVI files. However, I find that many files that I want to import do only result in MMA showing a blank frame or another placeholder. All or some of my video-players are able to show these AVI files.

This is an example where it works as advertised:

Import["http://er.jsc.nasa.gov/seh/jfkrice.avi", {"ImageList", 2}]

However,

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi", {"ImageList", 2}]

doesn't work (VLC does). I get an empty white frame.

More examples:

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/sine_wave.avi", {"ImageList", 2}]

(works on VLC and mediaplayer classic)

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/star_collapse.avi", {"ImageList", 2}]

BIG-126 MB!!
Should have looked like this: and works on Windows Media Player, VLC and mediaplayer classic, but I get a white box in MMA.

This is on windows 7-64/MMA8.04. I have the k-lite codecs pack installed.

UPDATE

Responding to Thomas' comment below I found most of the sample files I linked to in my original post gone. I tried to gather a new set and found some that worked and some that don't:

Importable:

Import[#, "VideoEncoding"] & /@
{"http://er.jsc.nasa.gov/seh/jfkrice.avi",
"http://redmine.yorba.org/attachments/615/MVI_0572.AVI",
"http://www-eng-x.llnl.gov/documents/a_video.avi",
"http://redmine.yorba.org/attachments/628/MVI_4981.AVI",
"http://www.csoft.co.uk/video/original/earth.avi",
"http://www.mysticfractal.com/video/fp.avi",
"http://www.softage.ru/files/video-codec/uncompressed/suzie.avi",

{"msvc", "MJPG", "msvc", "MJPG", "msvc", "msvc", "YUV", "Uncompressed"}

Don't import:

Import[#, "VideoEncoding"] & /@
{"https://code.ros.org/trac/opencv/export/7213/trunk/opencv/samples/cpp/tutorial_code/HighGUI/video-input-psnr-ssim/video/Megamind.avi",
"http://samples.mplayerhq.hu/avi/verona60avi56k.avi",
"http://samples.mplayerhq.hu/avi/filedoesitbetter.avi",
"http://www.infognition.com/ScreenPressor/browsing-divx.avi"
}

{"XVID", "MP42", "MJPG", "DX50"}

-
I think it has to do with how it is encoded. A lot of the video files you see floating on the internet with a .avi extension are not really AVI files. I'm don't think that mma utilizes the codecs in your system, whereas VLC does. As such, mma can only import videos that are truthful to the AVI specs (if there are any) –  rm -rf Jan 18 '12 at 14:53
@R.M The problem is that AVI is a container which may hold variously encoded material. Apparently, but not entirely understandable, MMA is not able to handle many codecs although they are present on my PC. However, I would have expected the Microsoft test movies to work, as they are explicitly meant as confirming to MS standards. –  Sjoerd C. de Vries Jan 18 '12 at 14:59
<insert "MS has standards?" joke here?> :) –  rm -rf Jan 18 '12 at 15:02
I am not that familiar with Windows's multimedia playback system, but my guess is this: Windows has two big frameworks: Video for Windows (VfW) and DirectShow (DS). VfW is outdated, but seems to be still used by Mathematica. When you install codecs, most of them are made for DS, and won't work with VfW. –  Szabolcs Jan 18 '12 at 15:28
This may be a Windows-specific issue: With Mathematica 8.0.4 under Mac OS X 10.6.8, I evaluated each of the Import commands you listed, and every one of them worked except for the final one -- the star collapse. But even for that one, I don't get a white box; rather, I get a black box with some variegated pixels in a band along the top of that box. I cannot test the Microsoft .avi's you cite, since the link you gave is to an .exe, and of course an .exe is not an OS X executable. –  murray Jan 18 '12 at 16:01

(Thanks to K.H. from Wolfram):

Even on Windows, if QuickTime is installed, Mathematica uses the native QuickTime install (excluding added, third-party codecs) to import AVI files, and the supported codecs are listed in:

Internal`$VideoEncodings (Without an installation of QuickTime, only uncompressed AVI files are supported by Mathematica.) For the file http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi, QuickTime on Windows can not open it, so Mathematica has problem importing it. - Thanks, that explains a lot. So, we must hope Apple upgrades QT a bit then... – Sjoerd C. de Vries Jan 20 '12 at 20:34 Given the current legal wranglings over video formats, and the fact that AVI is considered to be inferior to current formats, and the fact that Apple doesn't like supporting other people's technology, hope is the operative word... – cormullion Jan 21 '12 at 10:50 add comment I think the problem is the fact that you cannot open the video at http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi by QuickTime. Try it. Download free QuickTime and try to open it in that player and is most probably won't work. It means that to make the video a non-standard video encoding was used which is not a subset of formats understood by QuickTime. High chance this is why M. cannot open it - my best guess. - Well, I'm on windows and Quicktime player is not the first choice on that platform. Anyway, that doesn't show a thing. As I wrote above, all the movies listed can be played by one of more of the players I have on my PC. The codec pack installed on my PC makes an enormous amount of standard and exotic codecs available to Windows and any player/importer that uses the appropriate system API should be able to use them. It looks like MMA doesn't go through the codecs that are installed on the machine and instead only uses a few built-in codecs; apparently a very small set. – Sjoerd C. de Vries Jan 20 '12 at 18:47 add comment If you only need to access the video frame by frame you can use ffmpeg to decode the video. Reposting a previous answer: If you only want to read it linearly you can tell ffmpeg to dump the video to stdout and then read width*height*bytes-per-pixel bytes at a time to get the video frame-by-frame: openVideo[fname_, w_, h_] := Module[{video}, video["stream"] = OpenRead[ "!ffmpeg -i " ~~ fname ~~ " -loglevel quiet -f rawvideo -pix_fmt rgb24 -", BinaryFormat -> True]; video["SkipFrame", n_Integer: 1] := Skip[video["stream"], Byte, n*3*w*h]; video["NextFrame"] := Partition[Partition[ BinaryReadList[video["stream"], "Byte", 3*w*h] , 3], w]; video["NextFrame", n_Integer] := Table[video["NextFrame"], {n}]; video["NextImage"] := Image[video["NextFrame"], "Byte"]; video["NextImage", n_Integer] := Table[video["NextImage"], {n}]; video] Here's an example: (* Create a test movie *) file =$TemporaryPrefix <> "testvid.avi";
Export[file,Table[Rasterize[i, ImageSize -> {352, 200}], {i, 1, 50}]];

video = openVideo[file,352,200]
video["NextImage",2]
video["SkipFrame",10]
video["NextImage"]

For some reason Mathematica seems to read through the entire video when doing Close[video["stream"]] and it might be worth to kill -15 the ffmpeg process manually to speed it up for large files.

For a 720x404 h264 mp4 video on a modest laptop it takes 13s to skip 200 frames and 0.07s to read a frame. The skipping could possibly be sped up by starting a new ffmpeg process that begins at desired frame. I would compare speeds to Import but Mathematica can't read it (for the test video Import takes .2s for first frame compared to .02s ). So as a bonus with this way you can work with many many more video formats :)

-
Doesn't seem to work for me. I tested that ffmpeg is present; tried it either with specifying its full path, and without path but directory set with SetDirectory. Nothing. Also tested with ReadList["!\"c:\\Program Files (x86)\\ConvertHelper\\ffmpeg.exe\" -h \ ", String] that ffmpeg actually executes. It does. –  Sjoerd C. de Vries Apr 10 at 17:32
There is a minus sign without option letter at the end of the ffmpeg call. Is that intentional? –  Sjoerd C. de Vries Apr 10 at 19:56
@SjoerdC.deVries The last - makes it output to stdout, do you get anything interesting on stderr when running from a terminal without -loglevel quiet? (Whatever windows equivalent of ffmpeg -i video.avi -f rawvideo -pix_fmt rgb24 - >/dev/null is, maybe > nul) –  ssch Apr 11 at 22:47
For comparison, this is what I get for above test video: pastie.org/7458920 –  ssch Apr 11 at 22:58
I upgraded ffmpeg to a newer version. That did the trick. Thanks! An improvement would be to remove the need to enter the frame size. –  Sjoerd C. de Vries Apr 12 at 21:33