Importing videos in Mathematica

Question

Mathematica's Import command is purportedly able to import .AVI files. However, I find that many files that I want to import do only result in MMA showing a blank frame or another placeholder. All or some of my video-players are able to show these AVI files.

This is an example where it works as advertised:

Import["http://er.jsc.nasa.gov/seh/jfkrice.avi", {"ImageList", 2}]

Mathematica graphics

However,

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi", {"ImageList", 2}]

doesn't work (VLC does). I get an empty white frame.

More examples:

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/sine_wave.avi", {"ImageList", 2}]

(works on VLC and mediaplayer classic)

Import["http://people.sc.fsu.edu/~jburkardt/data/avi/star_collapse.avi", {"ImageList", 2}]

BIG-126 MB!!
Should have looked like this: Mathematica graphics and works on Windows Media Player, VLC and mediaplayer classic, but I get a white box in MMA.

This is on windows 7-64/MMA8.04. I have the k-lite codecs pack installed.

UPDATE

Responding to Thomas' comment below I found most of the sample files I linked to in my original post gone. I tried to gather a new set and found some that worked and some that don't:

Importable:

Import[#, "VideoEncoding"] & /@ 
 {"http://er.jsc.nasa.gov/seh/jfkrice.avi", 
  "http://redmine.yorba.org/attachments/615/MVI_0572.AVI",
  "http://www-eng-x.llnl.gov/documents/a_video.avi",
  "http://redmine.yorba.org/attachments/628/MVI_4981.AVI",
  "http://www.csoft.co.uk/video/original/earth.avi",
  "http://www.mysticfractal.com/video/fp.avi", 
  "http://www.softage.ru/files/video-codec/uncompressed/suzie.avi", 
  "http://archive.org/download/Architects_of_Tomorrow/2007-12-10-02-39-00.avi"}

{"msvc", "MJPG", "msvc", "MJPG", "msvc", "msvc", "YUV", "Uncompressed"}

Don't import:

Import[#, "VideoEncoding"] & /@
 {"https://code.ros.org/trac/opencv/export/7213/trunk/opencv/samples/cpp/tutorial_code/HighGUI/video-input-psnr-ssim/video/Megamind.avi",
  "http://samples.mplayerhq.hu/avi/verona60avi56k.avi",
  "http://samples.mplayerhq.hu/avi/filedoesitbetter.avi",
  "http://www.infognition.com/ScreenPressor/browsing-divx.avi"
  }

{"XVID", "MP42", "MJPG", "DX50"}

I think it has to do with how it is encoded. A lot of the video files you see floating on the internet with a .avi extension are not really AVI files. I'm don't think that mma utilizes the codecs in your system, whereas VLC does. As such, mma can only import videos that are truthful to the AVI specs (if there are any) — rm -rf, Commented Jan 18, 2012 at 14:53
@R.M The problem is that AVI is a container which may hold variously encoded material. Apparently, but not entirely understandable, MMA is not able to handle many codecs although they are present on my PC. However, I would have expected the Microsoft test movies to work, as they are explicitly meant as confirming to MS standards. — Sjoerd C. de Vries, Commented Jan 18, 2012 at 14:59
I am not that familiar with Windows's multimedia playback system, but my guess is this: Windows has two big frameworks: Video for Windows (VfW) and DirectShow (DS). VfW is outdated, but seems to be still used by Mathematica. When you install codecs, most of them are made for DS, and won't work with VfW. — Szabolcs, Commented Jan 18, 2012 at 15:28
This may be a Windows-specific issue: With Mathematica 8.0.4 under Mac OS X 10.6.8, I evaluated each of the Import commands you listed, and every one of them worked except for the final one -- the star collapse. But even for that one, I don't get a white box; rather, I get a black box with some variegated pixels in a band along the top of that box. I cannot test the Microsoft .avi's you cite, since the link you gave is to an .exe, and of course an .exe is not an OS X executable. — murray, Commented Jan 18, 2012 at 16:01

Simon Woods · Accepted Answer · 2018-12-01 16:15:09Z

64-bit Windows only

Note for Mathematica 11.3: There is a potential conflict between MathMF and the built-in MediaTools package. See here for details and here for an example of how to use MediaTools in place of MathMF.

Note for Mathematica version 10: The Wolfram Library has been updated in version 10 and you will need to recompile the MathMF DLL. This is most easily accomplished by evaluating "MathMF"//FindLibrary//DeleteFile prior to loading the package.

Link to package on GitHub

I have written a package called MathMF which uses a LibraryLink DLL to do frame-by-frame video import and export with Windows Media Foundation. It should be able to read a reasonable variety of movie files, including AVI, WMV and MP4. Exporting is currently limited to WMV and MP4 formats (AVI encoding is not natively supported by Media Foundation)

Here is the sort of code you can write with it. The code first opens a video file for reading, and creates a new video file for writing to. It then runs a loop in which each frame is sequentially read from the input stream, processed in Mathematica and then written to the output stream. So Mathematica is effectively being used as a video filter.

{duration, framerate, width, height} =
 MFInitSourceReader["C:\\Users\\Simon\\Desktop\\test1.wmv"];

MFInitSinkWriter["C:\\Users\\Simon\\Desktop\\filtered.wmv", 
 width, height, "FrameRate" -> framerate]

While[
  (image = MFGrabFrame["RealImage"]) =!= EndOfFile,
  MFSendFrame @ GradientFilter[image, 2]
  ] ~Monitor~ image

MFFinaliseSink[]

The package can be downloaded from the GitHub link at the top of this post, it is too large to include in full here.

The package includes the library source code, and on first use will attempt to compile the library locally. I believe the compilation should work if you have Visual Studio 2010 or later installed, and probably won't work if you use a different compiler. There is a pre-built DLL available if the compilation fails (see the readme on GitHub for more details)

I hope some people find this useful, it has been hovering in my mind as something to try to do for quite some time, hindered mainly by my total lack of experience with C++ and COM programming.

Excellent Simon! I had to download the Visual C++ Redistributable for Visual Studio 2012 before it worked for me, but after that, flawless. One small point: I don't think we should distribute software in the form of binaries, as that can have terrible security implications. — Sjoerd C. de Vries, Commented Apr 27, 2014 at 14:07
@Sjoerd, thank you. I agree about distributing the binary, I have now uploaded a .m package and the C++ source code. The package will attempt to build the library from source when it is first loaded. There are various header files which the compiler needs, I think any system with VS2012 or the Win7 SDK will have these available but I'm not sure. I'd be grateful if you could try it out when you have the chance. — Simon Woods, Commented Apr 27, 2014 at 20:34
as confirmed in chat, compiling works. Using avi to wmv converters available on the internet should allow avi import as well. — Sjoerd C. de Vries, Commented Apr 27, 2014 at 22:07
@Sjoerd, avi import should just work. MF supports reading avi, just not writing. — Simon Woods, Commented Apr 27, 2014 at 22:15
@Simon Woods: Please have a look at mathematica.stackexchange.com/questions/186349/…. Thanks. — mrz, Commented Nov 28, 2018 at 16:50

Arnoud Buzing · Accepted Answer · 2012-01-20 19:51:12Z

18

(Thanks to K.H. from Wolfram):

Even on Windows, if QuickTime is installed, Mathematica uses the native QuickTime install (excluding added, third-party codecs) to import AVI files, and the supported codecs are listed in:

Internal`$VideoEncodings

(Without an installation of QuickTime, only uncompressed AVI files are supported by Mathematica.) For the file http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi, QuickTime on Windows can not open it, so Mathematica has problem importing it.

edited Jan 20, 2012 at 19:51

answered Jan 20, 2012 at 19:36

Arnoud Buzing

9,8312 gold badges49 silver badges58 bronze badges

$\begingroup$ Thanks, that explains a lot. So, we must hope Apple upgrades QT a bit then... $\endgroup$
– Sjoerd C. de Vries
Commented Jan 20, 2012 at 20:34
1

$\begingroup$ Given the current legal wranglings over video formats, and the fact that AVI is considered to be inferior to current formats, and the fact that Apple doesn't like supporting other people's technology, hope is the operative word... $\endgroup$
– cormullion
Commented Jan 21, 2012 at 10:50

Add a comment |

Community · Accepted Answer · 2017-04-13 12:55:55Z

If you only need to access the video frame by frame you can use ffmpeg to decode the video.

Reposting a previous answer:

If you only want to read it linearly you can tell ffmpeg to dump the video to stdout and then read widthheightbytes-per-pixel bytes at a time to get the video frame-by-frame:

openVideo[fname_, w_, h_] := 
 Module[{video}, 
  video["stream"] = 
   OpenRead[
    "!ffmpeg -i " ~~ fname ~~ " -loglevel quiet -f rawvideo -pix_fmt rgb24 -", 
    BinaryFormat -> True];
  video["SkipFrame", n_Integer: 1] := Skip[video["stream"], Byte, n*3*w*h];
  video["NextFrame"] := Partition[Partition[
     BinaryReadList[video["stream"], "Byte", 3*w*h]
   , 3], w];
  video["NextFrame", n_Integer] := Table[video["NextFrame"], {n}];
  video["NextImage"] := Image[video["NextFrame"], "Byte"];
  video["NextImage", n_Integer] := Table[video["NextImage"], {n}];
  video]

Here's an example:

(* Create a test movie *)
file = $TemporaryPrefix <> "testvid.avi";
Export[file,Table[Rasterize[i, ImageSize -> {352, 200}], {i, 1, 50}]];

video = openVideo[file,352,200]
video["NextImage",2]
video["SkipFrame",10]
video["NextImage"]

first

second

For some reason Mathematica seems to read through the entire video when doing Close[video["stream"]] and it might be worth to kill -15 the ffmpeg process manually to speed it up for large files.

For a 720x404 h264 mp4 video on a modest laptop it takes 13s to skip 200 frames and 0.07s to read a frame. The skipping could possibly be sped up by starting a new ffmpeg process that begins at desired frame. I would compare speeds to Import but Mathematica can't read it (for the test video Import takes .2s for first frame compared to .02s ). So as a bonus with this way you can work with many many more video formats :)

Doesn't seem to work for me. I tested that ffmpeg is present; tried it either with specifying its full path, and without path but directory set with SetDirectory. Nothing. Also tested with ReadList["!\"c:\\Program Files (x86)\\ConvertHelper\\ffmpeg.exe\" -h \ ", String] that ffmpeg actually executes. It does. — Sjoerd C. de Vries, Commented Apr 10, 2013 at 17:32
There is a minus sign without option letter at the end of the ffmpeg call. Is that intentional? — Sjoerd C. de Vries, Commented Apr 10, 2013 at 19:56
@SjoerdC.deVries The last - makes it output to stdout, do you get anything interesting on stderr when running from a terminal without -loglevel quiet? (Whatever windows equivalent of ffmpeg -i video.avi -f rawvideo -pix_fmt rgb24 - >/dev/null is, maybe > nul) — ssch, Commented Apr 11, 2013 at 22:47
For comparison, this is what I get for above test video: pastie.org/7458920 — ssch, Commented Apr 11, 2013 at 22:58
A Library for importing using FFmpeg: mathematica.stackexchange.com/a/47545/4568 — Karolis, Commented May 15, 2015 at 14:32

Vitaliy Kaurov · Accepted Answer · 2012-01-20 16:35:24Z

3

I think the problem is the fact that you cannot open the video at

http://people.sc.fsu.edu/~jburkardt/data/avi/ccvt_box.avi

by QuickTime. Try it. Download free QuickTime and try to open it in that player and is most probably won't work. It means that to make the video a non-standard video encoding was used which is not a subset of formats understood by QuickTime. High chance this is why M. cannot open it - my best guess.

answered Jan 20, 2012 at 16:35

Vitaliy Kaurov

73.4k9 gold badges205 silver badges358 bronze badges

$\begingroup$ Well, I'm on windows and Quicktime player is not the first choice on that platform. Anyway, that doesn't show a thing. As I wrote above, all the movies listed can be played by one of more of the players I have on my PC. The codec pack installed on my PC makes an enormous amount of standard and exotic codecs available to Windows and any player/importer that uses the appropriate system API should be able to use them. It looks like MMA doesn't go through the codecs that are installed on the machine and instead only uses a few built-in codecs; apparently a very small set. $\endgroup$
– Sjoerd C. de Vries
Commented Jan 20, 2012 at 18:47

Add a comment |

Stack Exchange Network

Importing videos in Mathematica

4 Answers 4

Your Answer

Not the answer you're looking for? Browse other questions tagged
import
video
or ask your own question.

Linked

Hot Network Questions

Importing videos in Mathematica

4 Answers 4

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged importvideo or ask your own question.

Linked

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
import
video
or ask your own question.