11
$\begingroup$

I am trying to import a gif animation, which is about 80MB. The goal is to convert the image to grayscale, and then do some calculation on ImageData for each frame. The operations are in the sample code:

data = ColorConvert[Import["MOV_5705.gif"],"Grayscale"];
datag = RemoveAlphaChannel /@ data;
datag = ImageData /@ datag;
datag = Flatten /@ datag;

The issue is that Mathematica will use up all 32 GB of memory available to me by the end. I cannot even print out data from datag list. The gif file has 554 frames and each frame holds 2073600 values, which gives approximately 9 GB of double precision data. Is there any way to make processing the frames manageable. (the gif file can be downloaded here)

$\endgroup$
  • $\begingroup$ I can not give you a straight answer to your question but will like to offer light on a route to a good solution. I have a current research project that faces the same challenge. Go on using parallel processing. Oneway is to use a GPU card and its custom language. One excellent option is NVIDIA card with CUDA. Also, look at parallel processing using a separate computer with a Mathematica kernel each( or computers) network in a grid. Assign each machine a block of images for processing. I do not have specific examples to offer, but will be glad to share with you though this post at a later time $\endgroup$ – Jose Enrique Calderon Jun 28 '18 at 16:58
  • 4
    $\begingroup$ You can use the mathematica-ffmpeg package to load only a subset of the frames, then process them, then load another subset and so on until you are done. This way you don't have to have all of the frames in memory at the same time. You can save the modified frames to disk and then make an animation out of them using FFMPEG command line, if it can't be done with Mathematica. Optimally, we wouldn't have to involve FFMPEG but I'm not sure that the memory problem can be worked around using only Mathematica. $\endgroup$ – C. E. Jun 28 '18 at 17:05
9
$\begingroup$

I know halirutan has already provided an answer, but here's a nice way to deal with GIF import in a frame-wise manner (or really in chunks of frames).

Mathematica does a very dumb thing when it reads GIFs. It calls the function System`Convert`CommonGraphicsDump`IEImageRead which, if intelligent, would merely process the requested frames. Instead it always takes every single frame, which is returned as an Image3D calls Image3DSlices.

Why it does this is truly beyond me, but you can check it like so:

Block[{Image3DSlices = Echo[Head[#]] &},
 Import[gif, {"ImageList", 5}]
 ]

Image3D

Import::fmterr: Cannot import data as GIF format.

$Failed

Instead of using that we'll use the built in "GIFTools`" library which is intelligent about this. The core functions we'll want are these:

{
 GIFTools`Private`$ReadOneFrame,
 GIFTools`Private`$ReadPalettes,
 GIFTools`Private`$ReadAllFrames,
 GIFTools`Private`$ReadRasterBits,
 GIFTools`Private`$ReadFileMetadata, 
 GIFTools`Private`$ReadFrameMetadata,
 GIFTools`Private`$ReadGlobalPalette
 }

The most useful parts here are $ReadFileMetadata and $ReadOneFrame, but all of these can be used to dramatically outperform the default Import implementation.

Now we can effectively map a function over blocks of frames extracted using Range.

Here's the full code. Most of it is just making the interface nice, implementing protections and validations, etc.

gifBlockMapLoadLib[] :=
  (
   Replace[
     If[DownValues@
        System`Convert`CommonGraphicsDump`ImageReadMetadataGIF == {}, 
      Quiet@Import[
         "https://i.imgur.com/7Wkns4d.gif", {"GIF", "Elements"}];
      ];
     DownValues@System`Convert`CommonGraphicsDump`ImageReadMetadataGIF,
     {} :>
      Throw[
       Failure["BadGIFInit", <|
         "MessageTemplate" -> "GIF tools not loaded"|>], "GIFBlockMap"]
     ];
   );
gifBlockMapGetFile[gif_] :=

  With[{base = ExpandFileName@Replace[gif, File[f_] :> f]},
   If[! FileExistsQ@base,
    Throw[
     Failure["BadGIF", 
      <|"MessageTemplate" -> "GIF file `` does not exist", 
       "MessageParameters" -> {gif}|>
      ],
     "GIFBlockMap"
     ],
    base
    ]
   ];
gifBlockMapGetImageCount[gif_] :=
  Replace[
   Quiet@
    System`Convert`CommonGraphicsDump`ImageReadMetadataGIF[gif, 
     "ImageCount"],
   {
    {i_Integer} :> i,
    _ :>
     Throw[
      Failure["BadGIF", 
       <|"MessageTemplate" -> "`` is not a valid GIF file", 
        "MessageParameters" -> {gif}|>
       ],
      "GIFBlockMap"
      ]
    }
   ];
gifBlockMapGetBlockSpec[frameCount_, block_] :=
  Module[
   {
    start, stop, step
    },
   If[! IntegerQ@frameCount,
    Throw[
     Failure["BadGIF", 
      <|"MessageTemplate" -> "GIF frame count `` is not an integer", 
       "MessageParameters" -> {frameCount}|>
      ],
     "GIFBlockMap"
     ]
    ];
   If[block[[3]] <= 0,
    Throw[
     Failure["BadBlock", 
      <|"MessageTemplate" -> "Block step `` must be greater than 0", 
       "MessageParameters" -> {block[[3]]}|>
      ],
     "GIFBlockMap"
     ]
    ];
   {start, stop, step} =
    Replace[List @@ block, 
     {
      All -> frameCount,
      i_Integer?Negative :>
       frameCount - i,
      i_Integer?(# > frameCount &) :>
       Throw[
        Failure["BadBlock", 
         <|
          "MessageTemplate" -> 
           "Block parameter `` is larger than frame count ``", 
          "MessageParameters" -> {i, frameCount}|>
         ],
        "GIFBlockMap"
        ],
      Scaled[s_] :>
       Max@{Ceiling[s*frameCount], 1}
      },
     1
     ];
   If[start >= stop, 
    Throw[
     Failure["BadBlock", 
      <|"MessageTemplate" -> 
        "Block start `` must be smaller than block end ``", 
       "MessageParameters" -> {start, stop}|>
      ],
     "GIFBlockMap"
     ]
    ];
   {start, stop, step}
   ];
gifBlockMapRead["GIF", frame_, ___][gif_, ___] :=
  Replace[
   Switch[
    frame,
    _Integer,
    {GIFTools`Private`$ReadOneFrame[gif, frame]},
    {__Integer},
    GIFTools`Private`$ReadOneFrame[gif, #] & /@ frame,
    All,
    GIFTools`Private`$ReadAllFrames[gif],
    _,
    $Failed
    ],
   Except[{__Image}] :>
    Throw[
     Failure["BadFrames", 
      <|
       "MessageTemplate" ->
        "Can't read frames `` from ``", 
       "MessageParameters" -> {frame, gif}
       |>
      ],
     "GIFBlockMap"
     ]
   ];
gifBlockMap[
   func_,
   gifFile : (_String | _File), 
   mapFunc : Map | Scan : Map,
   blockSpec :
    Span[
     _Integer | Scaled[_?(0 <= # < 1 &)],
     _Integer | Scaled[_?(0 < # <= 1 &)] | All,
     _Integer?Positive | Scaled[_?(0 <= # <= 1 &)] | All
     ],
   sampling : _Integer?Positive : 1
   ] :=
  Function[Null, Catch[#, "GIFBlockMap"], HoldFirst ]@
   Block[
    {
     System`Convert`CommonGraphicsDump`IEImageRead = gifBlockMapRead
     },
    Module[
     {
      gif,
      frameCount,
      start,
      end,
      step,
      curFrame,
      oldFrame,
      frameRange
      },
     gifBlockMapLoadLib[];
     gif = gifBlockMapGetFile[gifFile];
     frameCount = gifBlockMapGetImageCount[gif ];
     {start, end, step} = 
      gifBlockMapGetBlockSpec[frameCount, blockSpec];
     curFrame = start;
     Switch[mapFunc, Map, Table, Scan, Do][
      oldFrame = curFrame;
      curFrame = 1 + Min@{curFrame + step, end};
      frameRange = Range[oldFrame, curFrame - 1, sampling];
      If[Length@frameRange > 0,
       func@Import[gif, {"GIF", "ImageList", frameRange}],
       Nothing
       ],
      Ceiling[frameCount/step]
      ]
     ]
    ];
gifBlockMap[
  func_,
  gifFile : (_String | _File), 
  mapFunc : Map | Scan : Map,
  blockSpec : _Integer?Positive | Scaled[_?(0 < # < 1 &)] | All : 10,
  sampling : _Integer?Positive : 1
  ] :=
 gifBlockMap[
  func, gifFile, mapFunc,
   1 ;; All ;; blockSpec, sampling
  ]

Then we can see this in action:

gif = URLDownload["https://i.imgur.com/7Wkns4d.gif"];

gifBlockMap[Blend, gif, Map, 10] // GraphicsRow[#, ImageSize -> 800] &

raccoon

It performs okay (at minimum it cuts down our memory consumption):

Blend /@ Partition[Import[gif, "ImageList"], 10] // 
  MaxMemoryUsed // AbsoluteTiming

{0.253601, 41266472}

gifBlockMap[Blend, gif, 10] // MaxMemoryUsed // AbsoluteTiming

{0.807644, 11808856}

We're just under four times slower and use bit under four times less memory.

Amusingly, this outperforms the direct "ImageList import, since we're being less general and don't use Image3D and things:

Import[gif, "ImageList"] // MaxMemoryUsed // AbsoluteTiming

{0.137272, 41266400}

gifBlockMap[Identity, gif, Map, All][[1]] // 
  MaxMemoryUsed // AbsoluteTiming

{0.118931, 20667552}

Even better though is to just directly call $ReadAllFrames:

Image3DSlices@GIFTools`Private`$ReadAllFrames[First@gif] // 
  MaxMemoryUsed // AbsoluteTiming

{0.034736, 41164192}

Or if you can work with the Image3D object instead:

GIFTools`Private`$ReadAllFrames[First@gif] // 
  MaxMemoryUsed // AbsoluteTiming

{0.017178, 20520712}

On the other hand for a big GIF this method makes things more tractable:

bigGIF =
  URLDownload["https://www.wolframcloud.com/objects/b3m2a1/chem/gradient_descent_walkers.gif"];

tm1 =
 gifBlockMap[Head, bigGIF, 35] // MaxMemoryUsed // AbsoluteTiming

{8.12303, 42231864}

tm2 =
 Head /@ Partition[Import[bigGIF, "ImageList"], UpTo[35]] // 
   MaxMemoryUsed // AbsoluteTiming

{1.83082, 470784832}

MapThread[
  ((# - #2)/Max@{##}) &,
  {tm1, tm2}
  ] // N

{0.774613, -0.910295}

It takes 77% longer but use 91% less memory

$\endgroup$
  • $\begingroup$ Wouldn't it be nice if ImageFile* worked on GIFs? Given that it can handle multi-page tiffs, I don't understand the omission. $\endgroup$ – bobthechemist Jul 1 '18 at 0:19
  • $\begingroup$ @bobthechemist yep :) why they do what they do is unclear $\endgroup$ – b3m2a1 Jul 1 '18 at 5:10
  • $\begingroup$ In which version did GIFTools appear? $\endgroup$ – Mr.Wizard Jul 1 '18 at 6:17
  • $\begingroup$ @Mr.Wizard I don't have 10 to check, but if it's not working it might also mean the library isn't loaded? I'll check on a fresh kernel if this works. $\endgroup$ – b3m2a1 Jul 1 '18 at 6:21
  • $\begingroup$ @Mr.Wizard my original loadLib trick wasn't enough. Should work now. $\endgroup$ – b3m2a1 Jul 1 '18 at 6:29
7
$\begingroup$

The bad news is that your code spiked at about 60 GB RAM on my machine before the result is returned. The problem is clearly the Import routine. I would like to offer an alternative if you have more than 22 GB of RAM available.

data = Import[file, {"GIF", "RawData"}];

This gives you the list of frames. Each frame is a matrix of color-indices. So to reconstruct the first frame, you need to extract the color-map.

colorMap = With[{values = Import[file, {"GIF", "GlobalColorMap"}]}, 
   Thread[Range[Length[values]] -> values]];

This needs another 6 GB of RAM, so if you are unsure, then first extract this map and store it as mx locally. The map itself is tiny but again, Import seems to like eating RAM.

Then you could do

data[[1]] /. rules // Image[#, "Byte"] &

to get the first frame as an image. But since you are not interested in the frames, but you only need a flattened list, you can directly work on data.

I tested it with Mathematica 11.3 on OS X.

$\endgroup$
  • 3
    $\begingroup$ It is really stupid realization of Import for gifs. MMa reads all the file even when I ask just a number of frames.. $\endgroup$ – Rom38 Jun 29 '18 at 5:09
  • $\begingroup$ Yep. I didn't check the specs for GIF, but I'm sure this is not necessary. I might be wrong though. $\endgroup$ – halirutan Jun 29 '18 at 5:11
  • $\begingroup$ You need to read just the header for this.. $\endgroup$ – Rom38 Jun 29 '18 at 10:24
  • $\begingroup$ @Rom38 I'm not sure it needs that. Looking at the import elements for GIF it seems to be able to do things in a targeted manner, if we know the right elements to pick. $\endgroup$ – b3m2a1 Jun 30 '18 at 20:23

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.