Follow each pixel brightness in set of images (Memory constrained)

Question

I have a set of 2700 consecutive png images (8bit, 2048*2048 pixels).

I would like to visualize the intensity variation of each pixel vs. image number.

Here is an example of 10 images: http://goo.gl/52McV3

At first I wanted to store the intensity of each pixel in each image into an array. Unfortunately the code below does not work properly. What am I doing wrong?

ChoiceDialog[{FileNameSetter[Dynamic[imageDir], "Directory"], 
   Dynamic[imageDir]}];
SetDirectory[imageDir];
fNames = FileNames["*.png"];
numFiles = Length[fNames];

SetSharedVariable[pixelIntensity];

pixelIntensity = Array[0 &, {2048, 2048, numFiles , 1}];

ParallelTable[

  image = Import[fNames[[i]]];

  ParallelTable[
   pixelIntensity[[r, c, i]] = 
    ImageData[ImageTake[image, {r, r}, {c, c}]],
   {r, 1, 2048}, {c, 1, 2048}
   ],

  {i, 1, numFiles}

  ]; 

I have RAM problems when many images (2700) have to be processed

Answer 1

The data

FileNames["*.png"]
(* {"image_01.png", "image_02.png", "image_03.png", \
"image_04.png", "image_05.png", "image_06.png", "image_07.png", \
"image_08.png", "image_09.png", "image_10.png"} *)

All at once

If there are no memory constraints, you can load all in a single array (read below for other cases).

data = ImageData[Import[#], "Byte"] & /@ FileNames["image_*.png"];

MinMax@Flatten[data]
(* {0, 97}*)

Manipulate[
 ListPlot[data[[All, x, y]], PlotRange -> {0, 100}]
 , {x, 1, Length[data[[1]]], 1}
 , {y, 1, Length[data[[1, 1]]], 1}
 ]

Efficient memory use

Loading data only for the requested pixel coordinates, by taking advantage of the options of Import that allow loading only specific parts of components using Import["file.png", {"Data", row, column}]

Also using Memoization, so if you call for the same data more than once, no work is repeated.

For a single coordinate pair

xyd[x_Integer, y_Integer] := 
 xyd[x, y] = 
  ParallelMap[Import[#, {"Data", x, y}] &, FileNames["image_*.png"]]

ListPlot[xyd[822, 920]]

For a list of coordinates

xyld[list_] :=
 xyld[list] =
  Transpose@ParallelMap[
    Diagonal@Import[#, {"Data", list[[All, 1]], list[[All, 2]]}] &
    , FileNames["*.png"]
    ]

Answer 2

This is what I'd do: You say most of the pixels are dark, and thus uninteresting, but some of them are bright. So I'd start by summing all images up to find the "bad" pixels:

files = FileNames[
   "*.png"];

totalBrightness = 0.0;
Monitor[Do[
   totalBrightness = ImageData[Import[f]] + totalBrightness, {f, 
    files}], f];

meanBrightness = totalBrightness/Length[files];    

brightestPixels = Reverse[Sort[Flatten[meanBrightness]]][[;; 100]];

ListLinePlot[brightestPixels, PlotRange -> All, 
 PlotLabel -> "Top 100 brightest pixel values (mean over all images)"]

Then you can load the images a second time and extract the pixel values for these positions:

brightestPixelLocations = 
  Position[meanBrightness, #][[1]] & /@ brightestPixels;

Monitor[pixelValues = 
   Table[Extract[ImageData[Import[f]], brightestPixelLocations], {f, 
     files}], f];

and plot them any way you like:

ListLinePlot[pixelValues\[Transpose][[;; 10]], 
 PlotLabel -> "Top 10 brightest pixels", PlotRange -> All, 
 AxesLabel -> {"Image Index", "Brightness"}]

ArrayPlot[pixelValues, ColorFunction -> GrayLevel, ImageSize -> 800, 
 FrameLabel -> {"Image index", "Top 100 Pixels"}]