5
$\begingroup$

As example I have the following two image sets (10 images each, png format, 8 bit gray scale) in two different directories and want to determine the mean intensity of each image:

set 1: https://drive.google.com/open?id=0B9wKP6yNcpyfRS1Pb1ctcmdFNUE

set 2: https://drive.google.com/open?id=0B9wKP6yNcpyfZDdjRk1ya0hqUWc

The code below produces an error. What is the reason that the files in dir_2 are not found?

ClearAll["Global`*"];

dirs = {"F:\\dir_1\\", "F:\\dir_2\\"};

ndir = Length[dirs];

SetSharedVariable[meanIntensity];

Do[

  SetDirectory[dirs[[j]]];

  dirBase = FileNameTake[dirs[[j]]];

  fNames = FileNames["*.png"];
  numFiles = Length[fNames];

  meanIntensity = Array[0 &, numFiles];

  ParallelDo[

   meanIntensity[[i]] = 
    ComponentMeasurements[Import[fNames[[i]], "png"], 
       "MeanIntensity"][[All, 2]][[1]];

   Print["j=", j, " i=", i, " meanIntensity=", meanIntensity[[i]]];

   , {i, 1, numFiles}

   ];

  , {j, 1, ndir}

  ];

The output is:

(kernel 2) j=1 i=5 meanIntensity=0.0228285

(kernel 4) j=1 i=1 meanIntensity=0.0225379

(kernel 3) j=1 i=3 meanIntensity=0.0227294

(kernel 1) j=1 i=6 meanIntensity=0.0227178

(kernel 4) j=1 i=2 meanIntensity=0.0225431

(kernel 3) j=1 i=4 meanIntensity=0.0224934

(kernel 2) j=1 i=7 meanIntensity=0.0223063

(kernel 1) j=1 i=8 meanIntensity=0.0227366

(kernel 4) j=1 i=9 meanIntensity=0.0227438

(kernel 3) j=1 i=10 meanIntensity=0.0225731

(kernel 4) Import::nffil :  File not found during Import.

(kernel 3) Import::nffil :  File not found during Import.

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 2) Import::nffil :  File not found during Import.

(kernel 1) Import::nffil :  File not found during Import.

(kernel 4) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 3) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

During evaluation of In[4]:= ComponentMeasurements::invarg1: Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

During evaluation of In[4]:= ComponentMeasurements::invarg1: Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

During evaluation of In[4]:= ComponentMeasurements::invarg1: Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

During evaluation of In[4]:= General::stop: Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 2) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 1) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 3) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) j=2 i=1 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 3) j=2 i=3 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 2) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 1) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 4) Import::nffil :  File not found during Import.

(kernel 3) Import::nffil :  File not found during Import.

(kernel 2) j=2 i=5 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 1) j=2 i=6 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 4) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 3) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 2) Import::nffil :  File not found during Import.

(kernel 1) Import::nffil :  File not found during Import.

(kernel 4) j=2 i=2 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 3) j=2 i=4 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 2) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 1) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 4) Import::nffil :  File not found during Import.

(kernel 3) Import::nffil :  File not found during Import.

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 3) Part::partd :  Part specification ComponentMeasurements[$Failed,MeanIntensity][[All,2]] is longer than depth of object.

(kernel 2) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 1) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 2) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 1) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 2) j=2 i=7 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 1) j=2 i=8 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 4) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 3) ComponentMeasurements::invarg1 :  Expecting a valid label matrix, an image, or a list of an image and a label matrix instead of $Failed.

(kernel 4) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 3) General::stop :  Further output of ComponentMeasurements::invarg1 will be suppressed during this calculation.

(kernel 4) j=2 i=9 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

(kernel 3) j=2 i=10 meanIntensity=ComponentMeasurements[$Failed,MeanIntensity]

When I use Do instead of ParallelDo everything is fine:

j=1 i=1 meanIntensity=0.0225379

j=1 i=2 meanIntensity=0.0225431

j=1 i=3 meanIntensity=0.0227294

j=1 i=4 meanIntensity=0.0224934

j=1 i=5 meanIntensity=0.0228285

j=1 i=6 meanIntensity=0.0227178

j=1 i=7 meanIntensity=0.0223063

j=1 i=8 meanIntensity=0.0227366

j=1 i=9 meanIntensity=0.0227438

j=1 i=10 meanIntensity=0.0225731

j=2 i=1 meanIntensity=0.0221354

j=2 i=2 meanIntensity=0.0223254

j=2 i=3 meanIntensity=0.0225109

j=2 i=4 meanIntensity=0.0220713

j=2 i=5 meanIntensity=0.0221103

j=2 i=6 meanIntensity=0.022274

j=2 i=7 meanIntensity=0.0221438

j=2 i=8 meanIntensity=0.0221835

j=2 i=9 meanIntensity=0.0223304

j=2 i=10 meanIntensity=0.0222357
$\endgroup$
  • 3
    $\begingroup$ I assume you would need to set the current directory on each subkernel as necessary. $\endgroup$ – ilian Sep 23 '16 at 16:25
  • $\begingroup$ How would that be done? $\endgroup$ – mrz Sep 23 '16 at 16:49
  • 2
    $\begingroup$ @mrz try to replace SetDirectory[dirs[j]] by With[{dirj = dirs[[j]]}, ParallelEvaluate@SetDirectory@dirj ] $\endgroup$ – Rolf Mertig Sep 23 '16 at 18:39
  • $\begingroup$ @Rolf Mertig: I tried your replacement but this even produces more errors: see the nb file: tinyurl.com/zsxxmy6 and the output: tinyurl.com/z4dynxg. $\endgroup$ – mrz Sep 24 '16 at 8:59
3
$\begingroup$

This works as expected. The key issue is to use the second argument of FileNames.

dirs={"F:\\dir_1\\","F:\\dir_2\\"};
ndir=Length[dirs];
SetSharedVariable[meanIntensity];
Do[
   fNames=FileNames["*.png",{dirs[[j]]}];
   numFiles=Length[fNames];
   meanIntensity=Array[0&,numFiles];
     ParallelDo[ meanIntensity[[i]]  = ComponentMeasurements[
                 Import[fNames[[i]],"png"],             
                 "MeanIntensity"][[All,2]][[1]];
       Print["j=",j," i=",i," meanIntensity=",meanIntensity[[i]] ]
     ,
     {i,1,numFiles}
    ]   
    ,
    {j,1,ndir}   
];
$\endgroup$
  • 1
    $\begingroup$ Great. Thanks a lot. Can you explain why this small change solves the problem? $\endgroup$ – mrz Sep 24 '16 at 9:22
  • 1
    $\begingroup$ It works because you give FileNames the real full file name and all Global` symbols are distributed over the parallel kernels. Why SetDirectory on parallel subkernels does not work, I do not know. Please ask or better call Wolfram support. They will certainly know more about the mysteries of differences of subkernels from the master kernel. $\endgroup$ – Rolf Mertig Sep 24 '16 at 13:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.