Greetings. This is my first post here, also, I am not a seasoned programmer, so any advice pertaining to coding that is not directly related to the problem (stated below) is welcome. I am having trouble with a piece of code that, seemingly, does not compute as I intend it to do. In fact, it seems to do nothing at a certain line. My question here actually contain two parts;
(i) My specific problem is located in that I am trying to make a computation on kernel C involving data gathered in kernel A and kernel B. The data gathered in A and B are complex numbers which I "send" to C so it can form a table of differences modulus squared, so the table of data that should result from the computation in C is just real numbers.
Some structure: (Defining relevant tables. Purpose will become clear as we go.)
datalist1={};
datalist2={};
datalist11={};
datalist22={};
dataC={};
(Letting kernels be aware of the definitions. Also, only letting certain tables be shared among kernels)
DistributeDefinitions[datalist1,datalist2,datalist11,datalist22,dataC];
SetSharedVariables[datalist11,datalist22];
(My code uses 3 parallel kernels as it is now. Two of them runs some rather heavy numerical integration, and the third kernel has been given the purpose as to control (when to stop them) these integrations via some control object. It is in NDSolve's StepMonitor that I gather data (complex numbers) in a table "datalist1" (and also "datalist2") from the integrations at points chosen to occur regularly. Further, at points in time, I pass this data over to the tables "datalist11" and "datalist22" which are shared among kernels. I want to use these tables of data in kernel C. I run into problems on kernel C when I want to make said table which will contain real numbers.)
(This "While" acts as a stop, not letting the code progress until conditions are met. "go1" and "go2" are shared real numbers that acts as control parameters, being set from kernels A and B. Piece of code below occurs in kernel C.)
While[go1 < 1 || go2 < 1, Pause[1/100000]];
Do[
AppendTo[dataC,
Sqrt[Sum[
Abs[(datalist22[[h, k]] - datalist11[[h, k]])]^2, {k, 1,
201}]]],
{h, 1, 8}];
Print["Broke through!"];
(In the above, the computation halts at the "Do" command, as in; Nothing more comes out of kernel C. I never get to see the message "Broke through!".)
I checked that, yes indeed, kernel C does get the data ("datalist11" and "22") as intended since I could let it print some numbers for me on screen which were ok. But then, why does kernel C halt itself at that point mentioned? The computation of the Do-loop in kernel C is usually a fast one, and no changes to the tables of data is done for a length of time which is more than enough (Do-loop take < second and changes are made to "datalist11" and "22" after about 40 or more seconds), so nothing should be lost in parallel.
The way I am doing the entire thing in parallel is through "ParallelTable". That is, each kernel I mention is set to work under ParallelTable. Three kernels are active because I have set three spaces on the table for computing (two for NDSolve and one for control computation). Is this perhaps a bad way to do it? I thought it was desirable from the standpoint of NDSolve.
(ii) Regarding parallel computing, communication/sharing of data between kernels seems rather stable with the tool SetSharedVariable[] (or the likes), but when incorporating data gathered in "kernel A" to be used in a computation in a parallel "kernel B" the situation seem to get more delicate. What are the "do's and dont's" in any such situation? I know I say that "things seem to be..", so a more concrete example would be my specific problem.
Thanks in advance for any advice. //Patrik
EDIT(1): (Response to Jagra)
@Jagra Hey there and thanks for the input. I am looking into the post you linked (although your link was "incomplete" with a missing "m", I'm guessing it was this post :)). The data I am gathering should indeed be gathered in the master kernel since no direct communication between kernel A and kernel B is made. I am under the impression that the function of SetSharedVariable[] is, that kernel A and B can read and alter any shared variable but that the alteration is made in the master kernel (aswell as parallel kernels?). The power in this should be that, in my case, kernel A and B produce tables of data that is set as shared variables, and with this, kernel C can read the data gathered in A and B from the master kernel; thus in kernel C there should be no hindrance to create new tables of data with this data from A and B. But this is where my code get stuck... It just won't create a simple table.
EDIT(2): Found it out.
I found out what was going wrong in this instance. I had in the problem details above written that the Do-loop concerned was not evaluating. On that I was wrong, it was evaluating, although in a subtle manner! The problem was it was taking too long so that the whole parallel process gave nothing useful because of kernels clashing in "which kernel made what changes and when". Why it was taking so long was because each time the Do-loop wanted to insert a value into a table (as specified) it had to get the whole list of data and then just pick the current one to go in. It takes considerably longer time to ask for the whole list of data in evaluated form, and I noticed that the time to go through the Do-loop approximately doubled each time (no good) since the lists of data kept growing. A solution here was to simply use Unevaluated[], so each time I wanted to write into said table I took only the element in interest in evaluated form. This reduced computing time back to constant each time the Do-loop computed.
The subtle part to this was that when the Do-loop was to be computed the processor usage went down to almost zero, jumping around somewhere in the interval 0-3%. This is why I naturally thought something died in the computation. But I guess the kernels were just communicating real slow through Master.
So, a tip when coding for parallelism, use Unevaluated[] to increase speed in instances like these. You do not read every book in the library each day you go there just to get to read the one book you actually wanted to read that day, but merely acknowledging the rest of the books and that they will be available some other day when you do want to read them.
