Christopher Bowman
Reputation
Top tag
Next privilege 100 Rep.
Edit community wikis
 Mar21 comment How do I find the best parameter to fit my data if the model is a interpolating function? NVM. I just realized it's the square of the residuals between what's predicted by the interpolation and the actual data. Mar21 comment How do I find the best parameter to fit my data if the model is a interpolating function? @chris What is the //#.#& at the end of the chi2 function? Jan15 comment Efficiently extracting an array subset given a separate array (cont'd) the result using your solution would be result = {{string1,145745,a}, {string3,56546,b}, {string7,145745,a}, {string7,12355,b}}, which is not correct since array2 does not have {string7,145745}. The reason it picked {string7,145745} is because the 145745 is also present with string1. My slight modification corrects this, and I must still commend you for coming up with your solution. Without it, I would not have been able to do this. Jan15 comment Efficiently extracting an array subset given a separate array Yes, the Intersection is not needed with the modifition, but the Mapped result will need to be Flattened. The reason the Intersection does not work, at least in my case, is that memberPositions is first mapped to select all stringN and then mapped to select all integers K without requiring that the strict combination of {stringN, K} be present. Thus, if array1 = {{string1, 145745, a},{string2, 56546, a},{string3, 56546, b},{string3, 246, b},{string7, 145745, a},{string7, 12355, b}} and array2 = {{string1, 145745, 3.14324},{string3, 56546, -0.34319},{string7, 12355, 0.23535}} Jan15 comment Efficiently extracting an array subset given a separate array To give others an idea of the incredible speed, Pick takes over 25 minutes for array1 size 1.5 million rows and array2 size of 8700. Shifrins' solution takes 3.47 seconds. I've also performed the solution on my total dataset (array1 has 19.7 million rows, array2 has 175,000 rows) and it took 121 seconds. Breaking array1 down into 21 chunks, it takes 72.1 seconds. This is tested on V6.0, Intel Corei7 x4 2.4GHz, 8GB RAM. I didn't test whuber's solution as I have V6.0 on this computer. Jan15 comment Efficiently extracting an array subset given a separate array I used array1[[Intersection @@ Map[memberPositions[array1[[All, #]], array2[[All, #]]] &, {{1, 2}}]]] Jan15 comment Efficiently extracting an array subset given a separate array This solution is elegant and FAST! I was thinking that I should label the positions in array1 as is done in memberPositions but thought it would hinder performance. I was certainly wrong. I did make a minor amendment to the solution, though. I Mapped the memberpositions to {{1,2}} rather than {1,2} to explicitly require that both the first and second column of any given row matches the a row in the other array. Otherwise, I might collect unwanted rows, such as {string7, 145745, a} if it were in array1 but not in array2. Jan15 comment Efficiently extracting an array subset given a separate array I also thought to use Intersection but then realized that, since array2[[All,{1,2}]] is a subset of array1[[All,{1,2}]], the Intersection is already simply array2[[All,{1,2}]]. Jan15 comment Efficiently extracting an array subset given a separate array @PinguinDirk Sorry for the confusion. VLC is correct in that I made a mistake, which has been corrected above. Previously, there was not {string7, 12355, a} element in array1, but there should have been and it is there now. Also, I have tried Select, Cases, and Position in various combinations identical or similar to yours, but, unfortunately, those setups are dismally slower than the Pick command I've implemented so far, around 1 order of magnitude slower at best. Oct5 comment Overlapping Red and Green ArrayPlots to show yellow intersections The retention of color dominance is extremely informative and useful. Oct4 comment Overlapping Red and Green ArrayPlots to show yellow intersections Brilliant! This is exactly what I was trying to figure out. Thanks! Oct4 comment Overlapping Red and Green ArrayPlots to show yellow intersections Yes, this is a good example, EXCEPT the heart of my question is how to specifically get YELLOW intersections where red and green intersect. "Stronger" overlap (defined by red and green addition) yields a "stronger" yellow color.