85 reputation
6
bio website
location
age
visits member for 1 year, 11 months
seen May 20 '13 at 16:39

Mar
21
comment How do I find the best parameter to fit my data if the model is a interpolating function?
NVM. I just realized it's the square of the residuals between what's predicted by the interpolation and the actual data.
Mar
21
comment How do I find the best parameter to fit my data if the model is a interpolating function?
@chris What is the //#.#& at the end of the chi2 function?
Jan
15
comment Efficiently extracting an array subset given a separate array
(cont'd) the result using your solution would be result = {{string1,145745,a}, {string3,56546,b}, {string7,145745,a}, {string7,12355,b}}, which is not correct since array2 does not have {string7,145745}. The reason it picked {string7,145745} is because the 145745 is also present with string1. My slight modification corrects this, and I must still commend you for coming up with your solution. Without it, I would not have been able to do this.
Jan
15
comment Efficiently extracting an array subset given a separate array
Yes, the Intersection is not needed with the modifition, but the Mapped result will need to be Flattened. The reason the Intersection does not work, at least in my case, is that memberPositions is first mapped to select all stringN and then mapped to select all integers K without requiring that the strict combination of {stringN, K} be present. Thus, if array1 = {{string1, 145745, a},{string2, 56546, a},{string3, 56546, b},{string3, 246, b},{string7, 145745, a},{string7, 12355, b}} and array2 = {{string1, 145745, 3.14324},{string3, 56546, -0.34319},{string7, 12355, 0.23535}}
Jan
15
comment Efficiently extracting an array subset given a separate array
To give others an idea of the incredible speed, Pick takes over 25 minutes for array1 size 1.5 million rows and array2 size of 8700. Shifrins' solution takes 3.47 seconds. I've also performed the solution on my total dataset (array1 has 19.7 million rows, array2 has 175,000 rows) and it took 121 seconds. Breaking array1 down into 21 chunks, it takes 72.1 seconds. This is tested on V6.0, Intel Corei7 x4 2.4GHz, 8GB RAM. I didn't test whuber's solution as I have V6.0 on this computer.
Jan
15
comment Efficiently extracting an array subset given a separate array
I used array1[[Intersection @@ Map[memberPositions[array1[[All, #]], array2[[All, #]]] &, {{1, 2}}]]]
Jan
15
comment Efficiently extracting an array subset given a separate array
This solution is elegant and FAST! I was thinking that I should label the positions in array1 as is done in memberPositions but thought it would hinder performance. I was certainly wrong. I did make a minor amendment to the solution, though. I Mapped the memberpositions to {{1,2}} rather than {1,2} to explicitly require that both the first and second column of any given row matches the a row in the other array. Otherwise, I might collect unwanted rows, such as {string7, 145745, a} if it were in array1 but not in array2.
Jan
15
comment Efficiently extracting an array subset given a separate array
I also thought to use Intersection but then realized that, since array2[[All,{1,2}]] is a subset of array1[[All,{1,2}]], the Intersection is already simply array2[[All,{1,2}]].
Jan
15
comment Efficiently extracting an array subset given a separate array
@PinguinDirk Sorry for the confusion. VLC is correct in that I made a mistake, which has been corrected above. Previously, there was not {string7, 12355, a} element in array1, but there should have been and it is there now. Also, I have tried Select, Cases, and Position in various combinations identical or similar to yours, but, unfortunately, those setups are dismally slower than the Pick command I've implemented so far, around 1 order of magnitude slower at best.
Oct
5
comment Overlapping Red and Green ArrayPlots to show yellow intersections
The retention of color dominance is extremely informative and useful.
Oct
4
comment Overlapping Red and Green ArrayPlots to show yellow intersections
Brilliant! This is exactly what I was trying to figure out. Thanks!
Oct
4
comment Overlapping Red and Green ArrayPlots to show yellow intersections
Yes, this is a good example, EXCEPT the heart of my question is how to specifically get YELLOW intersections where red and green intersect. "Stronger" overlap (defined by red and green addition) yields a "stronger" yellow color.