0
$\begingroup$

Consider the following code, which generates some data and performs some selections:

DataSamples = 
  Join[Partition[RandomReal[{0, 1}, 10^7], 1], 
   Partition[Sqrt[RandomReal[{0, 1}, 10^7]], 1], 2];
DataSamples2 = Exp[-DataSamples];
SelDataCompiled = 
 Hold@Compile[{{data, _Real, 
      2}, {index1, _Integer}, {index2, _Integer}, {value1, _Real}, \
{value2, _Real}}, 
    Select[data, #[[index1]] > value1 && #[[index2]] < value2 &], 
    CompilationTarget -> "C", RuntimeOptions -> "Speed"] // 
  ReleaseHold
index1=1;
index2=2;
SelDataCompiled[DataSamples2, index1, index2, 0.3, 
   0.7]; // AbsoluteTiming

I would like to make an association corresponding to different sets of the values value1, value2:

SelDataAssociation[data_, index1_, index2_, set_] := 
 Association[{{data, index1, index2, "1"} -> 
     SelDataCompiled[data, index1, index2, 0.5, 0.5], {data, index1, 
      index2, "2"} -> 
     SelDataCompiled[data, index1, index2, 0.3, 0.7], {data, index1, 
      index2, "3"} -> 
     SelDataCompiled[data, index1, index2, 0.1, 0.9]}][{data, index1, 
   index2, set}]

This code works a bit slower than if just calling the compiled function:

SelDataAssociation[DataSamples2, index1, index2, "1"]; // AbsoluteTiming
SelDataCompiled[DataSamples2, index1, index2, 0.5, 
       0.5]; // AbsoluteTiming

{2.24648,Null}

{0.652332,Null}

Could you please tell me how to use associations efficiently in the given case?

$\endgroup$

1 Answer 1

1
$\begingroup$

SelDataAssociation is 3x slower because it calls SelDataCompiled 3 times.

Are you looking for a convenient way to call your values? If so set up an association like mappings

keys = CharacterRange["1", "3"];
values = {{0.5, 0.5}, {0.3, 0.7}, {0.1, 0.9}};
mappings =  AssociationThread[keys, values] 
(* <|"1"->{0.5`,0.5`},"2"->{0.3`,0.7`},"3"->{0.1`,0.9`}|> *) 

foo[data_, index1_, index2_, key_] :=  SelDataCompiled[data, index1, index2, mappings[key][[1]], mappings[key][[2]]]

foo[DataSamples2, index1, index2, "1"]; // AbsoluteTiming (* 0.655975 *)

foo[DataSamples2, index1, index2, "1"] === 
 SelDataAssociation[DataSamples2, index1, index2, "1"] (* True *)

After thought

Personally I would prefer to redefine the selector function to take a list as an argument, eg:

f[data_, index1_, index2_, {value1_, value2_}]

instead of

f[data_, index1_, index2_, value1_, value2_]

and then use Part directly on values instead of creating an association.

ie:

values = {{0.5, 0.5}, {0.3, 0.7}, {0.1, 0.9}};

(* using Chris Degnen answer from https://mathematica.stackexchange.com/q/276400/60568 instead of SelDataCompiled *)
PickData2[data_, index1_, index2_, {value1_, value2_}] := 
 Pick[data, 
  UnitStep[-data[[All, index1]] + value1] + 
   UnitStep[data[[All, index2]] - value2], 0];

PickData2[DataSamples2, index1, index2, values[[1]]] === 
 SelDataAssociation[DataSamples2, index1, index2, "1"] (* True *)

This has the benefit of being able to use Map across values which is faster as it autocompiles.

PickData2[DataSamples2, index1, index2, #] & /@ values; // RepeatedTiming
(* 1.77766 *)

vs

SelDataAssociation[DataSamples2, index1, index2,"1"]; // RepeatedTiming
(* 2.30011 *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.