0
$\begingroup$

I have a large data set that I am calculating, and some elements do not return a value (meaning it does not exist and is valid, but returns a blank list that cannot be further evaluated in the module).

Here is a very small portion of the data.

data = 
  {{150, 5, 0.1, 1, {156751/500, 0}}, 
   {150, 5, 2., 1, {1/100 (20343 + (57 First[{}])/5), 57}}, 
   {150, 50, 0.1, 1, {154217/500, 0}}, {150, 50, 2., 1, {78353/250, 0}}}

As you can see, one of the elements could not complete the evaluation, and returns a First[{}]. What I want to do is identify all cases where this takes place (they are not identical) and replace them with a constant number, say 290.

The data set I am calculating is massive and I cannot effectively remove these through breaking up the evaluation around these points.

The extra level in the 5 position in each second-level sub-element lists the desired return value and how many data points had to be thrown out that did not meet a specific condition (i.e., statistical fluctuations resulted in false triggers of the critical level). This is important to identify how robust the result is for the other parameters listed in positions 1 through 4 in each second level of data.

I have tried various things, but I cannot find how to identify elements of a list that are not real (can do positive, negative, etc. as shown below).

x = {3, 4}
xx = {3, 4, p}
x /. x_?Positive -> 0
xx /. x_?NumericQ -> 0

Any help would be appreciated.

$\endgroup$
  • $\begingroup$ Do you want to replace the whole number, with 290, or just the First[{}]? $\endgroup$ – Itai Seggev Jul 20 '17 at 6:42
  • $\begingroup$ For one, you should be aware that First has a second argument that specifies a default. For example, First[{}] does not evaluate, but First[{}, "Error"] gives "Error". This could be useful for you here. For example, you could use First[..., Nothing], to automatically make elements where First doesn't work disappear. $\endgroup$ – Sjoerd Smit Jul 20 '17 at 11:04
  • $\begingroup$ @Kuba @Sjoerd. Here is part of the code in the Module I am using (too many characters to include it all). res[[k]] = First[Flatten[Position[zmov*MovingAverage[Table[Log[(okrate + ksinb)/ksinb] ntotal[[i]] - (okrate), {i,1, Length[Testsprt]}], zmov], val_ /; val > a]]]/div + (zmov - 1)/div]; pez = {Mean[DeleteCases[res[[All]], x_ /; x < 300]], Length[DeleteCases[res[[All]], x_ /; x > 300]]};pez. $\endgroup$ – ProfessorL Jul 20 '17 at 19:18
  • $\begingroup$ @Sjoerd Thanks. I should have looked at the input options of First more closely. In some cases the threshold was never reached, so position returned an empty list, and therefore we got First[{}]. By placing First[expr,290] I got what I wanted. Thanks. $\endgroup$ – ProfessorL Jul 20 '17 at 19:23
  • $\begingroup$ @ProfessorL consider adding working example to the question and then self answering with fixes that fit your needs. $\endgroup$ – Kuba Jul 20 '17 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.