0
$\begingroup$

I have a data file which contains thousands of lines and each line has eight elements. Here is a small sample of the data file

-4.00      -0.80   0.1886024468848907E+01   0.1467147621657460E+01   -.1217067274319363E+01   0.7206100000000000E+03   0.7693457688734395E-12    5
-4.00      -0.70   0.1430357986632780E+01   -.1404093461650013E+01   -.1742223680347601E+01   0.1824700000000000E+03   0.8439003681169850E-12    8
-4.00      -0.60   -.1324719768465547E+01   0.1740130076002850E+01   0.1497978622206873E+01   0.5479900000000000E+03   0.4264485578634903E-14    2
-4.00      -0.50   0.1358536876189560E+01   -.1580696533502541E+01   0.1621539980382560E+01   0.2881100000000000E+03   0.1319885603098060E-13    4
-4.00      -0.40   -.1588487538231399E+01   0.1275577589608218E+01   0.1707607247015512E+01   0.1337500000000000E+03   0.1057878487713421E-12    2
-4.00      -0.30   0.1755414125374284E+01   0.1332710827520201E+01   0.1477984475201826E+01   0.6426400000000000E+03   0.1459764022611444E-13    1
-4.00      -0.20   0.1245972697710741E+01   0.1540633564543885E+01   0.1777167629372046E+01   0.6112200000000000E+03   0.5718386586661092E-13    1
-4.00      -0.10   -.1311461418732105E+01   -.1594149065989313E+01   0.1661344176980193E+01   0.3507800000000000E+03   0.6765588799377521E-14    3

I read this file using

data = ReadList["data.out", Number, RecordLists -> True];

The total length of the list is obtained, of course as

ntot = Length[data];

The list contains eight elements per row and the last of them is an integer taking values in the interval [0,8]. What I want is the following:

(a). Count how many rows have 0 value at the last element (let's suppose there are n0), how many have 1, 2, 3, ... , 8. Then calculate the corresponding percentages per0 = n0/ntot, per1 = n1/ntot, etc. It could be nice if this was inside a DO loop with i = 0,8.

(b). Count again percentages but using more than one criteria this time. For example, count how many rows have 1 at the last element and the value of the seventh element is smaller than 10^{-4}.

Any suggestions?

EDIT

Using's @Kuba's solution we have

{{5, 656}, {8, 640}, {2, 673}, {4, 663}, {1, 673}, {3, 663}, {6, 656}, {7, 640}, {0, 19}}

Is it possible to divide each sum automatically with ntot thus obtaining the percentages?

{{5, 656/ntot}, {8, 640/ntot}, {2, 673/ntot}, {4, 663/ntot}, {1, 673/ntot}, {3, 663/ntot}, {6, 656/ntot}, {7,640/ntot}, {0, 19/ntot}}

Also it would be great if they were sorted from 0 to 8 not randomly as they are now.

$\endgroup$
7
  • 1
    $\begingroup$ try data[[ ;; , 8]] // Tally there is all what you need to calculate those %. $\endgroup$
    – Kuba
    Aug 18, 2013 at 7:26
  • $\begingroup$ @Kuba Not exactly what I need; see my EDIT. What about the case where two criteria are used? $\endgroup$
    – Vaggelis_Z
    Aug 18, 2013 at 7:41
  • 1
    $\begingroup$ The problem is the at each step you need functions that are called exactly as the procedure which you want to do (so they are easy to find). Earlier it was Count or Tally, now you are asking for SortBy and at the end Cases or Select. And this is a problem because it shows no research effort :/ $\endgroup$
    – Kuba
    Aug 18, 2013 at 7:46
  • $\begingroup$ @Kuba Not at all, I 'm just unfamiliar with list manipulation. Perhaps you should post a complete answer showing these functions so I can accept it. $\endgroup$
    – Vaggelis_Z
    Aug 18, 2013 at 7:51
  • 1
    $\begingroup$ I'm sure that you can handle this and post an answer. Also remember that Divide is Listable so you can just SortBy[data[[;; , 8]] // Tally, 1][[;;, 2]] / ntot. For extended criteria use Cases or Select. $\endgroup$
    – Kuba
    Aug 18, 2013 at 8:02

2 Answers 2

2
$\begingroup$

There are many ways to acheive this:

SetDirectory@NotebookDirectory[];
data = Import["list.txt", "Table"] (*I saved your sample in txt file*)

ntot = Length@data;

temp = SortBy[Tally@data[[;; , -1]], 1];
temp[[;; , 2]] = temp[[;; , 2]] 100/ntot // N; 
(*I've multiplied by 100 to get % value not the ratio*)
temp
{{1, 25.}, {2, 25.}, {3, 12.5}, {4, 12.5}, {5, 12.5}, {8, 12.5}}

What can be useful, but the difference is it will "count" also what is not there (e.g 7):

viahist = HistogramList[data[[;; , 8]], {1}, "Probability"]
{{0, 1, 2, 3, 4, 5, 6, 7, 8, 9},
  {0., 0.25, 0.25, 0.125, 0.125, 0.125, 0., 0., 0.125}}

notice that first part is a list of intervals limits so at the end:

{Most@#, #2 100} & @@ viahist // Transpose
{{0, 0.}, {1, 25.}, {2, 25.}, {3, 12.5}, {4, 12.5}, {5, 12.5}, 
{6, 0.}, {7, 0.}, {8, 12.5}}

For your second question, one way to do this is:

Length@Select[data, #[[7]] < 10^(-13) && #[[8]] == 2 &] 100 / ntot // N
12.5

$\endgroup$
3
  • $\begingroup$ @Vaggelis_Z but please, do not accept, this question is so broad that, unless it's closed, it can provide many interesting answers. I hope it will help you. $\endgroup$
    – Kuba
    Aug 18, 2013 at 8:41
  • $\begingroup$ Everything is great! But I cannot understand why you don't want to accept your answer? A minor question: is there any difference between Lenght@data and Length[data]? I mean is it more fast? I always use the standard [ ] notation. $\endgroup$
    – Vaggelis_Z
    Aug 18, 2013 at 8:45
  • $\begingroup$ @Vaggelis_Z Because better do not discourage others :) it is good habbit to wait with an accept. About @, take a look here. In this case it does not matter. $\endgroup$
    – Kuba
    Aug 18, 2013 at 8:48
1
$\begingroup$

You could write functions to generalize this but for your specific problem:

data = Import["list.txt", "Data"];
ntot = Length[data];

(a) to get percentages of elements in the last column (you could optionally Sort this list too)

Inner[Times, Tally[data[[All, -1]]], {1, 100./ntot}, List]

Note that I used Inner here but normally I use Part for this sort of thing but Kuba did something similar:

tmp = Tally[data[[All, -1]]];
tmp[[All, 2]] = tmp[[All, 2]]*100./ntot;

(b) to get percentage when last column is 1 and a condition is applied on the second last column

Length[Cases[data, {__, z_ /; z < 10^-4, 1}]]*100./ntot
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.