0
$\begingroup$

I'm working with a set of data binning values. I'm using bincount and binlists to find bins that meet a certain criteria and then testing a list to see which are within one of those bins. This list of edges is exported to python code to use in a python application.

When testing a set of values to identify which are within any of those bins my code produces a differing count than Total@binCount[mydata, binwidth] does.

data = {1, 1, 5, 5 , 12, 12}
data // Length
Total@BinCounts[data, 3]

My bin candidacy code is wrong. I'd like to get the edges of the bins, something like this:

data = {1, 1, 5, 5 , 12, 12}
BinEdges[data, 3]
{{0, 2}, {3, 5}, {6, 8}, {9, 11}, {12, 14}}

data // Length
Total@BinCounts[data, 3]

Ultimately my code will look something like this.

validdata = RandomInteger[{0, 1000}, 10000];
testdata = RandomInteger[{0, 1000}, 1000000];
MapIndexed[If[ #1 > 10, #2[[1]], Nothing] &, BinCounts[validdata, 10]]

Then identify which values in test data lay within a bin with > 10 values in it. These edges will be export to a python application for use.

Is this the correct logic for generating bin edges and testing inclusion in a bin?

dx = 10;
data = Range[1, 100];
BinCounts[data, dx]
BinLists[data, dx]
data // Length
Total@BinCounts[data, dx]
binedges = 
 MapIndexed[{#2, start + (dx*#1) - dx, (start + (dx*#1)) - 1} &, 
  Range[1, ((data // Length)/dx) + 1]]
numinbin[nums_, start_, end_] := Map[ start < # <= end &, nums];

Edit: As I'm working with bignums brute force testing against bins is too slow.

Improve this question
$\endgroup$
1
  • $\begingroup$ I'm still working on this. Any attempt I try gets a different result from BinCounts/binlists. I really do need to know what binlists uses foe edges. Please help. $\endgroup$
    – kernel density
    Commented Feb 29 at 2:49

2 Answers 2

Reset to default
2
$\begingroup$

You may define a function: "whichBin" that tells ayou in which bin a numbers goes:

whichBin[x_, data_, dx_] := 
 Module[{start = Ceiling[Min[data] - dx, dx], 
   end = Floor[Max[data] + dx, dx], i = 1},
  While[start + i  dx <= x, i++];
  i
  ]

To test we can apply it to the numbers 0..12:

data = {1, 1, 5, 5, 12, 12};
whichBin[#, data, 3] & /@ Range[0, 12]

{1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5}
Improve this answer
$\endgroup$
2
  • $\begingroup$ This does technically work. But I am using bignums, so this is so slow it never finishes even for small lists of numbers. I'm stuck finding the bin edges. $\endgroup$
    – kernel density
    Commented Feb 16 at 19:55
  • $\begingroup$ If I'm understanding your answer correctly, removing , i = 1 and replacing While[start + i dx <= x, i++]; i with 1 + Floor[(x - start)/dx seems to speed things up considerably for larger amounts of values. I'm also not seeing the need for end = Floor[Max[data] + dx, dx]. But also even after several reads of the OP's question, I still don't understand it. $\endgroup$
    – JimB
    Commented Jul 25 at 4:24
0
$\begingroup$

When counting integers, no binning is necessary and you can simply count the instances:

data = RandomInteger[{0, 20}, 30];

Lookup[Counts[data], Range[0, 20], 0]
(*    {1, 0, 0, 0, 2, 5, 2, 1, 1, 0, 3, 0, 2, 0, 2, 4, 0, 0, 3, 1, 3}    *)

You can also use this trick if the bin size is different from 1: just Round or Floor or Ceiling the numbers to the desired size before counting,

data = RandomReal[{0, 1}, 20];

Lookup[Counts[Round[7*data]], Range[0, 7], 0]
(*    {0, 2, 5, 4, 2, 3, 2, 2}    *)
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.