Seeing as my initial comment seems likeSide note
I have performed a both elegantlarger revision on this answer to reflect input from Leonid and fast solution I'll postfurther testing to highligh points raised in our discussion, it still contains the initial solutions and benchmarks, however leonids code was updated to his newest version as an asnwer. Basically it's just using Outerit behaves similarly to carry out the same sorthis original yet scales better with respect to number of unique elements.
Answer
My initial comment and naive solution posted was to simply perform andrews already stated double loop you wrote inover the last part of your question(subslists,uniqe) element pais using Outer:
Outer[{#2, Count[#1, #2]} &, myData, {1, 2, 3, 4}, 1]
I honestly didn't expect it to be faster than solutions written specifically for this task, just though I'd mention it, since this is in my oppinion a nicer way of doing a "double map" over each elementThis elegang solution performs quite well in the first list with each element inregime of the last list. But it seemsgiven example data compared to be acceptable when comparing speeds with the other posted solutions.
Timing given to this and the other solutions here (using Do[code,{10000}];//AbsoluteTimin)answer as will be documented, I currently get the following timingshowever given large number of unique elements or very long sublists, this solution will scale better:
vincent2 quickCount: 0.255000
jVincent quickCount[data_,elms_] Outer: = 0.302000
Andrew myFun: 0.344000
Leonids myFun:Transpose@{elms, #} 0.772077
kgulers& /@ (Map[Tally[Join[elms, #]] &, myTally:myData][[1 ;;, 0.828083
Mr.1 Wizards;;, countBy:2]] - 0.9650961)
So it seems to do just fine with respect to computational efficiency with respect toThe above also outperforms any other answer (as of writing) and the input listnaive solution for the regime of the example data. It's assumed that the data only contains elements found in elms.
The better solutionperformance testing
So I finaly mangagedThe last solution above was added after Mr. wizard in response to get handlingthe original solution raised concerns regarding computational efficiency for larger lists. The issue in short is that when we loop over the sublists elements performing Tallyn's output fast enough iterations and then for each iteration perform m comparisons taking the time T = (n*(loopT+m*checkT)) while for the count-based algorithm we carry out a loop for each unique element throughout the lists and then iterate through the sublist, thus: T=m*(n*(loopT+checkT)). Thus the latter has an additional temporal cost of (m-1)*loopT. Owing to make it fasterthe fact that loopT is small and that indeed in the givenquestion m was fixed at 4, I did not take issue with this scaling, but Mr.wizard is indeed correct that the asymtotic computational efficiency with respect to number of unique elements is worse for the Count-based solution than for the Tally based solution. If we assume that we are dealing with the 4 fixed unique elements listed in the myData test example, both solutions have the same asymptotic complexity in sublists lengths and number of subslist namely O(n) in both (verified below).
Now while it's nice to look at bigO, the real discussion arose because I had the aduacity to include test that showed that for the given usage case the naive algorithm outperformed Leonids Tally based solution. The performance meassurements are given below using two different input data, a randomized dataset of same dimensionality as myData, and a set containing much larger sublists (generation method is detailed at the end of the answer):
quickCount[data_]:= Map[Tally[Join[{1, 2, 3, 4},#]] &, data][[1 ;;, 1 ;;, 2]] - 1
vincent2 := quickCount[myData]
The unspoken detailtestdata was generated using SeedRandom[7529127];myData=RandomInteger[{1,4},{40,20}].
It is ofcausetrue however that this hasLeonids solution will become better if we use many more unique elements. My testing shows that starting from the myData sizes we see that Leonids solution outperforms the naive when (numElms>16,lengthList=20). Now a different output formatconsideration is the length of the input sublists. Even though both algorithms have the same asymptotic bigO scaling does not mean that they will both follow that scaling for small n, and to put it back into the same output format it becomes slowerexact prefactor needs to be know before any conclusion can be drawn.
testdata was generated using SeedRandom[7529127];myData=RandomInteger[{1,4},{40,500}];.
Looking at actual timings we see however that arguments based on asymptotic scaling don't really apply to Leonids solution until we reach lists that are more then 10^5 elements long, at which point we still can argue which is better since they all scale as O(n). The actual performance comparison becomes a question of finding the aboveprefactors which is seen to be much much larger for the Count-based solution, eg:specifically we see that in the regime of (numList=4,numList>~100) Leonids solution beats the naive, below the the naive outperforms his.
Transpose@{{1,2,3,4},#}& /@(Map[Tally[Join[{1, 2, 3, 4}, #]] &, myData][[1 ;;, 1 ;;, 2]] - 1)
runsThe above shows nicely Leonids objections, he was upset that the simpler solution was shown to be faster in the region of input similar to the example, 0.314031(region between 10-100), even though it scales better. I would characterize it as a case of "You pay for everything not just for what scales". His solution shows a nice scaling with respect to input list size, however it is still slower for the testtesting cases because of slow handling of the nicely scaling output of Tally. When dealing with actual usage such considerations should be taken into account, and it is not generally the case that a method exists that will outperform another in all regimes.
For completenessConcluding remarks
I chose originally notBigO scale reasoning is a great tool for improving algorithms, but should be backed up by actual testing with representative data in order to post this since it's just repeatingverify that indeed performs better. In particular when dealing with functions typically called with small input.
And on second note, one should never argue that timings based on actual use-cases aren't meaningful because they are not "general" with the codeimplied assumption that the "general" means representative of other answersvery large inputs, but since Leonid askedsimply for the sake of letting better scaling algorithms win the race. A timing showing MethodA faster than methodB for a given dataset means just that, here it is:even if the ordering would hot have been the same for a different use-case.
For completeness the profiled code bits
I chose originally not to post this since it' s just repeating the code of other answers, but since Leonid asked, here it is :
andrewMap[sublist_] := Map[{#,Count[sublist,#]}&,{1,2,3,4}]
andrew := andrewMap/@myData
vincentmyFun1[sublist_, elems_] :=
Outer[{#2,Count[#1,#2]}&,myData, Module[{1,2,3,4}tallyT = Transpose@Tally[elems ~ Join ~ sublist],1];
myFun[sublist_]: range = SortBy[Tally[sublist]Range[Length[elems]]},First]
myFun[sublist_ tallyT[[2,elems_] :range ]] = Replace[myFun[sublist~Join~elems],{el:Alternatives@@elems,n_}:>{eltallyT[[2,n range ]] -1} 1;
SortBy[Transpose[tallyT],1]; First]
];
leonid :=myFun[#= myFun1[#, {1,2,3,4}]&] & /@myData;@ myData;
countBy[dat_, bins_] := {bins, Tr /@ Reap[Sow[1, #], bins, Tr@#2 &][[2]]}\[Transpose] & /@ dat
wizard :=countBy[myData= countBy[myData, {1, 2, 3, 4}];
myTally[data_, elems_:{}] := Composition[# /. {e_, n_} :> {e, n - Boole[MemberQ[elems, e]]} &, Sort, Tally, Join][#, elems] & /@ data;
kguler := myTally[myData, {1, 2, 3, 4}];
Do[vincentAll the numbers for the timeings where found by looping many times over the code in order to get measurable timing per call. The random data was generated from SeedRandom[7529127].
1/10000*First[Do[jvincOuter, {10000}]; // AbsoluteTiming // FirstAbsoluteTiming]
Do[andrewThe asymptotic timing graph was created using(note the number of calls is larger when testing quicker calls):
testing=Hold[jvincOuter, jVquickCount,leonid2];
Table[
myData=RandomInteger[{100001,4}]; // AbsoluteTiming // First
Do[leonid, {1000020,10^n}]; // AbsoluteTiming // First
Do[wizard, {1000010^n,timePerCall[Max[1,10^(4-n+1)]][testing[[i]]]}]; // AbsoluteTiming // First
Do[kguler, {10000i,1,3}]; // AbsoluteTiming ,{n,1,6}]// FirstListLogLogPlot