How to efficiently select elements with the minimum value from a large list?

Question

How can I efficiently select sublists with the minimum value of the third elmement from the following list?

longList = {{1, 1, 12}, {1, 2, 10}, {1, 3, 12}, {1, 4, 12}, {1, 5, 10}, 
            {2, 1, 10}, {2, 2, 8}, {2, 3, 10}, {2, 4, 10}, {2, 5, 8}, 
            {3, 1, 12}, {3, 2, 10}, {3, 3, 12}, {3, 4, 12}, {3, 5, 10}};
Select[longList, #[[3]] == Min[longList[[All, 3]]] &]

Assume all numbers are integers and that longList is quite large, containing around 100 million elements.

Answer 1

Random data

longList = RandomInteger[100, {1000,5}];

Your solution

AbsoluteTiming[
   Select[longList, #[[3]] == Min[longList[[All, 3]]] &]
]

(* {0.017431,{{100,81,0,36,91},{6,33,0,76,0},{9,9,0,19,16},{56,81,0,16,10},{72,31,0,9,3},{34,97,0,38,79},{53,93,0,32,37},{98,94,0,84,60}}}*)

My solution

Part, Position

AbsoluteTiming[
   Part[#, Flatten@ Position[#[[All, 3]], Min[#[[All, 3]]]]]& @ longList
]

(* {0.000173,{{100,81,0,36,91},{6,33,0,76,0},{9,9,0,19,16},{56,81,0,16,10},{72,31,0,9,3},{34,97,0,38,79},{53,93,0,32,37},{98,94,0,84,60}}} *)

MinimalBy, Extract

MinimalBy[Extract[3]]@ longList

Benchmarks

theTrends =Transpose@Table[
    With[
        {longList = RandomInteger[100, {k,5}]},
        {
            {k, First@AbsoluteTiming@Select[longList, #[[3]] == Min[longList[[All, 3]]] &]}, (* OP *)
            {k, First@AbsoluteTiming[Part[#, Flatten@ Position[#[[All, 3]], Min[#[[All, 3]]]]]& @ longList]}, (* rhermans *)
            {k, First@AbsoluteTiming@MinimalBy[Extract[3]]@ longList}, (* rhermans *)
            {k, First@AbsoluteTiming@Pick[longList, longList[[All, 3]],   Min[longList[[All, 3]]]]}, (* "user1066" *)
            {k, First@AbsoluteTiming[MinimalBy[longList, #[[3]] &]@longList]} (* PlatoManiac *)
        }
    ]
    ,{k, PowerRange[2, 2^17, 2]}
];

Answer 2

With[{x = longList[[All, 3]]}, Pick[longList, x, Min[x]]]

(* {{4, 29, 0, 15, 40}, {31, 3, 0, 3, 61}, {86, 12, 0, 13, 54},
    {48, 56, 0, 4, 94}, {19, 94, 0, 56, 28}, {90, 57, 0, 91,51}, 
    {76, 14, 0, 70, 84}} *)

Original Answer

 Pick[longList, longList[[All, 3]], 
  Min[longList[[All, 3]]]] // AbsoluteTiming


(* {0.000064, {{4, 29, 0, 15, 40}, {31, 3, 0, 3, 61}, {86, 12, 0, 13, 
   54}, {48, 56, 0, 4, 94}, {19, 94, 0, 56, 28}, {90, 57, 0, 91, 
   51}, {76, 14, 0, 70, 84}}} *)

---

Using longlist as defined by @rhermans

longList = RandomInteger[100, {1000,5}];

and

Select[longList, #[[3]] == Min[longList[[All, 3]]] &] == 
  Pick[longList, longList[[All, 3]], Min[longList[[All, 3]]]]

(* True *) 

Answer 3

Your solution is not bad, but the problem with your solution is that Min[longList[[All, 3]]] is evaluated repeatedly for every element in longList. Calculate the minimum once:

longList = {{1, 1, 12}, {1, 2, 10}, {1, 3, 12}, {1, 4, 12}, {1, 5, 
    10}, {2, 1, 10}, {2, 2, 8}, {2, 3, 10}, {2, 4, 10}, {2, 5, 8}, {3,
     1, 12}, {3, 2, 10}, {3, 3, 12}, {3, 4, 12}, {3, 5, 10}};
min = Min[longList[[All, 3]]];
Select[longList, #[[3]] == min &] // AbsoluteTiming

{0.0000158, {{2, 2, 8}, {2, 5, 8}}}

Answer 4

This should also work reasonably fast and for large lists.

longList = RandomInteger[100, {100000000, 5}];
MinimalBy[longList, #[[3]] &]; // AbsoluteTiming

{5.28234, Null}

Answer 5

Grabbing the longlist defined by @rhermans:

longList = RandomInteger[100, {1000,5}];

Using PositionSmallest and Extract:

Extract[longList, Outer[List, PositionSmallest[longList[[All, 3]]]]] ==
Pick[longList, longList[[All, 3]], Min[longList[[All, 3]]]]

(*True*)

Answer 6

Using Reap, Sow and Scan:

 Reap[Scan[If[#[[3]] == Min[longList[[All, 3]]], Sow[#]] &, 
   longList]][[2, 1]]