# Filtering data in one step rather than two

When generating a table, whose elements are of varying length, is it possible to delete an element from this list, dynamically, based on its length?

For example, given a data set, that is divided up into blocks of equal length, and applying a simple Grubbs filter to each of the blocks to remove elements we are not interested in:

SeedRandom[1];
TestList = RandomVariate[NormalDistribution[0, 1], 100];
BlockedData = TakeList[TestList,{25,25,25,25}];

Table[
Select[BlockedData[[i]], -0.11 < # < +0.11 &],
{i, 1, Length[BlockedData]}]


This would give the result:

{{-0.0148679, 0.0206518, 0.0977624}, {0.0516533, -0.0570916}, {-0.0321673}, {}}

Here a quite aggressive filter has been applied, so most points have been removed. When unfiltered, each block would contain 25 points (in this example). We can see that four blocks remain, respectively containing 3, 2, 1, and 0 points.

How could I simply delete the elements that contain less than two points inside the table at the same time it is created?

Here are two ways to do what I think you want.

Module[{item},
Table[
item = Select[BlockedData[[i]], -.11 < # < .11 &];
If[Length @ item > 1, item, Nothing],
{i, 1, Length[BlockedData]}]]


and

Module[{item},
Function[u,
item = Select[u, -.11 < # < .11 &];
If[Length @ item > 1, item, Nothing]]]
/@
BlockedData


Both of these return

{{-0.0148679, 0.0206518, 0.0977624}, {0.0516533, -0.0570916}}

• Very nice answer. Out of curiosity, why is Module necessary? It isn't a problem for me I am just curious. Is it simply to keep everything inside the Module local, such that only the output is global to the notebook?
– Q.P.
Nov 27, 2019 at 16:15
• @Q.P., I just didn't want item to be a global variable. Nov 27, 2019 at 16:16