Questions tagged [filtering]

Questions on filtering or selecting data based on certain criteria. This is to be used with primary tags such as list-manipulation, string-manipulation or signal-processing, with the primary tag indicating the specific use case.

Filter by
Sorted by
Tagged with
38
votes
9answers
5k views

Removing elements from a list which appear in another list

There are two lists {a, b, c, a, d, a, e} and {a, c, a}. I need to remove those elements from the first list which appears in a second list, to get {b, d, a, e}
70
votes
6answers
5k views

Does Mathematica have advanced indexing?

I have two $M \times K$ arrays $L, T$ where I would like to set all the elements in $L$ to zero whenever the corresponding element of $T$ is greater than 15. The ...
95
votes
4answers
3k views

What best practices or performance considerations are there for choosing between Cases, Position, Pick and Select?

Cases, Select,Pick and Position each have different ...
17
votes
7answers
5k views

How to find rows that have maximum value?

Suppose if I have following list { {10,b,30}, {100,a,40}, {1000,b,10}, {1000,b,70}, {100,b,20}, {10,b,70} } How to find rows that have max value in ...
16
votes
5answers
4k views

Select/Delete with Sublist elements?

Probably easy and short question, I still didn't fully figure out how to easily select/delete sublists from a list. Example: tt = {{2, 4}, {4, 8}}; I want to ...
6
votes
5answers
391 views

Is there an analogue of the Variables command for general expressions? [duplicate]

The command Variables[poly] gives me a list of all variables that appear in the expression poly, which involved sums, ...
21
votes
13answers
4k views

How to get list of duplicates when using DeleteDuplicates?

This might be easy, but can't find a way to use DeleteDuplicates to get also list of the actual duplicates. Example: ...
6
votes
1answer
1k views

Removing an ordered pair from a list of ordered pairs if the second element in the list is Less than a Value

I have a list of ordered pairs from a piezoelectric sensor sampled at 2kH. The first element is time and the second, amplitude. I'm trying to remove all ordered pairs in the list of ordered pairs when ...
17
votes
7answers
953 views

How to find all the positions of max value of a list efficiently?

For Example, list=RandomInteger[{1,100},2000]. Yes,I know Position[list,Max[list]] can do. But it's based on pattern matching!...
24
votes
4answers
5k views

Savitzky-Golay Filter to smooth noisy data

I do have noisy data and want to smooth them by a Savitzky-Golay filter because I want to keep the magnitude of the signal. a) Is there a ready-to-use Filter available for that? b) what are ...
6
votes
3answers
414 views

Using Part rather than Extract

In my work I found that I needed the Extract function. However I was wondering if there was a way to use Part. Assume that <...
20
votes
2answers
487 views

Clean and handy Options management/filtering

the case I want to be able to create a function with some default options but also without need to add full explicit list of options available for it. And then inside I want to be able to filter ...
17
votes
3answers
5k views

How to remove outliers from data

I have a data with noise which some times includes significant outliers. The position of the outliers are random. For example: ...
10
votes
2answers
355 views

How to define patterns to filter lists of replacement rules?

I have a list of replacement rules like list˘of˘replacement˘rules = {a -> 1, b -> 2, c -> 3} and I want to filter all the elements where ‘b’ is replaced ...
4
votes
1answer
252 views

Why is there a difference when formulating patterns for Select and FirstPosition?

Select[{1, 2, 4, 7, 6, 2}, # > 2 &] Gives {4, 7, 6}. OK! So one should think that the following would work: ...
23
votes
3answers
1k views

Select performance

I am fairly new to mathematica and working my way thru Paul Wellins book on Mathematica programming so experimenting with various language constructs. I was experimenting with ...
33
votes
7answers
23k views

Selecting every n-th element from a list

I have a list: data = {4, 5, 7, 8, 9, 5, 3, 2, 1, 2, 13, 12}; I want to take those elements in the list which are located at equal distances, say, every third ...
9
votes
1answer
545 views

Smoothen the curve found by EdgeDetect

It seems that curves found by EdgeDetect always persist $C^{0}$ continuity only. Consider the following example: ...
8
votes
1answer
988 views

Finding outliers in 2D and 3D numerical data

I'm working on dataset which isn't normally distributed.which contains three dimensions like cost, discount and profit I'm trying to find outliers in all these dimensions. I used z score to find ...
13
votes
2answers
2k views

Filtering and Replacing outliers

I have a set of data that measurements of temperature vs time (date). It is easy to realize when the sensor went wrong as one might see a blip on the data. How to teach to Mathematica to: 1) Detect ...
5
votes
1answer
2k views

Data filtering using Butterworth filter

I want to filter data using Butterworth filter. I am currently using Matlab and I want to know how to do it in Mathematica. I have the following data: ...
0
votes
2answers
550 views

Interpolating noisy data

My data looks like this: https://pastebin.com/n59CTB3L Or if displayed in log chart: I want to create an interpolation of the data that would remove most of the noise. What are some good ways to ...
5
votes
2answers
235 views

Removing anomalous points from data

I have two sets of 2d-data points which can be transformed in each other by using a certain transformation function. The problem is that in both data sets there are points which do not have ...
36
votes
3answers
6k views

Remove noise from data

Hi, as you can see above I have some experimental data which has a large offset and shows clear noise fluctations around the tendency of the curve. I wanted to ask if someone could suggest me a method ...
21
votes
14answers
3k views

Pick elements of largest absolute value

For example, given list = {{1, -3, -5}, {2, 1, 6}, {0, 2, 4}, {-9, 2, 6}} should return: {-5, 6, 4, -9} Updated I found ...
13
votes
2answers
846 views

Using Except with strings

Suppose that I have the following sample list, called list: list = {"bBb", "aAa", "aaa"}; Now suppose I want to select the ...
19
votes
8answers
1k views

How do I delete all items that occur more than once?

DeleteDuplicates works fine but leaves a single copy of the duplicated item. I need to remove all items that occur more than once i.e. {{1,2},{1,2},{3,4}} -> ...
8
votes
4answers
2k views

Select a repeated element in a list

I want to select all elements that repeat 4 times in the list. I propose this code ...
6
votes
7answers
2k views

Can Mathematica delete some of the 0's from a list?

I have a list: {1,2,3,0,0,4,5,0,0,0}. I want to delete the 0's occurring after the last positive integer. In other words, I ...
2
votes
2answers
988 views

How can I select elements that are true

I have a list of lists. For example, list = {{0,1,0,1,true},{0,0,0,0,false},{0,1,1,1,true},{1,1,1,1,false},{2,2,2,2,false}} I'd like to find all lists that ...
9
votes
5answers
759 views

Select a particular rules out of a list

rule = {beta -> 4, alpha -> 2, x -> 4, z -> 2, w -> 0.8} Select[rule, beta] Select[rule, beta &] Select[rule, beta -> _ &] How would I ...
4
votes
1answer
4k views

Select row from a table by column value [duplicate]

I have the following: animals = {{dog, black, 200}, {cat, white, 100}, {moose, brown, 500}, {zebra,black,100}} If this came from a SQL table called animals, ...
4
votes
1answer
681 views

Butterworthfilter data filter

I am currently doing a signal analysis and filtering using Matlab. The filter I am using is Butterworthfilter model to filter scattered data (vector data). I am using Matlab as follows: ...
16
votes
4answers
1k views

How to get cell boundaries in the image?

I have a microscope image of some animal tissue and wish to get the contours for all the cells that are present in the image. the cells are connected to the neighbouring cells via these contours. At ...
17
votes
3answers
1k views

Performance of Select

I have a dataset of 3D coordinates with a length of about $ 4\times 10^6 $. From this volume I am sequentially selecting coordinates along one axis and manipulating this subset. My question: Can the ...
11
votes
2answers
720 views

Select cases from a list

I have a list in the form : {1,1,1,0,-1,-1,1,1,1,0,0,0,0,-1,-1} No I'm trying to select only those sequences which start with 1 have a zero (or many zeros) ...
11
votes
7answers
3k views

How to select all elements above the main diagonal of matrix?

How to select all elements above the main diagonal of matrix? I need to create a list of them.
11
votes
3answers
432 views

How to remove a line of a matrix if not all elements are numeric?

I have a matrix a n columns and m lines. I'd like to efficiently remove the lines that are not entirely composed of numeric ...
18
votes
2answers
1k views

Why the BandpassFilter is different than applying LowpassFilter then HighpassFilter

If I generate some data with noise, and use a filter to filter the noise, why does BandpassFilter not produce the same results as using ...
10
votes
4answers
1k views

Fast method to select matrix elements based on a vector of positions

There is an operation for which I have long wanted to find a better solution. Let: a be a matrix of dimensions $m\times n$ v ...
8
votes
6answers
1k views

How to select sub-lists having at least one element that matches a pattern

This will probably be a very easy question, yet although I did find a relatively similar one, I couldn't use the answers there to solve my problem (I'm still a novice). I'm simulating 10 blocks of ...
15
votes
11answers
1k views

Delete redundant {x,y} pairs

Working through the problems from Hazrat's Mathematica book and there's a simple exercise to find all the square numbers where $n^2+m^2=h^2$ yields $h$ as an integer (I think they're also called ...
7
votes
4answers
714 views

Filter list based on date

What I would like to do is remove entries from a list of instrument data when it is in maintenance. The maintenance data I have is a series of dates which look like this; ...
4
votes
4answers
766 views

More elegant way

I read ReplaceOrRemoveInvalidOrMissingData Mathematica documentation, and I tried to apply the same approach to my dataset. I've got two problems, but one is only a style problem! There is one more ...
5
votes
1answer
330 views

Filtering Query Nested Data (Dataset and Association)

I am trying to filter a nested Dataset. I have some data that looks like this (curated for this example): ...
5
votes
1answer
478 views

Select elements of list

I have a list of lists list={l1,l2,l3,l4,...}; and two elements e1,e2. I want to select from lists ...
3
votes
2answers
847 views

How to delete all rows in datafile that contain a string

I'm importing some data from a file. However, the file was created by an unreliable instrument, such that the data looks like this: ...
3
votes
4answers
243 views

Filtering the close values from the list? [duplicate]

I found some difficulty to flatten the data. Let's say we have a list such that, list={-0.0223, -0.0199, -0.0195, -0.0159, -0.0159, -0.0122, -0.0126} You can ...
3
votes
1answer
210 views

pure functions in nested select

I want to select certain rows from a list by using two nested Select commands ...
2
votes
1answer
255 views

Using DeleteCases with multiple AstronomicalData properties

I am attempting to use AstronomicalData and DeleteCases together to remove invalid cases of two properties while listing the ...