# Tagged Questions

Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

9k views

### How to perform a multi-peak fitting?

I am wondering how to implement the multi-peak detecting and fitting in Mathematica. Following is an example of fitting the data using three peaks (such that the data ~ peak1 + peak2 + peak3). The ...
2k views

### How to integrate functions of linearly interpolated data?

At first, consider integration of pure InterpolatingFunction. Importing some data (works in v.9, for earlier versions one can use this link to download zipped <...
2k views

### Smoothing ListContourPlot contours

I have some data of type {{x1, y1, f1}, {x2, y2, f2}, ...} and want to do ListContourPlot. However, there is a problem that the ...
2k views

### How to extract data from contour plot as a text file?

I have the following expression from which I want to extract data for a contour plot. ...
186 views

### Making large calculation results persistent

Very often, I need to do same calculations under different parameters for comparing the results. Every calculation may generate a large data set; for example, the calculation result of ...
3k views

### How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
2k views

### Graph databases / Nosql with Mathematica

I am very new to mathematica but it looks like an interesting language and I would like to explore it for a future project to create a web search engine. It seems like it would be fairly well suited ...
2k views

### Dealing with large files

I have a *.dat file in which the contents are basically one giant list: {1,2,3,4} {5,6,7,8} {9,10,11,12} and so on. I'd like to read in the first 500 members { }...
4k views

### How to get Matlab data imported with the same dimensions?

I have some Matlab image data with the following dimensions (output is from Matlab): >> size(im) ans = 86 86 3 45 The data imports just fine ...
859 views

### Data filtering using Butterworth filter

I want to filter data using Butterworth filter. I am currently using Matlab and I want to know how to do it in Mathematica. I have the following data: ...
252 views

### Removing background from peak function

Let's say I have a a very large list of data in the format {{n,f[n]}} so something like {{1,600},{2,700},{3,1100}...} and if this data is plotted, then it will look like a sum of Gaussian functions+a ...
1k views

### Parametric Interpolation of 2D data

I want to create an interpolation through the following 2dim data points: ...
828 views

### Correct fourier scaling and high-resolution frequency identification

I have a dataset of amplitude versus time $(t,A(t))$ and I need to extract the dominant frequency and amplitude, and also get the amplitude at one other specific frequency. My data looks like this: <...
385 views

### Storing data in the Notebook in readable form

The common problem in my work is a need to import some experimental data to work with. This means that my notebook depends on some external files which makes it difficult to share it with others and ...
507 views

### How to import files with headers?

R has excellent header supports such as read.table with header options. I don't want to mess up with text-processing or stream-processing. I am looking for a similar tool as in R because it saves a ...
473 views

### Turning a set of random walk data 45 degrees

I have a set of data that is just a "random" (generated by me, not by computer) sequence of length 2000 of 1's and (-1)'s. I used it to plot a 1-D random walk where +1 is step up, (-1) is step down, ...
306 views

### Importing multiple files with a loop

I have 101 files with data that I want to import into Mathematica. All the file names are numbered as follows datafile.20.0000.out, datafile.20.1000.out, datafile.20.2000.out and so on. I used the ...
410 views

### upper envelope of data

Here is a ListPlot[] of some data. Clearly, there is a fairly smooth upper envelope - the question is whether there is an nice way of extracting it...
1k views

### Plotting data points: Optimizing size and visuals

The following problem is one that may have showed up to you while plotting a large data collection. Suppose you have a data set of say 1 million points. ListLinePlot...
4k views

### Finding fourier transform of data and hence frequencies

I looked at all the other questions related to mine before posting this, and they didn't solve my problem. I have a large data set which can be downloaded from here. I'm using the following code to ...
1k views

### Calculate mean of values in bins

Problem description: I have a list of $\{x,y\}$ pairs. I'd like to divide $x$ into equal[*] bins, say $bx_1, bx_2, \ldots$, calculate $\left<y\right>$ for every bin and then plot the bin values ...
1k views

### Filtering and Replacing outliers

I have a set of data that measurements of temperature vs time (date). It is easy to realize when the sensor went wrong as one might see a blip on the data. How to teach to Mathematica to: 1) Detect ...
281 views

### Assign the contents of a “Copy” operation programmatically to a variable?

While an earlier question Cut and paste data from a spreadsheet nominally addresses the issue, a year has past since its OP asked it and I hope that others have come across simpler or more direct ...
934 views

### ListContourPlot interpolation screws up on concave set of data

I want to create ListContourPlots for sets of Data (from a physics simulation) given as Triplets {x, y, f(x, y)}. Certain parameter regions in the x-y-plane are ...
383 views

### Histogram of a very large dataset?

I have a very large dataset stored in a file (over 2GB). The file contains a tab-separated table of floating-point numbers. I want to do an Histogram of all the numbers in the table. But if I try to ...
230 views

### Deseasonalize the data of traffic growth for the past data

In another question (Using TimeSeriesForecast for forecasting the traffic growth) I asked to use the TimeSeriesForecast on this set of data the answer provided the plot with and without the ...
992 views

I want to do some large data analysis, and i want to be able to import files in a fast way. I tried Import, but it is quite slow; ReadList takes half the time. This is the code I was using: ...
616 views

### More elegant way

I read ReplaceOrRemoveInvalidOrMissingData Mathematica documentation, and I tried to apply the same approach to my dataset. I've got two problems, but one is only a style problem! There is one more ...
347 views

### Selecting for 2D points that are within a threshold distance of an upper- and lower-bound number of points

I have a very large set of 2D points: numberOf2DPoints = 10^6; pointList = RandomReal[{0, 1000}, {numberOf2DPoints, 2}]; I'd like to find a way to quickly ...
2k views

### How to plot data with error bars on a log scale in Mathematica 9?

I have data in the form d={{x1,y1,dy1},{x2,y2,dy2},..{xn,yn,dyn}}; so the error bars are different for each coordinate pair. I want to plot on a logarithmic x-...
1k views

### How can I control the number format of exported data?

I'm having issues while exporting a tables of numerical values to "*.dat" data files. Here's the thing, sometimes when I export a table, I get more decimals on some random values. The problem is that ...
614 views

### How do I find the maximum value from the given set of data?

I have a set of data like {{x1, y1},{x2, y2}, ...,{xn, yn}}. From this data can I find the value of x for which ...
192 views

### Animation of moving rectangles in 2D from data file

I am not very familiar with mathematica but I know the basics so I thought I give it a try. For a project I have to animate rectangles moving around in 2D (later also 3D). The data come from a C ...
1k views

### How to extract the edge from a set of points

I have an accumulation of points representing a body as you can see in the image below. From this data I want to generate a list of points or a curve, which is the envelope. The first idea was a ...
608 views

### Detecting components in timeseries

I'm looking at this sequence of values: I'd like to detect the points where the center of the time-series shifts (around x=1000 and x=2000). Many of the transforms and smoothing methods I have ...
1k views

### Filtering beat-to-beat heart rate data

From an experiment, I have a dataset of beat-to-beat heart rate data: a list of the time between each heart beat in [ms]. The data is measured using an infrared optic sensor at the finger tip. The ...
825 views

### Export dataset to JSON including Unicode Characters

I'm experiencing a weird issue. Consider a dataset defined like this: dataset = Dataset[{<|"a" -> 1, "b" -> 2|>, <|"a" -> 3, "b" -> 4|>}]; ...
893 views

### Relational joining of tables

A common thing to want to do with data is to combine it (at least I thought this was a common thing). In SQL there is the idea of table joins "select id,v1,v2 from A,B where A.id=B.id" kind of thing. ...
625 views

### Can A Plot Be Overlayed With A Map?

Some brief, only slightly important background. I am doing a research project using the data from NASA's GRACE mission. I wrote a short Perl script to take two data files and find the change in ...
1k views

### Determine the Number of Lines in a Text File

I have looked and looked but I do not see a simple efficient way to get MMA to return the number of lines in a text file? I thought about reading the file until it returned the EOF marker, but that ...
2k views

### Moving maximum function?

How can I construct a moving maximum function? For example, if I have a list of 12 values: { 5, 6, 9, 3, 2, 6, 7, 8, 1, 1, 4, 7 } and I want to maximize over 3 ...
669 views

### How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
401 views

### “NotAvailable” and “NotApplicable” data points returned by WeatherData[]

I am trying to use ListVectorPlot in combination with WeatherData and CountryData to show ...
623 views

### Map of Swedish regions?

There have been some questions dealing with how to draw individual US states since such data doesn't come with Mathematica through its CountryData which offers ...
152 views

### How can I export definitions of several symbols to a single file?

the problem related to this question What I want to do is to Export definition of several symbol into a single file. The reason I choose ...
372 views

### Using Periodogram to investigate the seasonality of the data

After asking how to get rid of the seasonality of this set of data, I'd like to ask how to investigate the seasonality in data using Mathematica. Which tool would you use to study seasonality? I.e. ...
136 views

### Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
378 views

### BarChart that lines up with dates in DateListPlot

I am trying to create a chart similar to the view offered by TradingChart, but with custom data not relating to finance. ...