Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

1
vote
3answers
441 views

How to import files with headers?

R has excellent header supports such as read.table with header options. I don't want to mess up with text-processing or stream-processing. I am looking for a similar tool as in R because it saves a ...
2
votes
1answer
92 views

Statistical Analyzis of Observed Data (Queuing Process)

Basically I have a process that belongs to a certain state at any point of time. However, the time intervals are not fixed (in fact they are exponentially distributed). Essentially, the data points ...
1
vote
1answer
592 views

How to append data in an already existing data .txt file in newline or in tab separtion?

Say I have an already existing .txt file containing data in the following format. $a \qquad b$ Now at different times I will successively have data like ...
2
votes
1answer
113 views

Struggling with FindFit

I'm trying to fit a sinusoidal wave to some numerical data. I can do it by hand using manipulate, so I know roughly the sort of parameter values that I should get back, but functions such as FindFit ...
5
votes
2answers
382 views

Getting the Cumulative Distribution Function of f(x) from a dataset of x

I've a data set $\mathcal{X}$ which consists of randomly generated numbers. Let's say for example data = RandomVariate[ExponentialDistribution[1], 10^4]; My aim ...
1
vote
2answers
89 views

How do you (sanely) fix up data in a csv/table?

I have data in a CSV file I've Import[]ed. Some of the data needs fixing up - date formats changed, data normalized, etc. After fixing up I'll insert it into an SQL database. My CSV file has ...
1
vote
1answer
74 views

Export data from NDSolve when they meet a certain condition

I need to export to a txt file the data from the following system of equations: ...
0
votes
3answers
244 views

Importing multiple files with a loop

I have 101 files with data that I want to import into Mathematica. All the file names are numbered as follows datafile.20.0000.out, datafile.20.1000.out, datafile.20.2000.out and so on. I used the ...
4
votes
2answers
193 views

unevenly spaced data

Suppose I have a (very large) noisy data set of points $(x_i, y_i)$ and I want to smooth it. Mathematica seems to have a number of smoothing schemes (...
1
vote
0answers
125 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
5
votes
0answers
119 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
0
votes
1answer
312 views

How to open *.sif files with Mathematica [duplicate]

While I am processing my data collect from an Andor EM-CCD, I find it diffcult to import the file into mathematica. I think mathematica can open this kind of files since that Matlab can open it. So if ...
9
votes
3answers
266 views

Improving selection of elements from large multidimensional array

Frequently in my profession I deal with large arrays of data. Typically these are mixed types including integers, strings, dates, and reals. To recreate a typical data array: ...
0
votes
1answer
109 views

Right data format when EmpiricalDistribution expects “Rectangular Array”

I have the following code to generate random data (these are the eigenvalues of a non-central Wishart matrix, and I am interested in their distribution). ...
5
votes
2answers
472 views

Exporting row vector as a column in excel

I want to export a vector in Mathematica to appear as a column in Excel. I am trying A={1, 2, 3}; Export["mfile.xls", {"A" -> A}] But it is showing 1,2,3 in ...
2
votes
1answer
230 views

Import selective elements from mixed (text and numbers) data file

How can I import only the 2nd and 5th elements (numbers, located in the 2nd and 5th columns) from all n raws of a data file (.xlxx) that contains both numbers and text. Imported data be inserted into ...
0
votes
1answer
56 views

Automatically restoring the state of a prior session [duplicate]

I have the following problem (I guess a very naive one): I work with some data and make a statistical analysis in Mathematica. I do it over several days and each time I open the file, I have to go ...
3
votes
1answer
106 views

Calculating groupwise mean in Mathematica

I have a huge panel data with more than 2.5 million observations and more than 15000 groups. I want to calculate group-wise mean and I also need Id variable in my result. This is what I have done. ...
4
votes
1answer
306 views

Importing FITS file from HEALPix

I am trying to read FITS file created by HEALPix software. The file contains a spherical map of a scalar field. There is the online support on how to use the file in IDL, but not for Mathematica. I ...
5
votes
3answers
669 views

Creating Correlation Table (and Working) With Missing Data

I am working with data that contains missing values (I have several variables (across the columns) and several observations (across the rows) in an excel spreadsheet). Example of the data structure: ...
5
votes
1answer
192 views

Tools for Importing/working with SEED seismographic data in Mathematica?

Do you know if exist a library or a package in order to import and work with SEED Seismographic data in Mathematica?
11
votes
2answers
594 views

Map of United States as a 3D histogram

By combining the state data from here and the extrusion code from here I have managed to make prisms of the various US states such as However, the extrusion method looses information about the ...
3
votes
1answer
277 views

Spectral Clustering

I am attempting to implement a spectral clustering routine using Mathematica, but I am encountering difficulties. Using the standard Gaussian similarity function found in section 2.2 of this paper ...
16
votes
3answers
760 views

Export dataset to JSON including Unicode Characters

I'm experiencing a weird issue. Consider a dataset defined like this: dataset = Dataset[{<|"a" -> 1, "b" -> 2|>, <|"a" -> 3, "b" -> 4|>}]; ...
4
votes
1answer
244 views

Extract data from WeatherData as a list of associations

Given weather data for 2013 and 2014 as follows: ...
6
votes
1answer
96 views

How to provide data to users in a way similar to Mathematica's curated data

How can one go about implementing a custom data backend usable by Mathematica? For example something like CountryData or ...
1
vote
0answers
39 views

Updating values in a Dataset [duplicate]

I'm starting to work with datasets. They seem like a useful addition to Mathematica. However, there's one fairly major stumbling block I can't get over. How do I update data in a dataset? I have a ...
4
votes
1answer
363 views

How to clean up messy data

The start of my question is about one specific situation - I have some erroneous lat/long values in a dataset which I need to clean up. But I'd like it to cover the overall topic of cleaning up messy ...
8
votes
3answers
375 views

“NotAvailable” and “NotApplicable” data points returned by WeatherData[]

I am trying to use ListVectorPlot in combination with WeatherData and CountryData to show ...
2
votes
1answer
113 views

Setting a Row from a Matrix as Variable

I'm not advanced in Mathematica but I'm trying to get better. I have a big set of Data and I want to Plot the first row against ...
1
vote
1answer
648 views

Convolving data set with gaussian profile

I have a spectrum looking like List, and I want to convole it with a gaussian of the form PDF[NormalDistribution[μ, σ], x] (where σ also depends on x). The raw ...
7
votes
1answer
440 views

How to create a real-time dashboard in Mathematica?

I am trying to figure out how to use Mathematica as a real-time interface. I've used Excel/ODBC connection to a SQL database to use the graphing functions within to get my real-time dashboard. I ...
1
vote
1answer
146 views

Distance between data and curve

I have two datasets: ...
2
votes
2answers
294 views

Obtain data points from a density plot

I have a density plot, for example DensityPlot[Sin[x*y],{x,0,2pi},{y,0,2pi}]. Then I would like to collect points from this densityplot into a file so that I ...
4
votes
1answer
360 views

BarChart that lines up with dates in DateListPlot

I am trying to create a chart similar to the view offered by TradingChart, but with custom data not relating to finance. ...
0
votes
1answer
150 views

Problem of how to manipulate Taylor/McLaurin series

I'm new to Mathematica so I don't know very well the program itself. I would like to manipulate as he does this person in youtube: http://www.youtube.com/watch?v=fCJHvQaGNiQ But I put all the ...
1
vote
0answers
119 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
102 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
0
votes
1answer
152 views

How to find a fit for a function?

I have the following function: ...
8
votes
3answers
818 views

How to parse information contained in a file name

I have a set of files containing my data, and the file-names contain relevant information that I would like to extract. What I have is: ...
4
votes
2answers
278 views

Dealing with a huge dataset

I have two ingredients here: a big dataset contained in a list, with ~ 20M values. a function that takes as each element of the list as input and yields True or False I want to save somewhere the ...
0
votes
1answer
2k views

How to plot data with error bars on a log scale in Mathematica 9?

I have data in the form d={{x1,y1,dy1},{x2,y2,dy2},..{xn,yn,dyn}}; so the error bars are different for each coordinate pair. I want to plot on a logarithmic ...
2
votes
2answers
110 views

Importing Data and Devoting Their Names to Them

There are some files in a directory. I have used of SetDiroctory to access all of them. SetDirectory["C:\\Users\\SE7EN\\Desktop\\mathematica\\test"]; I can visit ...
1
vote
2answers
201 views

dropping a row with at least one value larger than a particular value

So for instance I have M X N table. I want to drop any row which has an element larger than, say, one in the second or the third column. I guess I have to do some sort of pattern matching. But not ...
1
vote
1answer
532 views

Find polynomial equation for set of 3D data

data was given in columns in the order {y, x, z}, with y dependent on x, z ...
2
votes
2answers
184 views

Animation of moving rectangles in 2D from data file

I am not very familiar with mathematica but I know the basics so I thought I give it a try. For a project I have to animate rectangles moving around in 2D (later also 3D). The data come from a C ...
0
votes
1answer
168 views

How to load data into a Manipulate expression

I have data showing truncated octahedron geometry changing with time. How can I read those data into a Manipulate expression? To be specific, I want to manipulate ...
3
votes
4answers
399 views

random split a dataset

I've got a very simple problem, but not so trivial. I need to split a dataset (input and output data) in three part randomly: training set 70%, validation set 20% and is test set 10%. First of all ...
6
votes
3answers
234 views

Looking for ideas for handling a complicated fitting problem

I have the following data: ...
2
votes
2answers
197 views

Building adjacency matrix from set data

I have two sets: games and players. Players pick their games. As such I will have data say g1={p1,p3,p5},g2={p2,p4},g3={p2,p3,p5}. My interest though is to build the connections among players because ...