Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

1
vote
1answer
157 views

ReadList can't read “.csv” data with correct line number?

Well, the problem was solved today. Since "testAll.csv" is generated by Java/Eclipse, and I noticed that the LineSeperator is ...
3
votes
3answers
209 views

How to detect outlier in data of an unknown function $f(x)$

Suppose I have an unknown function $f(x)$ that is monotonic, convex and nonlinear by definition. I somehow evaluated it at 1000 nonuniform $x$ points. By ploting these data, I found a very obvious ...
0
votes
1answer
127 views

Fitting data with inclusion of constraints

How do I find a fit for the dataS (see below) with the constraint that the parameter np is an integer, and that the parameters la+lc = 100. A weak attempt at this is provided below for a specific ...
5
votes
1answer
259 views

Forcing NonLinearModelFit through a point

Like the title says, is there a way to make an NLM fit pass through a specific point.
1
vote
1answer
141 views

Country density plot of lines (highways)

I love solution from question (histogram as well as distrubution) http://mathematica.stackexchange.com/questions/31948/show-density-plot-on-us-map#=. But both cases are a bit more difficult in my ...
1
vote
3answers
472 views

How to import files with headers?

R has excellent header supports such as read.table with header options. I don't want to mess up with text-processing or stream-processing. I am looking for a similar tool as in R because it saves a ...
2
votes
1answer
95 views

Statistical Analyzis of Observed Data (Queuing Process)

Basically I have a process that belongs to a certain state at any point of time. However, the time intervals are not fixed (in fact they are exponentially distributed). Essentially, the data points ...
1
vote
1answer
650 views

How to append data in an already existing data .txt file in newline or in tab separtion?

Say I have an already existing .txt file containing data in the following format. $a \qquad b$ Now at different times I will successively have data like ...
2
votes
1answer
120 views

Struggling with FindFit

I'm trying to fit a sinusoidal wave to some numerical data. I can do it by hand using manipulate, so I know roughly the sort of parameter values that I should get back, but functions such as FindFit ...
5
votes
2answers
396 views

Getting the Cumulative Distribution Function of f(x) from a dataset of x

I've a data set $\mathcal{X}$ which consists of randomly generated numbers. Let's say for example data = RandomVariate[ExponentialDistribution[1], 10^4]; My aim ...
1
vote
2answers
89 views

How do you (sanely) fix up data in a csv/table?

I have data in a CSV file I've Import[]ed. Some of the data needs fixing up - date formats changed, data normalized, etc. After fixing up I'll insert it into an SQL database. My CSV file has ...
1
vote
1answer
75 views

Export data from NDSolve when they meet a certain condition

I need to export to a txt file the data from the following system of equations: ...
0
votes
3answers
263 views

Importing multiple files with a loop

I have 101 files with data that I want to import into Mathematica. All the file names are numbered as follows datafile.20.0000.out, datafile.20.1000.out, datafile.20.2000.out and so on. I used the ...
4
votes
2answers
193 views

unevenly spaced data

Suppose I have a (very large) noisy data set of points $(x_i, y_i)$ and I want to smooth it. Mathematica seems to have a number of smoothing schemes (...
1
vote
0answers
126 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
5
votes
0answers
121 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
0
votes
1answer
329 views

How to open *.sif files with Mathematica [duplicate]

While I am processing my data collect from an Andor EM-CCD, I find it diffcult to import the file into mathematica. I think mathematica can open this kind of files since that Matlab can open it. So if ...
9
votes
3answers
270 views

Improving selection of elements from large multidimensional array

Frequently in my profession I deal with large arrays of data. Typically these are mixed types including integers, strings, dates, and reals. To recreate a typical data array: ...
0
votes
1answer
114 views

Right data format when EmpiricalDistribution expects “Rectangular Array”

I have the following code to generate random data (these are the eigenvalues of a non-central Wishart matrix, and I am interested in their distribution). ...
5
votes
2answers
483 views

Exporting row vector as a column in excel

I want to export a vector in Mathematica to appear as a column in Excel. I am trying A={1, 2, 3}; Export["mfile.xls", {"A" -> A}] But it is showing 1,2,3 in ...
2
votes
1answer
236 views

Import selective elements from mixed (text and numbers) data file

How can I import only the 2nd and 5th elements (numbers, located in the 2nd and 5th columns) from all n raws of a data file (.xlxx) that contains both numbers and text. Imported data be inserted into ...
0
votes
1answer
57 views

Automatically restoring the state of a prior session [duplicate]

I have the following problem (I guess a very naive one): I work with some data and make a statistical analysis in Mathematica. I do it over several days and each time I open the file, I have to go ...
3
votes
1answer
107 views

Calculating groupwise mean in Mathematica

I have a huge panel data with more than 2.5 million observations and more than 15000 groups. I want to calculate group-wise mean and I also need Id variable in my result. This is what I have done. ...
4
votes
1answer
318 views

Importing FITS file from HEALPix

I am trying to read FITS file created by HEALPix software. The file contains a spherical map of a scalar field. There is the online support on how to use the file in IDL, but not for Mathematica. I ...
5
votes
3answers
713 views

Creating Correlation Table (and Working) With Missing Data

I am working with data that contains missing values (I have several variables (across the columns) and several observations (across the rows) in an excel spreadsheet). Example of the data structure: ...
5
votes
1answer
194 views

Tools for Importing/working with SEED seismographic data in Mathematica?

Do you know if exist a library or a package in order to import and work with SEED Seismographic data in Mathematica?
12
votes
2answers
604 views

Map of United States as a 3D histogram

By combining the state data from here and the extrusion code from here I have managed to make prisms of the various US states such as However, the extrusion method looses information about the ...
3
votes
1answer
289 views

Spectral Clustering

I am attempting to implement a spectral clustering routine using Mathematica, but I am encountering difficulties. Using the standard Gaussian similarity function found in section 2.2 of this paper ...
16
votes
3answers
779 views

Export dataset to JSON including Unicode Characters

I'm experiencing a weird issue. Consider a dataset defined like this: dataset = Dataset[{<|"a" -> 1, "b" -> 2|>, <|"a" -> 3, "b" -> 4|>}]; ...
4
votes
1answer
253 views

Extract data from WeatherData as a list of associations

Given weather data for 2013 and 2014 as follows: ...
6
votes
1answer
96 views

How to provide data to users in a way similar to Mathematica's curated data

How can one go about implementing a custom data backend usable by Mathematica? For example something like CountryData or ...
1
vote
0answers
40 views

Updating values in a Dataset [duplicate]

I'm starting to work with datasets. They seem like a useful addition to Mathematica. However, there's one fairly major stumbling block I can't get over. How do I update data in a dataset? I have a ...
4
votes
1answer
366 views

How to clean up messy data

The start of my question is about one specific situation - I have some erroneous lat/long values in a dataset which I need to clean up. But I'd like it to cover the overall topic of cleaning up messy ...
8
votes
3answers
379 views

“NotAvailable” and “NotApplicable” data points returned by WeatherData[]

I am trying to use ListVectorPlot in combination with WeatherData and CountryData to show ...
2
votes
1answer
113 views

Setting a Row from a Matrix as Variable

I'm not advanced in Mathematica but I'm trying to get better. I have a big set of Data and I want to Plot the first row against ...
1
vote
1answer
690 views

Convolving data set with gaussian profile

I have a spectrum looking like List, and I want to convole it with a gaussian of the form PDF[NormalDistribution[μ, σ], x] (where σ also depends on x). The raw ...
7
votes
1answer
452 views

How to create a real-time dashboard in Mathematica?

I am trying to figure out how to use Mathematica as a real-time interface. I've used Excel/ODBC connection to a SQL database to use the graphing functions within to get my real-time dashboard. I ...
1
vote
1answer
146 views

Distance between data and curve

I have two datasets: ...
2
votes
2answers
304 views

Obtain data points from a density plot

I have a density plot, for example DensityPlot[Sin[x*y],{x,0,2pi},{y,0,2pi}]. Then I would like to collect points from this densityplot into a file so that I ...
4
votes
1answer
367 views

BarChart that lines up with dates in DateListPlot

I am trying to create a chart similar to the view offered by TradingChart, but with custom data not relating to finance. ...
0
votes
1answer
158 views

Problem of how to manipulate Taylor/McLaurin series

I'm new to Mathematica so I don't know very well the program itself. I would like to manipulate as he does this person in youtube: http://www.youtube.com/watch?v=fCJHvQaGNiQ But I put all the ...
1
vote
0answers
123 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
104 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
0
votes
1answer
159 views

How to find a fit for a function?

I have the following function: ...
8
votes
3answers
850 views

How to parse information contained in a file name

I have a set of files containing my data, and the file-names contain relevant information that I would like to extract. What I have is: ...
4
votes
2answers
279 views

Dealing with a huge dataset

I have two ingredients here: a big dataset contained in a list, with ~ 20M values. a function that takes as each element of the list as input and yields True or False I want to save somewhere the ...
0
votes
1answer
2k views

How to plot data with error bars on a log scale in Mathematica 9?

I have data in the form d={{x1,y1,dy1},{x2,y2,dy2},..{xn,yn,dyn}}; so the error bars are different for each coordinate pair. I want to plot on a logarithmic ...
2
votes
2answers
111 views

Importing Data and Devoting Their Names to Them

There are some files in a directory. I have used of SetDiroctory to access all of them. SetDirectory["C:\\Users\\SE7EN\\Desktop\\mathematica\\test"]; I can visit ...
1
vote
2answers
204 views

dropping a row with at least one value larger than a particular value

So for instance I have M X N table. I want to drop any row which has an element larger than, say, one in the second or the third column. I guess I have to do some sort of pattern matching. But not ...
1
vote
1answer
549 views

Find polynomial equation for set of 3D data

data was given in columns in the order {y, x, z}, with y dependent on x, z ...