Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

2
votes
2answers
114 views

Best way to include a “singularity” in a SmoothHistogram?

I have a dataset like this: data = {0,0,1.1,1.3,0,0,3.4,0,0,2.3,0,0 .....} You can imagine that data is generated from a ...
18
votes
0answers
330 views

How does Dataset compare to DataFrame in pandas?

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data. Just like ...
6
votes
0answers
107 views

Can Predict/Classify be parallelized?

Being able to parallelize Predict /Classify would be useful on multiple fronts: Performance/Scale Streaming Pedagogical ...
5
votes
0answers
55 views

Built-in way to handle uncertainty in measurment and physical constants?

Is there a built-in or fairly elegant way to handle uncertainties in measurement data and/or physical constants? For instance, the 2010 CODATA for the elementary charge is given as $\mathsf{1.602 176 ...
5
votes
0answers
119 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
4
votes
0answers
92 views

Fitting data with unique variable per measurement

Imagine we are in the following situation, we have a nonlinear system of functions $f$ with coördinates $x_i$, and unknown parameters $\delta_i$'s and $t$. Let this system be defined as, ...
4
votes
0answers
73 views

File Formats for Scientific Data

I have a simulation written in another language which currently just dumps all its data (doubles, lists of doubles, lists of lists of doubles) in lines in ASCII, and then I read it into Mathematica by ...
4
votes
0answers
307 views

How to save (large) data generated from simulations

The following is a simple example of a much larger simulation that I want to run Define some processes ...
3
votes
0answers
85 views

WDF specification?

Motivated by a recent question on the meaning of all sorts of W* acronyms, I would like to get more information about the Wolfram Data Framework (WDF). On its web page ...
3
votes
0answers
91 views

Data smoothing in three dimensions

I have data with statistical fluctuations of the form $f(x_i,y_i)$ over a discrete grid. I now need the values of $f(x,y)$ also at points other than the grid points. That can be achieved by ...
3
votes
0answers
76 views

How to export to NetCDF format

Consider I have a scalar field on a grid f(x,y), where x and y can take values ...
3
votes
0answers
111 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
2
votes
0answers
45 views

Optimization for iterating and plotting a matrix product and its elements?

I have an iterative matrix product of this type Since i didnt knew how to do it, I explicitly wrote the product as {Ayn + Bxn, Cyn + Dxn} and evaluated it with the NestList construct ...
2
votes
0answers
74 views

How can I access the Criterion function of KI method in `FindThreshold` for a dataset?

I want to find a threshold for some data set, using Kittler-Illingworth minimum error thresholding method (KI method). To do so, I use the Mathematica built-in function ...
2
votes
0answers
489 views

Lazy evaluation/loading of parts of an expression

Is it possible to have an expression x such that its elements are only evaluated when they are individually accessed, for example when evaluating statements like ...
2
votes
0answers
121 views

Significant platform-dependent efficiency when importing data?

I'm curious, as I understand that answer this way: If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field ...
1
vote
0answers
56 views

Constructing the spectral density of white noise

My goal is to find the spectral density $S_{XX}(f)$ from a time domain noise signal $X(t)$. The approach should work for general time series, but to illustrate what I've been doing I'll keep it ...
1
vote
0answers
125 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
1
vote
0answers
120 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
103 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
1
vote
0answers
625 views

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type: I'm running a simulation with the OOMMF micromagnetics package http://math.nist.gov/oommf/ where are magnet is represented by a mesh of 3 million cells, it gets excited by a ...
1
vote
0answers
103 views

ReadList and multiple access of the same file

I have a file containing a large amount of data which can be processed independently. In order to process all data in a reasonable amount of time I distribute the jobs on three different machines. ...