Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

0
votes
1answer
32 views

Question about loading data by skipping

. It is often important to take data and measurements and give them a name which has a number, and manipulate them in a for loop fashion to manipulate a large data base efficently. The loop above ...
18
votes
0answers
363 views

How does Dataset compare to DataFrame in pandas?

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data. Just like ...
7
votes
0answers
120 views

Can Predict/Classify be parallelized?

Being able to parallelize Predict /Classify would be useful on multiple fronts: Performance/Scale Streaming Pedagogical ...
5
votes
0answers
120 views

WDF specification?

Motivated by a recent question on the meaning of all sorts of W* acronyms, I would like to get more information about the Wolfram Data Framework (WDF). On its web page http://www.wolfram.com/data-...
5
votes
0answers
57 views

Built-in way to handle uncertainty in measurment and physical constants?

Is there a built-in or fairly elegant way to handle uncertainties in measurement data and/or physical constants? For instance, the 2010 CODATA for the elementary charge is given as $\mathsf{1.602 176 ...
5
votes
0answers
127 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
4
votes
0answers
98 views

Fitting data with unique variable per measurement

Imagine we are in the following situation, we have a nonlinear system of functions $f$ with coördinates $x_i$, and unknown parameters $\delta_i$'s and $t$. Let this system be defined as, \begin{...
4
votes
0answers
77 views

File Formats for Scientific Data

I have a simulation written in another language which currently just dumps all its data (doubles, lists of doubles, lists of lists of doubles) in lines in ASCII, and then I read it into Mathematica by ...
4
votes
0answers
316 views

How to save (large) data generated from simulations

The following is a simple example of a much larger simulation that I want to run Define some processes ...
3
votes
0answers
63 views

What's the most efficient way to load a very large dispatch table?

This question is possibly related to this topic, although I think I have something a little more specific in mind. Recently I've been working with dispatch tables for a project, since it seemed like a ...
3
votes
0answers
95 views

Data smoothing in three dimensions

I have data with statistical fluctuations of the form $f(x_i,y_i)$ over a discrete grid. I now need the values of $f(x,y)$ also at points other than the grid points. That can be achieved by ...
3
votes
0answers
79 views

How to export to NetCDF format

Consider I have a scalar field on a grid f(x,y), where x and y can take values ...
3
votes
0answers
112 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
2
votes
0answers
45 views

Optimization for iterating and plotting a matrix product and its elements?

I have an iterative matrix product of this type Since i didnt knew how to do it, I explicitly wrote the product as {Ayn + Bxn, Cyn + Dxn} and evaluated it with the NestList construct ...
2
votes
0answers
78 views

How can I access the Criterion function of KI method in `FindThreshold` for a dataset?

I want to find a threshold for some data set, using Kittler-Illingworth minimum error thresholding method (KI method). To do so, I use the Mathematica built-in function ...
2
votes
0answers
515 views

Lazy evaluation/loading of parts of an expression

Is it possible to have an expression x such that its elements are only evaluated when they are individually accessed, for example when evaluating statements like <...
2
votes
0answers
122 views

Significant platform-dependent efficiency when importing data?

I'm curious, as I understand that answer this way: If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field ...
1
vote
0answers
66 views

Constructing the spectral density of white noise

My goal is to find the spectral density $S_{XX}(f)$ from a time domain noise signal $X(t)$. The approach should work for general time series, but to illustrate what I've been doing I'll keep it simple:...
1
vote
0answers
127 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
1
vote
0answers
125 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
104 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
1
vote
0answers
639 views

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type: I'm running a simulation with the OOMMF micromagnetics package http://math.nist.gov/oommf/ where are magnet is represented by a mesh of 3 million cells, it gets excited by a ...
1
vote
0answers
103 views

ReadList and multiple access of the same file

I have a file containing a large amount of data which can be processed independently. In order to process all data in a reasonable amount of time I distribute the jobs on three different machines. ...
0
votes
0answers
29 views

data fit using second order PDE

I'm trying to find the parameter for second order PDE using data. i don't know what is the error of that, ...
-2
votes
0answers
23 views

List Contour Plot neglect the lower number of events

I have data in which the maximum number of events are concentrated in small region at the center of the figure and other lower number of events are extended to large distances from the center of the ...