Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

0
votes
1answer
24 views

Question about loading data by skipping

. It is often important to take data and measurements and give them a name which has a number, and manipulate them in a for loop fashion to manipulate a large data base efficently. The loop above ...
18
votes
0answers
342 views

How does Dataset compare to DataFrame in pandas?

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data. Just like ...
6
votes
0answers
113 views

Can Predict/Classify be parallelized?

Being able to parallelize Predict /Classify would be useful on multiple fronts: Performance/Scale Streaming Pedagogical ...
5
votes
0answers
55 views

Built-in way to handle uncertainty in measurment and physical constants?

Is there a built-in or fairly elegant way to handle uncertainties in measurement data and/or physical constants? For instance, the 2010 CODATA for the elementary charge is given as $\mathsf{1.602 176 ...
5
votes
0answers
121 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
4
votes
0answers
96 views

Fitting data with unique variable per measurement

Imagine we are in the following situation, we have a nonlinear system of functions $f$ with coördinates $x_i$, and unknown parameters $\delta_i$'s and $t$. Let this system be defined as, ...
4
votes
0answers
74 views

File Formats for Scientific Data

I have a simulation written in another language which currently just dumps all its data (doubles, lists of doubles, lists of lists of doubles) in lines in ASCII, and then I read it into Mathematica by ...
4
votes
0answers
311 views

How to save (large) data generated from simulations

The following is a simple example of a much larger simulation that I want to run Define some processes ...
3
votes
0answers
59 views

What's the most efficient way to load a very large dispatch table?

This question is possibly related to this topic, although I think I have something a little more specific in mind. Recently I've been working with dispatch tables for a project, since it seemed like a ...
3
votes
0answers
87 views

WDF specification?

Motivated by a recent question on the meaning of all sorts of W* acronyms, I would like to get more information about the Wolfram Data Framework (WDF). On its web page ...
3
votes
0answers
91 views

Data smoothing in three dimensions

I have data with statistical fluctuations of the form $f(x_i,y_i)$ over a discrete grid. I now need the values of $f(x,y)$ also at points other than the grid points. That can be achieved by ...
3
votes
0answers
78 views

How to export to NetCDF format

Consider I have a scalar field on a grid f(x,y), where x and y can take values ...
3
votes
0answers
112 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
2
votes
0answers
45 views

Optimization for iterating and plotting a matrix product and its elements?

I have an iterative matrix product of this type Since i didnt knew how to do it, I explicitly wrote the product as {Ayn + Bxn, Cyn + Dxn} and evaluated it with the NestList construct ...
2
votes
0answers
74 views

How can I access the Criterion function of KI method in `FindThreshold` for a dataset?

I want to find a threshold for some data set, using Kittler-Illingworth minimum error thresholding method (KI method). To do so, I use the Mathematica built-in function ...
2
votes
0answers
500 views

Lazy evaluation/loading of parts of an expression

Is it possible to have an expression x such that its elements are only evaluated when they are individually accessed, for example when evaluating statements like ...
2
votes
0answers
122 views

Significant platform-dependent efficiency when importing data?

I'm curious, as I understand that answer this way: If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field ...
1
vote
0answers
62 views

Constructing the spectral density of white noise

My goal is to find the spectral density $S_{XX}(f)$ from a time domain noise signal $X(t)$. The approach should work for general time series, but to illustrate what I've been doing I'll keep it ...
1
vote
0answers
126 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
1
vote
0answers
123 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
104 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
1
vote
0answers
634 views

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type: I'm running a simulation with the OOMMF micromagnetics package http://math.nist.gov/oommf/ where are magnet is represented by a mesh of 3 million cells, it gets excited by a ...
1
vote
0answers
103 views

ReadList and multiple access of the same file

I have a file containing a large amount of data which can be processed independently. In order to process all data in a reasonable amount of time I distribute the jobs on three different machines. ...
0
votes
0answers
27 views

Extracting the data of each frame from an AVI file

What's the most straightforward way to "Import" the data of an AVI video? For instance, if I record a 30 frame per second video, then after 10 seconds I would have 300 frames. How can I import the ...