Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

1
vote
1answer
137 views

ReadList or any other command to read in blocks of very large numbers

I have a data file, data.txt, having following data: ...
0
votes
1answer
78 views

Binomial Regression where n > 1

I am not well versed in Generalized Linear Models, but am trying to use this approach to model some discrete data. I have binomial data where the sample size for each observation is the same size, ...
-1
votes
1answer
80 views

NonlinearModelFit errors and precision

For this problem, I use a data set, which I've made available here. ...
9
votes
0answers
70 views

How does Dataset compare to DataFrame in pandas?

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data. Just like ...
5
votes
0answers
47 views

Built-in way to handle uncertainty in measurment and physical constants?

Is there a built-in or fairly elegant way to handle uncertainties in measurement data and/or physical constants? For instance, the 2010 CODATA for the elementary charge is given as $\mathsf{1.602 176 ...
5
votes
0answers
91 views

Dropbox datastore

Dropbox seems to have a feature where it can be used as a database. This would be really cool to use for memoized functions, that can be restored on several computers and sessions. I have a function ...
4
votes
0answers
88 views

How to efficiently read data from any part inside huge file

I have a data file (over 2.5 GB) and I want to do calculations on the data. At first I used. data=ReadList["file", Record,n] I still have three problems: 1- ...
3
votes
0answers
210 views

Exporting Specific Cells of Many Workbooks into One using Mathematica

I currently have a large number of detailed files from which I would like to extract certain cells' information to make an Archive spreadsheet. There are approximately 9 parameters of each ...
3
votes
0answers
105 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
3
votes
0answers
284 views

How to save (large) data generated from simulations

The following is a simple example of a much larger simulation that I want to run Define some processes ...
2
votes
0answers
96 views

Fitting local regression through data

Is there a way to fit local regression through data in Mathematica (equivalent to say "smooth" in ggplot-R)? I've literally spent the last hour searching the internet and documentation centre and I ...
2
votes
0answers
90 views

Cloud cover fraction

I need to make use of Mathematica's WeatherData functionality, the CloudCoverFraction property, specifically. However, it does ...
2
votes
0answers
316 views

Lazy evaluation/loading of parts of an expression

Is it possible to have an expression x such that its elements are only evaluated when they are individually accessed, for example when evaluating statements like ...
2
votes
0answers
114 views

Significant platform-dependent efficiency when importing data?

I'm curious, as I understand that answer this way: If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field ...
1
vote
0answers
115 views

Taking a derivative, then squaring a two dimensional numerical function

Context I am interested in first finding an interpolating function of the solution to the linearly damped wave equation. Here is the solution to the LDWE with smooth square inital function : ...
1
vote
0answers
98 views

Datastream to Mathematica

I currently use data from Thomson Reuters Datastream. With an add-in in Excel, the data can be requested. Because I need to use the data in Mathematica, and I need to update the data frequently: Is it ...
1
vote
0answers
80 views

Historical Prices via FinancialData

I downloaded the historical stock prices for some stocks in the period 1990-2013 via FinancialData some months ago and computed the financial returns. Now I downloaded the data again for the same ...
1
vote
0answers
533 views

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type: I'm running a simulation with the OOMMF micromagnetics package http://math.nist.gov/oommf/ where are magnet is represented by a mesh of 3 million cells, it gets excited by a ...
1
vote
0answers
100 views

ReadList and multiple access of the same file

I have a file containing a large amount of data which can be processed independently. In order to process all data in a reasonable amount of time I distribute the jobs on three different machines. ...
0
votes
0answers
60 views

Data manipulatation with MapAt and Join

I would like to make data like: {{a1, a2}, {b1, b2, b3}, {c1, c2}, {d1, d2, d3}} -> {{a1, a2, b1, b2, b3}, {c1, c2, d1, d2, d3}} by using ...
0
votes
0answers
49 views

Solve an integral equation: to fit the given data with an integral of two functions?

I am trying to find an efficient way to solve the following equation $$h\left(b\right)=\int_{0}^{b}f\left(\frac{b-c}{1-c}\right)\frac{g\left(c\right)}{1-c}dc$$ where for $h(b)$ I have the data ...
0
votes
0answers
40 views

Handling *.in and *.out files

im trying to use Mathematica in solving of algorithm tasks like this one: http://poj.org/problem?id=1274 When i was doing those in c/c++ i was able to easily pass input/output in Windows command line ...
0
votes
0answers
57 views

How to handle substrings of a matrix by a named index

I am using open source code to run stochastic simulations. The output looks like this: ...
0
votes
0answers
100 views

System stochastic nonlinear differential equation, Ito process?

I'm trying to solve a system of nonlinear stochastic differential equation. It is a discrete nonlinear schrodinger system of n equation, with n independet weiner process. I'm actually using the Ito ...
0
votes
0answers
59 views

How to determine if a point is within a geopositioned RegionMember

I have a KML file which contains an area of interest, I also have a number of lat/long geo positions. I'm trying to filter out the points not within my AOI. So I have my KML file. I convert it to a ...
0
votes
0answers
189 views

How to open *.sif files with Mathematica

While I am processing my data collect from an Andor EM-CCD, I find it diffcult to import the file into mathematica. I think mathematica can open this kind of files since that Matlab can open it. So if ...
0
votes
0answers
75 views

Convergence Problem using FindFit

I have the following input where there is a convergence problem in the FindFit as shown in the attached figure.The figure shows 2 plots: one of the real data and the other of the fitted line. Any ...
0
votes
0answers
73 views

How to save all data in all variables in context different than “global”?

I have seen questions on saving data on "global" context. Now if I set my notebook's evaluation context "unique to this notebook". the global context doesn't work quite well. Any idea what should i do ...
0
votes
0answers
98 views

Discrete Fourier Transform and data manipulate

I solve a system od ODE and i save the discrete solution (i.e.: x,y,vx,vy, or a subset of it) in a file. Then i try to call the DFT on this file but the problem is that i must have also the time in ...
0
votes
0answers
584 views

histogram fit for probability density

I am trying to fit the (probability density) histogram of some data with the exponential function, but I am having difficulties. My raw data are in file called ...
0
votes
0answers
365 views

Performance in parameter estimation from ParametricNDSolve using varied initial conditions

I have 250 data points from a timecourse exeriment in a list, with columns specifying (1) time, (2-4) initial conditions, (5) absorbance reading. I want to fit 4 parameters (k1, k2, k3, k4) in a DAE ...