Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

2
votes
0answers
532 views

Lazy evaluation/loading of parts of an expression

Is it possible to have an expression x such that its elements are only evaluated when they are individually accessed, for example when evaluating statements like <...
0
votes
2answers
1k views

How to generate a random data set with determined expected value and variance? [closed]

Recently I am doing my math homework and there is a question asking me to Come up with a dataset X that has at least 10 distinct numbers in it such that Expect[X] = 3.7, and Var[X] = 2.1. Actually I ...
2
votes
1answer
213 views

Importing data from Excel for DateListPlot

I am importing data from Excel and plotting it with DateListPlot. My code is ...
33
votes
5answers
9k views

How to perform a multi-peak fitting?

I am wondering how to implement the multi-peak detecting and fitting in Mathematica. Following is an example of fitting the data using three peaks (such that the data ~ peak1 + peak2 + peak3). The ...
0
votes
1answer
141 views

Column specific data inputs

My data is in the form of 6 columns in a notepad file. I want to import, only from a specific column,say, the third column into Mathematica in the form of a list, i.e., i want commas to be placed ...
0
votes
2answers
254 views

Visualizing multi-state lattice from a data set

Is there any way to make Mathematica visualize a multistate square lattice? That is, I have a data set from a simulation that is x coloumns wide and y rows tall (just a simple .dat file), where each ...
2
votes
1answer
3k views

Using NonlinearModelFit to fit data with errors

Assume we have some data of measurements SeedRandom[9]; data = Table[{i,2*(1 + 5*RandomReal[])*Exp[-0.01*(1 + RandomReal[])*i]}, {i, {0, 100, 200, 400, 700}}]; ...
2
votes
2answers
421 views

Importing LTSpice data into mathematica

I am working with LTSpice (Discret electrical circuit ( Spice) simulator). I would like to import LTSpice ouput data into mathematica for processing and plots. An example of a file can be found here (...
0
votes
0answers
381 views

Performance in parameter estimation from ParametricNDSolve using varied initial conditions [duplicate]

I have 250 data points from a timecourse exeriment in a list, with columns specifying (1) time, (2-4) initial conditions, (5) absorbance reading. I want to fit 4 parameters (k1, k2, k3, k4) in a DAE ...
4
votes
1answer
185 views

Saving and Reading data across computers

I DumpSaved some data from my home laptop, uploaded it onto google drive and now I am trying to open it at my school computer but I am getting errors, even though the data is in the correct directory. ...
2
votes
0answers
123 views

Significant platform-dependent efficiency when importing data?

I'm curious, as I understand that answer this way: If you are using a Linux derivative or something like cygwin on Windows, then this is an efficient solution for reading only the third field ...
1
vote
2answers
278 views

Efficient Data Entry / Manipulation with Many Datasets

I currently have a simple task of graphing many different datasets (26 here, with many different instances where I'll need to do, say, 26 different ones from the same template). I'm trying to ...
2
votes
2answers
1k views

Importing several huge files with headers & footers

having here several files in ASCII format http://ge.tt/91rv0Jf/v/0 I want to import, while removing headers and footers and then join the several lists. ...
15
votes
3answers
669 views

How to Import random elements of huge data files

I am calculating huge data files with an external program. I would like then to import the data into Mathematica for analysis. The files are 2 columns and up to many millions of rows. So for ...
1
vote
1answer
73 views

Substituting Data from File

I have sets of Diff. Eq. that I'd like to solve symbolically, then substitute in numerical values for some of the symbols. I'd like to store the value for each symbol in a file, e.g., an excel ...
1
vote
1answer
196 views

Table/Tree of values

My question: Is there a way to create a tree of values? Something like the output of the command TreeForm, but with values in the nodes? Why do I want this? I'm trying to do a complete program to ...
16
votes
6answers
1k views

Determine the Number of Lines in a Text File

I have looked and looked but I do not see a simple efficient way to get MMA to return the number of lines in a text file? I thought about reading the file until it returned the EOF marker, but that ...
7
votes
2answers
123 views

How can I recover a graph from a graphical output in a notebook after resetting the kernel?

I have an old Mathematica notebook with an output for a graph structure $G$ I would like to recover. However, the routine for building up the graph is stochastic, so I can't simply rerun the code. ...
1
vote
2answers
184 views

Request help for reading text data files with headers and 2 sections

I just started with Mathematica. I need it for many purposes, but the important one is to read a text data file and extract column values as variables. I have successfully tested the tab delimited ...
1
vote
0answers
645 views

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type: I'm running a simulation with the OOMMF micromagnetics package http://math.nist.gov/oommf/ where are magnet is represented by a mesh of 3 million cells, it gets excited by a ...
7
votes
5answers
385 views

Storing data in the Notebook in readable form

The common problem in my work is a need to import some experimental data to work with. This means that my notebook depends on some external files which makes it difficult to share it with others and ...
15
votes
4answers
1k views

Plotting data points: Optimizing size and visuals

The following problem is one that may have showed up to you while plotting a large data collection. Suppose you have a data set of say 1 million points. ListLinePlot...
1
vote
1answer
306 views

Automating sequential import of data from web

I am trying to import data from multiple web pages hosted by a single online source. The data is posted by the source as one data set per web page for each week of the year. I would like to import the ...
1
vote
2answers
314 views

Running out of memory when filtering a large data set

My goal is to read list0.dat, filter the data with a function f, and then write the transformed data to ...
4
votes
1answer
4k views

Transform 3D data into 2D color map like Matlab's surf/imagesc

I have a bunch of 3D data (x, y coordinates and values) which is generated in Matlab and I need to render them in a 2D color map with certain axis ranges. Here are the Matlab code and the generated ...
4
votes
2answers
899 views

Check Google PageRank for a given URL

Does anyone know how to write a Mathematica function that returns the Google PageRank of a given URL? I found a tutorial for achieving this in a bash script here, but I have no idea how to port this ...
3
votes
1answer
692 views

NonlinearModelFit returning nonsense for very simple data

In fitting the following data, if I first log transform the y-values and use LinearModelFit I get a set of parameters that fits the data: ...
5
votes
2answers
1k views

How can I control the number format of exported data?

I'm having issues while exporting a tables of numerical values to "*.dat" data files. Here's the thing, sometimes when I export a table, I get more decimals on some random values. The problem is that ...
1
vote
2answers
5k views

Export plot data in Mathematica [duplicate]

How can I export data, which is plotted using something such as: Plot[Γ[x]/(1.6*10^(-19)), {x, 0.1, 1.2}] into a .txt file in Mathematica?
-2
votes
1answer
502 views

Non-regular grid data

I have issue with non-regulary spaced data. I have set of 3 numbers {x, y, z} in my list. ...
5
votes
1answer
233 views

WolframAlpha Selective SubPod Content

I am sending queries to Wolfram | Alpha from Mathematica V9 and and importing the results back into Mathematica, but I am running into a challenge in formatting the request. Specifically, I would like ...
6
votes
2answers
934 views

ListContourPlot interpolation screws up on concave set of data

I want to create ListContourPlots for sets of Data (from a physics simulation) given as Triplets {x, y, f(x, y)}. Certain parameter regions in the x-y-plane are ...
3
votes
1answer
747 views

Importing data sets from plain-text files and performing operations on them

I am working with a lot of daily weather files in the txt format. I need to calculate a daily quantity from each of them, e.g., daily light intensity, which ...
3
votes
1answer
744 views

Read and filter a text file without importing

I have a large text file of a million sections. Each section contains A lines of header and B lines of numeric data. I want to read the first and last line of B from each section (to have a list of 2 ...
11
votes
3answers
1k views

Approximately Fit Data Without FindFit

I have some noisy data, and I'd like to analyze the derivative of the function the data approximately matches. Interpolation is normally great, but doesn't work ...
2
votes
1answer
143 views

Plot[] breaks behavior of custom data handling function - Recursion

I am working on a utility to analyze a set of data. I want to process the data with a sliding window, in such a way that there is an output associated with each sample of data. To start, I have a ...
22
votes
4answers
2k views

How to integrate functions of linearly interpolated data?

At first, consider integration of pure InterpolatingFunction. Importing some data (works in v.9, for earlier versions one can use this link to download zipped <...
1
vote
0answers
103 views

ReadList and multiple access of the same file

I have a file containing a large amount of data which can be processed independently. In order to process all data in a reasonable amount of time I distribute the jobs on three different machines. ...
1
vote
3answers
828 views

How to export a header before the data in the result file?

I would like to export a phrase to my result data as a heading. I'm exporting the data correctly with Export["file.dat", table]. But I'm doing several ...
2
votes
3answers
332 views

Making data persistent, as in xmgrace

I've been using xmgrace for quite a long time as my favorite solution for scientific data manipulation and plotting. These data often come from home-made fortran programs, in the form of, e.g., a ...
1
vote
1answer
694 views

Optimizing the fitting of Ordinary Differential Equation(NDsolve) involving interpolation functions

I am a newbie to this forum. I am trying to use NDsolve to solve some ODEs involving the interpolation function. Then, I will compare the fitted curve and data to determine the ODE coefficients. ...
5
votes
1answer
325 views

LinearModelFit with millions of rows

I'd like to do a simple regression with millions of rows of (x,y) data. It seems to be too long for Mathematica. Is there a way to speed it up? It shouldn't be that hard to do, I am not sure what ...
4
votes
3answers
661 views

How do I make a frequency heatmap?

I have an array of data that has repeats in it. I want to generate a heatmap depending on how much repetition each point has, as in how many times it is repeated.
0
votes
1answer
507 views

importing and manipulating data file

I have a large text file containing numeric data of 1000 groups each of 200 lines. The lines are unsorted. There is a 9 line header at the beginning of each group, i.e., 209 ⨉ 1000 lines altogether. ...
16
votes
2answers
3k views

How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
3
votes
0answers
112 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
8
votes
2answers
6k views

How do I plot the derivative of a set of experimental points?

I have a set of points like this: P = {{$x_1$,$y_1$},{$x_2$,$y_2$},...,{$x_n$,$y_n$}}. I want to plot them and also plot the derivative of that graph. How can I do this with Mathematica? Is there a ...
8
votes
3answers
1k views

Analytically finding critical values in a hysteresis curve made with ListPlot

Let me start with a super basic explanation of what I'm trying to do, and I will follow with a more detailed discussion afterwards. From data in the form shown below (a ...
17
votes
2answers
2k views

Dealing with large files

I have a *.dat file in which the contents are basically one giant list: {1,2,3,4} {5,6,7,8} {9,10,11,12} and so on. I'd like to read in the first 500 members { }...
29
votes
4answers
1k views

Filtering beat-to-beat heart rate data

From an experiment, I have a dataset of beat-to-beat heart rate data: a list of the time between each heart beat in [ms]. The data is measured using an infrared optic sensor at the finger tip. The ...