Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

1
vote
3answers
562 views

How to export a header before the data in the result file?

I would like to export a phrase to my result data as a heading. I'm exporting the data correctly with Export["file.dat", table]. But I'm doing several ...
2
votes
3answers
288 views

Making data persistent, as in xmgrace

I've been using xmgrace for quite a long time as my favorite solution for scientific data manipulation and plotting. These data often come from home-made fortran programs, in the form of, e.g., a ...
1
vote
1answer
567 views

Optimizing the fitting of Ordinary Differential Equation(NDsolve) involving interpolation functions

I am a newbie to this forum. I am trying to use NDsolve to solve some ODEs involving the interpolation function. Then, I will compare the fitted curve and data to determine the ODE coefficients. ...
5
votes
1answer
292 views

LinearModelFit with millions of rows

I'd like to do a simple regression with millions of rows of (x,y) data. It seems to be too long for Mathematica. Is there a way to speed it up? It shouldn't be that hard to do, I am not sure what ...
4
votes
3answers
536 views

How do I make a frequency heatmap?

I have an array of data that has repeats in it. I want to generate a heatmap depending on how much repetition each point has, as in how many times it is repeated.
0
votes
1answer
435 views

importing and manipulating data file

I have a large text file containing numeric data of 1000 groups each of 200 lines. The lines are unsorted. There is a 9 line header at the beginning of each group, i.e., 209 ⨉ 1000 lines altogether. ...
15
votes
2answers
2k views

How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
3
votes
0answers
105 views

External file Sub Matrix Import

Is there a Built-In Mathematica Function to extract a Sub matrix from a huge external file? Functions getting pieces of internal nested Lists, (arrays) are in place. Example: ...
6
votes
2answers
4k views

How do I plot the derivative of a set of experimental points?

I have a set of points like this: P = {{$x_1$,$y_1$},{$x_2$,$y_2$},...,{$x_n$,$y_n$}}. I want to plot them and also plot the derivative of that graph. How can I do this with Mathematica? Is there a ...
8
votes
3answers
992 views

Analytically finding critical values in a hysteresis curve made with ListPlot

Let me start with a super basic explanation of what I'm trying to do, and I will follow with a more detailed discussion afterwards. From data in the form shown below (a ...
15
votes
2answers
1k views

Dealing with large files

I have a *.dat file in which the contents are basically one giant list: {1,2,3,4} {5,6,7,8} {9,10,11,12} and so on. I'd like to read in the first 500 members { ...
28
votes
4answers
1k views

Filtering beat-to-beat heart rate data

From an experiment, I have a dataset of beat-to-beat heart rate data: a list of the time between each heart beat in [ms]. The data is measured using an infrared optic sensor at the finger tip. The ...
3
votes
0answers
284 views

How to save (large) data generated from simulations

The following is a simple example of a much larger simulation that I want to run Define some processes ...
0
votes
1answer
424 views

How to create function that classifies sample data

I want to create expression, that classifies samples ( 4 weather features -> decision). Example training data: For the values that i have data for, I can use rules: ...
10
votes
2answers
3k views

How to get Matlab data imported with the same dimensions?

I have some Matlab image data with the following dimensions (output is from Matlab): >> size(im) ans = 86 86 3 45 The data imports just fine ...