Questions on systematic data handling and analysis, including data correction, manipulation, or selection of user-data or built-in data available with Mathematica.

learn more… | top users | synonyms

6
votes
3answers
130 views

Fitting data with interdependence

I have an issue, of which I'm not sure it can be solved, with or without Mathematica. I feel as if it should be possible, but I'm quite clueless as to how. Some introduction is required. I'm working ...
3
votes
2answers
138 views

Automatically detect higher slope in numerical data (within a certain range)

Suppose the dataset data is part of some simulations whose results should follow a certain model. Here is data and its ...
0
votes
1answer
91 views

Binomial Regression where n > 1

I am not well versed in Generalized Linear Models, but am trying to use this approach to model some discrete data. I have binomial data where the sample size for each observation is the same size, ...
6
votes
1answer
91 views

How to avoid creating negative values when using Interpolation?

I'm having some problems with Interpolation. I'm trying to interpolating some measurements data into mathematica. But using Interpolation will create some negative data, which is actually not possible ...
5
votes
0answers
60 views

Can Predict/Classify be parallelized?

Being able to parallelize Predict /Classify would be useful on multiple fronts: Performance/Scale Streaming Pedagogical ...
3
votes
0answers
41 views

How to export to NetCDF format

Consider I have a scalar field on a grid f(x,y), where x and y can take values ...
1
vote
2answers
38 views

Converting data set from one form to another

I wish to convert Data to Data2. Both sets are provided below. Note that the second set has an arbitrary parameter "t". How can this be done. ...
1
vote
1answer
148 views
0
votes
1answer
39 views

Determine what a sound snippet contains?

The function SystemDialogInput["RecordSound"] brings up an interface that lets you record a sound object. Let's say we first record a one second long snippet of ...
1
vote
1answer
70 views

Performance of Extract-Transform operations

I'm doing simple data analysis, where I load some "date-time" data from a file. The data is a simple list, but there are 41,000 values in total. Here's a short sample: ...
1
vote
4answers
5k views

Creating a data1 versus data2 plot?

I have a table of data with multiple rows and columns, and I have to make a versus data table using two of these columns. Essentially, I have to set one column of data (column 3) as the y-axis and ...
0
votes
1answer
29 views

How can I plot a function of imported data that ends up nested in lists four deep? [closed]

I imported a file of data from a model that I am running for my research. It produces the following output in Mathematica: ...
11
votes
0answers
102 views

How does Dataset compare to DataFrame in pandas?

Pandas is a Python package providing fast, flexible, and expressive data structures designed to make working with “relational” or “labeled” data. Just like ...
10
votes
2answers
4k views

How to get Matlab data imported with the same dimensions?

I have some Matlab image data with the following dimensions (output is from Matlab): >> size(im) ans = 86 86 3 45 The data imports just fine ...
6
votes
3answers
2k views

Find a best fitting curve for some data with no regular pattern

Suppose some data with no regular pattern is given, so common patterns of functions all fail to fit it. Then how can one find the best fitting curve? I believe there will be some data for which we can ...
3
votes
6answers
562 views
2
votes
1answer
79 views

Still more problems with Fitting, please help

I'm in a little over my head here and I'm fully aware of it, so I'm hoping for some help. I've been assigned to a project where I am given data: http://www.pastebin.ca/3061136 (Example Data) I ...
2
votes
1answer
34 views

BinaryWrite negative numbers [closed]

testList = {1, 2, 3, 4}; file = "test1"; BinaryWrite[file, testList]; BinaryReadList[file] Close[file]; Gives {1, 2, 3, 4} ...
3
votes
3answers
80 views

Orderless storage of tabulated function values

I would like to elaborate on my previous question Function of arguments given in non-fixed order and a relevant related one Best practice of passing a large number of parameters to functions I find ...
-1
votes
1answer
85 views

NonlinearModelFit errors and precision

For this problem, I use a data set, which I've made available here. ...
9
votes
1answer
212 views

Median of Dataset in Mathematica 10

I'm having trouble calculating the median of a Dataset[] in Mathematica 10. The situation is as follows. Consider a dataset that was defined as follows: ...
1
vote
1answer
53 views

Importing a spreadsheet with infinity values from Excel

I have imported a spreadsheet from Excel that I want to use as an adjacency matrix. I want to use this matrix as the argument in WeightedAdjacencyGraph, so I have ...
1
vote
2answers
101 views

Plotting data from multiple text files

I'm new to mathematica and stack exchange, so any help would be appreciated. I want to plot a bunch of data from text files that contain a lot of extraneous data. I have a hundred files with different ...
0
votes
2answers
66 views

how to read a list from a file which has mixed types of data?

I need to read in a file with the first several lines reading as ...
6
votes
2answers
111 views

Removing a non linear noise trend from data

I have a dataset that looks as follows Now, this is from an actual measurement (of the reflection coefficient of a microchip with four LC resonators connected in a periodic fashion) so there is ...
1
vote
1answer
62 views

Writing to a serial device failing mysteriously

I have a device which communicates with the computer through serial. The device waits until it receives the character 'a', and then begins to stream data. I have been able to connect to the device and ...
17
votes
4answers
691 views

How: 2D scatterplots with quantitative density-dependent coloring

Consider the following scatterplot of a 50K-point dataset: ...
0
votes
0answers
61 views

Data manipulatation with MapAt and Join

I would like to make data like: {{a1, a2}, {b1, b2, b3}, {c1, c2}, {d1, d2, d3}} -> {{a1, a2, b1, b2, b3}, {c1, c2, d1, d2, d3}} by using ...
0
votes
1answer
96 views

Matching entries then finding positions of two lists

I'm trying to analyse data with Mathematica. I have a big array with various columns representing timestamps and then lots of data. I have some code that will produce a list of timestamps when the ...
6
votes
2answers
298 views

Find fit of an experimental PDF

After quite some struggle, I managed to put up a code that is syntaxically correct. The goal is to find the parameters for a gamma distribution that best match experimental data. The data is a set of ...
17
votes
2answers
441 views

Can A Plot Be Overlayed With A Map?

Some brief, only slightly important background. I am doing a research project using the data from NASA's GRACE mission. I wrote a short Perl script to take two data files and find the change in ...
15
votes
3answers
1k views

Parametric Interpolation of 2D data

I want to create an interpolation through the following 2dim data points: ...
2
votes
2answers
98 views

How do I partition a matrix?

I have a data file with fields x y z. When I import it to "data", I get a matrix if the form: ...
5
votes
1answer
350 views

Why should we make sure the calculated MD5 is of length 32?

I see when people write code on calculating MD5, (even in the documentation), they write like this: IntegerString[Hash[something, "MD5"], 16, 32] But I think ...
2
votes
2answers
77 views

FindDistributionParameters gives an error for a mixture of user defined sinh-arcsinh distributions

I have some data (1593 values). I first binned it using the Knuth rule and fitted a function that is a linear combination of two sinh-arcsinh functions. I then ...
3
votes
1answer
221 views

Spectral Clustering

I am attempting to implement a spectral clustering routine using Mathematica, but I am encountering difficulties. Using the standard Gaussian similarity function found in section 2.2 of this paper ...
0
votes
0answers
52 views

Solve an integral equation: to fit the given data with an integral of two functions?

I am trying to find an efficient way to solve the following equation $$h\left(b\right)=\int_{0}^{b}f\left(\frac{b-c}{1-c}\right)\frac{g\left(c\right)}{1-c}dc$$ where for $h(b)$ I have the data ...
0
votes
0answers
41 views

Handling *.in and *.out files

im trying to use Mathematica in solving of algorithm tasks like this one: http://poj.org/problem?id=1274 When i was doing those in c/c++ i was able to easily pass input/output in Windows command line ...
2
votes
2answers
264 views

Using Periodogram to investigate the seasonality of the data

After asking how to get rid of the seasonality of this set of data, I'd like to ask how to investigate the seasonality in data using Mathematica. Which tool would you use to study seasonality? I.e. ...
15
votes
2answers
2k views

How do you deal with very large datasets in Mathematica?

This question is in two related parts. The first is about dataset size, the second data wrangling. Part 1: I've built a naive Bayesian classifier on Mathematica. It uses ...
1
vote
2answers
245 views

How to Fit experimental data with 4D ParametricNDSolve?

Let me first explain the background problem. I'm trying to solve a problem with a projectile motion in 4D such that the position vector is: $\boldsymbol{r}(t)=\boldsymbol{r}[x(t),y(t),z(t)]$ I have ...
1
vote
2answers
102 views

Slicing multidimensional table

Say I have a multidimensional list created in the following way grid= Table[{i, j+1, k+2}, {i, 0, 1}, {j, 0, 1}, {k, 0, 1}] grid={{{{0, 1, 2}, {0, 1, 3}}, {{0, 2, 2}, {0, 2, 3}}}, {{{1, ...
1
vote
1answer
77 views

Importing Data Issue, Breaking Up Elements

I am trying to import a comma delimited text file as a Table and I am having two issues: First, the file imports all of the numbers that should be separate columns as a single string. This is not ...
0
votes
1answer
75 views

Loop and list all p-values

I have the following mathematica code (see below). Is there a way to iterate this approach in such a way to get the p-value results of each test in a list? I would like to get say 1000 p-values for ...
3
votes
2answers
78 views

Manipulation of .csv imported data — no blanks in either of two lists

Above is a .csv file. I am trying to construct two lists of male and female math test scores such that only districts which include info for both male and female students are included i.e. neither ...
2
votes
2answers
59 views

Extracting coordinates from a ListLinePlot

I used an iterative method to resolve equations, then i plotted the results using ListLinePlot. Unfortunately, now I need to use some of the data that was plot. I know the X-coordinate, and need to ...
5
votes
0answers
47 views

Built-in way to handle uncertainty in measurment and physical constants?

Is there a built-in or fairly elegant way to handle uncertainties in measurement data and/or physical constants? For instance, the 2010 CODATA for the elementary charge is given as $\mathsf{1.602 176 ...
0
votes
0answers
57 views

How to handle substrings of a matrix by a named index

I am using open source code to run stochastic simulations. The output looks like this: ...
4
votes
2answers
159 views

Deseasonalize the data of traffic growth for the past data

In another question (Using TimeSeriesForecast for forecasting the traffic growth) I asked to use the TimeSeriesForecast on this set of data the answer provided the plot with and without the ...
19
votes
5answers
2k views

Moving maximum function?

How can I construct a moving maximum function? For example, if I have a list of 12 values: { 5, 6, 9, 3, 2, 6, 7, 8, 1, 1, 4, 7 } and I want to maximize over 3 ...