# Tag Info

24

Given that your data is on an integer grid, it may be sufficient to define a boundary point as one which does not have 4 neighbours (up, down, left, right) boundary = Complement[spiel, Intersection @@ Outer[Plus, {{0, 1}, {0, -1}, {1, 0}, {-1, 0}}, spiel, 1]]; ListPlot[boundary]

21

test = {5, 6, 9, 3, 2, 6, 7, 8, 1, 1, 4, 7} MaxFilter[test, 1] (* {6, 9, 9, 9, 6, 7, 8, 8, 8, 4, 7, 7} *) You can also use Max /@ Transpose[{Rest[Append[#, 0]], #, Most[Prepend[#, 0]]}] &[yourList] which is competitive with the MM MaxFilter, but will allow you to change the 'slide' (e.g.pad with zeroes, or other arbitrary 'start').

20

You are looking for ideas, so I will venture a partial solution in the hope it might inspire something useful. The idea presented here is to exploit a statistical model of the heartbeat intervals as a way to test the goodness of any attempted clustering of the data. The approach is general, because it is based on maximum likelihood methods, but I will ...

19

It is possible to include the number of peaks (denoted $n$ below) in minimum searching. First we create some test data: peakfunc[A_, μ_, σ_, x_] = A^2 E^(-((x - μ)^2/(2 σ^2))); dataconfig = {{.7, -12, 1}, {2.2, 0, 5}, {1, 9, 2}, {1, 15, 2}}; datafunc = peakfunc[##, x] & @@@ dataconfig; data = Table[{x, Total[datafunc] + .1 RandomReal[{-1, 1}]}, {x, ...

15

One very simple basic approach is the following (I assume you already loaded the variable spiel) With[{m = Normal@SparseArray[spiel -> 1]}, boundary = Position[m - Erosion[m, 1, Padding -> 0], 1]; ListPlot[{Position[m, 1], boundary}, PlotStyle -> {Gray, Red}]]

12

NIntegrate has many advanced options that let you control which algorithms and strategies it will use. I'm quite sure that you can find a set of options that will make NIntegrate work well enough for the desired task, but of course these numerical algorithms will never be quite as fast and precise as an exact solution, which your sums and Integrates results ...

12

Seeing as how someone has been nice enough to write the C-code for you, you could just use that. Assuming you have a C compiler on your machine, here's how you use it within Mathematica. Note that code is defined below. (* Be sure to define code first! *) Needs["CCompilerDriver"]; url = "http://www.stackexchange.com/"; checksum = CreateExecutable[code, ...

12

Looking at your plotted data you can see about 40 cycles of the dominant frequency, this tells you that the peak will appear somewhere around the 40th element of the DFT. That's in the region where your plot of the DFT is clipped, so it's no wonder you can't see the peak. Looking at the relevant part of the DFT you can see the peak quite clearly: ...

11

You can read lines from an InputStream strm (opened with OpenRead) in batches: lines=ReadList[strm, "String", 4000] You can vary the chunk size based on your application, 4000 is a number I found to work well for reading web server logs with lines that aren't crazy-long. You can also reposition for random access on startup. Version 9 improves the use of ...

11

I am not sure what exactly that rank mean. But here's direct rough porting of code: ConvertStrToInt[url_String, init_, factor_] := Fold[FromDigits[IntegerDigits[#1*factor + #2, 16, 8], 16] &, init, ToCharacterCode[url]]; HashURL[url_String]:= Block[{c1, c2,t1, t2}, c1=ConvertStrToInt[url, 5381,33]; c2=ConvertStrToInt[url, 0,65599]; ...

11

My interpretation of your question is that you want to fit a linear combination of peaked functions with non-negative coefficients. Beware: The minimum misfit solution with non-negative coefficients is a few isolated delta-functions. Therefore, allowing peaks widths is useless, whether for least square or least absolute error, because the minimum allowed ...

11

Using the fourth and fifth arguments of Partition gives you exactly what you want lis = {5, 6, 9, 3, 2, 6, 7, 8, 1, 1, 4, 7} Max @@@ Partition[lis, 3, 1, {2, 2}, {}] Gives: {6, 9, 9, 9, 6, 7, 8, 8, 8, 4, 7, 7} Update As Simon Wood suggested in the comment below (I also know this but on my system the difference isn't that much), Maping Max instead ...

11

Another option is DeveloperPartitionMap. In RunnyKine's solution we first partition the list and sweep through it to add Max to every element. With Developer`PartitionMap we can do both at the same time, which is faster. Here's a table for reference. My first table was incorrect and I apologize for that, it was an honest mistake which I am not sure how it ...

10

You want to remove high-frequency noise while retaining the low-frequency signal. This is a job for a bandpass filter. A simple one is the MovingAverage, which you can apply like so: xsi = Interpolation[MovingAverage[data, 20]] Plot[{Derivative[1][xsi][t], Cos[t]}, {t, 0, 6.25}, PlotRange -> {-1.1, 1.1}]

10

This filters out your points by their EuclideanDistance. I think it makes a pretty good job preserving the curve features with very few points: rx[n_] := Accumulate[Prepend[RandomVariate[ExponentialDistribution[1000], n], 0]] ry[n_] := Accumulate[Prepend[RandomVariate[NormalDistribution[0, .001], n], 0]] c = Transpose[{rx@#, ry@#}] &@100000; t = ...

10

Here is a modified version of belisarius' answer I ended up using. Instead of using the euclidean distance to find out if two points are close to each other I set a fixed rectangle in an area and collapse it into a line. For instance, consider the same example. rx[n_] := Accumulate[Prepend[RandomVariate[ExponentialDistribution[1000], n], 0]] ry[n_] := ...

10

Actually, it is all about packing. By using RandomReal you an generating packed sub-arrays even if the complete array is not packed (and can't be, due to irregular shape): Map[PackedArrayQ, testList, {2}] // Short {{True,True},{True,True},{True,True},{True,True},{True,True},<<91>>,{True,True}, {True,True},{True,True},{True,True}} Let's ...

9

LinearModelFit does too much--it computes residuals, etc., etc. When working with large problems, just compute what you need when you need it. It all begins with the fit itself, which should be done with LeastSquares: First[Timing[ LeastSquares[{ConstantArray[1, #], RandomReal[NormalDistribution[], #]}\[Transpose], RandomReal[NormalDistribution[], #]]]] ...

9

Using your values for data, this seems to work: error = StandardDeviation[data]; data //. {a___, b_, c__, d___} /; Abs[b + c - Median[data]] < error :> {a, b + c, d} I used StandardDeviation[data] because that's what you used, but you can put in whatever error bound you think is best there. Also note that I replaced Mean[data] with Median[data], ...

9

A simple Mathematica-only solution is: CountLines[file_String /; FileExistsQ[file]] := Module[{counter = 0, str = OpenRead@file}, While[ Read[str, Record, NullRecords -> True] =!= EndOfFile, counter++ ]; Close[str]; counter]; which is quite slow of course, so 123 MB (1978142 ...

9

The question is not so innocent as is appears. Without a penalty on the number of peaks the "best" model is overfitting the data. The answer by Silvia demonstrates this already. And, think about it, you got what you wanted: adding more peaks will fit the data better. Always! One may revert to adding an ad-hoc penalty function on the number of peaks. But ...

9

FullDefinition You can use the undocumented(?) multi-argument syntax of FullDefinition: ExportString[FullDefinition[ff, gg], "Package"] (* Created by Wolfram Mathematica 7.0 : www.wolfram.com *) ff[1, 1] = 1 ff[1, 2] = 2 gg[1, 1] = 3 gg[1, 2] = 4 You will notice that any arguments of FullDefinition after the first will be colored, by default in ...

9

This one is fairly fast. I used GatherBy to collect like data rows and kept the ones that matched another. (I assumed that the id entries of a and b are unique in each table.) The appropriate entries are then extracted. On 10000/5000 entries: a = Table[{x, RandomReal[]}, {x, 1, 10000}]; b = Table[{x, RandomReal[]}, {x, 1, 10000, 2}]; ...

9

I don't think there are "general methods". Normally interpolation or curve fitting can be used. Let's see your particular problem. You have five distinct samplings: ListLinePlot[data[[# ;; -1 ;; 5]] & /@ Range@5] The first one shows the problem: ListLinePlot[data[[1 ;; -1 ;; 5]]] Let's see which point is the outlier: ...

9

update Just to clean things up a bit, we can use the discussion here to make a couple functions that help extract the frequency data from this dataset. I define two functions findPeriod and reconstruct: Clear[findPeriod]; findPeriod[data_, threshold_] := Module[{fs, s1, s = {}, i, a0f, af, pf, pos, fr, frpos, fdata, fdatac, n, per}, n = ...

9

Here are the two ways I have successfully used MongoDB with Mathematica. AFAIK much of this would also apply to similar DBs such as CouchDB. A. Easy way One working example used a third party Mongo supplier such as 28msec.io and use URLFetch e.g. With this supplier you can both read from and write to the database using HTTP GET and simple query strings. ...

9

N.B. Your actual data calls for a more sophisticated approach than the quick hack in my original answer, so I've replaced it with a much better and quite general solution. There are two things that make your actual data harder to work with than the toy example. First, it is highly irregular and nonuniformly distributed: ListPlot[Most /@ data, AspectRatio ...

8

You can try to force NIntegrate into doing a similar approach than you did by calculating the sum directly. This reduces your error quite a bit, although I don't understand the reasons behind this exercise: ni = NIntegrate[int[ww], {ww, dataF[[1, 1]], dataF[[-1, 1]]}, Method -> {"TrapezoidalRule", "Points" -> Length[dataF]}, MaxRecursion -> ...

8

f = Interpolation[l3= (Last /@ Sort /@ GatherBy[l[[All, 1 ;; 2]], #[[1]] &])] ListContourPlot[l, RegionFunction -> (#2 < f[#1] &)] Edit If you want a smoother curve, you could use for example whuber's method here for getting something similar to an "envolvent" curve: l4 = {(#[[1]] - 14) 2 + 1, #[[2]]} & /@ l3; nrow = 32; i = ...

8

The simplest and most efficient way would probably be using the common wc external utility. For example, In[33]:= Import["!wc ~/test.m", "Table"] Out[33]= {{6, 5, 56, "/Users/szhorvat/test.m"}} You'll get wc by default on Linux/OSX, but you can install it on Windows too.

Only top voted, non community-wiki answers of a minimum length are eligible