Hot answers tagged

37

UPDATE I thought it would be neat to try and animate the thing, so I let the $a$ parameter run between $-\pi$ and $\pi$. I generated 600 images and put them together using ffmpeg. Check it out on youtube. It might not be in the spirit of Mathematica Stack Exchange, but allow me an objection - stuff that is slow in Mathematica should be kept out of it. To ...


36

As @RahulNarain says, forming the image point by point saves significant memory because the number of image pixels is typically much smaller than the hundreds of millions of iterations that compose it. Therefore, iterate the attractor equations, and for each point generated, find its location within the image matrix. Colour coding of the number of hits in ...


34

The behavior described here is the same from 10.0.0 up to at least 10.3. Summary We can look at the code of DeleteDuplicatesBy and it turns out it uses GroupBy. The test cases proposed by Mr.Wizard are all handled by some part of the code of DeleteDuplicatesBy. Other parts of this code also seem to have some issues. Most of the members of the *By family of ...


30

Chunks of weak compositions Here is slightly modified version of algorithm used in Combinatorica`NextComposition converted to a LibraryFunction. Needs["CCompilerDriver`"] " #include \"WolframLibrary.h\" DLLEXPORT mint WolframLibrary_getVersion() { return WolframLibraryVersion; } DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) { ...


28

Is there a possibility to disable stack tracing, but keep messages? Internal`$MessageMenu = False reverts back to the old messages. Seems to do the trick and prevent the leak from my testing.


25

I revisited this problem - this time in pure Mathematica. The trick to any kind of performance is the Compile[] function, which in itself can be a bit moody - so you need to set global options to warn you when it refuses compilation and work around that. The performance I'm seeing is on the order of magnitude slower than that I get from C++, and two orders ...


24

Analysis current as of Mathematica version 11.0.1 and 11.1.0. We can disable the Show Stack Trace item in the new message menu as follows: MessageMenu`$PruneStack; MessageMenu`Dump`$IncludeStack = False; The reference to MessageMenu`$PruneStack is there to ensure that the message menu packages have been autoloaded (otherwise our setting will be lost ...


23

I've always wondered about the scalability of MathLink (now officially "Wolfram Symbolic Transfer Protocol"). This is the protocol used by Mathematica to communicate between the front end and the kernel, and the basis of the Parallel` package. It has quite low bandwidth and high latency relative to, for example, MPI libraries. I also wonder how many MathLink ...


22

Timing charts updated for version 10.0.2. The behavior remains unchanged. Attempting to analyze the performance of this function in the manner that Taliesin Beynon did for PositionIndex I shall use the same tools. The old method that will be compared in all cases below: myDeDupeBy[x_, f_] := GatherBy[x, f][[All, 1]] Speed A BenchmarkPlot of ...


22

I decided to take one of my large packages and Remove all symbols in the Private` context that have no definitions attached to them ... And is there any risk if only those symbols that are not used globally or that do not have definitions are carefully removed? In general, the complexity of the Mathematica language and the intricacies of the evaluation ...


21

Your function F is implemented really, really inefficiently. By quite simple means and in the proposed situation, it can be sped up by a factor of 20000. The key is to start with calculations in machine precision as early as possible and to store frequently used data in packed arrays. n = 100; mlist = Range[-1. + 2/n, 1. - 2/n, 2./n]; m2list = mlist^2; ...


18

The memory leak in NIntegrate is a bug and has been fixed as of version 10.2.0. Earlier versions would lose around 720 bytes per evaluation for this example, which could not be recovered without restarting the kernel. ClearSystemCache[] should be used to make sure the memory is released. Using version 10.2: NI[z_?NumericQ, b0_?NumericQ] := NIntegrate[E^-...


18

Cause Under the hood System`SquaresR is still calling functions in the context NumberTheory`. Partial output of: Needs["GeneralUtilities`"] PrintDefinitions @ SquaresR SquaresR[2, NumberTheory`SquaresRDump`n _ Integer?Positive] := Block[{NumberTheory`SquaresRDump`res}, NumberTheory`SquaresRDump`res = NumberTheory`SquaresRDump`squaresR2[...


18

I suggest not to use Graph to render, as it is slow. Instead, compute the vertex coordinates, then render manually. The rough steps are as follows: Create an "index graph", i.e. a graph where the vertex names coincide with the vertex indices: n = 10000; g = IndexGraph@Graph[ Flatten[(Rule @@@ Partition[#, 2, 1]) & /@ CollatzSequence@Range[n]...


17

ClearAll clears all definitions associated with the symbol. However, the symbol remains in the symbol table, so all references to that symbol from other symbols (their definitions) remain fully valid. The symbol can then acquire new rules or other global properties associated with it. Remove removes the symbol from the symbol table. More precisely, it ...


17

MyImagePartition In the meantime, before Mathematica 10 will come out, you can enjoy my on-foot solution MyImagePartition, which both saves memory and time using the PartitionMap function from the Developer context: MyImagePartition[im_, wh_, dwdh_List: {0, 0, 0}] := Module[{it = ImageType@im, cs = First@Options[im, ColorSpace], il = ...


17

$HistoryLength is just a global variable. It not clear the history. You can still access the history by ByteCount@Out[12] ByteCount@%10 It can be cleared by Unprotect[In, Out] Clear[In, Out] However, it would be better if you set $HistoryLength=0 before your resource-intensive code. P.S. It would be great to have $HistoryMemoryLimit or something like ...


17

Update: The issue has been fixed in Mathematica 10.1! :) Besides: Interesting enough, the suggestion bar's first option after evaluating the statements given above now is "solve for …", so: The assumption, that in 10.0.2 Mathematica's suggestion bar started out trying to actually solve for all variables, which, as I pointed out, is quite impossible ...


15

I am sure you can easily install also Linux on it and then you could contact Vladyslav Shtabovenko, the current maintainer of FeynCalc (https://github.com/vsht) and ask him about hard problems in High Energy Physics he would like to benchmark on such a King-Kong machine. Either him or somebody else could also provide you with more complicated examples of ...


14

This is not a direct answer to your question but it can help you. I see your question is about adaptive thresholding. I propose finding threshold values without exact partitioning. img = Import["http://homepages.inf.ed.ac.uk/rbf/HIPR2/images/son1.gif"] The simplest is GaussianFilter which is analog to $T = mean$ in your link. GF = GaussianFilter[img, 30] ...


14

This function will be rewritten in C for 10.0.2 and should come down to average-case complexity of $O(n)$ from its current $O(n \log(n))$. Note that the version most users will be bothered to write (and the way we advertized this before in the docpage for DeleteDuplicates) is $O(n^2)$, so most users are probably already winning. In the meantime, my advice ...


13

tl;dr I think it's a memory leak (bug) and you should report it to Wolfram Support (please do!) According to my reading, you were saying that after importing JSON files many times, the kernel memory usage reported by the operating system (or some task manager program) was growing to unreasonable levels. However, the memory usage reported by the kernel ...


13

This is also not an answer (I think you found the culprit and it needs to be solved by WRI) but a suggestion for a workaround. The idea is to run the memory leaking code in an extra kernel and restart that kernel every once in a while. Using the parallel functionality of current Mathematica releases this is pretty simple and by choosing an appropriate value ...


13

Please report this to support. You can get rid of some of the memory leak by executing the following: cleanupTextSentences[] := Remove[ "NaturalLanguageProcessing`TextCases`PackagePrivate`interpretations$@", "NaturalLanguageProcessing`TextCases`PackagePrivate`probabilities$@", "NaturalLanguageProcessing`TextPosition`PackagePrivate`...


13

Special case Best way would be to not store this thing at all. It has so much structure that a[[i,j]] is cheaply computable on the fly. Integer and double arithmetic in the CPU are orders of magnitude faster than memory access. Here is how to compute entries on the fly for the given example: MaxLength = 5; a = Table[Tuples[{1, 2, 3}, {n}], {n, 1, MaxLength}];...


12

Let me put my comment into an answer, because I think we might have misunderstood each other. You answered in the comment However, it would be great to do whole process (preparation of the matrix, solving the eigensystem, and further analysis) in Mathematica. That exactly was my idea. You only write some lines of C-Code which are compiled into a ...


12

4 makes sense to me. But first I need to explain how I believe conditions on the rhs work. In a definition like f[x_]:=rhs[x], when Mathematica sees f[something] it immediately replaces it with rhs[something] and continues its evaluation. This is not as trivial as it sounds. This is what makes g[0]:=0; g[n_]:=g[n-1]; g[10000] be an iteration and not a ...


12

Yves showed you how to shave off a factor of 10, and in the following I'll shave off another factor of 10. First, the original code (plus some data of mine to make it work): points = RandomReal[{0, 10}, {1000, 2}]; n = 500; p = .99; g1 = Graphics[{PointSize[1/n], White, g = Point /@ Join @@ {points, points.{{1, 0}, {0, -1}}}, Array[Rotate[g, Pi #/...


12

As a Palette 1) Using ListLinePlot: CreatePalette[{ DynamicModule[{miu, timeStep = 0.1}, Grid[{ {Row[{"Current MemoryInUse [MiB]: ", Dynamic[Style[miu[[-1]], Bold], TrackedSymbols :> {miu}]}], Item[Row[{"time step: ", RadioButtonBar[Dynamic[timeStep], {0.1, 0.5, 1}], Spacer[35]}], Alignment -> Right]}, {Dynamic[...


Only top voted, non community-wiki answers of a minimum length are eligible