# Tag Info

50

Streaming module - general, and the case at hand Starting with V10.1, there is an undocumented support for certain lazy operations in Mathematica. However, the primary goal of Streaming is to support out of core computations reasonably efficiently, and lazy operations are only the secondary goal. Example: lazy infinite lists and an analog of enumerate ...

30

Chunks of weak compositions Here is slightly modified version of algorithm used in CombinatoricaNextComposition converted to a LibraryFunction. Needs["CCompilerDriver"] " #include \"WolframLibrary.h\" DLLEXPORT mint WolframLibrary_getVersion() { return WolframLibraryVersion; } DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) { ...

25

Since Mathematica 8 it is possible generate the elements of any group one by one with GroupElements. Here's for example a randomly chosen element of the permutation group on 20 elements: GroupElements[SymmetricGroup[20], {10^6 + 1}] {Cycles[{{11, 13, 19}, {12, 18, 17, 16}, {15, 20}}]} The result is immediately; there's no need to build up the full group ...

22

AbsoluteTiming[Total[Range[1, 100000000]^2]] (* {26.0686, 333333338333333350000000} *) or AbsoluteTiming[#.# &@Range[1, 100000000]] (* {35.2883, 333333338333333350000000} *) or AbsoluteTiming@Total[Range[1, 100000000] /. {x_ -> x^2}] (* {30.9893, 333333338333333350000000} *) To compare with @corey979 solution out = 0; Do[out = out + i^2, {i, 1, ...

20

Subsets function takes optional third argument with standard sequence specification. Using this third argument you can take subsets "in chunks". For example, following code gives three 5-combinations from positions 90000 to 90002, from all 8 trillions 5-combinations of set of 1000 elements: Subsets[Range[1000], {5}, {90000, 90002}] (* {{1, 2, 3, 98, 845}, {...

16

Download a version of this post as a Mathematica notebook by evaluating the following expression in a fresh notebook: Import["http://goo.gl/NaH6rM"]["http://i.stack.imgur.com/Yfan3.png"] Folds over Lists, Lazy Lists (Iteractive) and Tasks (Reactive) We'd like to have a symmetric set of operations over Lists, lazy lists (streams, iterables, or iteractive [...

15

The patch Here is a way to get Streaming working: on a fresh kernel execute the following: Import["https://raw.githubusercontent.com/lshifr/StreamingPatch/master/StreamingPatchBootstrap.m"] DownloadAndInstallStreamingPatch[] Get["StreamingPatch"] If this worked, you should be able to use Streaming. Note however, that the framework itself changed ...

15

The implementation of lazy tuples here pretty much contains the solution to the lazy Outer problem. I will take the relevant parts from that code. The following code constructs a function take, which would, given the start and end positions in the flat list of the resulting combinations, extract the elements: ClearAll[next]; next[{left_, _}, dim_] := {...

12

To overcome the memory issues, let's store as few data in the memory as possible: out = 0; Do[out = out + i^2, {i, 1, 100000000}]; // AbsoluteTiming out {58.7571, Null} 333333338333333350000000 I observed 100-200 MB memory usage in the system monitor. For comparison, the original approach from OP's question: AbsoluteTiming[Total[Map[#^2 &, ...

12

Overview Here is a refinement of @Leonid's approach that is a bit faster. The basic idea is to create a TuplesFunction that encapsulates the tuples information, and can be applied to Part type specs. That is, instead of using: Tuples[lists][[part]] you would use: tf = TuplesFunction[lists]; tf[part] The goal is to make TuplesFunction[__] as small as ...

12

I recently published a lazy list package on GitHub, so I might as well show it off here: https://github.com/ssmit1986/lazyLists Installation instructions can be found on the GitHub landing page above (or in the README.md file). My code overloads the normal list processing functions like Map, MapIndexed, MapThread, FoldList, Pick, Cases, and Select. ...

11

Here are some ways this could be done: list = CharacterRange["a", "g"]; Thread[{Range[Length@list], list}] Transpose[{Range@Length@list, list}] Table[{j, list[[j]]}, {j, Length@list}]; MapIndexed[{#2[[1]], #1} &, list] Inner[List, Range@Length@list, list, List] You could re-index by using Range[0, Length@list-1] or i = 1; {i++, #} & /@ list or ...

10

Chunks of permutations Here is a LibraryFunction implementation of "CoolMulti" algorithm generating permutations of multisets. The algorithm is described in: A. Williams: Loopless generation of multiset permutations by prefix shifts /* permutations.c */ #include "WolframLibrary.h" DLLEXPORT mint WolframLibrary_getVersion() { return ...

9

Migrated from here Mathematica is quite likely to be slow at generating permutations. Here is a Java-based approach. It is based on Java reloader and this nice Java code, which I slightly extended with a getNextMultiple method. So, load the Java reloader first (run the code from that post). Then, execute the following: JCompileLoad @ " import java.util....

8

In my lazyLists package mentioned by the OP, you would do something like this to find out if a list is monotonic: << lazyLists n = 100000; (* lazy representation of the example input *) input = lazyCatenate[{{3, 4, 2}, lazyGenerator[# &, 1, 1, n, 1]}]; monotonicQ[lz_lazyList, test_] := Catch[ FoldList[ If[TrueQ @ test[#2, #1], #2, Throw[False, ...

8

This addresses the underlying problem, which implies a really fast way is not likely to be found, and shows yet another way to sum a list, Tr. One reason the OP's sum is slow is that the sum is bigger than a machine integer (which is what I think BlacKow was guessing). Even the little difference of a few hundred terms makes a big difference in time: Tr[...

8

I will show how similar tasks can be done using (undocumented) Streaming framework. The usual caveat applies: since this is undocumented functionality, there is no guarantee that it will exist in the future versions in the same exact form, or at all. But I thought it may be a nice application to illustrate some of the ideas behind Streaming, as well as to ...

7

You could use LinearProgramming. To use LinearProgramming, convert the list of lists into a single list. For your example we create the list {1, 2, 3, 4, 5, 6, 7, 1, 3, 2, 4, 6}. Since there is no criteria for which tuple to return, I use a cost vector of all 1s. Then, LinearProgramming will try to find a vector v whose dot product with this cost vector is ...

7

Between fast, but memory hungry, vectorized operations represented by Total[Range[10^8]^2] and slow, but memory efficient, top level iteration represented by Do solution, there's a middle ground. One can generate data in chunks of specified length and use vectorized operations on those chunks. This way you can get constant memory usage dictated by size of a ...

7

Since OP's core complaint appears to be about memory use, I've generated memory usage statistics to match BlacKow's data: d[n_] := #.# &@Range[n]; t[n_] := Total[Range[n]^2]; op[n_] := Total[Map[#^2 &, Range[n]]]; re[n_] := Total[Range[n] /. {x_ -> x^2}]; s2[n_] := Sum[i^2, {i, 1, n}]; memUsed[f_, n_] := Module[{ low, mid, up, try }, ...

6

ParallelSum[i^2, {i, 100000000}, Method -> "CoarsestGrained"] // AbsoluteTiming {0.128204, 333333338333333350000000} The parallelized version of the Do loop in the answer by corey979: AbsoluteTiming[ ParallelEvaluate[out = 0]; ParallelDo[out += i^2, {i, 100000000}]; Total@ParallelEvaluate[out] ] {35.3163, 333333338333333350000000}

6

Applying DeMorgan's law to the logic simplifies things a bit: With[{ d = Differences[input] }, Nand[AnyTrue[d, # < 0 &], AnyTrue[d, # > 0 &]] ] The idiomatic™ way to solve this is with SequenceCases to report the first case where an element is smaller than the previous one: ismontoneinc[list_] := SequenceCases[list, {x_, y_} /; y < x, 1] ==...

6

Here's an answer that seems pretty fast and small. It essentially just implements a simple depth-first search, with the current row of s being investigated being at index i, and the current index investigated in each row j being a[j]. It also handles your kmin case, with the default (omitted argument) being k. It stores some found values, but not many—only ...

5

A lot of times, we don't necessarily need all the permutations, the real purpose is to select a part of interest according to condition. I wrote a function selectPermutations, it's quite efficient and take up very little memory, sometimes would be useful. ClearAll[selectPermutations]; Options[selectPermutations]={CompilationTarget->"WVM"}; ...

5

I like WReach's answer because it shows how to compose an expression that is similar to the example in the question. I like Sal Mangano's answer because it is concise and shows how to make something that behaves like a C# enumerator. I hope to provide a little of each. Enumerator[state_:0, increment_:(# + 1 &)] := Module[ {s = state}, (s = ...

4

Here is a recursive implementation. The recursion is stopped when the first result is found using Throw. For more results the recursion can be stopped later. k is the number of sub lists and kmin is the minimal number of terms in the sum of n: s = {{1, 2, 3}, {4, 5, 6, 7}, {1, 3}, {2, 4, 6}}; n = 19; (*searched sum*) k = Length[s]; kmin = k; (* minimal ...

3

Also you can use this simple "trick" : given you example list : a = {{"11", "12", "13"}, {"21", "22"}, {"31"}, {"41", "42"}}; the corresponding 12 (3x2x1x2) unique combinations correspond also to the unique 12combinations of the elements position in their respective list: (for example here above, for the entry #9, 13 is at position 3in the {"11", "12", "...

2

I don't know why I missed this question, but the Streaming framework computes this out of the box. Unfortunately, it has to be patched in modern versions of Mathematica, so I refer to this answer for a patch. Assuming that that code has been executed, let us create a lazy range: lr = StreamingListAPILazyLazyRange[1, 100000000] Then check its length: ...

1

First: Your integral equals 0 ! Integrate[II, {\[Phi], 0, 2 \[Pi]} ] (*0*) Second: Mathematica evaluates in finite time (~84s) the integral if you provide additional condition R>0 Integrate[II, {\[Phi], 0, 2 \[Pi]} , {\[Theta], 0,\[Pi]}, Assumptions -> R > 0] (*0*)

1

You seem to have about 23 digits of precision here. The result of a controlled precision calculation? x=1.449778855312947885432923 (* 1.4497788553129478854329 *) Anyway, if you want subsequent calculations to be fast, simply use N: N[x] (* 1.44978 *) Note that the actual precision here is about 16 digits, but Mathematica only displays 6. Calculations ...

Only top voted, non community-wiki answers of a minimum length are eligible