Reputation
65,463
Next tag badge:
207/100 score
19/20 answers
Badges
6 194 285
Newest
 Nice Answer
Impact
~408k people reached

Nov
26
comment Reading periodic elements from a large file
It seems to me that the whole point of the question was to avoid allocating large space on the heap, even if that is only needed until the computation is finished. When you have hundreds of millions of rows, you'll likely run out of memory during the computation, it won't even finish reading. Avoiding this is the whole point of using streams in this context.
Nov
26
comment Failed to connect remote Mathematica kernel using J/Link
Hi Todd, nice to see you here! +1.
Nov
26
comment Speeding up numerical Fourier Transform
Hi Mike, welcome to Mathematica at Stack Exchange! I've been reading your blog with great interest for a while now. It's great that you joined, and I hope you will like the community here. Gives me a pleasure to give you the first upvote on this site,+1.
Nov
26
comment Reading periodic elements from a large file
Actually having an all-Mathematica solution is very good. Besides, I am not sure how memory-efficient my solution is. It should be ok, but I did not really measure that. It may very well be that yours is better in this respect. I personally was more interested in pushing the speed envelope. Once I tried RandomAccessFile and FileInputStream vs BinaryReadList and found that they are on par. Now I know that the buffered read via Java can still offer a speed advantage. And, another use case for my Java reloader :)
Nov
26
comment Reading periodic elements from a large file
+1. I was thinking in this direction as well, actually my Java code implements the same idea. But since I started with Java, I wanted to get as fast as I could :)
Nov
26
revised Reading periodic elements from a large file
added 247 characters in body
Nov
26
answered Reading periodic elements from a large file
Nov
25
comment Precise Timing for nested functions, how to?
Related questions: profiling-from-mathematica, and workbench-profile-question. Not fully sure, but this one may be a dupe of one of those.
Nov
25
comment clean functional way to get first n rows that yield maximum rank
I certainly agree. My comment was rather aimed at the OP's general request for the better myTake function, which can be sort of read off from your code.
Nov
25
comment clean functional way to get first n rows that yield maximum rank
This may be OK for smaller matrices, but due to copying a list at every step, has generally the quadratic complexity in the matrix size (or list size in general). This problem is unavoidable in the OP's formulation though, since the crit function demands partial lists at every step, so +1.
Nov
25
comment clean functional way to get first n rows that yield maximum rank
@J.M. TakeWhile by itself is not quite what is being asked, since the criteria must be applied to the taken list as a whole, not individual elements.
Nov
25
comment Cannot calculate with Dynamic variables in a grid
@Rojo Good point, thanks. I will edit the post some time soon to incorporate this.
Nov
24
comment Cannot calculate with Dynamic variables in a grid
@Rojo I don't know, it just didn't work for me. When enabled in my version of the code, it prevented me from changing the original value in an input field. I did not look very hard at it though, perhaps I overlooked something obvious.
Nov
20
answered Cannot calculate with Dynamic variables in a grid
Nov
20
comment How can I know if a pattern corresponds to a string pattern?
+1. A look at the implementation reveals some unpleasant behavior,e.g. stpatt = "**"; System`Dump`validStringExpressionQ[stpatt] returns False, while System`Dump`validStringExpressionQ[Evaluate[stpatt]] returns True. I would prefer to get True in both cases.
Nov
17
awarded  Favorite Question
Nov
15
comment Dealing with large files
My framework has not been yet optimized to work with lists where every part is very small but the number of parts is just huge. OTOH, it is relatively straightforward to do. The framework itself need not be modified, since this scenario can be addressed by writing some code on top of it. I will try to work out some example tomorrow.
Nov
15
comment File-backed lists/variables for handling large data
@b.gatessucks ... /example (say, as a separate question), I can try to illustrate how it can be done with some specific code. Once you convert your file to the file-backed list and save it, you can work with it fully within the framework and get all the benefits it can give you.
Nov
15
comment File-backed lists/variables for handling large data
@b.gatessucks Yes, as long as you have some means to read that file into Mathematica piece by piece. This can be, for example, some custom file-reading function based on streams, either pure Mathematica (a.g. using BinaryRead or BinaryReadList), or say Java. Then, you just have to read the next element of the list (assuming your file represents one large list - which is the current assumption of the framework), and use appendTo from the above API to append it to the file-backed list (or you can use appendList to append many elements at once). If you give a specific file format /...
Nov
14
comment How to work with Application Project files in Wolfram Workbench?
@rm-rf I've added an answer. Hope this will help! Let me know if you'd like to see something there which I missed, and I will edit it.