Reputation
80,208
Next tag badge:
288/100 score
18/20 answers
Badges
8 235 330
Newest
 Good Answer
Impact
~555k people reached

Jul
3
comment Lazy evaluation/loading of parts of an expression
No problem. Whatever works for you.
Jul
3
comment Paging RAM in case of memory shortage issue
I could help you with formatting :) If you wish, I can edit your toHD with what my code formatter outputs for it. If you don't like it, you could roll back.
Jul
3
awarded  Enlightened
Jul
2
awarded  Nice Answer
Jul
2
comment Lazy evaluation/loading of parts of an expression
As a simple alternative, you can use Hold or HoldComplete as your container, instead of List: x= Hold[Get[...],...,Get[...]]. Perhaps this is as transparent as it gets, but I am not really sure what this buys you that can not be totally covered by overloading functions such as Part.
Jul
2
comment Lazy evaluation/loading of parts of an expression
What I would do would be to try supporting the core functions (there will likely be fewer of them than one might think), and see whether or not that would cover most of the use cases of interest to you. You can always have a fall-back case where the lazy stream is converted to a normal list (expression) for functions which have not been overloaded explicitly. Note however that many functions will also need a different algorithm when overloaded on lazy streams, to take an advantage of them (e.g.Sort, etc). Making lazy streams an integral and transparent part of the language is no small task.
Jul
2
comment Lazy evaluation/loading of parts of an expression
Yes, you will have to know the functions. Or, support all core functions in Mathematica, which is doable but a lot of work. Basically, their is no magic spell here - for example, in order to fully support packed arrays when they were introduced, a huge amount of work was required to overload all important functions in Mathematica, on this (then new) data structure. A lazy stream is no different - since it is not yet offered as a part of the language, one would need to define those functions one want to work on it, oneself. In Java, one would have to implement an interface, etc.
Jul
2
comment Lazy evaluation/loading of parts of an expression
Not sure whether to consider this one a duplicate of the one on file-backed lists, let's see what others think.
Jul
2
comment Lazy evaluation/loading of parts of an expression
The other limitation of the first linked answer is that it only deals with specific expressions - lists with large sub-lists. Again, this can be generalized to arbitrary expressions and their parts, and I actually plan to do so, but this has not been done yet.
Jul
2
comment Lazy evaluation/loading of parts of an expression
The answer is yes. Basically,something similar is done in this answer. It can be combined with this great answer on lazy lists, to make the lazy nature of data loading more explicit, but this may not be necessary. To rephrase, the first linked answer currently lacks generic stream interface and centralized memory manager for streams - both can be written rather easily on top of it though.
Jul
2
comment Improving the source-code indentation in Wolfram Workbench
Returning back to this question, I have now the palette tied to my code formatter, and in fact I already used it also to format code in WorkBench. Basically, I was copying code to FE cells, then using the palette, then copying it back. Right now the palette uses a formatting function that keeps trailing brackets, but I do have another function which does not, and extending the palette to optionally use that one is a very simple matter.
Jul
2
awarded  Nice Answer
Jul
2
comment A fast, robust DropWhile
By the way, it just crossed my mind that a better design would have been to optimize LengthWhile and then always use the simple solution with that optimized lengthWhile - this way we get LengthWhile (and potentially also TakeWhile) optimizations for free.
Jul
2
comment A fast, robust DropWhile
@Pillsy See my edit.
Jul
2
revised A fast, robust DropWhile
Added a JIT-based solution
Jul
2
answered A fast, robust DropWhile
Jul
1
comment Next highly composite number?
So, using RLink is not an option, I guess?
Jul
1
comment Find names of the functions defined in a file
@Liam In your case, the only non-system symbol you have is test, and you use it anyway. To get the output, you need to use ReleaseHold, since the result of loadFile is code wrapped in HoldComplete. In more complicated cases, where you may have many non-system symbols in your file, all of them will be created in the current working context, unless fully-qualified names were used for them (which is not typical). Generally, such namespace pollution is not what one wants.
Jul
1
comment Convert list of lists to cumulative join
This is nice, +1. For very large list of lists, you may want to memoize the partial linked lists instead, and then Flatten them and use a single Join when the function is later called for a particular n. This would make the initial call very fast (linear in the number of lists), for the price of some (not very significant) added overhead for the subsequent calls for specific n. As of now, your initial call is O(m^2), where m is the total number of elements in all lists.
Jul
1
answered Find names of the functions defined in a file