Skip to main content
19 votes

Stream reading 1000 times faster in Python

FindList is meant to deal with such tasks. Not sure whether it works with all types of files but for plaintext files like in the case of CSV file it should work.
azerbajdzan's user avatar
  • 19.9k
16 votes

Can Mathematica edit large binary files in-place?

We can do it straightforward in Java using JLink: ...
BlacKow's user avatar
  • 6,438
16 votes
Accepted

How to read a named pipe on Windows?

Short Version On Windows, unlike Unix, named pipes are special and do not behave like regular stdio streams. Programs that use named pipes require custom logic. As a rule the Wolfram Language I/O ...
WReach's user avatar
  • 69.1k
11 votes

Is building an event-based parser possible/reasonable in Mathematica (WL)?

Note: If you want this type of parser added for JSON, I think WRI could do it easily. They already have Developer`ReadRawJSONStream so I think adding some event-...
b3m2a1's user avatar
  • 47k
10 votes
Accepted

Is there a way to create ZIP OutputStream ?

As mentioned already it's easy to implement this functionality with JLink. But once you use Java's ZipOtputStream you will need to convert your data to list of ...
BlacKow's user avatar
  • 6,438
7 votes
Accepted

how do I to create a function similar to feof in C?

Your ideas are too C-oriented. The Wolfram Language is not at all like C despite some syntactic sugar built into it to make some constructs C-like; In my opinion, these features more often lead ...
m_goldberg's user avatar
  • 108k
7 votes

Reading from a socket stream

Beginning with Mathematica 10 there is a new function ReadString to get the (current) available data from an open socket connection by setting ...
TheQuant's user avatar
7 votes
Accepted

Can Read[] apply a custom function automatically to the read values?

Yes! This is documented under Details: You can use Read to get objects to insert into any expression structure, not necessarily a list. Example: ...
7 votes

Can Mathematica edit large binary files in-place?

I have noticed that NETLink does not get a lot of love on this site (at least not as much as JLink does), so let me just post ...
RunnyKine's user avatar
  • 33.2k
6 votes

Is there a way to create ZIP OutputStream ?

Neither the import format ZIP (nor any of those similar to it like GZIP, TAR or BZIP) nor the function CreateArchive do have any documented functionality which ...
Albert Retey's user avatar
  • 23.6k
6 votes

Vector field Stream?

Here are the streamlines using the centers of the arrows in the vector plot as initial conditions: ...
Michael E2's user avatar
  • 239k
5 votes

Opening a pipe with OpenRead

Use sockets. For example, set up a listening connection in shell with nc -l 2222 Then in Mathematica ...
Daniel Genin's user avatar
5 votes
Accepted

SliceStreamPlot3D or StreamPlot3D

Obviously, my earlier first suggestion is exactly what SliceVectorPlot3D does, a function I was not aware of. Therefore, here is my suggestion how to convert 2d <...
halirutan's user avatar
  • 113k
5 votes
Accepted

Matrix Vector multiplication for very big matrices (>>RAM)

You may use ParallelTable and other functions from the Parallel Computation Setup & Configuration guide. With ...
Edmund's user avatar
  • 42.4k
5 votes

Read not advancing StreamPosition

Having written it up, I might as well answer having double-checked my thinking before posting, just in case I was about to make a fool of myself... The problem lies on this line: ...
Julian Moore's user avatar
  • 2,560
5 votes
Accepted

What is the fastest way to read the new lines in the dynamically updatable file?

Wolfram Support Answer The input stream is "buffered" by C, which is to say the file's data is read into memory when the stream is opened. Then all "read" operations work on the buffered data in ...
Alexey Golyshev's user avatar
5 votes
Accepted

How do I reliably read from stdin using wolframscript?

The implementation of $ScriptInputString in wolframscript is buggy. I have found that providing ...
Theelepel's user avatar
  • 328
5 votes
Accepted

How to read doubles from specific location of a large file

Make sure you use the right format for you floating point reals. E.g. with "Real64", where every number take 8 Bytes, we can write 3 numbers and then jump to the second number by: ...
Daniel Huber's user avatar
  • 53.5k
5 votes
Accepted

Memory preserving computation in chunks

In general, a good strategy to free memory when no longer needed is to give the variables a local scope. However, when applying this too finely grained, this may cause many costly allocations and ...
Henrik Schumacher's user avatar
5 votes
Accepted

Plotting phase plane in Filippov systems

This should be more straightforward with my EcoEvo package. First, install package (once ever): ...
Chris K's user avatar
  • 20.3k
3 votes

Explicitly control newline behaviour when writing strings with arbitary encoding

The trouble is it seems when you open BinaryFormat->True it ignores the CharacterEncoding directive. This is a hack writing ...
george2079's user avatar
3 votes
Accepted

How to get the standard error stream for a ProcessObject?

Use ProcessConnection. process = StartProcess[$SystemShell] ProcessConnection[process, "StandardError"] This returns an <...
Szabolcs's user avatar
  • 236k
3 votes

Can Mathematica edit large binary files in-place?

Mathematica is not a great tool to edit large files; the documentation essentially admits this. The obvious answer is to work in another language; I use Java, where it would be trivial to read a file ...
CElliott's user avatar
  • 560
3 votes
Accepted

How to read *all* input from stdin?

I am not aware of any pure Mathematica solution. The usual way is to read from the stream stream = OpenRead["!cat"] You can do this directly with any read ...
Szabolcs's user avatar
  • 236k
3 votes
Accepted

How do I import a 1 gig text file by only importing the last 3 lines of every 10 line chunk?

Maybe something along these lines would work? ...
MassDefect's user avatar
  • 10.1k
3 votes
Accepted

Phase portrait on the sphere

So, one way to do this is to project your vector field (pointwise) onto the vector field that goes around the origin, and then only plot the streams that go through points on the circle—we'll take a ...
thorimur's user avatar
  • 9,020
3 votes

How can I make the vectors (vector heads) larger in a StreamPlot?

StreamScale will affect the size of the arrowheads: ...
Brett Champion's user avatar
3 votes

How to wait for new input from file InputStream

If your system has tail or alternatives: TailProcess = StartProcess[{"tail", "-f", "data.csv"}]; ...
Dan Oak's user avatar
  • 638
2 votes

How to output an expression to an external file in "plain text" format without breaking lines?

Using undocumented FrontEnd`ExportPacket command you can get exactly the same formatting as with Copy As ► Plain Text (i.e. ...
Alexey Popkov's user avatar
2 votes

how do I to create a function similar to feof in C?

I don't think it is possible to check for the end of file on a generic stream (which may be a pipe, or other thing form $InputStreamMethods) as opposed to a file (...
Szabolcs's user avatar
  • 236k

Only top scored, non community-wiki answers of a minimum length are eligible