Questions tagged [streams]

A source of input or output: files and pipes are both examples of general Mathematica objects known as streams.

Filter by
Sorted by
Tagged with
1
vote
2answers
56 views

How to wait for new input from file InputStream

I'm trying to open a file to read and do some processing indefinitely until I decide to abort: ...
0
votes
0answers
20 views

Combine two streamplots with different position dependent color functions

Building from replies to other posts I have come up with a combination of two stream plots which has a position dependent color function for stream lines which the same for the two plots a way to &...
0
votes
1answer
21 views

How can I make the vectors (vector heads) larger in a StreamPlot?

In my StreamPlots, the vectors can hardly be seen, in particular, since the heads of the vectors are so small. Can I change this? Example: ...
2
votes
1answer
89 views

Phase portrait on the sphere

I have that $(x,y)\in S^1$, i.e. on the sphere, and in Mathematica, the ODE system ...
2
votes
2answers
113 views

Converting Strings to Reals in a List

I am new to Mathematica and working with streamed data: ...
0
votes
1answer
52 views

How to do computations with a specific column of a stream?

I'm trying to calculate the mean of a specific column of a huge dataset (given as .csv-file) using a stream in mathematica. I tried the following: ...
3
votes
2answers
116 views

How do I import a 1 gig text file by only importing the last 3 lines of every 10 line chunk?

I am trying to get the last three lines of every 10 line chunk into mathematica as a data variable. The text file has 3 columns. For example: ...
2
votes
1answer
44 views

StreamDensityPlot and its legends

I'm trying to plot this simple code ...
1
vote
1answer
53 views

write output to stream and also print it

I tried to save the Print[] output to a file using this code: ...
2
votes
0answers
63 views

How to read input stream with no newlines?

I have a script named stdout.wl: ...
0
votes
0answers
34 views

Selective read/write on a mx file

Consider an mx file to which has been saved the following data ...
4
votes
0answers
30 views

Is there an equivalent to “Open Update”

I'm writing a binary file that contains a header followed by the data. The header contains the length of the data. The data is written first, skipping past the header. Once done, I would like to ...
3
votes
0answers
97 views

Reading through file one line at a time, running out of RAM [closed]

I have a handful of files (some of which are very large), and I would like to loop through them one line at a time, run a calculation, and then write the result of the calculation to a new output file:...
1
vote
0answers
47 views

Issue with StreamDensityPlot

I'm plotting a complicated vector field that is yielding the following particular StreamDensityPlot. I'm using the following command to plot it: ...
6
votes
1answer
147 views

How do I reliably read from stdin using wolframscript?

I am trying to direct a stream of data to a wls script in a command line environment. Wolframscript provides the ...
2
votes
1answer
172 views

Vector field Stream?

The following graph shows the vector plot. How can I plot the solid lines which represents how the vectors are spatially varying and both on the same graph (vector plot and stream field)? ...
10
votes
1answer
3k views

How to read a named pipe on Windows?

Tried to open an Out named pipe from Powershell ...
5
votes
0answers
138 views

Optimize Use of PeakDetect in Mathematica

I am quite new to Mathematica and I am looking for some advice in order to improve and optimize my code. Here is what I am trying to do: Open a stream Read the stream line by line and store it in "...
1
vote
2answers
132 views

Reading CSV data from a stream [duplicate]

I'm trying to figure out how many rows/records my csv file has, which I have to stream because it's quite large. Can someone tell me if my approach works and how I can get it to loop through the ...
2
votes
0answers
194 views

How to force Close of opened stream when stream write previously failed (bug in opening stream twice)?

I opened a stream like: str = OpenWrite[NotebookDirectory[] <> "myStream.tsv"]; and made an error when writing the first line (meant to be the tsv headers):...
9
votes
2answers
120 views

What is the fastest way to read the new lines in the dynamically updatable file?

I'm writing a log to a simple .txt file and want to read the last few records using Mathematica. But there are some problems. f = OpenRead["logs.txt"]; Skip[f, ConstantArray[Number, 2], 1]; Read[f, ...
2
votes
1answer
140 views

Streams in parallel kernels. Writing in multiple files in ParallelDo loop in Mathematica

I need to write data in multiple files in ParallelDo loop: ...
2
votes
2answers
50 views

Naming indexed output streams

I need to write into multiple files in a Do loop. Hence, I tried to open multiple streams: ...
3
votes
2answers
187 views

Join two datasets with a function similar to Vlookup [Big Dataset and Efficiency issue] - Wizards are welcome!

I am working on a huge dataset and I have a lot of data formatted in XML, we are talking about a couple of GBs. For this reason I am currently using the stream function to read them and extract the ...
2
votes
0answers
72 views

Why does ReadString complain about the stream not being opened in binary format?

This works fine: ...
11
votes
0answers
129 views

LibraryLink: How to read from a Mathematica stream in C?

Mathematica has a flexible streams interface. Functions like Read can read from a "stream" that may be backed by a file, pipe, string, network resource, etc. We can ...
3
votes
1answer
315 views

How many records? In a file (streaming)

I want to know how many records a file (large data) contains... with streaming...like this: ...
7
votes
0answers
515 views

Communicating with server over Websocket using SocketConnect

I am trying to connect to a web socket feed of Gdax. The documentation is here, https://docs.gdax.com/#websocket-feed. There is no account needed and the information is public. I have successfully ...
0
votes
0answers
92 views

Is it possible to parallelize reading in Mathematica?

I have a gigantic data file that needs to be processed. I am reading it in chunks using ReadList with an InputStream. In my ...
3
votes
2answers
94 views

Explicitly control newline behaviour when writing strings with arbitary encoding

On Windows system, Mathematica by default opens stream which will automatically transform any "\n" into "\r\n". But I want pure ...
20
votes
1answer
378 views

Is building an event-based parser possible/reasonable in Mathematica (WL)?

Too long? Go to Problems section Background I like to learn by doing small, sometimes artificial, projects. Among other things I had 'parsers', and 'stream methods / streams in MMA' on my to learn ...
5
votes
1answer
414 views

Matrix Vector multiplication for very big matrices (>>RAM)

Problem Suppose a matrix-vector product of the form $M\vec{v}$ should be calculated where the amount of storage needed to store $M$ is noticeably larger than the available RAM on a machine. What is ...
1
vote
0answers
33 views

How to export result from SQLResultSet into a csv

I have a sql query which produces very large data from a remote server but the data is too large to fit into memory all at once. So i want to read and export it sequentially into a csv. I am using the ...
1
vote
1answer
37 views

Read not advancing StreamPosition [closed]

[MMA 11.0, Win7 64bit] I have a number of CVS files of ~1-2GB, millions of rows, so I want to read them in records, chunks building on Albert Retey's answer to this question I assumed, and the ...
5
votes
0answers
63 views

Why ReadList ignores NullRecords for Number?

When thinking on this recent question the immediately obvious solution which came to my mind was to use ReadList with options ...
6
votes
1answer
185 views

Is there a way to redirect the output from `Echo`?

The newish function Echo is tremendously useful in quick-and-dirty debugging, since you can stick it pretty much anywhere in an expression and get feedback about ...
3
votes
0answers
37 views

Unexpected behaviour with WordSeparators and Read [duplicate]

I am confused about how Read separates elements. The following all work fine: ...
4
votes
1answer
57 views

How to get the standard error stream for a ProcessObject?

When you run an external program using RunProcess, the program will run to completion and you will receive an association containing the exit code, stuff that went ...
6
votes
1answer
199 views

wolframscript creates tons of log files

On Linux, wolframscript seems to create log files in .Mathematica/ApplicationData/ProcessLink/Streams for example: ...
1
vote
0answers
101 views

Why does using BinaryWrite make my file bigger? [closed]

I have a binary file 00010005.dat. I want to normalize the matrix so I use the following code: ...
7
votes
1answer
1k views

SliceStreamPlot3D or StreamPlot3D

I know that neither of the two functions listed in the title exist, but I basically replaced vector for stream to give you an idea of what I want. The most ideal would be ...
3
votes
0answers
946 views

How to visualize the vorticity of a 3D velocity field? [closed]

I sample a snapshot from a system with 50 particles including their position and instant velocities. The snapshot is a list of 3 position and 3 velocities values per particle. i.e. 50 lines each of ...
9
votes
2answers
283 views

Is there a way to create ZIP OutputStream ?

I want to write data to a file using compression. Is it possible to do that just in time (like in java ZipOutputStream). E.g. what I want: ...
27
votes
3answers
616 views

Can Mathematica edit large binary files in-place?

I have a very large binary data file (several hundred GB) which I would like to edit using Mathematica. The only edits required are a few isolated byte replacements—no insertions or deletions. Is it ...
3
votes
0answers
272 views

How to flush stdout stream?

I am aware that one can write to a terminal standard output like so: WriteString["stdout","Some stuff","\r"]; Note here that I am using a carriage return on ...
4
votes
1answer
225 views

How to read *all* input from stdin?

The answers given to this similar question won't do, because those solutions read only one line from stdin. I want to read all available lines, pretty much in the ...
4
votes
3answers
707 views

how do I to create a function similar to feof in C?

Using C language,I can use feof to test if the pointer at end of file. while(!feof(fp)) { //to do } But in the mma,I ...
2
votes
1answer
354 views

How to Terminate Read at EndOfFile

We are working on a project that reads in a stream, extracts information from it, and then needs to stop the calculation when it reaches EndOfFile. So for example,...
6
votes
1answer
74 views

Can Read[] apply a custom function automatically to the read values?

Read[stream, Number] will return a single read number. We can use a number of standard, documented types instead of Number, ...
11
votes
0answers
458 views

How to read and write data to open socket?

I am trying to open connection with socket. I am using this: ...