Questions tagged [streams]
A source of input or output: files and pipes are both examples of general Mathematica objects known as streams.
99
questions
0
votes
0
answers
55
views
How to plot field line with exact starting and ending position continuously
As the title suggested, I want to plot the electric field lines with exact starting and ending position continuously.
The code is like this:
...
- 55
0
votes
2
answers
99
views
Memory preserving computation in chunks
I would like to do the following:
Make a table
tab1={{i1,b1,c1,d1},{i1,b1,c1,d2},...,{i1,b1,c2,d1},...,{i1,b2,c1,d1},...{i2,b1,c1,d1},...}
for the given range imin < i < imax.
Make some ...
- 3,544
2
votes
1
answer
42
views
How to read doubles from specific location of a large file
I have a 1GB file in a proprietary format that is a mix of ASCII characters and doubles (floating point numbers). I need to read in a sequence of twelve doubles. They are in consecutive bytes of the ...
- 6,630
0
votes
1
answer
43
views
Errors with opening and closing file streams
I'm fairly new to Mathematica programming. I'm writing a small program that needs to handle text file streams to write output to and read input from them.
Although I'm closing the file streams at the ...
3
votes
0
answers
52
views
How can I save the output from AudioRecord?
I'm trying to save a recording using AudioRecord. My recording gets overwritten even if I save the output in a fresh cell. This illustrates the issue:
...
- 1,027
1
vote
0
answers
171
views
How to stream NSE option chain data into mathematica
I'm trying to stream option chain data into mathematica from the
url: https://www.nseindia.com/api/option-chain-indices?symbol=BANKNIFTY
...
- 91
3
votes
1
answer
91
views
How to get StreamPlots where the density of arrows adapted (discretization ) adapts to the presence of funnels, saddle-points, etc?
I want to draw a phase portrait where the density of arrows adapted (discretization ) adapts to the presence of funnels, saddle points, and other phenomena. For example, in the code attached, the aim ...
- 123
1
vote
2
answers
104
views
How to wait for new input from file InputStream
I'm trying to open a file to read and do some processing indefinitely until I decide to abort:
...
- 618
1
vote
0
answers
34
views
Combine two streamplots with different position dependent color functions
Building from replies to other posts I have come up with a combination of two stream plots which has
a position dependent color function for stream lines which the same for the two plots
a way to &...
- 404
0
votes
1
answer
23
views
How can I make the vectors (vector heads) larger in a StreamPlot?
In my StreamPlots, the vectors can hardly be seen, in particular, since the heads of the vectors are so small.
Can I change this?
Example:
...
- 193
2
votes
1
answer
127
views
Phase portrait on the sphere
I have that $(x,y)\in S^1$, i.e. on the sphere, and in Mathematica, the ODE system
...
- 193
2
votes
2
answers
120
views
Converting Strings to Reals in a List
I am new to Mathematica and working with streamed data:
...
- 89
0
votes
1
answer
56
views
How to do computations with a specific column of a stream?
I'm trying to calculate the mean of a specific column of a huge dataset (given as .csv-file) using a stream in mathematica.
I tried the following:
...
- 45
3
votes
2
answers
136
views
How do I import a 1 gig text file by only importing the last 3 lines of every 10 line chunk?
I am trying to get the last three lines of every 10 line chunk into mathematica as a data variable. The text file has 3 columns. For example:
...
- 65
2
votes
1
answer
55
views
1
vote
1
answer
84
views
write output to stream and also print it
I tried to save the Print[] output to a file using this code:
...
- 121
2
votes
0
answers
65
views
4
votes
0
answers
30
views
Is there an equivalent to "Open Update"
I'm writing a binary file that contains a header followed by the data.
The header contains the length of the data.
The data is written first, skipping past the header.
Once done, I would like to ...
- 41
3
votes
0
answers
125
views
Reading through file one line at a time, running out of RAM [closed]
I have a handful of files (some of which are very large), and I would like to loop through them one line at a time, run a calculation, and then write the result of the calculation to a new output file:...
- 31
1
vote
0
answers
51
views
Issue with StreamDensityPlot
I'm plotting a complicated vector field that is yielding the following particular StreamDensityPlot. I'm using the following command to plot it:
...
- 499
6
votes
1
answer
175
views
How do I reliably read from stdin using wolframscript?
I am trying to direct a stream of data to a wls script in a command line environment. Wolframscript provides the ...
- 318
2
votes
1
answer
212
views
Vector field Stream?
The following graph shows the vector plot. How can I plot the solid lines which represents how the vectors are spatially varying and both on the same graph (vector plot and stream field)?
...
- 177
10
votes
1
answer
4k
views
5
votes
0
answers
146
views
Optimize Use of PeakDetect in Mathematica
I am quite new to Mathematica and I am looking for some advice in order to improve and optimize my code.
Here is what I am trying to do:
Open a stream
Read the stream line by line and store it in "...
- 103
1
vote
2
answers
163
views
Reading CSV data from a stream [duplicate]
I'm trying to figure out how many rows/records my csv file has, which I have to stream because it's quite large.
Can someone tell me if my approach works and how I can get it to loop through the ...
2
votes
0
answers
231
views
How to force Close of opened stream when stream write previously failed (bug in opening stream twice)?
I opened a stream like:
str = OpenWrite[NotebookDirectory[] <> "myStream.tsv"];
and made an error when writing the first line (meant to be the tsv headers):...
- 395
10
votes
2
answers
141
views
What is the fastest way to read the new lines in the dynamically updatable file?
I'm writing a log to a simple .txt file and want to read the last few records using Mathematica. But there are some problems.
f = OpenRead["logs.txt"];
Skip[f, ConstantArray[Number, 2], 1];
Read[f, ...
- 9,086
2
votes
1
answer
161
views
Streams in parallel kernels. Writing in multiple files in ParallelDo loop in Mathematica
I need to write data in multiple files in ParallelDo loop:
...
2
votes
2
answers
60
views
Naming indexed output streams
I need to write into multiple files in a Do loop.
Hence, I tried to open multiple streams:
...
3
votes
2
answers
201
views
Join two datasets with a function similar to Vlookup [Big Dataset and Efficiency issue] - Wizards are welcome!
I am working on a huge dataset and I have a lot of data formatted in XML, we are talking about a couple of GBs. For this reason I am currently using the stream function to read them and extract the ...
- 169
2
votes
0
answers
76
views
Why does ReadString complain about the stream not being opened in binary format?
This works fine:
...
- 224k
12
votes
0
answers
143
views
LibraryLink: How to read from a Mathematica stream in C?
Mathematica has a flexible streams interface. Functions like Read can read from a "stream" that may be backed by a file, pipe, string, network resource, etc. We can ...
- 224k
3
votes
1
answer
323
views
How many records? In a file (streaming)
I want to know how many records a file (large data) contains...
with streaming...like this:
...
- 31
7
votes
0
answers
549
views
Communicating with server over Websocket using SocketConnect
I am trying to connect to a web socket feed of Gdax. The documentation is here, https://docs.gdax.com/#websocket-feed. There is no account needed and the information is public. I have successfully ...
- 1,269
0
votes
0
answers
99
views
Is it possible to parallelize reading in Mathematica?
I have a gigantic data file that needs to be processed. I am reading it in chunks using ReadList with an InputStream. In my ...
- 291
3
votes
2
answers
97
views
Explicitly control newline behaviour when writing strings with arbitary encoding
On Windows system, Mathematica by default opens stream which will automatically transform any "\n" into "\r\n". But I want pure ...
- 15.6k
20
votes
1
answer
391
views
Is building an event-based parser possible/reasonable in Mathematica (WL)?
Too long? Go to Problems section
Background
I like to learn by doing small, sometimes artificial, projects. Among other things I had 'parsers', and 'stream methods / streams in MMA' on my to learn ...
Kuba♦
- 132k
5
votes
1
answer
473
views
Matrix Vector multiplication for very big matrices (>>RAM)
Problem
Suppose a matrix-vector product of the form $M\vec{v}$ should be calculated where the amount of storage needed to store $M$ is noticeably larger than the available RAM on a machine. What is ...
- 822
1
vote
0
answers
34
views
How to export result from SQLResultSet into a csv
I have a sql query which produces very large data from a remote server but the data is too large to fit into memory all at once. So i want to read and export it sequentially into a csv. I am using the ...
- 8,105
1
vote
1
answer
38
views
Read not advancing StreamPosition [closed]
[MMA 11.0, Win7 64bit]
I have a number of CVS files of ~1-2GB, millions of rows, so I want to read them in records, chunks building on Albert Retey's answer to this question
I assumed, and the ...
- 2,410
5
votes
0
answers
63
views
Why ReadList ignores NullRecords for Number?
When thinking on this recent question the immediately obvious solution which came to my mind was to use ReadList with options ...
- 55.7k
6
votes
1
answer
213
views
Is there a way to redirect the output from `Echo`?
The newish function Echo is tremendously useful in quick-and-dirty debugging, since you can stick it pretty much anywhere in an expression and get feedback about ...
- 17.8k
3
votes
0
answers
37
views
Unexpected behaviour with WordSeparators and Read [duplicate]
I am confused about how Read separates elements.
The following all work fine:
...
- 224k
4
votes
1
answer
65
views
How to get the standard error stream for a ProcessObject?
When you run an external program using RunProcess, the program will run to completion and you will receive an association containing the exit code, stuff that went ...
- 17.8k
6
votes
1
answer
209
views
wolframscript creates tons of log files
On Linux, wolframscript seems to create log files in .Mathematica/ApplicationData/ProcessLink/Streams
for example:
...
- 1,143
1
vote
0
answers
107
views
Why does using BinaryWrite make my file bigger? [closed]
I have a binary file 00010005.dat. I want to normalize the matrix so I use the following code:
...
- 6,656
7
votes
1
answer
1k
views
SliceStreamPlot3D or StreamPlot3D
I know that neither of the two functions listed in the title exist, but I basically replaced vector for stream to give you an idea of what I want. The most ideal would be ...
- 533
3
votes
0
answers
1k
views
How to visualize the vorticity of a 3D velocity field? [closed]
I sample a snapshot from a system with 50 particles including their position and instant velocities.
The snapshot is a list of 3 position and 3 velocities values per particle. i.e. 50 lines each of ...
- 519
9
votes
2
answers
293
views
Is there a way to create ZIP OutputStream ?
I want to write data to a file using compression. Is it possible to do that just in time (like in java ZipOutputStream). E.g. what I want:
...
27
votes
3
answers
651
views
Can Mathematica edit large binary files in-place?
I have a very large binary data file (several hundred GB) which I would like to edit using Mathematica. The only edits required are a few isolated byte replacements—no insertions or deletions. Is it ...
- 2,287