2
$\begingroup$

I have a large data file (.txt) with bivariate data in the following form

x1,y1
x2,y2
x3,y3
.
.
.
xn,yn
---------F 0
x1, y1
x2, y2
.
.
.
xn, yn
--------F 1
.
.
.

and so on.

What I want to do is import the X, Y data into Mathematica up to the first "------- F " and plot a Histogram3D. This, I would repeat until the end of the file, each time plotting a Histogram3D. In the end, I would like to make an Animation of the Histogram3Ds that I plotted.

How should I go about this?

I do 

data = Import[
  "C:\\Users\\Debajyoti \
Sengupta\\Documents\\Processing\\collide_pde\\spatial_density.txt", \
{"Data", Range[10]}]
Histogram3D[data,Automatic]

to import and plot a Histogram3D for the first 10 data points. I understand that I need to do a loop somehow and store the Histogram3Ds somehow, but I am not sure how to do that.

The files are really large (~2 GB). Any ideas about speeding up import are welcome.

Thanks

Improve this question
$\endgroup$
4
  • $\begingroup$ Hi @Debajyoti and welcome. A minimal data file for doing a working example would be nice for helping you... $\endgroup$
    – José Antonio Díaz Navas
    Feb 16, 2019 at 11:35
  • $\begingroup$ I can't find the option to upload the data file here. Here's a pastebin.com/Nrk7SeTi pastebin link containing the data. $\endgroup$
    – Debajyoti Sengupta
    Feb 16, 2019 at 11:55
  • $\begingroup$ It seems to me the easiest way is to use ReadLine[] and loop through all the lines of the input file. $\endgroup$
    – Somos
    Feb 16, 2019 at 15:27
  • $\begingroup$ I have to save n entries to the data variable and keep doing till EOF. I will also need to skip the line where it goes ------ F. $\endgroup$
    – Debajyoti Sengupta
    Feb 16, 2019 at 16:00

1 Answer 1

Reset to default
1
$\begingroup$

Import the raw data from your pastebin as comma-separated values:

list = Import["https://pastebin.com/raw/Nrk7SeTi", "CSV"];

This will give a list of lists with the following structure:

{{290.73, 62.61}, {71.91, 122.72}, {548.1, 326.38}, {331.88, 358.37}, {109.96, 74.38}, 
 {179.66, 364.96}, {221.43, 166.61}, {177.08, 188.76}, {331.75, 207.65}, {247.19, 277.06}, 
 {"------------ F 0"}, {291.21, 62.95}, {71.42, 122.6}, {548.02, 325.96}, {332.3, 358.63}, 
 {109.88, 73.98}, {180.1, 365.03}, {221.55, 166.49}, {177.15, 188.68}, {331.88, 207.85}, 
 {247.21, 277.53}, {"------------ F 1"}, 
 ...

You can exploit the fact that these are lists of pairs (i.e. elements with Length = 2), separated by a list with a single element in it, the separator string, i.e. an element with Length = 1. You can use SplitBy[list, Length] to get separated lists. Then you have to remove the separators, which are still lists with one element, so you can use Select with a length criterion: 

split = Select[SplitBy[list, Length], Length[#] != 1 &];

Finally, generate BarCharts from the pairs of values, and animate the list of plots using ListAnimate:

ListAnimate[BarChart /@ split]

Animated bar charts

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hello @MacroB, Thank you for your reply. The following code works perfectly. But what I really wanted to plot was a Histogram3D (or a SmoothDensityHistogram) so I do - ListAnimate[SmoothDensityHistogram /@ split]. It produces the desired output, except that it keeps readjusting the axes scale every now and then. Is there a way to fix the axes scale? Say Y range - 0 : 700, X range : 0 - 600? $\endgroup$
    – Debajyoti Sengupta
    Feb 17, 2019 at 10:08
  • $\begingroup$ Also, Is there a way I can export the ListAnimate to a gif or a mov? I tried Export["evo.mov",%] but it seems to take forever $\endgroup$
    – Debajyoti Sengupta
    Feb 17, 2019 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.