Timeline for Speeding up Import and Export in CSV format
Current License: CC BY-SA 3.0
18 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 6, 2016 at 16:05 | comment | added | RunnyKine |
@SPIL Thanks. Export is slower because it tries to do everything (Jack of all trade type thing). Surprisingly, in some cases for which it is optimized (like writing text files), it's actually hard to beat it's performance. I learned about this approach from experience and The Mathematica Book V5, which you can download for free.
|
|
| May 6, 2016 at 15:43 | comment | added | SPIL | Awesome solution - the Mathematica solution is great - so much faster. How did you find out about this? Why is the standard Mathematica export so much slower. | |
| Nov 5, 2013 at 15:42 | history | edited | RunnyKine | CC BY-SA 3.0 |
deleted 1 characters in body
|
| Nov 5, 2013 at 9:05 | comment | added | RunnyKine | @Richard. Glad I could help. | |
| Nov 5, 2013 at 9:03 | vote | accept | Richi W | ||
| Nov 5, 2013 at 9:03 | comment | added | Richi W | @RunnyKine Thanks for your solution. I prefer the pure Mathematica solution. It tremendously speeded up my routines. | |
| Nov 5, 2013 at 8:24 | history | edited | RunnyKine | CC BY-SA 3.0 |
deleted 25 characters in body
|
| Nov 5, 2013 at 5:52 | history | edited | RunnyKine | CC BY-SA 3.0 |
deleted 22 characters in body
|
| Nov 5, 2013 at 5:29 | history | edited | RunnyKine | CC BY-SA 3.0 |
added 3 characters in body
|
| Nov 4, 2013 at 22:10 | history | edited | RunnyKine | CC BY-SA 3.0 |
added 46 characters in body
|
| Nov 4, 2013 at 22:05 | comment | added | RunnyKine |
@LeonidShifrin. Ah okay, then it must be the \n RecordSeparator I forgot to add before.
|
|
| Nov 4, 2013 at 21:53 | comment | added | Leonid Shifrin | Yes, I also meant the numerics only. I did not mean the headers - I ignore them too. | |
| Nov 4, 2013 at 20:42 | comment | added | RunnyKine |
@LeonidShifrin. Thanks for checking it. I think the reason is I forgot to add the \n RecordSeparator. I've updated my code to include this and also include the first row (heading) of the .CSV file
|
|
| Nov 4, 2013 at 20:35 | history | edited | RunnyKine | CC BY-SA 3.0 |
added 625 characters in body
|
| Nov 4, 2013 at 20:11 | comment | added | Leonid Shifrin |
I did compare this to my method, calling your code as DeleteCases[readYourCSV["~/Downloads/returns_out.csv", 1419], {(Null | EndOfFile) ..}]. The strange thing was that your result was one row less (missed the very first one, it seems), and also that the results agreed only to 10^-3.
|
|
| Nov 4, 2013 at 19:09 | comment | added | Leonid Shifrin | But the large file contains many more columns (1419, to be precise). | |
| Nov 4, 2013 at 18:49 | comment | added | Leonid Shifrin |
This does not give the same result as Import["~/Downloads/returns_out_small.csv"], even if we compare purely numerical part.
|
|
| Nov 4, 2013 at 18:33 | history | answered | RunnyKine | CC BY-SA 3.0 |