Timeline for Importing a large Excel file
Current License: CC BY-SA 3.0
32 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 22, 2017 at 14:03 | comment | added | Edmund | This issue was brought up in the Import and Export talk at WTC 2017 this week. The developer said he was working very hard to increase performance. | |
| Oct 20, 2017 at 12:40 | answer | added | C. K. Hackenberg | timeline score: 1 | |
| Aug 10, 2017 at 20:35 | answer | added | George Wolfe | timeline score: 2 | |
| Apr 11, 2016 at 15:33 | history | edited | J. M.'s missing motivation♦ |
edited tags
|
|
| Apr 11, 2016 at 15:17 | answer | added | halirutan | timeline score: 10 | |
| Dec 22, 2014 at 23:06 | history | edited | Mr.Wizard | CC BY-SA 3.0 |
added 21 characters in body
|
| Dec 22, 2014 at 22:57 | comment | added | Mr.Wizard | @ArgentoSapiens I merged another question into this one so that relevant answers are gathered in one place. I made a minor edit to your question to allow all answers to be applicable while still hopefully preserving your original intent. | |
| Dec 22, 2014 at 22:55 | history | edited | Mr.Wizard | CC BY-SA 3.0 |
adapt for merge
|
| Dec 22, 2014 at 22:54 | history | reopened | Mr.Wizard | ||
| Dec 22, 2014 at 22:53 | history | post merged (destination) | |||
| Dec 22, 2014 at 21:05 | history | closed |
Oleksandr R. Sjoerd C. de Vries Albert Retey Öskå bobthechemist |
Duplicate of Import Large Excel File | |
| Dec 22, 2014 at 18:15 | review | Close votes | |||
| Dec 22, 2014 at 21:05 | |||||
| Dec 22, 2014 at 12:31 | answer | added | CrustyNoodle | timeline score: 12 | |
| Feb 20, 2014 at 13:55 | comment | added | george2079 |
@HDF5 actually for speed alone .mx is slightly better. Of course there is a file size trade off the .gz compression makes the file a good bit smaller.
|
|
| Feb 20, 2014 at 6:48 | answer | added | RMMA | timeline score: 13 | |
| Feb 12, 2014 at 1:53 | comment | added | george2079 |
correcting myself - Import[.xlsm] is slow as molasses when it works. Gotta use AbsoluteTiming not Timing..
|
|
| Feb 11, 2014 at 18:59 | comment | added | george2079 | Can you actually Import that? In my experience Importing .xlsx is considerably faster than .csv -- when it works . The native excel import gets flaky with jave heap errors and such with large files though (while .csv import is more robust..) | |
| Feb 11, 2014 at 14:46 | comment | added | Ajasja |
I usually export to mx.gz (not that this format is not necessary compatible among versions and platforms, but it's an order of magnitude faster to work with). You might also take a look at HDF5.
|
|
| Feb 11, 2014 at 3:02 | comment | added | RunnyKine | See this related question Speeding up Import and Export in CSV Format, if that does not solve your problem you'll have to give a sample of your file. | |
| Apr 4, 2012 at 9:24 | answer | added | Albert Retey | timeline score: 7 | |
| Apr 3, 2012 at 14:32 | history | edited | ArgentoSapiens | CC BY-SA 3.0 |
Removed some rudeness.
|
| Apr 3, 2012 at 11:13 | comment | added | Lou | @kguler it worked for me on XLS's but not the large ones. Apparently just 4GB of main memory doesn't get in done. The XML.EXE process eats up all resources disk start swapping at the end freezes up my laptop. But nice workaround! +1 | |
| Apr 3, 2012 at 3:08 | history | tweeted | twitter.com/#!/StackMma/status/187013621149204480 | ||
| Apr 2, 2012 at 21:26 | history | edited | ArgentoSapiens | CC BY-SA 3.0 |
added 370 characters in body
|
| Apr 2, 2012 at 21:20 | comment | added | ArgentoSapiens |
@LeonidShifrin and kguler, I can directly Import the XML object inside the xlsx file. I'm looking for a way where I don't have to do a bunch of pre- and post-processing like this.
|
|
| Apr 2, 2012 at 20:02 | answer | added | F'x | timeline score: 16 | |
| Apr 2, 2012 at 19:43 | comment | added | kglr | Can you check if the method (import the appropriate xml object in the zipped xls/xlsx file) described in this answer works for you | |
| Apr 2, 2012 at 18:14 | comment | added | Lou | It seems this just doesn't work with large XLS or XLSX files no matter how large I define the Javaw to be (running Home edition). A fully loaded XLS with 65536 rows to IV collumns fails to load (and takes 10 minutes+) but a CSV loads in a couple of seconds. | |
| Apr 2, 2012 at 18:01 | comment | added | Leonid Shifrin |
I would suggest to try something similar to what I suggested in this answer, as another possible work-around. In the case of that discussion, this was vastly more efficient than Import both run-time and memory-wise, and actually made it possible to import the file. Of course, I was relying on CSV format specifics, so this may not work for your case directly.
|
|
| Apr 2, 2012 at 17:41 | comment | added | ArgentoSapiens | That works, but is an unacceptable workaround. | |
| Apr 2, 2012 at 17:37 | comment | added | s0rce | Can you save it as a CSV in excel and then try importing it? | |
| Apr 2, 2012 at 16:49 | history | asked | ArgentoSapiens | CC BY-SA 3.0 |