4
$\begingroup$

I need to import data from various database sources. At the moment I plan on importing files via a CSV file or XLS. All the data if in XLS would be on one sheet (notebook). Are there any advantages of using one format over the other? Thanks. David

$\endgroup$
  • 1
    $\begingroup$ It's probably good to know about this. When the importer tries to be too smart, it can work against you. $\endgroup$ – Szabolcs Nov 14 '14 at 15:37
  • $\begingroup$ Very helpful. That is a gotcha that could take a long time to figure out! $\endgroup$ – David Kerr Nov 14 '14 at 20:13
7
$\begingroup$

Both have advantages and disadvantages.

Importing XLS is done using a Java-based application. In my experience, it is terribly slow and eats up enormous amounts of memory to the extent that it blows up your Java heap space, requiring a manual adjustment of this heap space (quite often to no avail). It happens to me all the time that a data structure that I know should fit in memory cannot be imported along the XLS route just because of these memory issues. It does, however, have conveniences such as the ability to read and automatically convert dates and times, select indivudal cells for import, and you can read both formulas and their results.

Importing CSV files with Import can be much faster and takes less time and memory, although I often resort to reading line by line or as a block using ReadList and do my own conversions when needed.

$\endgroup$
  • $\begingroup$ Those names are single fields according to RFC 4180: ietf.org/rfc/rfc4180.txt $\endgroup$ – Alan Nov 14 '14 at 16:16
  • $\begingroup$ I think the point you just removed is quite important. Yes, RFC4180 is there as a recommendation, but in practice there's no single standard that CSV readers/writers (or people piecing together CSV files) follow, which causes all kinds of trouble. $\endgroup$ – Szabolcs Nov 14 '14 at 16:23
  • $\begingroup$ Again, very helpful. Thank you. $\endgroup$ – David Kerr Nov 14 '14 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.