4
$\begingroup$

I'm trying to import CSV file with the following format:

20150803 00:00:01.649,1.09675,1.09678

using this code:

Import[filename, 
  "DateStringFormat" -> {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}]

But this does not import Time part - it only imports the Date:

{{DateObject[{2015, 8, 4}], 1.09675, 1.09682}, 
 {DateObject[{2015, 8, 3}], 1.09675, 1.09678},
 ...

How can I import all of the information from my file?

$\endgroup$

3 Answers 3

2
$\begingroup$

Import does not support the time portion of DateStringFormat. (This is a deficiency in the docs if not a bug ) If you know the specific column(s) where your date appears you can map datelist like this:

MapAt[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
 "Minute", ":", "Second", ".", "Millisecond"}}] & , 
      Import[filename.csv], {All, 1}]

In case you do not know where the strings might occur you can do this:

Map[Quiet@
  Check[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}}], #] & , 
       Import[filename.csv], {-1}]

I would guess this will really kill performance on a large file though.

$\endgroup$
0
$\begingroup$

I found solution at this answer:

And also improved loading speed of my 250Mb CSV file

$\endgroup$
3
  • $\begingroup$ 250 Gb, would that mean Giga-Bit or was it a type and would read like GB (Giga-Byte) or are it rather Mega-Bytes really? I would wonder on what machine Mathematica would be able to handle a 250 Gb (30GB ?) CSV file :-) $\endgroup$ Sep 17, 2015 at 14:31
  • $\begingroup$ Oh, I'm mistaked - my CSV file 250Mb, not 250Gb :) $\endgroup$ Sep 19, 2015 at 11:23
  • $\begingroup$ OK, that sounds somewhat less demanding :-) $\endgroup$ Sep 19, 2015 at 13:11
0
$\begingroup$
in = StringToStream[StringJoin[
    "20150803 00:00:01.649,1.09675,1.09678\r\n", 
    "20170930 23:59:59.649,1.19675,1.29678\r\n"]]
    
file = (StringSplit[#1, ","] & ) /@ 
   StringSplit[ReadString[in], RegularExpression[
     "[\r\n]+"]]
     
Close[in]

getNumber = Interpreter["Number"]

data = ({FromDateString[#1[[1]]], 
     getNumber[#1[[2]]], getNumber[
      #1[[3]]]} & ) /@ file
InputStream[Name: String
Unique ID: 3

]
{{20150803 00:00:01.649,1.09675,1.09678},{20170930 23:59:59.649,1.19675,1.29678}}
String
Interpreter[Number]
{{Mon 3 Aug 2015 00:00:01GMT-4,1.09675,1.09678},{Sat 30 Sep 2017 23:59:59GMT-4,1.19675,1.29678}}

The first cell would be replaced normally by an OpenRead to the CSV file.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.