4
$\begingroup$

I'm trying to import CSV file with the following format:

20150803 00:00:01.649,1.09675,1.09678

using this code:

Import[filename, 
  "DateStringFormat" -> {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}]

But this does not import Time part - it only imports the Date:

{{DateObject[{2015, 8, 4}], 1.09675, 1.09682}, 
 {DateObject[{2015, 8, 3}], 1.09675, 1.09678},
 ...

How can I import all of the information from my file?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
2
$\begingroup$

Import does not support the time portion of DateStringFormat. (This is a deficiency in the docs if not a bug ) If you know the specific column(s) where your date appears you can map datelist like this:

MapAt[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
 "Minute", ":", "Second", ".", "Millisecond"}}] & , 
      Import[filename.csv], {All, 1}]

In case you do not know where the strings might occur you can do this:

Map[Quiet@
  Check[DateList[{#, {"Year", "Month", "Day", " ", "Hour", ":", 
    "Minute", ":", "Second", ".", "Millisecond"}}], #] & , 
       Import[filename.csv], {-1}]

I would guess this will really kill performance on a large file though.

Improve this answer
$\endgroup$
0
$\begingroup$

I found solution at this answer:

And also improved loading speed of my 250Mb CSV file

Improve this answer
$\endgroup$
3
  • $\begingroup$ 250 Gb, would that mean Giga-Bit or was it a type and would read like GB (Giga-Byte) or are it rather Mega-Bytes really? I would wonder on what machine Mathematica would be able to handle a 250 Gb (30GB ?) CSV file :-) $\endgroup$
    – Albert Retey
    Sep 17, 2015 at 14:31
  • $\begingroup$ Oh, I'm mistaked - my CSV file 250Mb, not 250Gb :) $\endgroup$
    – Slava Bogdanov
    Sep 19, 2015 at 11:23
  • $\begingroup$ OK, that sounds somewhat less demanding :-) $\endgroup$
    – Albert Retey
    Sep 19, 2015 at 13:11
0
$\begingroup$ 
in = StringToStream[StringJoin[
    "20150803 00:00:01.649,1.09675,1.09678\r\n", 
    "20170930 23:59:59.649,1.19675,1.29678\r\n"]]
    
file = (StringSplit[#1, ","] & ) /@ 
   StringSplit[ReadString[in], RegularExpression[
     "[\r\n]+"]]
     
Close[in]

getNumber = Interpreter["Number"]

data = ({FromDateString[#1[[1]]], 
     getNumber[#1[[2]]], getNumber[
      #1[[3]]]} & ) /@ file

InputStream[Name: String
Unique ID: 3

]
{{20150803 00:00:01.649,1.09675,1.09678},{20170930 23:59:59.649,1.19675,1.29678}}
String
Interpreter[Number]
{{Mon 3 Aug 2015 00:00:01GMT-4,1.09675,1.09678},{Sat 30 Sep 2017 23:59:59GMT-4,1.19675,1.29678}}

The first cell would be replaced normally by an OpenRead to the CSV file.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.