Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I need to convert tons of date strings into absolute times. This generates an example data set of 10,000 strings:

dateStrings=DateString[#,{"Year","-","Month","-","Day"," ","Hour",":","Minute",":","Second"}]&/@
              Range[3576302100-10000,3576302100];

I think the simplest way is

AbsoluteTime /@ dateStrings

and this gives exactly the result I want. But on lists of hundreds of thousands of dates, it is quite slow. On the example data set, it takes about 5 s. I can speed it up it a bit by telling AbsoluteTime what date format to expect:

AbsoluteTime[{#, {"Year", "-", "Month", "-", "Day", " ", 
                  "Hour", ":", "Minute", ":", "Second"}}
] & /@ dateStrings;

And this takes about 3.8 s.

How can the performance be improved further?

share|improve this question
    
Did you try to evaluate this function? Something like f[x_Integer] := Evaluate[DateString[ x, {"Year", "-", "Month", "-", "Day", " ", "Hour", ":", "Minute", ":", "Second"}]]; and f /@ Range[3576302100 - 10000, 3576302100]? For me this is about 4 times faster. But there is a hint that x can not be interpreted as a date or time input. Perhaps there is a way to define f in a way that this warning does not occur. –  chromate Apr 30 '13 at 20:48
    
@chromate, thanks, but you're performing the reverse of the desired conversion. The input data are strings. –  ArgentoSapiens Apr 30 '13 at 20:55
    
Ahh sorry, my fault. Unfortunately the same approach in the other direction is not faster than your solution. Perhaps somebody else knows how to speed things up in this case. –  chromate Apr 30 '13 at 21:13

2 Answers 2

up vote 4 down vote accepted

AbsoluteTime is faster on DateLists as compared to arbitrary DateStrings. Since the date format is known in this case, converting the strings to DateLists first gives a speed improvement of a factor of 10:

AbsoluteTime /@ (ToExpression /@ (StringSplit[#, {" ", "-", ":"}] & /@ dateStrings));

Not as fast as the java programming variant, but good enough for government work ;-)

share|improve this answer
1  
both StringSplit and ToExpression will work for a list of strings without the Map which isn't only slightly shorter but also a little faster than your version: AbsoluteTime /@ ToExpression[StringSplit[dateStrings, {" ", "-", ":"}]] –  Albert Retey May 5 '13 at 21:04

You can get about 100 times faster by using Java, without any particular tuning, but you will have to provide the date format explicitly. Here is the solution based on Java reloader.

Implementation

  1. Load the Java reloader

  2. Compile the following class:

    JCompileLoad@
    "
      import java.text.ParseException;
      import java.text.SimpleDateFormat;
      import java.util.*;
    
      public class JAbsoluteTimeFinder{  
        public static double[] absTime(String[] dates,String format) throws ParseException{
         Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone(\"GMT\"));
         double[] result = new double[dates.length];
         int ctr = 0;
         SimpleDateFormat formatter = new SimpleDateFormat(format);
         for(String d: dates){
             Date dt = formatter.parse(d);    
             calendar.setTime(dt);            
             result[ctr++]= (long) (calendar.getTimeInMillis()/1000);
         }   
         return result;      
      }
    }";
    
  3. Define the top-level Mathematica function as

    ClearAll[getJAbsoluteTime];
    getJAbsoluteTime[dates : {__String}, format_String] :=
       JAbsoluteTimeFinder`absTime[dates, format] 
          + AbsoluteTime[{1970, 1, 1, 0, 0, 0}] + 3600*$TimeZone;
    

Benchmarks

AbsoluteTime/@dateStrings;//AbsoluteTiming

(* {5.554688,Null} *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"];//AbsoluteTiming

(* {0.043945,Null}  *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"]==AbsoluteTime/@dateStrings

(* True *)

It may well be that one can get this still faster by tuning the solution or using a different Java library.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.