How can this AbsoluteTime call be made faster?

Question

I need to convert tons of date strings into absolute times. This generates an example data set of 10,000 strings:

dateStrings=DateString[#,{"Year","-","Month","-","Day"," ","Hour",":","Minute",":","Second"}]&/@
              Range[3576302100-10000,3576302100];

I think the simplest way is

AbsoluteTime /@ dateStrings

and this gives exactly the result I want. But on lists of hundreds of thousands of dates, it is quite slow. On the example data set, it takes about 5 s. I can speed it up it a bit by telling AbsoluteTime what date format to expect:

AbsoluteTime[{#, {"Year", "-", "Month", "-", "Day", " ", 
                  "Hour", ":", "Minute", ":", "Second"}}
] & /@ dateStrings;

And this takes about 3.8 s.

How can the performance be improved further?

Answer 1

AbsoluteTime is faster on DateLists as compared to arbitrary DateStrings. Since the date format is known in this case, converting the strings to DateLists first gives a speed improvement of a factor of 10:

AbsoluteTime /@ (ToExpression /@ (StringSplit[#, {" ", "-", ":"}] & /@ dateStrings));

Not as fast as the java programming variant, but good enough for government work ;-)

Answer 2

You can get about 100 times faster by using Java, without any particular tuning, but you will have to provide the date format explicitly. Here is the solution based on Java reloader.

Implementation

1. Load the Java reloader

2. Compile the following class:

   JCompileLoad@
   "
     import java.text.ParseException;
     import java.text.SimpleDateFormat;
     import java.util.*;
   
     public class JAbsoluteTimeFinder{  
       public static double[] absTime(String[] dates,String format) throws ParseException{
        Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone(\"GMT\"));
        double[] result = new double[dates.length];
        int ctr = 0;
        SimpleDateFormat formatter = new SimpleDateFormat(format);
        for(String d: dates){
            Date dt = formatter.parse(d);    
            calendar.setTime(dt);            
            result[ctr++]= (long) (calendar.getTimeInMillis()/1000);
        }   
        return result;      
     }
   }";
   

3. Define the top-level Mathematica function as

   ClearAll[getJAbsoluteTime];
   getJAbsoluteTime[dates : {__String}, format_String] :=
      JAbsoluteTimeFinder`absTime[dates, format] 
         + AbsoluteTime[{1970, 1, 1, 0, 0, 0}] + 3600*$TimeZone;
   

Benchmarks

AbsoluteTime/@dateStrings;//AbsoluteTiming

(* {5.554688,Null} *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"];//AbsoluteTiming

(* {0.043945,Null}  *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"]==AbsoluteTime/@dateStrings

(* True *)

It may well be that one can get this still faster by tuning the solution or using a different Java library.