# How can this AbsoluteTime call be made faster?

I need to convert tons of date strings into absolute times. This generates an example data set of 10,000 strings:

dateStrings=DateString[#,{"Year","-","Month","-","Day"," ","Hour",":","Minute",":","Second"}]&/@
Range[3576302100-10000,3576302100];


I think the simplest way is

AbsoluteTime /@ dateStrings


and this gives exactly the result I want. But on lists of hundreds of thousands of dates, it is quite slow. On the example data set, it takes about 5 s. I can speed it up it a bit by telling AbsoluteTime what date format to expect:

AbsoluteTime[{#, {"Year", "-", "Month", "-", "Day", " ",
"Hour", ":", "Minute", ":", "Second"}}
] & /@ dateStrings;


And this takes about 3.8 s.

How can the performance be improved further?

-
Did you try to evaluate this function? Something like f[x_Integer] := Evaluate[DateString[ x, {"Year", "-", "Month", "-", "Day", " ", "Hour", ":", "Minute", ":", "Second"}]]; and f /@ Range[3576302100 - 10000, 3576302100]? For me this is about 4 times faster. But there is a hint that x can not be interpreted as a date or time input. Perhaps there is a way to define f in a way that this warning does not occur. –  chromate Apr 30 '13 at 20:48
@chromate, thanks, but you're performing the reverse of the desired conversion. The input data are strings. –  ArgentoSapiens Apr 30 '13 at 20:55
Ahh sorry, my fault. Unfortunately the same approach in the other direction is not faster than your solution. Perhaps somebody else knows how to speed things up in this case. –  chromate Apr 30 '13 at 21:13

AbsoluteTime is faster on DateLists as compared to arbitrary DateStrings. Since the date format is known in this case, converting the strings to DateLists first gives a speed improvement of a factor of 10:

AbsoluteTime /@ (ToExpression /@ (StringSplit[#, {" ", "-", ":"}] & /@ dateStrings));


Not as fast as the java programming variant, but good enough for government work ;-)

-
both StringSplit and ToExpression will work for a list of strings without the Map which isn't only slightly shorter but also a little faster than your version: AbsoluteTime /@ ToExpression[StringSplit[dateStrings, {" ", "-", ":"}]] –  Albert Retey May 5 '13 at 21:04

You can get about 100 times faster by using Java, without any particular tuning, but you will have to provide the date format explicitly. Here is the solution based on Java reloader.

### Implementation

2. Compile the following class:

JCompileLoad@
"
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.*;

public class JAbsoluteTimeFinder{
public static double[] absTime(String[] dates,String format) throws ParseException{
Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone(\"GMT\"));
double[] result = new double[dates.length];
int ctr = 0;
SimpleDateFormat formatter = new SimpleDateFormat(format);
for(String d: dates){
Date dt = formatter.parse(d);
calendar.setTime(dt);
result[ctr++]= (long) (calendar.getTimeInMillis()/1000);
}
return result;
}
}";

3. Define the top-level Mathematica function as

ClearAll[getJAbsoluteTime];
getJAbsoluteTime[dates : {__String}, format_String] :=
JAbsoluteTimeFinderabsTime[dates, format]
+ AbsoluteTime[{1970, 1, 1, 0, 0, 0}] + 3600*\$TimeZone;


### Benchmarks

AbsoluteTime/@dateStrings;//AbsoluteTiming

(* {5.554688,Null} *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"];//AbsoluteTiming

(* {0.043945,Null}  *)

getJAbsoluteTime[dateStrings,"yyyy-MM-dd HH:mm:ss"]==AbsoluteTime/@dateStrings

(* True *)
`

It may well be that one can get this still faster by tuning the solution or using a different Java library.

-