DateList coupled with String Operations

Question

Through some webscraping, I have the following list:

dates={{"<h3>January 18, 2001</h3>"}, {"<h3>February 1, 2001</h3>"},
{"<h3>February 2, 2001</h3>"}, {"<h3>February 24, 2001</h3>"},
{"<h3>February 26, 2001</h3>"}, {"<h3>March 1, 2001</h3>"},
{"<h3>March 2, 2001</h3>"}, {"<h3>March 31, 2001</h3>"}, {"<h3>April
5, 2001</h3>"}, {"<h3>April 18, 2001</h3>"}, {"<h3>May 6, 2001</h3>"}};

I am trying to make this into a DateList[] list. I can make it human readable in the following fashion:

datelist = StringReplace[
  ToString[dates], "<" ~~ Except[">"] .. ~~ ">" -> ""]

{{January 18, 2001}, {February 1, 2001}, {February 2, 2001}, \ {February 24, 2001}, {February 26, 2001}, {March 1, 2001}, {March 2, \ 2001}, {March 31, 2001}, {April 5, 2001}, {April 18, 2001}, {May 6, \ 2001}

This brings up a list of nice dates. The only problem is that they are now a string. The list operations that I have tried to use (Cases, Pick, etc.) all deal with the whole part of a list, rather than sub-sets of each part.

Is there a quick way to convert what I have in dates to a Mathematica-readable DateList[]?

Answer 1

The format specification for DateList is pretty flexible. Since we know that we have <h3> tags wrapped around things, we can just account for them:

DateList[{#, {"<h3>", "MonthName", "Day", ",", "Year", "</h3>"}}] & /@ Flatten[dates]

(*
==> {{2001, 1, 18, 0, 0, 0.}, {2001, 2, 1, 0, 0, 0.}, {2001, 2, 2, 0, 0, 0.},
     {2001, 2, 24, 0, 0, 0.}, {2001, 2, 26, 0, 0, 0.}, {2001, 3, 1, 0, 0, 0.}, 
     {2001, 3, 2, 0, 0, 0.}, {2001, 3, 31, 0, 0, 0.}, {2001, 4, 5, 0, 0, 0.}, 
     {2001, 4, 18, 0, 0, 0.}, {2001, 5, 6, 0, 0, 0.}}
*)

Answer 2

First one has to clean the starting list and replace the "\n" character with a space (if, as suggested by R.M, the extra \n came in from pasting the code then the StringReplace bit is not necessary).

DateList[{StringSplit[#, {"<h3>", "</h3>"}][[1]][[1]], {"MonthName", " ", "Day", ", ", 
 "Year"}}] & /@ (StringReplace[#, {"\n" -> " "}] & /@ dates)

Answer 3

One can extend and modify your code a little to get this:

datelist = 
 DateList@StringReplace[ToString[#], 
     "<" ~~ Except[">"] .. ~~ ">" -> ""] & /@ Flatten[dates]

(*
==> {{2001, 1, 18, 0, 0, 0.}, {2001, 2, 1, 0, 0, 0.}, {2001, 2, 
  2, 0, 0, 0.}, {2001, 2, 24, 0, 0, 0.}, {2001, 2, 26, 0, 0, 
  0.}, {2001, 3, 1, 0, 0, 0.}, {2001, 3, 2, 0, 0, 0.}, {2001, 3, 31, 
  0, 0, 0.}, {2001, 4, 5, 0, 0, 0.}, {2001, 4, 18, 0, 0, 0.}, {2001, 
  5, 6, 0, 0, 0.}}
*)

Answer 4

You only need a slight modification to your StringReplace to get rid of the HTML tags:

datestr = StringReplace[#, "<h3>" | "</h3>" :> ""] & /@ dates
(* {{"January 18, 2001"}, {"February 1, 2001"}, {"February 2, 2001"}, {"February 24, 2001"}, 
    {"February 26, 2001"}, {"March 1, 2001"}, {"March 2, 2001"}, {"March 31, 2001"}, 
    {"April 5, 2001"}, {"April 18, 2001"}, {"May 6, 2001"}} *)

This result can be interpreted as a valid date string and converted to a DateList as follows:

DateList[{#, {"MonthName", "Day", "Year"}}] & @@@ datestr
(* {{2001, 1, 18, 0, 0, 0.}, {2001, 2, 1, 0, 0, 0.}, {2001, 2, 2, 0, 0, 0.}, 
    {2001, 2, 24, 0, 0, 0.}, {2001, 2, 26, 0, 0, 0.}, {2001, 3, 1, 0, 0, 0.}, 
    {2001, 3, 2, 0, 0, 0.}, {2001, 3, 31, 0, 0, 0.}, {2001, 4, 5, 0, 0, 0.}, 
    {2001, 4, 18, 0, 0, 0.}, {2001, 5, 6, 0, 0, 0.}} *)

Answer 5

Flatten[DateList /@ StringReplace[#, {"<h3>" -> "", "</h3>" -> ""}] & /@ dates, 1]

Answer 6

This is a separate answer as it is not a modification of your original attempt. It uses StringCases, and manually constructs a valid DateList. Since DateList was added in v6, this should also work for older versions. Note that one can replace the months in my With block with a manual list of 12 names.

With[{months = System`DateStringDump`getCachedMonths@""},
    (StringCases[#, "<h3>" ~~ m : LetterCharacter .. ~~ " " ~~ d : DigitCharacter .. ~~ 
        ", " ~~ y : DigitCharacter .. ~~ "</h3>" :> 
        {ToExpression@y, Position[months, m][[1, 1]], ToExpression@d, 0, 0, 0}
        ] & /@ dates
    )~Flatten~2
]

(* {{2001, 1, 18, 0, 0, 0.}, {2001, 2, 1, 0, 0, 0.}, {2001, 2, 2, 0, 0, 0.}, 
    {2001, 2, 24, 0, 0, 0.}, {2001, 2, 26, 0, 0, 0.}, {2001, 3, 1, 0, 0, 0.}, 
    {2001, 3, 2, 0, 0, 0.}, {2001, 3, 31, 0, 0, 0.}, {2001, 4, 5, 0, 0, 0.}, 
    {2001, 4, 18, 0, 0, 0.}, {2001, 5, 6, 0, 0, 0.}} *)

Answer 7

(DateList@StringTrim[First@#, "<h3>" | "</h3>"]) & /@ dates

or

DateList /@ StringTake[First /@ dates, {5, -6}]

or

(DateList @@ StringCases[First@#, "<h3>" ~~ a__ ~~ "</h3>" :> a]) & /@ dates

or

StringCases[First /@ dates, "<h3>" ~~ a__ ~~ "</h3>" :> DateList[a]]

or

Sequence @@@ StringReplace[First /@ dates, "<h3>" ~~ a__ ~~ "</h3>" :> DateList[a]]

all give

(* {{2001, 1, 18, 0, 0, 0.}, {2001, 2, 1, 0, 0, 0.}, {2001, 2, 2, 0, 0, 0.}, 
{2001, 2, 24, 0, 0, 0.}, {2001, 2, 26, 0, 0, 0.}, {2001, 3, 1, 0, 0, 0.}, 
{2001, 3, 2, 0, 0, 0.}, {2001, 3, 31, 0, 0, 0.}, {2001, 4, 5, 0, 0, 0.}, 
{2001, 4, 18, 0, 0, 0.}, {2001, 5, 6, 0, 0, 0.}} *)