How to Import Apple emlx files into Mathematica?

Question

A few OS X versions ago, Apple changed the storage format of their e-mail client Mail.app from a single, standards based, mbox format to a proprietary format using a message per file approach.

Mathematica, as a result, cannot import these files except as lists, strings, or html depending on the contents of the message itself. This also means that some examples of personal analytics are not "out of the box" compatible with OS X.

How does one read a file with the "emlx" extension into Mathematica so that they can be used in the various examples of Personal Analytics with a minimum of hassle?

Answer 1

The following function will read an emlx file into Mathematica in a format compatible with examples that assume an EML format.

readEMLX[filename_String] := Module[ {instream},
    ReadLine[instream = OpenRead[filename]];
    Import[instream, {"EML", "FullMessageElements"}]
]

The emlx file format consists of a right space padded character representation of message length, the rfc822 message body immediately following (beginning on 'line 2'). At the end of the rfc822 message body and before EOF, is an Apple xml property list. The latter contains information used by Mail.app including read status and other 'interesting' information. If this information is necessary, it can be obtained by the following:

readEMLXXML[filename_String] := Module [ {instream
 , sentinelString = "\"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">"},
 Find[instream = OpenRead[filename], sentinelString];
 Import[instream, "XML"]
]

This xml property list may be further processed using the approach discussed HERE.

It is important to note that if you wish to process the entirety of the plist XML embedded within the file, you must change the sentinelstring and pass ReadDTD-> "False" to Import . Unless this option in set, Mathematica fails when it encounters the DTD.

Also, within the property list is a field called flags which encodes the various statuses (read, answered, junk) as well as some numeric information. You can find more information about this as well as other parts of the emlx format from HERE.

As an aside, If someone writes an efficient/elegant/concise way to decode the flags, please post it here. I do have code to do this, yet its extremely ugly.

As I learn more about Mathematica and how to use it to integrate with Mail.app, I shall post it here.