9
$\begingroup$

I have several .eml email files to import. Some of them were saved as quoted-printable 7-bit encoded.

I skipped through the Mathematica documentation and found ToCharacterCode and CharacterEncoding. However, 

Import[foo.eml, CharacterEncoding -> "PrintableASCII"]

does not lead to conversion of the 7-bit characters to the default set (8-bit I guess).

In particular, I want to get rid of encodings like =FC, which is a German Umlaut-u (ü). After that I would like to be able to proceed with normal text analysis. Does anybody have experience with the right functions?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
11
$\begingroup$

I have had some success with the following function that I wrote a couple of years ago to battle quoted printables on MathGroup:

translateQuotedPrintable[str_String] := 
  StringReplace[str, {"=" ~~ c1:HexadecimalCharacter~~c2:HexadecimalCharacter :> 
    FromCharacterCode[FromDigits[c1 <> c2, 16], "Math1"],"=" ~~ EndOfLine -> ""}]

translateQuotedPrintable["=FCberhaupt"]

"überhaupt"

More about this in my original Stackoverflow posting.

The "Math1" argument of FromCharacterCode could perhaps be dropped. I used it in the context of the MathGroup where a lot of Mathematica symbols got quoted printabled. For normal text, the default translation should be fine.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Great solution. What a pity that the Import[] does not auto. convert the characters. Other languages have their function for this:-( -- btw: works with ease. $\endgroup$
    – Frank
    May 26, 2013 at 16:25
  • $\begingroup$ Yesterday I found out, that the German a-Umlaut is translated to an imaginary part double-i. I removed the parameter "Math1" and obtained the correct result. Moreover, in my case the equal sign at the end of the line introduced a line wrap at column 75. Thus I changed the 2nd rule to "=\n" -> "" $\endgroup$
    – Frank
    May 28, 2013 at 8:27
  • 1
    $\begingroup$ A shorter replacement equivalent to the original is: StringReplace[str, {"=" ~~ hex : Repeated[HexadecimalCharacter, {2}] :> FromCharacterCode[FromDigits[hex, 16], "Math1"], "=" ~~ EndOfLine -> ""}] $\endgroup$
    – Mr.Wizard
    May 29, 2013 at 5:15
  • $\begingroup$ @Mr.Wizard & Sjoerd, online converters are giving an empty string for =00 while this gives \\.00 which looks like a whitespace. What have I missed? $\endgroup$
    – Kuba ♦
    Mar 1, 2016 at 15:36
  • $\begingroup$ @kuba =00 is the NULL character, which in many languages is used to delimit a character string. Whether this should yield an empty string or the NULL character seems like a matter of taste to me. $\endgroup$
    – Sjoerd C. de Vries
    Mar 1, 2016 at 18:58
5
$\begingroup$

While I'm not a fan of posting a java based answer when a nice and clean Mathematica solution like Sjoerd's exists, over time I've begun to appreciate Leonid's frequent appeals to learn to "steal code" from Java and use it via JLink. Your comment:

Other languages have their function for this:-(

prompted me to post this answer, to mostly serve as a nudge/reminder that in case you're stuck at a problem and know how to do it in of the several other languages that can be interfaced with Mathematica, then the most productive use of your time would be to quickly whip it up in language X and use it inside Mathematica using the appropriate X-Link application. In that spirit, I'll present the following simple solution using Java/JLink:

Needs["JLink`"]
InstallJava[];
decodeQP[str_] := JavaBlock[
    QPC = JavaNew["org.apache.commons.codec.net.QuotedPrintableCodec"];
    QPC@decode[str, "ISO-8859-1"]
]

Now you can easily use it as decodeQP["=FCberhaupt"], using Sjoerd's example, and it should give you the translated string.

Note that you can't convert directly to "UTF-8" as asked in the question and need to convert it to ISO Latin 1 first (credit goes to Sjoerd for spotting this).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Nice, simple example of a well reasoned argument. +1 :-) $\endgroup$
    – Mr.Wizard
    May 29, 2013 at 5:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.