# Importing formatted, highlighted text from : HTML vs RTF vs. XML

We would like the ability to import and analyze ASCII documents that have been manually segmented by highlighting and changing font colors of various sections, eg:

The original line breaks, spacing and other formatting need to be preserved upon import, and in addition, need to extract the highlight or font color.

To identify a suitable format we are using MS Word for highlighting because we can then save as .rtf, .htm and .xml.

Clearly importing plain text won't contain highlight info, so on to structured formats:

So far not having much luck with MMA9's RTF:

Import[NotebookDirectory[] <> "/example-highlighting-word.rtf", "RTF"]


gives:

Notebook[{Cell[
TextData[{"I found out some info",
" on this type of acrylic..  It is called an exotic type of \
acrylic that shows a colored edge when it is cut.  Comes in different \
color types but is not a stock item at San Diego Plastics and is \
pretty expensive.", "

", "Also, the cost of ",
"a 4' x 4' sheet of 1/2\" acrylic for my discount is \$150.

", "Did you figure out the exact measurements of the two \
pieces.", "

", StyleBox["I ran some test pieces and they fit. ",
FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox[
" I think we will have to get some new bolts for the larger \
pieces because the heads a", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox["re to", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox["o", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox[
" tall and I would have to make too deep of a cut to hide the \
bolt head; which would end up making a weaker attachment point.",
FontColor -> RGBColor[1., 0.4, 0.]], StyleBox["
", FontSize -> 12, FontColor -> RGBColor[1., 0.4, 0.]]}],
"Input", CharacterEncoding -> "MacintoshRoman"]},
WindowSize -> {740, 867},
WindowMargins -> {{910, Automatic}, {Automatic, 239}},
FrontEndVersion ->
"9.0 for Mac OS X x86 (32-bit, 64-bit Kernel) (November 20, 2012)",
StyleDefinitions -> "Default.nb"]


Clearly the generated Notebook structure doesn't match the original highlighting segments.

Next we saved and then imported XML:

Import[NotebookDirectory[] <> "/example-highlighting-word.xml", "XML"]


And won't copy the output but it's lengthy, it breaks up individual sentences and the data schema is binary so would be difficult to write pattern rules to extract content and highlights.

Finally, saving and importing HTML as "Text" looks promising:

Import[NotebookDirectory[] <>
"/example-highlighting-word.htm", "Text"]


The output is less voluminous than XML (though still lengthy), but there are still Word-specific HTML tags to deal with, for example:

<p class=MsoNormal \
style='mso-pagination:none;mso-layout-grid-align:none;
text-autospace:none'><span style='font-size:16.0pt;font-family:"Times \
New Roman";
background:red;mso-highlight:red'>I found out some info on this type \
of acrylic<span
class=GramE>..</span>&nbsp; It is called an exotic type of acrylic \
that shows a
colored edge when it is cut.&nbsp; Comes in different color types but \
is not a
stock item at San Diego Plastics and is pretty expensive.</span><span
style='font-size:16.0pt;font-family:"Times New \
Roman"'><o:p></o:p></span></p>


Question - besides is RTF broken? - Is there documentation to handle and pattern match the MS-generated HTML tags like , and to discover potential variations in tags. For example, we'd like to recover the number of new-lines separating paragraphs (as well as indentations), but are faced with composite tags like the following that seems to represent a single newline in the original document:

<p class=MsoNormal \
style='mso-pagination:none;mso-layout-grid-align:none;
text-autospace:none'><span style='font-size:16.0pt;font-family:"Times \
New Roman"'>&nbsp;<o:p></o:p></span></p>


Any better ways to go about it - ie, other formats or Import options? (It's not necessary to use MS Word, maybe other rich text editors are more conducive to proper RTF?)

-

Seems to work fine for me. I placed a simple RTF file (created by TextEdit on my Mac) on my webspace so you can try it.

NotebookPut[
Import["http://facstaff.unca.edu/mcmcclur/temp.rtf"] /.
Cell[td_TextData, "Input", opts___] ->
Cell[td, "Text", opts], WindowTitle -> "ImportedRTF.nb"];


The one problem I did have was that the resulting cell was an Input cell, where I suppose that Text cell makes more sense. I added a bit of code to convert any cell of the form Cell[td_TextData, ___] to a Text cell. The result looks like so:

If you require further diagnosis, you might want to make your /example-highlighting-word.rtf file publicly available.

-
Great thanks, +1 - I didn't mention however that the files we're dealing with contain protected health information and are located on PC/Windows machines. We will try to use TextEdit, but did you try the same content on MS Word? –  alancalvitti Dec 10 '12 at 3:58
in addition to font color, is it possible to highlight as per MS Word? –  alancalvitti Dec 10 '12 at 18:31
@alancalvitti It is certainly possible to highlight text in a Mathematica notebook. I don't actually have MSWord, though, so it's a little hard for me know. I did quickly create a document with highlighting using Google Drive and, after export to RTF, the highlighting was lost. Does RTF even support highlighting? As I said before, you should really make your own sample file available. –  Mark McClure Dec 10 '12 at 19:21
What I meant is - is it possible to import already highlighted text (preserving the markup)? Word enables highlighting (but importing it creates the issues I described above), but your example only shows font color changes. –  alancalvitti Dec 10 '12 at 19:25
@alancalvitti Yes, that's what I understood you to mean. I'm telling you that, in spite of the fact that one can highlight text in the Mathematica Notebook, there's no way for me to answer your question without access to your "already highlighted text". –  Mark McClure Dec 10 '12 at 19:41
show 1 more comment