We would like the ability to import and analyze ASCII documents that have been manually segmented by highlighting and changing font colors of various sections, eg:
The original line breaks, spacing and other formatting need to be preserved upon import, and in addition, need to extract the highlight or font color.
To identify a suitable format we are using MS Word for highlighting because we can then save as .rtf, .htm and .xml.
Clearly importing plain text won't contain highlight info, so on to structured formats:
So far not having much luck with MMA9's RTF:
Import[NotebookDirectory[] <> "/example-highlighting-word.rtf", "RTF"]
gives:
Notebook[{Cell[
TextData[{"I found out some info",
" on this type of acrylic.. It is called an exotic type of \
acrylic that shows a colored edge when it is cut. Comes in different \
color types but is not a stock item at San Diego Plastics and is \
pretty expensive.", "
", "Also, the cost of ",
"a 4' x 4' sheet of 1/2\" acrylic for my discount is $150.
", "Did you figure out the exact measurements of the two \
pieces.", "
", StyleBox["I ran some test pieces and they fit. ",
FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox[
" I think we will have to get some new bolts for the larger \
pieces because the heads a", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox["re to", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox["o", FontColor -> RGBColor[1., 0.4, 0.]],
StyleBox[
" tall and I would have to make too deep of a cut to hide the \
bolt head; which would end up making a weaker attachment point.",
FontColor -> RGBColor[1., 0.4, 0.]], StyleBox["
", FontSize -> 12, FontColor -> RGBColor[1., 0.4, 0.]]}],
"Input", CharacterEncoding -> "MacintoshRoman"]},
WindowSize -> {740, 867},
WindowMargins -> {{910, Automatic}, {Automatic, 239}},
FrontEndVersion ->
"9.0 for Mac OS X x86 (32-bit, 64-bit Kernel) (November 20, 2012)",
StyleDefinitions -> "Default.nb"]
Clearly the generated Notebook structure doesn't match the original highlighting segments.
Next we saved and then imported XML:
Import[NotebookDirectory[] <> "/example-highlighting-word.xml", "XML"]
And won't copy the output but it's lengthy, it breaks up individual sentences and the data schema is binary so would be difficult to write pattern rules to extract content and highlights.
Finally, saving and importing HTML as "Text" looks promising:
Import[NotebookDirectory[] <>
"/example-highlighting-word.htm", "Text"]
The output is less voluminous than XML (though still lengthy), but there are still Word-specific HTML tags to deal with, for example:
<p class=MsoNormal \
style='mso-pagination:none;mso-layout-grid-align:none;
text-autospace:none'><span style='font-size:16.0pt;font-family:"Times \
New Roman";
background:red;mso-highlight:red'>I found out some info on this type \
of acrylic<span
class=GramE>..</span> It is called an exotic type of acrylic \
that shows a
colored edge when it is cut. Comes in different color types but \
is not a
stock item at San Diego Plastics and is pretty expensive.</span><span
style='font-size:16.0pt;font-family:"Times New \
Roman"'><o:p></o:p></span></p>
Question - besides is RTF broken? - Is there documentation to handle and pattern match the MS-generated HTML tags like , and to discover potential variations in tags. For example, we'd like to recover the number of new-lines separating paragraphs (as well as indentations), but are faced with composite tags like the following that seems to represent a single newline in the original document:
<p class=MsoNormal \
style='mso-pagination:none;mso-layout-grid-align:none;
text-autospace:none'><span style='font-size:16.0pt;font-family:"Times \
New Roman"'> <o:p></o:p></span></p>
Any better ways to go about it - ie, other formats or Import options? (It's not necessary to use MS Word, maybe other rich text editors are more conducive to proper RTF?)
