Unwanted extra spaces at HTML tag boundaries when using ImportString

Question

What causes the following unexpected behavior?

In[1]:= ImportString["a<span>b</span>c", "HTML"]
Out[1]= "a b c"

In[2]:= ImportString["a<foo>b</foo>c", "HTML"]
Out[2]= "a bc"

In both cases, I would expect the returned string to be "abc".

What is the best way to work around it?

Answer 1

I don't know what causes the issue (which looks like a bug to me), but I generally find the best way to deal with exerting control over Import not doing quit the right thing with HTML is to use the "XMLObject" element and use ordinary Mathematica functions on it, like so:

In[1]:= ImportString@
         ExportString[
          ImportString["a<span>b</span>c", {"HTML", "XMLObject"}] /. 
           XMLElement[
             Except["html" | "body" | "p" | "li" | "table" | "tr" | "td" | 
               "br" | "ul" | "li"], {___}, {stuff___}] :> stuff, "XML"]
Out[1]= "abc"

It's kinda cumbersome to round-trip like that, but on the upside it's very flexible.

EDITed to strip out a wider range of tags.

Answer 2

jSoupLink can be used as a workaround, like this:

<< jSoupLink`
html2text[html_] := First@jSoupLink`ParseHTMLFragment[
   "<div id=\"container\">" <> html <> "</div>", "#container", "text"
   ]

html2text["a<span>b</span>c"]
(* Out: abc *)

html2text["a<foo>b</foo>c"]
(* Out: abc *)

Answer 3

One can import raw text and use this example to remove HTML tags

StringReplace[  #, "<" ~~ Except[">"] .. ~~ ">" -> ""] &@
 ImportString["a<bar>b</bar>c", {"HTML", "Source"}]
(* "abc" *)