7
$\begingroup$

I know that I can use WebExecute to retrieve WebElementObject text, as in this example:

StartWebSession["Chrome"]
WebExecute["OpenPage"->"https://en.wikipedia.org/wiki/List_of_countries_and_dependencies_by_population"]
table = First@WebExecute["LocateElements"->"XPath"->"//*[@id=\"mw-content-text\"]/div/table"]
WebExecute["ElementText"->table]

And that I can retrieve the complete page HTML:

html = WebExecute["JavascriptExecute"->"return document.documentElement.outerHTML;"]

The question is: How can I get WebElementObject table data as HTML instead of text?

I explored some WebUnit functions, but without success.

COMPLEMENT

You can convert your HTML to data using:

ImportString[getAttribute[table, "innerHTML"],  {"HTML", "Data"}]
$\endgroup$

2 Answers 2

7
$\begingroup$

Had to dig a bit, but since it's just using the W3C WebDriver stuff you can do it by hand since WebExecute doesn't seem to expose all the requests:

getWebElementPropertyURL[el_, prop_] :=
  URLBuild@
   ReplacePart[
    URLParse[el["URL"]],
    "Path" -> {"session", el["SessionID"], "element", el["ElementId"], 
      "property",
      prop
      }
    ];
getWebElementProperty[el_, prop_] :=
 Import[getWebElementPropertyURL[el, prop], "RawJSON"]["value"]

getWebElementProperty[table, "innerHTML"]

"#cite_note-218\">[197]</a></sup>
</td></tr>
</tbody><tfoot><tr>
<th></th>
<th style=\"text-align:left\"><span class=\"flagicon\" style=\"padding-left:25px;\">&nbsp;</span>World</th>
<th style=\"text-align:left\">7,771,541,000</th>
<th style=\"text-align:left\">100%</th>
<th style=\"text-align:left\"><span data-sort-value=\"000000002020-03-14-0000\" style=\"white-space:nowrap\">14 Mar 2020</span></th>
<th style=\"text-align:left\">UN projection<sup id=\"cite_ref-219\" class=\"reference\"><a href=\"#cite_note-219\">[198]</a></sup>
</th></tr></tfoot>"

You could generalize this stuff to any request you want, as documented here

$\endgroup$
1
  • $\begingroup$ Cool! This answer is another option to get element properties using WebUnit. $\endgroup$
    – Murta
    Commented Mar 14, 2020 at 11:54
2
$\begingroup$

Another option is to import the HTML as a Wolfram XML expression and pull out the bits you need and then convert them back to a string again:

xml = ImportString[html, {"HTML", "XMLObject"}];
FirstCase[xml, data : XMLElement["table", __] :> ExportString[data, "XML"], Missing[], Infinity]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.