# Pulling sections from a webpage

I'm trying to get specific sections from a webpage using Mathematica. I've started by importing the webpage, and getting the section "headers" that I want:

code = Import[
5baa4ee99a35/", "CSV"]
codeF = Map[ToString, Flatten[code]];
Select[codeF, #=="<section class=\"contents\">" &]


And I can also get the positions of these elements in the list. However, I want to get everything between that section header and it's closing </section>. The only problem I'm having is things like this:

<section class="contents">
<section class="subcontents">
</section>
</section>


So I can't simply search through until the next </section>. How can I extract everything in a specified section?

If you like, I can elaborate later on this. I would not import it as "CSV" but as "XMLObject". With a real Mathematica expression at hand, you can use Cases to extract complete subtrees.

That beeing said, start by using

Import["http://your.page.html", "XMLObject"]


and look at the expression you get. Then play with Cases and you should have a look at the Mathematica documentation.

• Ah! I tried that, but I was trying to import as "XML", not "XMLObject". Thank you! – NMister May 26 '18 at 16:20

I developed jsoupLink for this purpose. Here is an example of how to select all the section nodes with the class "contents" and then get all the text from the first of those nodes:

Code in text format:

Needs["jsoupLink"]
html = Import[
`