I've run into a little question while trying to parse XML. This seems like a rather simple task, but for the life of me I cannot figure it out. I pull up the XML of the website using the following code:
url = "https://investorshub.advfn.com/Twilio-TWLO-31145/?NextStart=0";
getXml[anyUrl_] := (
current = url;
page = URLFetch[url];
If[StringLength@page == 0, getXml@current,
Return@ImportString[page, "XMLObject"]]);
largeXml = getXml[url];
This returns to me a very long XML Tree. But what I am looking at specifically is this piece:
XMLElement[
"a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub",
"href" ->
"/boards/read_msg.aspx?message_id=131676511"}, {"wow this thing \
trades all over the place."}]
The overarching goal is to extract the part of the XML that says "wow this thing trades all over the place." I've done that with the following code:
firstScreen = Cases[largeXml,
XMLElement[
"a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub",
"href" -> "/boards/read_msg.aspx?message_id=131676511"}, {___}],
Infinity]
secondScreen = Cases[#,
XMLElement[
"a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub",
"href" ->
"/boards/read_msg.aspx?message_id=131676511"}, {comment_}] ->
comment, 1] &@firstScreen
"wow this thing trades all over the place" is a subject header on the website that leads into a comment page. There are various subject headers on the webpage all with the same "href" except for the id #. In other words, all of the comments have the XML pattern:
XMLElement[
"a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub",
"href" ->
"/boards/read_msg.aspx?message_id=131676511"}, {comment_}]
The thing that differentiates them is the id # ... "message_id = something different for each subject header". Is there a way I can make it so that I can pull up all the subject headers at once? I've thought of this:
firstScreen = Cases[largeXml,
XMLElement[
"a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub",
"href" -> "/boards/read_msg.aspx?message_id=___"}, {___}],
Infinity]
But that does not return anything. Also, I thought of RegEx but have also heard that RegEx cannot be used to parse HTML or XML. Any help would be greatly appreciated, thank you!