2
$\begingroup$

I've run into a little question while trying to parse XML. This seems like a rather simple task, but for the life of me I cannot figure it out. I pull up the XML of the website using the following code:

url = "https://investorshub.advfn.com/Twilio-TWLO-31145/?NextStart=0";
getXml[anyUrl_] := (
   current = url;
   page = URLFetch[url];
   If[StringLength@page == 0, getXml@current, 
    Return@ImportString[page, "XMLObject"]]);
largeXml = getXml[url];

This returns to me a very long XML Tree. But what I am looking at specifically is this piece:

  XMLElement[
   "a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
    "href" -> 
     "/boards/read_msg.aspx?message_id=131676511"}, {"wow this thing \
trades all over the place."}]

The overarching goal is to extract the part of the XML that says "wow this thing trades all over the place." I've done that with the following code:

firstScreen = Cases[largeXml,
  XMLElement[
   "a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
    "href" -> "/boards/read_msg.aspx?message_id=131676511"}, {___}], 
  Infinity]
secondScreen = Cases[#,
    XMLElement[
      "a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
       "href" -> 
        "/boards/read_msg.aspx?message_id=131676511"}, {comment_}] -> 
     comment, 1] &@firstScreen 

"wow this thing trades all over the place" is a subject header on the website that leads into a comment page. There are various subject headers on the webpage all with the same "href" except for the id #. In other words, all of the comments have the XML pattern:

XMLElement[
          "a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
           "href" -> 
            "/boards/read_msg.aspx?message_id=131676511"}, {comment_}] 

The thing that differentiates them is the id # ... "message_id = something different for each subject header". Is there a way I can make it so that I can pull up all the subject headers at once? I've thought of this:

firstScreen = Cases[largeXml,
  XMLElement[
   "a", {"shape" -> "rect", "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
    "href" -> "/boards/read_msg.aspx?message_id=___"}, {___}], 
  Infinity]

But that does not return anything. Also, I thought of RegEx but have also heard that RegEx cannot be used to parse HTML or XML. Any help would be greatly appreciated, thank you!

$\endgroup$
  • $\begingroup$ My mistake, the gv_ct103 will be different for other headers. For instance, others could be gv_ct102, gv_ct103, gv_ct104, etc. $\endgroup$ – Vic Jongmin Youn Jun 22 '17 at 15:06
2
$\begingroup$
xml = { XMLElement[    "a", {
     "shape" -> "rect", 
     "id" -> "ctl00_CP1_gv_ctl03_hlSub", 
     "href" -> "/boards/read_msg.aspx?message_id=131676511"
   }, 
 {comment_}]
};

Cases[xml
  , XMLElement[_, {___, "href" -> (
        s_?(StringStartsQ["/boards/read_msg.aspx?message_id="])
      ), ___}, _
    ] :> s
  , ∞
]
{"/boards/read_msg.aspx?message_id=131676511"}

or:

Cases[xml, XMLElement[_,  KeyValuePattern[ 
  "href" -> (s_?(StringStartsQ["/boards/read_msg.aspx?message_id="]))
 ], _] :>   s, ∞
]
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.