Tag Info

Hot answers tagged

14

You can always do Import["http://wsj.com","XMLObject"]. That has the side effect of producing some irregular XML whenever the underlying HTML doesn't quite map cleanly to XML, but it mostly produces an XMLObject[] expression tree that you can match over and extract data from, and I've never seen a web page for which it won't return something.


13

I think Mathematica with its symbolic and pattern-matching capabilites is well suited to tackle parser and tree searching. It's better to check, though. Here are two quick proof-of-concept of the toy engine. One ad-hoc, the other using functional parsers. It' s readily apparent that every simple selector can be mapped to some form of XMLElement pattern: ...


12

I agree wholeheartedly with the comment of celtschk to the OP. Both journals have RSS feeds (with pointers at the bottom of their main pages) that are designed exactly for the purpose that you describe. I doubt that either journal wants you to "scrape" their content; scraping is specifically forbidden by the WSJ Terms of Use. I don't know how much easier ...


11

The reason your approach fails is because Cases works slightly differently than what you've intended in the question. Cases does a depth-first scanning and once it finds its first match, it transforms it and starts traversing the tree backwards, looking for other matches. Consider this simple example: list = {p[1, 2], q, {p[3, 4], p[5, p[6, 7]]}}; ...


7

We can start by importing the file as an XMLObject: $url = "https://dl.dropboxusercontent.com/u/1012958/iTunes%20Library.xml"; $xml = Import[$url, {"XML", "XMLObject"}]; Short[$xml, 4] (* XMLObject[Document][ { XMLObject[Declaration][Version->1.0,Encoding->UTF-8] , XMLObject[Doctype][plist,Public->-//Apple Computer//DTD PLIST ...


7

If you are familiar with AppleScript, you could try an approach like this: (* from http://github.com/fmeinberg/AppleScript *) AppleScript["RunFile", file_] := Run["osascript " <> file] AppleScript["RunScript", script_] := Block[{file = ToFileName[$TemporaryDirectory, "script.txt"]}, Export[file, script, "String"]; AppleScript["RunFile", file]] ...


6

This is the reply from Wolfram Research Technical Support [CASE:631799]: After consultation with a developer, there is nothing really that can be done with the Export function directly in terms of simplifying the group structure of SVG files. You may be able to develop some system by programmatically parsing the file down, but this would likely ...


6

Update I have found a new method for converting numeric strings using an internal function that was previously unknown to me. It is quite flexible, but its use requires care as bad input will crash the kernel. I have detailed my present understanding of the function here: ParseTable syntax. Does this work for you? hexNodeData /. XMLElement[a_, {}, ...


5

Just import the source of the page instead of its rendered content: Import["http://nyt.com", "Source"]


5

Mathematica does match all instances. There is, however, one XMLElement (current_observation) that contains all other XMLElements. So after the deepest XMLElements are matched MMA goes one step up and matches the containing XMLElement, which is then returned, and it shows the other XMLElements. A simple example: Cases[ {XMLElement["test", "BlahBlah", ...


5

I have found the solution. From the Documentation, Some documents use names in a non-namespace-compliant fashion, because the XML namespace recommendation, which extends XML, was made after the initial XML recommendation. "IncludeNamespaces"->"Unparsed" is provided to allow parsing of these documents. The name is always represented as the ...


5

Here's an easy way: body //. XMLElement[_, _, t_] :> t // Flatten // StringJoin


5

I can't try it out because your snippet lacks some brackets, but perhaps this will do the trick: Cases[snippet, XMLElement["designInfo", {"studyType" -> atrib_, ___}, ___] :> atrib, \[Infinity]]


4

I am using Mathematica 8.0.4 and have no URLFetch command, so I have used Wget to download this 5915505 bytes file: wget http://uscode.house.gov/popularnames/popularnames.htm After downloading I evaluated popNamesXML = Import["popularnames.htm", "XMLObject"] And got the XMLObject without any errors. Then I Imported this file as "Text" and tried to get ...


4

This is one way. Using data=xmlInfo nodes=Cases[data,XMLElement[{_,"Object"},{"ObjectType"->"Target","Name"->target_,"Id"->id_},{x__}]:> {target,id,{x}} ,\[Infinity]]; getColors=Cases[#,XMLElement[{_,color_/;MemberQ[{"Cyan","Yellow","Black","Magenta"},color]},{},{value_}]-> {color,value},\[Infinity]]&; ...


3

As the result of the RSS is just a notebook expression, you can perform any transformation of that expression before you show the notebook. Here I replace the buttonbox as which the hyperlink is represented with a graphics cell that shows the image: blognb = Import[ "http://gregmankiw.blogspot.com/feeds/posts/default?alt=rss", "RSS"]; blognb /. { ...


3

This seems to be a normal way to extract the data:- xml = XMLObject["Document"][{XMLObject["Declaration"] ["Version" -> "1.0", "Encoding" -> "UTF-8", "Standalone" -> "yes"]}, XMLElement["file", {"name" -> "filename.xml"}, {XMLElement["tekst", {"write" -> "test1"}, {XMLElement["relation", {"type" -> "co-synonyms", ...


3

This is not a complete answer but has too much for a comment. I don't have Numbers so do not know what is in the file you linked but the following information can be obtained: Import["PathTo/example.numbers", "ZIP"] (* {"QuickLook/Thumbnail.jpg", "buildVersionHistory.plist", "index.xml"} *) Now grab the XML: Import["PathTo/example.numbers", ...


3

In this case what you want is simple/regular enough that you can define the XMLElement etc. as functions that return what you want inside a Block: Block[{XMLObject, XMLElement}, XMLObject["Document"] = #2 &; XMLElement["tekst", {"write" -> t_}, data_] := {t, Sequence @@ data}; XMLElement["lu", attr_, data_] := "id" /. attr; XMLElement[tag_, ...


2

Assuming that the example XML shown is representative, then we can exploit the fact the write and id attribute names are unambiguous within tekst elements: Cases[test , XMLElement["tekst", a_, c_] :> Sequence["write" /. a, Cases[c, ("id" -> i_) :> i, -1]] , -1 ] (* { "test1", {"AA-504771", "BB-59362", "CC-59362"}, "test2", {"AA-20210", ...


2

The following can be used to extract the identifiers used in a given test case, which matches the "write" attribute of an individual XMLElement. Note that the attribute names "write" and "id" are hard-coded, which is not ideal. GetIdentifiers[data_, test_String] := Module[{elem, attr}, (*find test case*) elem = Cases[data, XMLElement[___, {___, ...


2

Thanks to rcollyer's comments and some of the documentation (which was confusing to a person new to this side of things, i.e. this), I figured this out. On the off chance that anybody stumbles along here and has the same issue, here is what worked for me. Export["test.xml", XMLObject["Document"][{}, XMLElement[ "website", {}, {XMLElement[ ...


2

Seems to work fine for me. I placed a simple RTF file (created by TextEdit on my Mac) on my webspace so you can try it. NotebookPut[ Import["http://facstaff.unca.edu/mcmcclur/temp.rtf"] /. Cell[td_TextData, "Input", opts___] -> Cell[td, "Text", opts], WindowTitle -> "ImportedRTF.nb"]; The one problem I did have was that the resulting ...



Only top voted, non community-wiki answers of a minimum length are eligible