Hot answers tagged

15

You can always do Import["http://wsj.com","XMLObject"]. That has the side effect of producing some irregular XML whenever the underlying HTML doesn't quite map cleanly to XML, but it mostly produces an XMLObject[] expression tree that you can match over and extract data from, and I've never seen a web page for which it won't return something.


14

I think Mathematica with its symbolic and pattern-matching capabilites is well suited to tackle parser and tree searching. It's better to check, though. Here are two quick proof-of-concept of the toy engine. One ad-hoc, the other using functional parsers. It' s readily apparent that every simple selector can be mapped to some form of XMLElement pattern: ...


13

I agree wholeheartedly with the comment of celtschk to the OP. Both journals have RSS feeds (with pointers at the bottom of their main pages) that are designed exactly for the purpose that you describe. I doubt that either journal wants you to "scrape" their content; scraping is specifically forbidden by the WSJ Terms of Use. I don't know how much easier ...


13

The reason your approach fails is because Cases works slightly differently than what you've intended in the question. Cases does a depth-first scanning and once it finds its first match, it transforms it and starts traversing the tree backwards, looking for other matches. Consider this simple example: list = {p[1, 2], q, {p[3, 4], p[5, p[6, 7]]}}; ...


12

We can start by importing the file as an XMLObject: $url = "https://dl.dropboxusercontent.com/u/1012958/iTunes%20Library.xml"; $xml = Import[$url, {"XML", "XMLObject"}]; Short[$xml, 4] (* XMLObject[Document][ { XMLObject[Declaration][Version->1.0,Encoding->UTF-8] , XMLObject[Doctype][plist,Public->-//Apple Computer//DTD PLIST ...


7

If you are familiar with AppleScript, you could try an approach like this: (* from http://github.com/fmeinberg/AppleScript *) AppleScript["RunFile", file_] := Run["osascript " <> file] AppleScript["RunScript", script_] := Block[{file = ToFileName[$TemporaryDirectory, "script.txt"]}, Export[file, script, "String"]; AppleScript["RunFile", file]] ...


6

Here's an easy way: body //. XMLElement[_, _, t_] :> t // Flatten // StringJoin


6

Update I have found a new method for converting numeric strings using an internal function that was previously unknown to me. It is quite flexible, but its use requires care as bad input will crash the kernel. I have detailed my present understanding of the function here: ParseTable syntax. Does this work for you? hexNodeData /. XMLElement[a_, {}, ...


6

Mathematica does match all instances. There is, however, one XMLElement (current_observation) that contains all other XMLElements. So after the deepest XMLElements are matched MMA goes one step up and matches the containing XMLElement, which is then returned, and it shows the other XMLElements. A simple example: Cases[ {XMLElement["test", "BlahBlah", ...


6

This is the reply from Wolfram Research Technical Support [CASE:631799]: After consultation with a developer, there is nothing really that can be done with the Export function directly in terms of simplifying the group structure of SVG files. You may be able to develop some system by programmatically parsing the file down, but this would likely ...


6

I can't try it out because your snippet lacks some brackets, but perhaps this will do the trick: Cases[snippet, XMLElement["designInfo", {"studyType" -> atrib_, ___}, ___] :> atrib, \[Infinity]]


5

First, let me tell you that following your code my output is quite different. g=Graphics[{Disk[{-1,0},1/2],Rectangle[]}]; ExportString[g,"SVG"] Output <?xml version="1.0" encoding="UTF-8"?> <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="361pt" height="223pt" viewBox="0 0 361 223" version="1.1"> <g ...


5

Just import the source of the page instead of its rendered content: Import["http://nyt.com", "Source"]


5

I have found the solution. From the Documentation, Some documents use names in a non-namespace-compliant fashion, because the XML namespace recommendation, which extends XML, was made after the initial XML recommendation. "IncludeNamespaces"->"Unparsed" is provided to allow parsing of these documents. The name is always represented as the ...


5

I don't know what causes the issue (which looks like a bug to me), but I generally find the best way to deal with exerting control over Import not doing quit the right thing with HTML is to use the "XMLObject" element and use ordinary Mathematica functions on it, like so: In[1]:= ImportString@ ExportString[ ...


5

JLink` Here is a solution that uses JLink` to leverage Java's support of XML Schema: Needs["JLink`"] InstallJava[]; LoadJavaClass["javax.xml.validation.SchemaFactory", AllowShortContext -> False]; LoadJavaClass["javax.xml.XMLConstants", AllowShortContext -> False]; validateXml[xsd_String, xml_String] := JavaBlock @ Module[{factory, xsdSource, ...


4

As the result of the RSS is just a notebook expression, you can perform any transformation of that expression before you show the notebook. Here I replace the buttonbox as which the hyperlink is represented with a graphics cell that shows the image: blognb = Import[ "http://gregmankiw.blogspot.com/feeds/posts/default?alt=rss", "RSS"]; blognb /. { ...


4

This is one way. Using data=xmlInfo nodes=Cases[data,XMLElement[{_,"Object"},{"ObjectType"->"Target","Name"->target_,"Id"->id_},{x__}]:> {target,id,{x}} ,\[Infinity]]; getColors=Cases[#,XMLElement[{_,color_/;MemberQ[{"Cyan","Yellow","Black","Magenta"},color]},{},{value_}]-> {color,value},\[Infinity]]&; ...


4

I think I can understand this latter version of the question better. Unfortunately though I don't speak jquery, so I don't really understand your example, therefore I will leave the applications to you, and limit this answer to getting an XML form of a Mathematica expression. As it happens, achieving something along those lines is pretty simple, since ...


4

I am using Mathematica 8.0.4 and have no URLFetch command, so I have used Wget to download this 5915505 bytes file: wget http://uscode.house.gov/popularnames/popularnames.htm After downloading I evaluated popNamesXML = Import["popularnames.htm", "XMLObject"] And got the XMLObject without any errors. Then I Imported this file as "Text" and tried to get ...


4

In this case what you want is simple/regular enough that you can define the XMLElement etc. as functions that return what you want inside a Block: Block[{XMLObject, XMLElement}, XMLObject["Document"] = #2 &; XMLElement["tekst", {"write" -> t_}, data_] := {t, Sequence @@ data}; XMLElement["lu", attr_, data_] := "id" /. attr; XMLElement[tag_, ...


4

This seems to be a normal way to extract the data:- xml = XMLObject["Document"][{XMLObject["Declaration"] ["Version" -> "1.0", "Encoding" -> "UTF-8", "Standalone" -> "yes"]}, XMLElement["file", {"name" -> "filename.xml"}, {XMLElement["tekst", {"write" -> "test1"}, {XMLElement["relation", {"type" -> "co-synonyms", ...


4

This problem appears to be localized to HTMLFragment, possibly because HTML specifications before HTML5 did not use self-closing tags like XHTML and XML always have. Exporting the string as XML appears to solve the problem: ExportString[XMLElement["test", {"attr" -> "val"}, {}], "XML"] "<test attr='val' />"


3

Combining Michael E2's answer to another question, that redefines XMLElement, and MarcoB's answer above, here is another approach. To get the Wolfram expression of the messy XML that will recreate the original Wolfram expression (there is probably a better way): ImportString[ExportString[Hold[Times[a, Rational[1, -2], "test"]], "XML"], "XML"] This ...


3

This is not a complete answer but has too much for a comment. I don't have Numbers so do not know what is in the file you linked but the following information can be obtained: Import["PathTo/example.numbers", "ZIP"] (* {"QuickLook/Thumbnail.jpg", "buildVersionHistory.plist", "index.xml"} *) Now grab the XML: Import["PathTo/example.numbers", ...


3

Try ExportString[ImportString["<mtext> </mtext>", {"MathML", "XMLElement"}, "NormalizeWhitespace" -> False], "XML"] From XML/tutorial/ImportingXML. I thought the default for "NormalizeWhitespace" was Automatic. However the documentation states that "True" is the default, where "all the whitespace inside an element is normalized (default)".


3

You should use Condition with StringMatchQ: Cases[dataXML, XMLElement[tag_String /; StringMatchQ[tag, "project*"], __], Infinity] because Cases doesn't support string patterns. As to why it is designed in such a way, I would cite Leonid Shifrin: I would say that the reason is dead simple <…>. Cases and DeleteCases work on parsed expressions, ...


2

Thanks to rcollyer's comments and some of the documentation (which was confusing to a person new to this side of things, i.e. this), I figured this out. On the off chance that anybody stumbles along here and has the same issue, here is what worked for me. Export["test.xml", XMLObject["Document"][{}, XMLElement[ "website", {}, {XMLElement[ ...


2

Seems to work fine for me. I placed a simple RTF file (created by TextEdit on my Mac) on my webspace so you can try it. NotebookPut[ Import["http://facstaff.unca.edu/mcmcclur/temp.rtf"] /. Cell[td_TextData, "Input", opts___] -> Cell[td, "Text", opts], WindowTitle -> "ImportedRTF.nb"]; The one problem I did have was that the resulting ...


2

The following can be used to extract the identifiers used in a given test case, which matches the "write" attribute of an individual XMLElement. Note that the attribute names "write" and "id" are hard-coded, which is not ideal. GetIdentifiers[data_, test_String] := Module[{elem, attr}, (*find test case*) elem = Cases[data, XMLElement[___, {___, ...



Only top voted, non community-wiki answers of a minimum length are eligible