Hot answers tagged

19

We can start by importing the file as an XMLObject: $url = "https://dl.dropboxusercontent.com/u/1012958/iTunes%20Library.xml"; $xml = Import[$url, {"XML", "XMLObject"}]; Short[$xml, 4] (* XMLObject[Document][ { XMLObject[Declaration][Version->1.0,Encoding->UTF-8] , XMLObject[Doctype][plist,Public->-//Apple Computer//DTD PLIST 1.0//EN,&...


17

You can always do Import["http://wsj.com","XMLObject"]. That has the side effect of producing some irregular XML whenever the underlying HTML doesn't quite map cleanly to XML, but it mostly produces an XMLObject[] expression tree that you can match over and extract data from, and I've never seen a web page for which it won't return something.


16

I agree wholeheartedly with the comment of celtschk to the OP. Both journals have RSS feeds (with pointers at the bottom of their main pages) that are designed exactly for the purpose that you describe. I doubt that either journal wants you to "scrape" their content; scraping is specifically forbidden by the WSJ Terms of Use. I don't know how much easier ...


15

I think Mathematica with its symbolic and pattern-matching capabilites is well suited to tackle parser and tree searching. It's better to check, though. Here are two quick proof-of-concept of the toy engine. One ad-hoc, the other using functional parsers. It' s readily apparent that every simple selector can be mapped to some form of XMLElement pattern: "...


14

The reason your approach fails is because Cases works slightly differently than what you've intended in the question. Cases does a depth-first scanning and once it finds its first match, it transforms it and starts traversing the tree backwards, looking for other matches. Consider this simple example: list = {p[1, 2], q, {p[3, 4], p[5, p[6, 7]]}}; Cases[...


9

I'm going to focus on the convenience part of the question. jsoupLink can work well for a lot of situations. Let's say, for example, that we want to parse RSS. RSS is an XML format, so we can do it like this: << jSoupLink` root = Import["http://packagedata.net/index.php/feed", "HTMLDOM"] items = root["Select", "item"]; What this shows is that using ...


8

This is the reply from Wolfram Research Technical Support [CASE:631799]: After consultation with a developer, there is nothing really that can be done with the Export function directly in terms of simplifying the group structure of SVG files. You may be able to develop some system by programmatically parsing the file down, but this would likely not ...


8

JLink` Here is a solution that uses JLink` to leverage Java's support of XML Schema: Needs["JLink`"] InstallJava[]; LoadJavaClass["javax.xml.validation.SchemaFactory", AllowShortContext -> False]; LoadJavaClass["javax.xml.XMLConstants", AllowShortContext -> False]; validateXml[xsd_String, xml_String] := JavaBlock @ Module[{factory, xsdSource, ...


7

If you are familiar with AppleScript, you could try an approach like this: (* from http://github.com/fmeinberg/AppleScript *) AppleScript["RunFile", file_] := Run["osascript " <> file] AppleScript["RunScript", script_] := Block[{file = ToFileName[$TemporaryDirectory, "script.txt"]}, Export[file, script, "String"]; AppleScript["RunFile", file]] ...


7

I can't try it out because your snippet lacks some brackets, but perhaps this will do the trick: Cases[snippet, XMLElement["designInfo", {"studyType" -> atrib_, ___}, ___] :> atrib, \[Infinity]]


7

Update I have found a new method for converting numeric strings using an internal function that was previously unknown to me. It is quite flexible, but its use requires care as bad input will crash the kernel. I have detailed my present understanding of the function here: ParseTable syntax. Does this work for you? hexNodeData /. XMLElement[a_, {}, {...


6

I have found the solution. From the Documentation, Some documents use names in a non-namespace-compliant fashion, because the XML namespace recommendation, which extends XML, was made after the initial XML recommendation. "IncludeNamespaces"->"Unparsed" is provided to allow parsing of these documents. The name is always represented as the exact ...


6

Here's an easy way: body //. XMLElement[_, _, t_] :> t // Flatten // StringJoin


6

Mathematica does match all instances. There is, however, one XMLElement (current_observation) that contains all other XMLElements. So after the deepest XMLElements are matched MMA goes one step up and matches the containing XMLElement, which is then returned, and it shows the other XMLElements. A simple example: Cases[ {XMLElement["test", "BlahBlah", ...


5

First, let me tell you that following your code my output is quite different. g=Graphics[{Disk[{-1,0},1/2],Rectangle[]}]; ExportString[g,"SVG"] Output <?xml version="1.0" encoding="UTF-8"?> <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="361pt" height="223pt" viewBox="0 0 361 223" version="1.1"> <g ...


5

Just import the source of the page instead of its rendered content: Import["http://nyt.com", "Source"]


5

I am using Mathematica 8.0.4 and have no URLFetch command, so I have used Wget to download this 5915505 bytes file: wget http://uscode.house.gov/popularnames/popularnames.htm After downloading I evaluated popNamesXML = Import["popularnames.htm", "XMLObject"] And got the XMLObject without any errors. Then I Imported this file as "Text" and tried to get ...


5

I don't know what causes the issue (which looks like a bug to me), but I generally find the best way to deal with exerting control over Import not doing quit the right thing with HTML is to use the "XMLObject" element and use ordinary Mathematica functions on it, like so: In[1]:= ImportString@ ExportString[ ImportString["a<span>b<...


5

If you look at the URL that you found, you'll notice a parameter pn. If you change this parameter you get another set of t-shirts. To get all t-shirts you need to download that URL for all allowed pn. Let's say pn stands for "page number". Now, how many pages are there? We have to remember that this is part of an API. Another programmer used the response of ...


4

In this case what you want is simple/regular enough that you can define the XMLElement etc. as functions that return what you want inside a Block: Block[{XMLObject, XMLElement}, XMLObject["Document"] = #2 &; XMLElement["tekst", {"write" -> t_}, data_] := {t, Sequence @@ data}; XMLElement["lu", attr_, data_] := "id" /. attr; XMLElement[tag_, attr_,...


4

This seems to be a normal way to extract the data:- xml = XMLObject["Document"][{XMLObject["Declaration"] ["Version" -> "1.0", "Encoding" -> "UTF-8", "Standalone" -> "yes"]}, XMLElement["file", {"name" -> "filename.xml"}, {XMLElement["tekst", {"write" -> "test1"}, {XMLElement["relation", {"type" -> "co-synonyms", "...


4

Had to add the "ReadDTD" -> False option to $xml = Import[ "/Volumes/WDC3TBRAID/Downloads/iTunes Music Library.xml", {"XML", "XMLObject"}, "ReadDTD" -> False] to get WReach's solution to work with Mathematica 11.0. Rest of his solution works fine after that.


4

I think I can understand this latter version of the question better. Unfortunately though I don't speak jquery, so I don't really understand your example, therefore I will leave the applications to you, and limit this answer to getting an XML form of a Mathematica expression. As it happens, achieving something along those lines is pretty simple, since ...


4

As the result of the RSS is just a notebook expression, you can perform any transformation of that expression before you show the notebook. Here I replace the buttonbox as which the hyperlink is represented with a graphics cell that shows the image: blognb = Import[ "http://gregmankiw.blogspot.com/feeds/posts/default?alt=rss", "RSS"]; blognb /. { ...


4

This is one way. Using data=xmlInfo nodes=Cases[data,XMLElement[{_,"Object"},{"ObjectType"->"Target","Name"->target_,"Id"->id_},{x__}]:> {target,id,{x}} ,\[Infinity]]; getColors=Cases[#,XMLElement[{_,color_/;MemberQ[{"Cyan","Yellow","Black","Magenta"},color]},{},{value_}]-> {color,value},\[Infinity]]&; nodes[[All,3]]=getColors/@nodes[[All,...


4

This is not a complete answer but has too much for a comment. I don't have Numbers so do not know what is in the file you linked but the following information can be obtained: Import["PathTo/example.numbers", "ZIP"] (* {"QuickLook/Thumbnail.jpg", "buildVersionHistory.plist", "index.xml"} *) Now grab the XML: Import["PathTo/example.numbers", {"ZIP","...


4

Try ExportString[ImportString["<mtext> </mtext>", {"MathML", "XMLElement"}, "NormalizeWhitespace" -> False], "XML"] From XML/tutorial/ImportingXML. I thought the default for "NormalizeWhitespace" was Automatic. However the documentation states that "True" is the default, where "all the whitespace inside an element is normalized (default)".


4

This problem appears to be localized to HTMLFragment, possibly because HTML specifications before HTML5 did not use self-closing tags like XHTML and XML always have. Exporting the string as XML appears to solve the problem: ExportString[XMLElement["test", {"attr" -> "val"}, {}], "XML"] "<test attr='val' />"


4

First of all, I think you can make your life a bit easier by using RawJSON as input format specification. It basically makes Mathematica use Associations where appropriate when importing JSON. url = "http://finance.services.appex.bing.com/Market.svc/ChartDataV5?symbols=126.1.BY.NYS&chartType=1y&isEOD=False&lang=en-US&isCS=true&isVol=true"...


4

To get the ball rolling, here's my incomplete parser. And here's a minimal markdown sample. This is what my markdown parser makes of it, used like so: CloudExport[ MarkdownToXML[ Import["https://raw.githubusercontent.com/b3m2a1/mathematica-BTools/\ master/Resources/Templates/PacletServer/content/pages/About.md", "Text" ], "...


Only top voted, non community-wiki answers of a minimum length are eligible