TL;DR Is there any way that we can parse HTML using CSS selectors in Mathematica, the way it is done in for example jQuery?

Extracting information from websites, i.e. web-scraping, in Mathematica can be time-consuming. The traditional techniques described in

are simply not enough for most serious web-scraping tasks. Since the most common technique is to import HTML as symbolic XML and then parse the XML with Cases another user had the idea to abstract this method into a package that would turn CSS rules into patterns that can parse symbolic XML:

Although the effort is praiseworthy there are a couple of drawbacks with his solution, primarily because it is only a proof-of-concept. Unfortunately it would take an unreasonable amount of time to build a solution based on this start that is even close to as good as what is already out there for other langauges, such as jQuery or PHP Simple HTML DOM parser.

Is there any way we can get comparable functionality in Mathematica?

The questions on this site alone show that there is a demand for a solution to this problem. A solution would make it possible to provide elegant answers to at least the following questions:

There is also this question which demonstrates Leonid Shifrin's HTML parser. It could also have been avoided by starting from a jQuery-like HTML parser.

  • $\begingroup$ node.js is an option. Not exactly small overhead, but it is cross operating system and I believe might be faster then most java implementations. stackoverflow.com/questions/1801160/… $\endgroup$
    – William
    Commented Feb 10, 2015 at 1:01
  • 2
    $\begingroup$ I actually went some way to building a XPATH engine in pure Mathematica using XMLElements. I just found the query semantics somewhat hard to understand at first pass. Maybe I would have been more successful with a CSS selector engine. I don't rule out that we would do this in the future. $\endgroup$ Commented Feb 10, 2015 at 5:34

2 Answers 2



This answer pertains to the original release of jsoupLink. The interface changed completely in a later version. Please see the Github page for the current interface.


As much as I would like to see a solution to this problem written in Mathematica, this is very unlikely given the scope of the problem. I would like to share a way to solve this using JLink, in the hope that it may help someone.

JLink, for those who don't know, is a package that comes with Mathematica. It allows you to execute Java code from within Mathematica. This means you can use any Java library out there to solve your problems without leaving the notebook interface. For this particular problem I will use jSoup, which is a parser just like the ones mentioned in the question.

Downloading and installing the package

You can download the latest version as a zip file from here.

It is important that the files are kept in the correct folder, otherwise Mathematica will not be able to locate the Java files. Therefore, to install the package start by evaluating

FileNameJoin[{$UserBaseDirectory, "Applications"}]

in Mathematica and unzip the zip file you downloaded into this folder. Then use Needs["`jSoupLink`"] to load the package.


The package contains three functions: ParseHTML, ParseHTMLString and ParseHTMLFragment. Some information about these is contained in their usage messages, which, if you have loaded the package, you can view using for example


Typically you will use ParseHTML to download HTML source code from a website and then select a few elements. From these elements you will then extract some data. The general syntax is like this:

website address,
CSS selector,
data elements to extract

website address is any URL, for example http://mathematica.stackexchange.com. CSS selector is basically any valid CSS3 selector. There is a list of CSS3 selector in jSoup's documentation. Data elements to extract can be almost anything contained by the elements that you've selected. Most commonly you'll want to extract attributes such as src if you've selected img elements or href if you've selected links (a elements). There are a few keywords that aren't attributes such as text to select the text contained by a selected element (some text in <p>some text</p>) or html to select the HTML contained by a selected element. You can glean the complete list from the package source code, and look them up in jSoup's documentation if you're not sure what they are.


Selecting images from Wikipedia

urls = jSoupLink`ParseHTML[
   "http://en.wikipedia.org/wiki/Sweden", (* URL *)
   "table.infobox img", (* CSS selector *)
   "src" (* Attribute to retrieve *)
Partition[Import /@ urls, 2] // Grid

Example images from Wikipedia

Select headlines (both text and URL) from NYT

headlines = Rest@jSoupLink`ParseHTML[
    "h2 a, h3 a",
    {"text", "href"}
Take[headlines, 5] // TableForm

NYT headlines

Build a database with information about Swedish municipalities, using data on Wikipedia

headers = jSoupLink`ParseHTML[
   "table.wikitable.sortable th",
headers = StringReplace[#, "(" ~~ __ ~~ ")" -> ""] & /@ headers; (* Remove units *)
headers = StringReplace[#, WordBoundary ~~ x_ :> ToUpperCase[x]] & /@ headers; (* Capitalize *)
headers = StringReplace[#, " " -> ""] & /@ headers;(* Remove spaces *)

municipalities = jSoupLink`ParseHTML[
   "table.wikitable.sortable td",
municipalities = Partition[municipalities, 9];

ds = Dataset@Composition[
     Map[(headers -> #) &]

Now if you want to select all municipalities that belong to the county Västra Götaland you just have to type

ds[Select[#County == "Västra Götaland County" &], "Municipality"] // Normal

{"Ale Municipality", "Alingsås Municipality", "Bengtsfors \ Municipality", "Bollebygd Municipality", ...

  • $\begingroup$ @Kuba Thank you, I changed it. $\endgroup$
    – C. E.
    Commented Oct 26, 2015 at 11:29
  • $\begingroup$ It gives an error "A Java exception occurred: "java.lang.ClassNotFoundException: \ org.jsoup.Jsoup" when loading the module $\endgroup$
    – Prastt
    Commented Dec 12, 2015 at 22:15
  • $\begingroup$ @Prastt My instructions for how to load the package were not very good, please look at the new instructions in this post. $\endgroup$
    – C. E.
    Commented Dec 12, 2015 at 22:36
  • $\begingroup$ @C.E. Hi can you check that this works on v12? It's giving me A Java exception occurred: MathLinkException: 49: Unable to convert from MathLink encoding to requested character encoding. $\endgroup$
    – user5601
    Commented Jan 31, 2020 at 1:24
  • 1
    $\begingroup$ @NamNguyen Not that I know of but I haven't kept up to date on this, do you have reason to believe that it is? $\endgroup$
    – C. E.
    Commented May 10 at 12:55

If you are looking for a simple quick fix solution.

  1. Download http://nodejs.org/
  2. run npm install jquery
  3. Example http://pastebin.com/raw.php?i=E2t9hSYu


(function () {    
  var env = require('jsdom').env,
      // first argument can be html string, filename, or url
      html = '<html><body><h1>Hello World!</h1><p class="hello">Heya Big World!</body></html>';

  env(html, function (errors, window) {

    var $ = require('jquery')(window);

Then you can call the script using something like

Import["!node script.js", "Text"]

The data should be outputted to the console. There is a slight bug listed here on Windows

Run Command Not Executing Node

  • 1
    $\begingroup$ I think the question was about how to access the functionality from Mathematica and how to get back the results to Mathematica again. $\endgroup$
    – Szabolcs
    Commented Feb 10, 2015 at 1:36
  • $\begingroup$ Yes, what Szabolcs said; I often use jQuery to quickly extract some piece of information from HTML, but here I'm looking for ways to integrate jQuery style selectors seamlessly with Mathematica. $\endgroup$
    – C. E.
    Commented Feb 10, 2015 at 1:48
  • $\begingroup$ @Pickett Your answer is very well done compared to mine. If you use Watch File Directory and fs.writeFile(filename, data[, options], callback) it should be seamless with a temporary file. I am not familiar enough with the C hooks. $\endgroup$
    – William
    Commented Feb 10, 2015 at 1:59
  • $\begingroup$ @William Can you please update your wonderful answer's installation steps? I follow them but get this at the Needs step: Get::noopen: Cannot open GlobaljSoupLink. $\endgroup$
    – user5601
    Commented Jan 31, 2020 at 0:59
  • $\begingroup$ @William Actually that's not true, the Needs works... but when I call jSoupLink`ParseHTML in v12 on macos 10.15.2 I get this: Java::excptn: A Java exception occurred: MathLinkException: 49: Unable to convert from MathLink encoding to requested character encoding. $\endgroup$
    – user5601
    Commented Jan 31, 2020 at 1:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.