# Creating a hyperlink graph from Wikipedia

This goes out to all the Mathematica experts - I'm hoping to get some helpful comments, ideas or code for that matter.

In my attempts to maximize learning efficiency I had the idea to create an (undirected) graph of Wikipedia hyperlinks of a specific topic (e.g. probability theory).

This then could be used to identify central nodes/topics which are necessary (i.e. highly connected to other nodes) to understand the rest of the topic more easily, instead of more or less arbitrarily browsing through Wikipedia. For example would it be more efficient to start from a central node/topic since other topics are based on an understanding of this node/topic.

I have trouble reading out the hyperlinks from Wikipedia (e.g. up to a some depth) and I don't even know how to start with the graph, so any help (or examples) to get me started would be appreciated.

• Version 10.1 has WikipediaData that can do this. Jun 5, 2015 at 13:24
• I guess you can make graphs that always lead to Philosophy! Jun 5, 2015 at 20:28

Yep, as @Sjoerd mentions in the comments, 10.1 includes the WikipediaData functionality. You can use like this:

links = WikipediaData["Mathematica", "BacklinksRules", "MaxLevelItems" -> 10, "MaxLevel" -> 2]
Graph[links, VertexLabels -> Placed["Name", Tooltip], VertexStyle -> {"Mathematica" -> Red}]


• Unfortunately only have Mathematica 9, but now I need M10 :D..thanks! Jun 5, 2015 at 14:26

If you wanted to do this same kind of parsing for a website for which a built-in function was not available, you could read in the contents of the web page (using URLFetch) and then search this for links, which usually start with "href". For example

str = "https://en.wikipedia.org/wiki/Main_Page";
q = URLFetch[str];
ind = Position[StringPosition[StringSplit[q], "href"], {1, 4}][[All, 1]];
StringSplit[q][[#]] & /@ ind


creates a long list showing all the links on the page indicated in str, in this case, on the main page of wikipedia.

• For external links, Import["https://en.wikipedia.org/wiki/Main_Page" , "Hyperlinks"] should work too Jan 7, 2019 at 20:25