Creating a hyperlink graph from Wikipedia

Question

This goes out to all the Mathematica experts - I'm hoping to get some helpful comments, ideas or code for that matter.

In my attempts to maximize learning efficiency I had the idea to create an (undirected) graph of Wikipedia hyperlinks of a specific topic (e.g. probability theory).

This then could be used to identify central nodes/topics which are necessary (i.e. highly connected to other nodes) to understand the rest of the topic more easily, instead of more or less arbitrarily browsing through Wikipedia. For example would it be more efficient to start from a central node/topic since other topics are based on an understanding of this node/topic.

I have trouble reading out the hyperlinks from Wikipedia (e.g. up to a some depth) and I don't even know how to start with the graph, so any help (or examples) to get me started would be appreciated.

Answer 1

Yep, as @Sjoerd mentions in the comments, 10.1 includes the WikipediaData functionality. You can use like this:

links = WikipediaData["Mathematica", "BacklinksRules", "MaxLevelItems" -> 10, "MaxLevel" -> 2]
Graph[links, VertexLabels -> Placed["Name", Tooltip], VertexStyle -> {"Mathematica" -> Red}]

Answer 2

If you wanted to do this same kind of parsing for a website for which a built-in function was not available, you could read in the contents of the web page (using URLFetch) and then search this for links, which usually start with "href". For example

str = "https://en.wikipedia.org/wiki/Main_Page";
q = URLFetch[str];
ind = Position[StringPosition[StringSplit[q], "href"], {1, 4}][[All, 1]];
StringSplit[q][[#]] & /@ ind

creates a long list showing all the links on the page indicated in str, in this case, on the main page of wikipedia.