Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

I would like to import data from this page through Mathematica,

http://www.oddsportal.com/soccer/netherlands/eredivisie/ .

I tried importing through the default Import function

d = Import["http://www.oddsportal.com/soccer/netherlands/eredivisie/","Data"[[3, 1, 1, 2]];
Select[d, ! StringFreeQ[#[[1]], " - "] &] // Quiet // TableForm

I got a simular table like this where all the interessting data is replaced by a '-'.

Heracles - Cambuur      -   -   -
Willem II - AZ Alkmaar  -   -   -
Heerenveen - Excelsior  -   -   -
G.A. Eagles - Dordrecht -   -   -
Zwolle - Vitesse        -   -   -
Breda - Twente          -   -   -
Den Haag - Groningen    -   -   -
Feyenoord - Utrecht     -   -   -
Ajax - PSV              -   -   -

I've been told this is because the webpage updates the data through a JavaScript, that fetches the information form a database. So is there a way of letting the webpage load it's data before importing it?

Futhermore, I found a solution for Windows users, where the webpage is loaded with Internet Explorer by NETLink. But since I mostly use a OSX machine, this won't work for me.

Needs["NETLink`"];
InstallNET[];
ie = CreateCOMObject["InternetExplorer.Application"];
ie@Navigate[
"http://www.oddsportal.com/soccer/netherlands/eredivisie/"];
Pause[3];
doc = ie@Document;
html = doc@body@innerHTML;

This made able to construct a table with all the data, after combing through the html code.

AZ Alkmaar - Ajax       2.96    3.47    2.26
Excelsior - G.A. Eagles 2.38    3.37    2.86
Utrecht - Willem II     2.19    3.4     3.16
Groningen - Heracles    1.7     4.      4.41

So again, is there a way of executing the JavaScript on a page before importing the data? Maybe through a JLink equivalent of the code above?

share|improve this question

1 Answer 1

In this case it would be far more reliable to request the information from the source directly. You can find the source using your web browser's developer tool, in Google Chrome you find this information under "Network." It should look something like:

source

Importing that data as JSON gives us a nested list of rules containing all the data:

rawData = Import["http://fb.oddsportal.com/ajax-sport-country-tournament/1/d0aaiJN1/X0/1/?_=1408357250275"];
data = ImportString[StringReplace[rawData, "globals.jsonpCallback('/ajax-sport-country-tournament/1/d0aaiJN1/X0/1/', " ~~ d__ ~~ ");" :> d], "JSON"]

Each game has its own ID that is used in the data source. You can find the ID of the game by looking at the URL of the game's page, for example we have

http://www.oddsportal.com/soccer/netherlands/eredivisie/heracles-cambuur-h2uJFB4h/

which tells us that the ID of the Heracles-Cambuur game is h2uJFB4h.

share|improve this answer
    
This would indeed be the more reliable solution. The only thing I'm concerned with is, whether or not the source URL changes over a timespan. But in anycase, this works like charme, so thank you! –  user19218 Aug 18 at 11:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.