Take the 2-minute tour ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.

Does anyone know how to write a Mathematica function that returns the Google PageRank of a given URL?

I found a tutorial for achieving this in a bash script here, but I have no idea how to port this over to Mathematica.

share|improve this question
add comment

2 Answers 2

up vote 11 down vote accepted

I am not sure what exactly that rank mean. But here's direct rough porting of code:

ConvertStrToInt[url_String, init_, factor_] :=
 Fold[FromDigits[IntegerDigits[#1*factor + #2, 16, 8], 16] &, init, 
  ToCharacterCode[url]];

HashURL[url_String]:=
    Block[{c1, c2,t1, t2},
     c1=ConvertStrToInt[url, 5381,33];
     c2=ConvertStrToInt[url, 0,65599];
     c1=BitShiftRight[c1,2];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],67108800],BitAnd[c1,63]];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],4193280],BitAnd[c1,1023]];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],245760],BitAnd[c1,16383]];

     t1=BitShiftLeft[BitAnd[c1,960],4];
     t1=BitOr[t1,BitAnd[c1,60]];
     t1=BitOr[BitShiftLeft[t1,2],BitAnd[c2,3855]];

     t2=BitShiftLeft[BitAnd[c1,4294950912],4];
     t2=BitOr[t2,BitAnd[c1,15360]];
     t2 = BitOr[BitShiftLeft[t2,10],BitAnd[c2,252641280]];

     BitOr[t1,t2]
 ]

CheckHash[int_] :=
  Block[{check, flag, remainder, hashInt},
   hashInt = int;
   flag = False;
   check = 0;
   NestWhile[
    (remainder = Mod[#, 10];
      hashInt = Quotient[#, 10];
      If[flag, remainder += remainder;
       remainder = Quotient[remainder, 10] + Mod[remainder, 10]];
       check += remainder;
      flag = ! flag;
      hashInt
      ) &, int, (# != 0 &)];

   check = Mod[check, 10];
   If[check != 0,
    check = 10 - check;
    If[flag,
     If[1 == Mod[check, 2], check += 9];
     check = BitShiftRight[check, 1];
     ]
    ];
   FromCharacterCode[check + 48]
  ]

GooglePageRanking[url_String] :=
  Block[{checksum, hashInt, page},
   hashInt = HashURL[url];
   checksum = 
    StringJoin[{"7", CheckHash[hashInt], ToString[hashInt]}];
   page = StringJoin[{"http://toolbarqueries.google.com/tbr?client=navclient-auto&ch=", checksum, 
      "&ie=UTF-8&oe=UTF-8&features=Rank&q=info:", 
      StringReplace[url, {":" -> "%3A", "/" -> "%2F"}]}];
   Import[page]
   ]

In[485]:= GooglePageRanking["http://www.google.com"]

Out[485]= "Rank_1:1:9"
share|improve this answer
add comment

Seeing as how someone has been nice enough to write the C-code for you, you could just use that. Assuming you have a C compiler on your machine, here's how you use it within Mathematica. Note that code is defined below.

(* Be sure to define code first! *)
Needs["CCompilerDriver`"];
url = "http://www.stackexchange.com/";
checksum = CreateExecutable[code, "checksum"];
thisChecksum = Import["!" <> QuoteFile[checksum] <> " " <> url, "Text"]

(* Out: "792928720298" *)

I guess that the point of this thing is that it can be used in conjunction with Google's API to get very rough info on a sites rank. We can use it like so.

queryURL = "http://toolbarqueries.google.com/tbr";
queryURL = queryURL <> "?client=navclient-auto&features=Rank&ch=";
queryURL = queryURL <> thisChecksum <> "&q=info:" <> url;
Import[queryURL]

(* Out: "Rank_1:1:6" *)

This could be to generate an image like so:

enter image description here

Here's code. It's almost exactly the code referenced by the OP - I just escaped a few characters in the string literals.

code = "
#include <stdio.h>
#include <stdbool.h>

int ConvertStrToInt(char *pStr, int Init, int Factor)
{
    while (*pStr) {
        Init *= Factor;
        Init += *pStr++;
    }
    return Init;
}

int HashURL(char *pStr)
{
    unsigned int C1, C2, T1, T2;

    C1 = ConvertStrToInt(pStr, 0x1505, 0x21);
    C2 = ConvertStrToInt(pStr, 0, 0x1003F);
    C1 >>= 2;
    C1 = ((C1 >> 4) & 0x3FFFFC0) | (C1 & 0x3F);
    C1 = ((C1 >> 4) & 0x3FFC00) | (C1 & 0x3FF);
    C1 = ((C1 >> 4) & 0x3C000) | (C1 & 0x3FFF);

    T1 = (C1 & 0x3C0) << 4;
    T1 |= C1 & 0x3C;
    T1 = (T1 << 2) | (C2 & 0xF0F);

    T2 = (C1 & 0xFFFFC000) << 4;
    T2 |= C1 & 0x3C00;
    T2 = (T2 << 0xA) | (C2 & 0xF0F0000);

    return (T1 | T2);
}

char CheckHash(unsigned int HashInt)
{
    int Check = 0;
    bool Flag = false;
    int Remainder;

    do {
        Remainder = HashInt % 10;
        HashInt /= 10;
        if (Flag){
            Remainder += Remainder;
            Remainder = (Remainder / 10) + (Remainder % 10);
        }
        Check += Remainder;
        Flag = !Flag;
    } while( 0 != HashInt);

    Check %= 10;
    if (0 != Check) {
        Check = 10 - Check;
        if (Flag) {
            if (1 == (Check % 2)) {
                Check += 9;
            }
            Check >>= 1;
        }
    }
    Check += 0x30;
    return Check;
}

int main(int argc, char* argv[])
{
    unsigned int HashInt;

    HashInt = HashURL(argv[1]);
    printf(\"7%c%u\\n\", CheckHash(HashInt), HashInt);
    return 0;
}
";

Incidentally, another very nice tool for obtaining page rank information is via WolframAlpha:

WolframAlpha[
  "www.stackexchange.com", {{"WebSiteStatisticsPod:InternetData", 1},
  "ComputableData"}] // Column

enter image description here

EDIT:

If you're on Windows platform and you're using MSVS compiler, there is no C99 support. That is why above code will fail to compile, because of the lack of stdbool.h

A workaround for this is:

code = "
#include <stdio.h>
#ifdef _Win32
typedef int bool;
#define false 0
#define true 1
#else
#include <stdbool.h>
#endif
// ...

Needs["CCompilerDriver`"];

Now you need to specify the _Win32 preprocessor definition, in order to exclude the stdbool.h block:

checksum = CreateExecutable[code, "checksum", Defines -> {"_Win32"}];
share|improve this answer
    
@Stefan Thanks - Wish I could vote for you! –  Mark McClure Mar 12 '13 at 14:22
    
You could vote for my -insane-speedlight-dayOfWeek-idontfearleaks- implementation instead ;) Thank you for your answer. I wouldn't have come up to your idea for solving this. Your answer is pretty decent; in my eyes. –  Stefan Mar 12 '13 at 18:52
    
@Stefan Done! If it's good enough for Leonid, then it's certainly good enough for me. Incidentally, I used to have MSVS on Windows in a VirtualBox on my Mac, just so I could get my C code working on colleagues computers. I had forgotten what a pain working with preprocessor could be, even when working with very low level code - no GUI or system interface type stuff at all. –  Mark McClure Mar 13 '13 at 0:07
    
Thanks a lot for the detailed answer. I checked this out on my home PC and it indeed works. I am going to accept halmir's response as I do not have a C compiler available on the office machine and his solution has no external dependency. –  Ubiquitous Mar 15 '13 at 9:13
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.