Check Google PageRank for a given URL

Question

Does anyone know how to write a Mathematica function that returns the Google PageRank of a given URL?

I found a tutorial for achieving this in a bash script here, but I have no idea how to port this over to Mathematica.

Answer 1

I am not sure what exactly that rank mean. But here's direct rough porting of code:

ConvertStrToInt[url_String, init_, factor_] :=
 Fold[FromDigits[IntegerDigits[#1*factor + #2, 16, 8], 16] &, init, 
  ToCharacterCode[url]];

HashURL[url_String]:=
    Block[{c1, c2,t1, t2},
     c1=ConvertStrToInt[url, 5381,33];
     c2=ConvertStrToInt[url, 0,65599];
     c1=BitShiftRight[c1,2];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],67108800],BitAnd[c1,63]];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],4193280],BitAnd[c1,1023]];
     c1=BitOr[BitAnd[BitShiftRight[c1,4],245760],BitAnd[c1,16383]];

     t1=BitShiftLeft[BitAnd[c1,960],4];
     t1=BitOr[t1,BitAnd[c1,60]];
     t1=BitOr[BitShiftLeft[t1,2],BitAnd[c2,3855]];

     t2=BitShiftLeft[BitAnd[c1,4294950912],4];
     t2=BitOr[t2,BitAnd[c1,15360]];
     t2 = BitOr[BitShiftLeft[t2,10],BitAnd[c2,252641280]];

     BitOr[t1,t2]
 ]

CheckHash[int_] :=
  Block[{check, flag, remainder, hashInt},
   hashInt = int;
   flag = False;
   check = 0;
   NestWhile[
    (remainder = Mod[#, 10];
      hashInt = Quotient[#, 10];
      If[flag, remainder += remainder;
       remainder = Quotient[remainder, 10] + Mod[remainder, 10]];
       check += remainder;
      flag = ! flag;
      hashInt
      ) &, int, (# != 0 &)];

   check = Mod[check, 10];
   If[check != 0,
    check = 10 - check;
    If[flag,
     If[1 == Mod[check, 2], check += 9];
     check = BitShiftRight[check, 1];
     ]
    ];
   FromCharacterCode[check + 48]
  ]

GooglePageRanking[url_String] :=
  Block[{checksum, hashInt, page},
   hashInt = HashURL[url];
   checksum = 
    StringJoin[{"7", CheckHash[hashInt], ToString[hashInt]}];
   page = StringJoin[{"http://toolbarqueries.google.com/tbr?client=navclient-auto&ch=", checksum, 
      "&ie=UTF-8&oe=UTF-8&features=Rank&q=info:", 
      StringReplace[url, {":" -> "%3A", "/" -> "%2F"}]}];
   Import[page]
   ]

In[485]:= GooglePageRanking["http://www.google.com"]

Out[485]= "Rank_1:1:9"

Answer 2

Seeing as how someone has been nice enough to write the C-code for you, you could just use that. Assuming you have a C compiler on your machine, here's how you use it within Mathematica. Note that code is defined below.

(* Be sure to define code first! *)
Needs["CCompilerDriver`"];
url = "http://www.stackexchange.com/";
checksum = CreateExecutable[code, "checksum"];
thisChecksum = Import["!" <> QuoteFile[checksum] <> " " <> url, "Text"]

(* Out: "792928720298" *)

I guess that the point of this thing is that it can be used in conjunction with Google's API to get very rough info on a sites rank. We can use it like so.

queryURL = "http://toolbarqueries.google.com/tbr";
queryURL = queryURL <> "?client=navclient-auto&features=Rank&ch=";
queryURL = queryURL <> thisChecksum <> "&q=info:" <> url;
Import[queryURL]

(* Out: "Rank_1:1:6" *)

This could be to generate an image like so:

Here's code. It's almost exactly the code referenced by the OP - I just escaped a few characters in the string literals.

code = "
#include <stdio.h>
#include <stdbool.h>

int ConvertStrToInt(char *pStr, int Init, int Factor)
{
    while (*pStr) {
        Init *= Factor;
        Init += *pStr++;
    }
    return Init;
}

int HashURL(char *pStr)
{
    unsigned int C1, C2, T1, T2;

    C1 = ConvertStrToInt(pStr, 0x1505, 0x21);
    C2 = ConvertStrToInt(pStr, 0, 0x1003F);
    C1 >>= 2;
    C1 = ((C1 >> 4) & 0x3FFFFC0) | (C1 & 0x3F);
    C1 = ((C1 >> 4) & 0x3FFC00) | (C1 & 0x3FF);
    C1 = ((C1 >> 4) & 0x3C000) | (C1 & 0x3FFF);

    T1 = (C1 & 0x3C0) << 4;
    T1 |= C1 & 0x3C;
    T1 = (T1 << 2) | (C2 & 0xF0F);

    T2 = (C1 & 0xFFFFC000) << 4;
    T2 |= C1 & 0x3C00;
    T2 = (T2 << 0xA) | (C2 & 0xF0F0000);

    return (T1 | T2);
}

char CheckHash(unsigned int HashInt)
{
    int Check = 0;
    bool Flag = false;
    int Remainder;

    do {
        Remainder = HashInt % 10;
        HashInt /= 10;
        if (Flag){
            Remainder += Remainder;
            Remainder = (Remainder / 10) + (Remainder % 10);
        }
        Check += Remainder;
        Flag = !Flag;
    } while( 0 != HashInt);

    Check %= 10;
    if (0 != Check) {
        Check = 10 - Check;
        if (Flag) {
            if (1 == (Check % 2)) {
                Check += 9;
            }
            Check >>= 1;
        }
    }
    Check += 0x30;
    return Check;
}

int main(int argc, char* argv[])
{
    unsigned int HashInt;

    HashInt = HashURL(argv[1]);
    printf(\"7%c%u\\n\", CheckHash(HashInt), HashInt);
    return 0;
}
";

Incidentally, another very nice tool for obtaining page rank information is via WolframAlpha:

WolframAlpha[
  "www.stackexchange.com", {{"WebSiteStatisticsPod:InternetData", 1},
  "ComputableData"}] // Column

EDIT:

If you're on Windows platform and you're using MSVS compiler, there is no C99 support. That is why above code will fail to compile, because of the lack of stdbool.h

A workaround for this is:

code = "
#include <stdio.h>
#ifdef _Win32
typedef int bool;
#define false 0
#define true 1
#else
#include <stdbool.h>
#endif
// ...

Needs["CCompilerDriver`"];

Now you need to specify the _Win32 preprocessor definition, in order to exclude the stdbool.h block:

checksum = CreateExecutable[code, "checksum", Defines -> {"_Win32"}];