# Check Google PageRank for a given URL

Does anyone know how to write a Mathematica function that returns the Google PageRank of a given URL?

I found a tutorial for achieving this in a bash script here, but I have no idea how to port this over to Mathematica.

-

I am not sure what exactly that rank mean. But here's direct rough porting of code:

ConvertStrToInt[url_String, init_, factor_] :=
Fold[FromDigits[IntegerDigits[#1*factor + #2, 16, 8], 16] &, init,
ToCharacterCode[url]];

HashURL[url_String]:=
Block[{c1, c2,t1, t2},
c1=ConvertStrToInt[url, 5381,33];
c2=ConvertStrToInt[url, 0,65599];
c1=BitShiftRight[c1,2];
c1=BitOr[BitAnd[BitShiftRight[c1,4],67108800],BitAnd[c1,63]];
c1=BitOr[BitAnd[BitShiftRight[c1,4],4193280],BitAnd[c1,1023]];
c1=BitOr[BitAnd[BitShiftRight[c1,4],245760],BitAnd[c1,16383]];

t1=BitShiftLeft[BitAnd[c1,960],4];
t1=BitOr[t1,BitAnd[c1,60]];
t1=BitOr[BitShiftLeft[t1,2],BitAnd[c2,3855]];

t2=BitShiftLeft[BitAnd[c1,4294950912],4];
t2=BitOr[t2,BitAnd[c1,15360]];
t2 = BitOr[BitShiftLeft[t2,10],BitAnd[c2,252641280]];

BitOr[t1,t2]
]

CheckHash[int_] :=
Block[{check, flag, remainder, hashInt},
hashInt = int;
flag = False;
check = 0;
NestWhile[
(remainder = Mod[#, 10];
hashInt = Quotient[#, 10];
If[flag, remainder += remainder;
remainder = Quotient[remainder, 10] + Mod[remainder, 10]];
check += remainder;
flag = ! flag;
hashInt
) &, int, (# != 0 &)];

check = Mod[check, 10];
If[check != 0,
check = 10 - check;
If[flag,
If[1 == Mod[check, 2], check += 9];
check = BitShiftRight[check, 1];
]
];
FromCharacterCode[check + 48]
]

Block[{checksum, hashInt, page},
hashInt = HashURL[url];
checksum =
StringJoin[{"7", CheckHash[hashInt], ToString[hashInt]}];
page = StringJoin[{"http://toolbarqueries.google.com/tbr?client=navclient-auto&ch=", checksum,
"&ie=UTF-8&oe=UTF-8&features=Rank&q=info:",
StringReplace[url, {":" -> "%3A", "/" -> "%2F"}]}];
Import[page]
]

Out[485]= "Rank_1:1:9"

-

Seeing as how someone has been nice enough to write the C-code for you, you could just use that. Assuming you have a C compiler on your machine, here's how you use it within Mathematica. Note that code is defined below.

(* Be sure to define code first! *)
Needs["CCompilerDriver"];
url = "http://www.stackexchange.com/";
checksum = CreateExecutable[code, "checksum"];
thisChecksum = Import["!" <> QuoteFile[checksum] <> " " <> url, "Text"]

(* Out: "792928720298" *)


I guess that the point of this thing is that it can be used in conjunction with Google's API to get very rough info on a sites rank. We can use it like so.

queryURL = "http://toolbarqueries.google.com/tbr";
queryURL = queryURL <> "?client=navclient-auto&features=Rank&ch=";
queryURL = queryURL <> thisChecksum <> "&q=info:" <> url;
Import[queryURL]

(* Out: "Rank_1:1:6" *)


This could be to generate an image like so:

Here's code. It's almost exactly the code referenced by the OP - I just escaped a few characters in the string literals.

code = "
#include <stdio.h>
#include <stdbool.h>

int ConvertStrToInt(char *pStr, int Init, int Factor)
{
while (*pStr) {
Init *= Factor;
Init += *pStr++;
}
return Init;
}

int HashURL(char *pStr)
{
unsigned int C1, C2, T1, T2;

C1 = ConvertStrToInt(pStr, 0x1505, 0x21);
C2 = ConvertStrToInt(pStr, 0, 0x1003F);
C1 >>= 2;
C1 = ((C1 >> 4) & 0x3FFFFC0) | (C1 & 0x3F);
C1 = ((C1 >> 4) & 0x3FFC00) | (C1 & 0x3FF);
C1 = ((C1 >> 4) & 0x3C000) | (C1 & 0x3FFF);

T1 = (C1 & 0x3C0) << 4;
T1 |= C1 & 0x3C;
T1 = (T1 << 2) | (C2 & 0xF0F);

T2 = (C1 & 0xFFFFC000) << 4;
T2 |= C1 & 0x3C00;
T2 = (T2 << 0xA) | (C2 & 0xF0F0000);

return (T1 | T2);
}

char CheckHash(unsigned int HashInt)
{
int Check = 0;
bool Flag = false;
int Remainder;

do {
Remainder = HashInt % 10;
HashInt /= 10;
if (Flag){
Remainder += Remainder;
Remainder = (Remainder / 10) + (Remainder % 10);
}
Check += Remainder;
Flag = !Flag;
} while( 0 != HashInt);

Check %= 10;
if (0 != Check) {
Check = 10 - Check;
if (Flag) {
if (1 == (Check % 2)) {
Check += 9;
}
Check >>= 1;
}
}
Check += 0x30;
return Check;
}

int main(int argc, char* argv[])
{
unsigned int HashInt;

HashInt = HashURL(argv[1]);
printf(\"7%c%u\\n\", CheckHash(HashInt), HashInt);
return 0;
}
";


Incidentally, another very nice tool for obtaining page rank information is via WolframAlpha:

WolframAlpha[
"www.stackexchange.com", {{"WebSiteStatisticsPod:InternetData", 1},
"ComputableData"}] // Column


EDIT:

If you're on Windows platform and you're using MSVS compiler, there is no C99 support. That is why above code will fail to compile, because of the lack of stdbool.h

A workaround for this is:

code = "
#include <stdio.h>
#ifdef _Win32
typedef int bool;
#define false 0
#define true 1
#else
#include <stdbool.h>
#endif
// ...

Needs["CCompilerDriver"];


Now you need to specify the _Win32 preprocessor definition, in order to exclude the stdbool.h block:

checksum = CreateExecutable[code, "checksum", Defines -> {"_Win32"}];

-
@Stefan Thanks - Wish I could vote for you! – Mark McClure Mar 12 '13 at 14:22
You could vote for my -insane-speedlight-dayOfWeek-idontfearleaks- implementation instead ;) Thank you for your answer. I wouldn't have come up to your idea for solving this. Your answer is pretty decent; in my eyes. – Stefan Mar 12 '13 at 18:52
@Stefan Done! If it's good enough for Leonid, then it's certainly good enough for me. Incidentally, I used to have MSVS on Windows in a VirtualBox on my Mac, just so I could get my C code working on colleagues computers. I had forgotten what a pain working with preprocessor could be, even when working with very low level code - no GUI or system interface type stuff at all. – Mark McClure Mar 13 '13 at 0:07
Thanks a lot for the detailed answer. I checked this out on my home PC and it indeed works. I am going to accept halmir's response as I do not have a C compiler available on the office machine and his solution has no external dependency. – Ubiquitous Mar 15 '13 at 9:13