13
$\begingroup$

Is there a built-in function for decoding encoded URLs in Mathematica? I would like to convert an encoded URL, for example https%3A%2F%2Fwww.google.co.uk%2Fimages%2Fsrpr%2Flogo4w.png into https://www.google.co.uk/images/srpr/logo4w.png as attempting to directly access the encoded URL with Import[] fails.

|improve this question
$\endgroup$
8
$\begingroup$

Works nicely:

PercentDecode[s_String] := StringReplace[s, RegularExpression["%([[:xdigit:]]{2})"] :> 
                                         FromCharacterCode[FromDigits["$1", 16]]]

PercentEncode[s_String] := StringReplace[s, {" " -> "+", 
   RegularExpression["[^\\w/\\?=:._~]"] :> 
    StringJoin["%", ToUpperCase[IntegerString[First[ToCharacterCode["$0"]], 16]]]}]

Test:

PercentDecode["http://www.test.com/~a_b?i=Cos%5B2%5D"]
   "http://www.test.com/~a_b?i=Cos[2]"

PercentEncode["http://www.test.com/~a_b?i=Cos[2]"]
   "http://www.test.com/~a_b?i=Cos%5B2%5D"
|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for the function! Works nicely. $\endgroup$ – March Ho Apr 4 '13 at 12:39
  • $\begingroup$ I'm not an expert for this, and the relevant standards have changed once in a while. But concerning PercentEncode maybe is worth noting that as far as I understand the decoding should always give unambiguous results while the encoding is somewhat arbitrary: it has not always been exactly specified which characters have to be encoded and which not, the latest RFCs about this seem to be more strict. So encoding with different functions/services might give different results, but I don't think that means there is an error as long as the decoding results in the same string. $\endgroup$ – Albert Retey Apr 4 '13 at 12:51
  • $\begingroup$ @Albert: True, for the encoding I went with the characters that are frequently encoded by browsers. It should not be too hard to modify PercentEncode[], of course. $\endgroup$ – J. M. will be back soon♦ Apr 4 '13 at 12:55
  • $\begingroup$ @J.M.: to my understanding encoding is necessary for exactly those characters that are not "unreserved", everything else must be encoded if they are not to be interpreted with their "reserved" special meaning. So whether or not a character needs to be encoded depends on what they should be interpreted, if e.g. you want to send an url as an argument to a cgi script you would (unlike your PercentEncode does) need to encode "/". The uncertainty seems to be with all those characters that are not explicitly "reserved" or "unreserved"... $\endgroup$ – Albert Retey Apr 4 '13 at 14:05
16
$\begingroup$

There's also an internal function that does it:

ExternalService`DecodeString["https%3A%2F%2Fwww.google.co.uk%2Fimages%2Fsrpr%2Flogo4w.png"]

At one time, the implementation of that function used J/Link code similar to Albert's code above, but now it is done in top-level Mathematica.

ExternalService`EncodeString does URL-encoding, if you want to go in the other direction.

|improve this answer
$\endgroup$
10
$\begingroup$

There might be undocumented internal functions burried somewhere to do this, but it is relatively straightforward to get a result via JLink:

Needs["JLink`"];
InstallJava[];
LoadJavaClass["java.net.URLDecoder"];
url = "https%3A%2F%2Fwww.google.co.uk%2Fimages%2Fsrpr%2Flogo4w.png";
java`net`URLDecoder`decode[url]

see the Java documentation for more information about the java.net package and the java.net.URLDecoder class. The class java.net.URLEncoder might also be of interest...

With the information in e.g. wikipedia about "percent encoding" and the links to the relevant RFCs it is actually not difficult to build something based on StringReplace -- as J.M.'s answer shows in a very elegant way.

|improve this answer
$\endgroup$
8
$\begingroup$

URLDecode and URLEncode were introduced in Mathematica 10.0:

url = "https%3A%2F%2Fwww.google.co.uk%2Fimages%2Fsrpr%2Flogo4w.png";
URLDecode[url]
(*"https://www.google.co.uk/images/srpr/logo4w.png"*)

The symbol definition can be accessed as follows, which is scarily close to J. M.'s code above. 

URLDecode[url];
Unprotect[URLDecode];
ClearAttributes[URLDecode, ReadProtected];
DownValues[URLDecode][[2]]

(*HoldPattern[
  URLDecode[URLUtilities`Encoding`PackagePrivate`s_String, 
   OptionsPattern[]]] :> 
 FromCharacterCode[
  List @@ StringReplace[
    URLUtilities`Encoding`PackagePrivate`s, {RegularExpression[
       "%([0-9a-fA-F]{2})"] :> FromDigits["$1", 16], 
     "+" :> First[ToCharacterCode[" "]], 
     URLUtilities`Encoding`PackagePrivate`char_ :> 
      First[ToCharacterCode[
        URLUtilities`Encoding`PackagePrivate`char]]}], 
  OptionValue[CharacterEncoding]]*)
|improve this answer
$\endgroup$
  • 2
    $\begingroup$ Aww, I'm chuffed that I beat them to the implementation… :D The regex they picked is rather clunky-looking, tho. $\endgroup$ – J. M. will be back soon♦ Aug 12 '15 at 1:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.