9
$\begingroup$

With Wolfram Language:

In[7]:= ExportString["foobar中文", "Base64"]

Out[7]= "Zm9vYmFyXDo0ZTJkXDo2NTg3"

With the closest command-line software I could think of

echo -n 'foobar中文' | openssl base64                                                                       
Zm9vYmFy5Lit5paH

Why the difference? What's the best way to reproduce Mathematica's behavior?

$\endgroup$
0
17
$\begingroup$

When you use the "Base64" you are encoding a particular file format. For Export calls the file format is inferred from the file name given, like Export["file.ext.b64", expr] uses the ext format.

When used with ExportString, the format is chosen automatically. You can see here that the automatically chosen format is "String":

ExportString["foobar中文", {"Base64", "String"}] === 
 ExportString["foobar中文", "Base64"]
(* True *)

To understand the output you need to understand what string is being encoded:

ExportString["foobar中文", "String"]
(* "foobar\\:4e2d\\:6587" *)

If I take that and encode it from the command line we have a match:

echo -n 'foobar\:4e2d\:6587' | openssl base64 
Zm9vYmFyXDo0ZTJkXDo2NTg3

You should think of the "Base64" export format as an extension of Export and ExportString, they first export to a string and then encode.

For a pure string-to-base64 you can use BaseEncode and StringToByteArray:

BaseEncode@StringToByteArray@"foobar中文"
(* "Zm9vYmFy5Lit5paH" *)
$\endgroup$
7
  • 2
    $\begingroup$ Can you perhaps also explain why becomes XDo0ZTJk? When using ExportString["中", "Base64"] I don't get that result no matter how I encode and convert it to base64. $\endgroup$
    – Max1
    May 7 '20 at 21:03
  • 1
    $\begingroup$ Try echo -n '\:4e2d' | openssl base64 $\endgroup$
    – Jason B.
    May 7 '20 at 21:34
  • 1
    $\begingroup$ @Max1 I edited the answer to the best of my knowledge. I made the answer a community wiki so people will feel free to edit it. Character encoding and everything related has always been a blind spot of mine. $\endgroup$
    – Jason B.
    May 8 '20 at 1:37
  • 1
    $\begingroup$ Thanks for your great answer! I think I also get now what happens. The string \:4e2d is literally translated to an UTF-8 byte sequence. Afterwards every 3 bytes of data will be translated to 4 6-bit characters (since base 64 is 6 bit). Since every element of \:4e2d is an element of the ASCII set they will all be encoded with 1 byte. In total we have a 6 byte UTF-8 sequence which is translated to 8 characters in a 6-bit alphabet (base64). If I find some time I might add the conversion \:4e2d -> UTF-8 bytes -> base64 characters to the community wiki. $\endgroup$
    – Max1
    May 8 '20 at 5:52
  • 2
    $\begingroup$ Take my upvote :) $\endgroup$ May 8 '20 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.