17
$\begingroup$

If you run this code on 11.2

    hmacF[method_, message_, key_] := Module[

  {keyLen, dkey, opad, ipad, blocksize},

  blocksize =
   If[method === "SHA384" || method === "SHA512",
    128,
    64
    ];

  keyLen = StringLength[key];

  dkey = If[keyLen > blocksize,
    IntegerString[Hash[key, method], 16],
    StringPadRight[key, blocksize, FromCharacterCode[0]]
    ];

  {opad, ipad} = 
   FromCharacterCode[
      BitXor[ToCharacterCode@
        StringRepeat[FromCharacterCode[FromDigits[#, 16]], blocksize],
        ToCharacterCode@dkey]] & /@ {"5c", "36"};

  IntegerString[
   Hash[
    StringJoin[opad, 
     ExportString[
      IntegerDigits[Hash[StringJoin[ipad, message], method], 256], 
      "Binary"]],
    method],
   16, blocksize]
  ]

On something we know the answer to https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md

hmacF["SHA256","symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559","NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

you get the correct answer

(*c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71*)

Run the same code on 11.3 and you get

(*88fdd5c8c2226ff025e290b37dec36d4bc6f84affb9928950fd865f9a0fd475e*)

Why the difference?? 11.3 is not the correct HMAC result.

$\endgroup$
  • $\begingroup$ Well break the function down and find out which piece changed between versions... it's likely a single function giving a different output, my guess is Hash... $\endgroup$ – user6014 Mar 11 '18 at 1:38
  • $\begingroup$ I tested it, 11.2 on desktop, 11.3 on cloud, and yes, Hash seems to have changed in some manner? If you think there is a bug, minimize it and report the issue to support@wolfram.com. $\endgroup$ – user6014 Mar 11 '18 at 1:44
  • $\begingroup$ Looking at the Hash documentation page, there does seem to have been a handful of changes to the function for 11.3. Perhaps some of these adversely affected your code. $\endgroup$ – user6014 Mar 11 '18 at 1:46
  • $\begingroup$ @user6014 I did read the documentation. None of the changes to Hash should have changed the output, hence my post. In your opinion, is it a bug? Does my function need to be written differently in 11.3? There should be no reason why the function needs to change. $\endgroup$ – Ray Troy Mar 11 '18 at 2:00
  • 2
    $\begingroup$ Here is the minimal example demonstrating the difference: Hash[ExportString[{217, 73, 200, 195, 231, 110, 83, 36, 139, 50, 62, 251, 63, 221, 85, 205, 192, 34, 244, 161, 198, 42, 132, 228, 7, 215, 112, 249, 161, 228, 204, 113}, "Binary"], "SHA256"] $\endgroup$ – user6014 Mar 11 '18 at 2:04
21
$\begingroup$

Indeed, Hash has changed for 11.3 in several ways.

The one relevant here is that strings are now hashed according to their UTF-8 representation.

As a simple example of the difference, consider the following string, which has only one character.

c = FromCharacterCode[217]; (* "\[CapitalUGrave]" *)

Its UTF-8 representations consists of the two (decimal) bytes {195, 153}, so in version 11.3 the hash would be computed from those two bytes, while in 11.2 the hash would be a computed from the single (decimal) byte 217.

Then the two corresponding hash results would be different, namely

(* new hash value *)
Hash[c, "SHA256"]
Hash[StringToByteArray[c], "SHA256"] 
Hash[ByteArray[{195, 153}], "SHA256"]

(* 115129295529739515318262490854329296079790445441298973994137838100166123599005 *)
(* or as a hex string: fe88df3f083e08f99f796b5ba7f6eedbe14d45241cf5e0944caf17f55ac2349d *)

which agrees with various online hash calculators, e.g. http://new.md5calc.com/hash-calc/sha256/%C3%99 and

(* old hash value *)
Developer`LegacyHash[c, "SHA256"]
Hash[StringToByteArray[c, "ISO8859-1"], "SHA256"]
Hash[ByteArray[{217}], "SHA256"]

(* 11345241613194184164858110515603144575751235911333201083896508434490695406828 *)
(* or as a hex string: 19152ddfba193b5b09fcb80d1bba5248f36027c06e81670db5a7146fb654d4ec *)

where all inputs were evaluated with 11.3.0.

The hmacF implementation uses strings in a somewhat awkward way as data containers. It is easier to take advantage of several new Hash enhancements instead, like operating on byte arrays and returning the hashes in different formats.

For example,

hmacFnew[method_, message_, key_] := 
 Module[{dkey, opad, ipad, blocksize}, 
  blocksize = If[method === "SHA384" || method === "SHA512", 128, 64];
  dkey = StringToByteArray[key];
  If[Length[dkey] > blocksize, dkey = Hash[dkey, method, "ByteArray"]];
  dkey = Normal[dkey];
  If[Length[dkey] < blocksize, dkey = PadRight[dkey, blocksize, 0]];
  {opad, ipad} = ByteArray[BitXor[dkey, ConstantArray[#, blocksize]]] & /@ {92, 54};
  Hash[Join[opad, Hash[Join[ipad, StringToByteArray[message]], method, "ByteArray"]], 
    method, "HexString"]
]

hmacFnew["SHA256", \
"symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.\
1&recvWindow=5000&timestamp=1499827319559", \
"NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)

If you don't feel like modifying your code, another way to get the previous behavior is to use Developer`LegacyHash, say

Block[{Hash = Developer`LegacyHash}, 
 hmacF["SHA256", 
  "symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=\
0.1&recvWindow=5000&timestamp=1499827319559", 
  "NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)
$\endgroup$
  • 1
    $\begingroup$ I was on the way to posting a similar answer. Just one comment, the correct way of padding a key: PadRight[Normal@ If[keyLen > blocksize, Hash[StringToByteArray@key, method, "ByteArray"], StringToByteArray@key], blocksize, 0]; $\endgroup$ – swish Mar 11 '18 at 2:45
  • $\begingroup$ Thanks @swish, I was just going by OP's code. I hope it looks correct now. $\endgroup$ – ilian Mar 11 '18 at 2:57
  • 1
    $\begingroup$ That part would be more elegant if PadRight and BitXor accepted byte arrays, but unfortunately they don't. $\endgroup$ – ilian Mar 11 '18 at 5:28
  • $\begingroup$ Thank you! This was really throwing me for a loop. $\endgroup$ – Ray Troy Mar 11 '18 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.