18
$\begingroup$

If you run this code on 11.2

    hmacF[method_, message_, key_] := Module[

  {keyLen, dkey, opad, ipad, blocksize},

  blocksize =
   If[method === "SHA384" || method === "SHA512",
    128,
    64
    ];

  keyLen = StringLength[key];

  dkey = If[keyLen > blocksize,
    IntegerString[Hash[key, method], 16],
    StringPadRight[key, blocksize, FromCharacterCode[0]]
    ];

  {opad, ipad} = 
   FromCharacterCode[
      BitXor[ToCharacterCode@
        StringRepeat[FromCharacterCode[FromDigits[#, 16]], blocksize],
        ToCharacterCode@dkey]] & /@ {"5c", "36"};

  IntegerString[
   Hash[
    StringJoin[opad, 
     ExportString[
      IntegerDigits[Hash[StringJoin[ipad, message], method], 256], 
      "Binary"]],
    method],
   16, blocksize]
  ]

On something we know the answer to https://github.com/binance-exchange/binance-official-api-docs/blob/master/rest-api.md

hmacF["SHA256","symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559","NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

you get the correct answer

(*c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71*)

Run the same code on 11.3 and you get

(*88fdd5c8c2226ff025e290b37dec36d4bc6f84affb9928950fd865f9a0fd475e*)

Why the difference?? 11.3 is not the correct HMAC result.

$\endgroup$
6
  • $\begingroup$ Well break the function down and find out which piece changed between versions... it's likely a single function giving a different output, my guess is Hash... $\endgroup$
    – ktm
    Mar 11, 2018 at 1:38
  • $\begingroup$ I tested it, 11.2 on desktop, 11.3 on cloud, and yes, Hash seems to have changed in some manner? If you think there is a bug, minimize it and report the issue to support@wolfram.com. $\endgroup$
    – ktm
    Mar 11, 2018 at 1:44
  • $\begingroup$ Looking at the Hash documentation page, there does seem to have been a handful of changes to the function for 11.3. Perhaps some of these adversely affected your code. $\endgroup$
    – ktm
    Mar 11, 2018 at 1:46
  • $\begingroup$ @user6014 I did read the documentation. None of the changes to Hash should have changed the output, hence my post. In your opinion, is it a bug? Does my function need to be written differently in 11.3? There should be no reason why the function needs to change. $\endgroup$
    – Ray Troy
    Mar 11, 2018 at 2:00
  • 3
    $\begingroup$ Here is the minimal example demonstrating the difference: Hash[ExportString[{217, 73, 200, 195, 231, 110, 83, 36, 139, 50, 62, 251, 63, 221, 85, 205, 192, 34, 244, 161, 198, 42, 132, 228, 7, 215, 112, 249, 161, 228, 204, 113}, "Binary"], "SHA256"] $\endgroup$
    – ktm
    Mar 11, 2018 at 2:04

1 Answer 1

22
$\begingroup$

Indeed, Hash has changed for 11.3 in several ways.

The one relevant here is that strings are now hashed according to their UTF-8 representation.

As a simple example of the difference, consider the following string, which has only one character.

c = FromCharacterCode[217]; (* "\[CapitalUGrave]" *)

Its UTF-8 representation consists of the two (decimal) bytes {195, 153}, so in version 11.3 the hash would be computed from those two bytes, while in 11.2 the hash would be computed from the single (decimal) byte 217.

Then the two corresponding hash results would be different, namely

(* new hash value *)
Hash[c, "SHA256"]
Hash[StringToByteArray[c], "SHA256"] 
Hash[ByteArray[{195, 153}], "SHA256"]

(*115129295529739515318262490854329296079790445441298973994137838100166123599005*)
(* or as a hex string: *)
(* fe88df3f083e08f99f796b5ba7f6eedbe14d45241cf5e0944caf17f55ac2349d *)

which agrees with various online hash calculators, e.g. http://new.md5calc.com/hash-calc/sha256/%C3%99 and

(* old hash value *)
Developer`LegacyHash[c, "SHA256"]
Hash[StringToByteArray[c, "ISO8859-1"], "SHA256"]
Hash[ByteArray[{217}], "SHA256"]

(* 11345241613194184164858110515603144575751235911333201083896508434490695406828*)
(* or as a hex string: *)
(* 19152ddfba193b5b09fcb80d1bba5248f36027c06e81670db5a7146fb654d4ec *)

where all inputs were evaluated with 11.3.0.

The hmacF implementation uses strings in a somewhat awkward way as data containers. It is easier to take advantage of several new Hash enhancements instead, like operating on byte arrays and returning the hashes in different formats.

For example,

hmacFnew[method_, message_, key_] := 
 Module[{dkey, dmsg, opad, ipad, blocksize}, 
  blocksize = If[method === "SHA384" || method === "SHA512", 128, 64];
  {dkey, dmsg} = StringToByteArray /@ {key, message};
  If[Length[dkey] > blocksize, dkey = Hash[dkey, method, "ByteArray"]];
  dkey = Normal[dkey];
  If[Length[dkey] < blocksize, dkey = PadRight[dkey, blocksize, 0]];
  {opad, ipad} = ByteArray[BitXor[dkey, ConstantArray[#, blocksize]]]& /@ {92,54};
  Hash[Join[opad,Hash[Join[ipad,dmsg], method, "ByteArray"]], method, "HexString"]
]

hmacFnew["SHA256", \
"symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.\
1&recvWindow=5000&timestamp=1499827319559", \
"NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)

If you don't feel like modifying your code, another way to get the previous behavior is to use Developer`LegacyHash, say

Block[{Hash = Developer`LegacyHash}, 
 hmacF["SHA256", 
  "symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=\
0.1&recvWindow=5000&timestamp=1499827319559", 
  "NhqPtmdSJYdKjVHjA7PZj4Mge3R5YNiP1e3UZjInClVN65XAbvqqM6A7H5fATj0j"]]

(* c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71 *)
$\endgroup$
4
  • 1
    $\begingroup$ I was on the way to posting a similar answer. Just one comment, the correct way of padding a key: PadRight[Normal@ If[keyLen > blocksize, Hash[StringToByteArray@key, method, "ByteArray"], StringToByteArray@key], blocksize, 0]; $\endgroup$
    – swish
    Mar 11, 2018 at 2:45
  • $\begingroup$ Thanks @swish, I was just going by OP's code. I hope it looks correct now. $\endgroup$
    – ilian
    Mar 11, 2018 at 2:57
  • 1
    $\begingroup$ That part would be more elegant if PadRight and BitXor accepted byte arrays, but unfortunately they don't. $\endgroup$
    – ilian
    Mar 11, 2018 at 5:28
  • $\begingroup$ Thank you! This was really throwing me for a loop. $\endgroup$
    – Ray Troy
    Mar 11, 2018 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.