2
$\begingroup$

I'm able to post strings (text files) to AWS S3 using REST API's, but I can't post photos/files.

Because the "content-length" or ByteCount of a Wolfram Language expression is larger than the actual photo or data, I'm importing the photo with "Byte":

mContent = Import[aPath, "Byte"];
mByteLength = Length[mContent];

When I generate my Hash for mContent and try to "PUT" it like this:

URLExecute[HTTPRequest["http://myBucket.s3.amazonaws.com", <|Method->"PUT","Headers"->mHeader,
"Body"->mContent]

where mHeader is a complex AWS header, I get the error

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>XAmzContentSHA256Mismatch</Code><Message>The provided 'x-amz-content-sha256' header does not match what was computed.</Message
<ClientComputedContentSHA256>a440d87eb4b5da1b8decf760ad3f8350bae7a83ccfcdd2eda029f3dde51b663a</ClientComputedContentSHA256
<S3ComputedContentSHA256>e2bae210a3ee03a2d2987d63639a17d5206e7660fe758cc17ecb4fc7137c9010</S3ComputedContentSHA256><RequestId>44B77755E6F6BC81</RequestId>
<HostId>Jd8MRLSYntEaguxJPJ/dbt+Ri4aZjk7etQp/O+dK6+XehJQ2sFF3lABDlciGBBGyRFuOSg0dzAo=</HostId></Error>

Example of mHeader

Here's the header I'm using. Not sure how helpful this is:

<|"X-Amz-Content-SHA256" -> "a440d87eb4b5da1b8decf760ad3f8350bae7a83ccfcdd2eda029f3dde51b663a",
"content-length" -> 82384, "content-type" -> "image/jpeg", 
"X-Amz-Date" -> "20190321T035330Z", 
"Host" -> "myBucket.s3.amazonaws.com", 
"Authorization" -> "AWS4-HMAC-SHA256
    Credential=***publicKey***/us-east-1/s3/aws4_request,
    SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date,
    Signature=3ac3e35626c32c011eaAUTHORIZATION-HASH"|>

Additional information

I tried changing the "Body" specification, but I can't figure out exactly what to do. Here's another example:

URLExecute[HTTPRequest["https://mybucket.s3.amazonawx.com", <|Method -> "PUT", "Headers" -> mHeader,
    "Body" -> {"image" -> <|"MIMEType" -> "image/jpeg", "Content"->File[aPath]|>}|>]

This results in

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>SignatureDoesNotMatch</Code><Message>The request \
signature we calculated does not match the signature you provided. \
Check your key and signing \
method.</Message><AWSAccessKeyId>*******public key*******</AWSAccessKeyId>\
<StringToSign>AWS4-HMAC-SHA256
20190321T033955Z
20190321/us-east-1/s3/aws4_request
f26d4ff70df95f868dd54f9496ad211255a909e59d1386814549aece5c823a8f</\
StringToSign><SignatureProvided>\
cbeaff51f2459efff00c39b15d953b90f5270662b6e36d9f94cd9ab9f7217aa5</\
SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 \
48 41 32 35 36 0a 32 30 31 39 30 33 32 31 54 30 33 33 39 35 35 5a 0a \
...morebytes...
61 39 30 39 65 35 39 64 31 33 38 36 38 31 34 35 34 39 61 65 63 65 35 \
63 38 32 33 61 38 66</StringToSignBytes><CanonicalRequest>PUT
/test.jpeg

content-length:82384
content-type:image/jpeg; \
boundary=------------------------13896c956eada0be
host:afslabdb.s3.amazonaws.com
x-amz-content-sha256:\
a440d87eb4b5da1b8decf760ad3f8350bae7a83ccfcdd2eda029f3dde51b663a
x-amz-date:20190321T033955Z

content-length;content-type;host;x-amz-content-sha256;x-amz-date
a440d87eb4b5da1b8decf760ad3f8350bae7a83ccfcdd2eda029f3dde51b663a</\
CanonicalRequest><CanonicalRequestBytes>50 55 54 0a 2f 74 65 73 74 2e \
6a 70 65 67 0a 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a 38 32 \
...lots of bytes...
62 61 65 37 61 38 33 63 63 66 63 64 64 32 65 64 61 30 32 39 66 33 64 \
64 65 35 31 62 36 36 33 \
61</CanonicalRequestBytes><RequestId>4D0FBF28400578D9</RequestId><\
HostId>X7HAdHvlQlUQy6d4v2gC7uzh7rdJ3aFfUZL8xSby9Ecw3ep+cG/\
1Zfv2mskN9kEr9/S3SXlXFwI=</HostId></Error>

The documentation for HTTPRequest isn't quite as thorough as I think I need, so I'd appreciate some help from Wolfram Language experts/developers.

$\endgroup$
2
$\begingroup$

The problem is with my content hash not matching the AWS calculated hash (as you can see in the response from AWS). I didn't figure out how to get them to match, but I found an option in the AWS documentation to use UNSIGNED-PAYLOAD. So instead of calculating a content hash for the key x-aws-content-sha256, simply assign the value of that key to the literal "UNSIGNED-PAYLOAD".

Everything else is hashed and authorized, so there isn't a major security issue here.

After using UNSIGNED-PAYLOAD my file went straight to S3 using

<|Method -> "PUT", "Headers" -> mHeader, "Body" -> mContent|>

as the association in the HTTPRequest. mContent here is Import[myFile,"Byte"] and the header is similar to the one given in the example above but with "UNSIGNED-PAYLOAD" used for the content hash (x-amz-content-sha256).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.