# How to read list of UnsignedInteger16 from base64 string in mathematica

I am struggling to convert a base64 string to a list of UnsignedInteger16 values. I have limited experience with mathematica, so please excuse me if this should be obvious.

I read the base64 string from an XML file and ultimately into a variable base64String. This is a long string with 7057 UnsignedInteger16 values, but it starts off as: "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSop..."

Can convert to ByteArry with n = BaseDecode[base64String] // Normal;

Then n has the expected values {243, 41, 231, 41, 219, 41, 206, 41, 194, 41, 183, 41, 173, 41, 161, ...

However, what I am looking for is a list with the values {243 + 256 * 41, 231 + 256 * 41, 219 + 256 * 41, ...

i.e.: {10739, 10727, 10727, ...

Thank you in advance for you suggestions.

Edit: Thank you for the suggestion to use Partition. Works well. For the interests in improving my understanding, can any one suggest why I cannot get ImportString[...] to work.

My source data is XML file with data along the lines of the following where I truncated the Base64 string for clarity.:

<?xml version="1.0" encoding="utf-8" ?>
<BPplus version="5.0" filename="BPplus_00100.xml">
<MeasDataLogger guid="ee7bee36-ffd6-30ae-53f1-257f0eab7ddd">
<Sys>130</Sys>
<Dia>77</Dia>
<Map>101</Map>
<PressureWaves>
<RawPressureWave>
<RawPressure>NIBP</RawPressure>
<RawSampleCount>7057</RawSampleCount>
<RawCuffPressureWave>8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUk... bU1mjV+NWM1RjU=</RawCuffPressureWave>
</RawPressureWave>
</PressureWaves>
</MeasDataLogger>
</BPplus>


The script I then run with the C1, C2 & C3 suggestions.

sampleFilenameString = "C:\\BPPdata\\BPplus_00112.xml";
xmldata = Import[sampleFilenameString, "XML"];
RawCuffPressureWave =
Cases[xmldata, XMLElement["RawCuffPressureWave", _, _], Infinity];
nibpxml = RawCuffPressureWave[[2]];
base64String = ToString[nibpxml[[3]]];
C1 = Partition[Normal@BaseDecode[base64String], 2].{1, 256};
C2 = ImportByteArray[BaseDecode[base64String], "UnsignedInteger16"];
C3 = ImportString[base64String, {"Base64", "UnsignedInteger16"},
ByteOrdering -> -1]
C1 == C2
C1 == C3
C1


produces the following output

{17956, 26977, 25964}
True
False
{10739, 10727, 10715, 10702, 10690, 10679, 10669, 10657, ... }


I have tried to make the first parameter ToString[base64String], but that did not change the outcome.

If I manually define the string $base64 = "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSo.... It works as expected and C1 = C4 C4 = ImportString[$base64, {"Base64", "UnsignedInteger16"},
ByteOrdering -> -1];
C1 == C4


Can anyone explain the difference why $base64 variable works but base64String does not? Edit: base64String is not a string! Not sure if this is the best way to convert, but it works in Mathematica 11.x and higher. sampleFilenameString = "C:\\BPPdata\\BPplus_00112.xml"; xmldata = Import[sampleFilenameString, "XML"]; RawCuffPressureWave = Cases[xmldata, XMLElement["RawCuffPressureWave", _, _], Infinity]; nibpxml = RawCuffPressureWave[[2]]; (* convert to string then base64 decode *) base64Data = DeveloperDecodeBase64ToByteArray[ nibpxml //. XMLElement[_, _, t_] :> t // Flatten // StringJoin] // Normal; C1 = Partition[base64Data, 2].{1, 256};  • ImportByteArray[BaseDecode[str], "UnsignedInteger16"] ? Jul 20, 2020 at 15:31 • Or Partition[n, 2].{1, 256} Jul 20, 2020 at 15:42 • Looks like we need to use something along the lines of the following to convert the XmlElement data to Text (* convert to string then base64 decode *) base64Data = DeveloperDecodeBase64ToByteArray[ ToString[ nibpxml //. XMLElement[_, _, t_] :> t // Flatten // StringJoin]] // Normal; C1 = Partition[base64Data, 2].{1, 256}; Jul 21, 2020 at 2:49 • The issue is that the variable base64String in the question contains extraneous braces, i.e. "{8ynnKdsp...}". This is not a valid Base64 string -- but apparently BaseDecode will ignore such errors and process it anyway. Note how the content portion of the XML element is actually a list: XMLElement["RawCuffPressureWave", {}, {"8ynnKdsp..."}]. The call to ToString converts the entire list to a string, braces and all. Assuming the content of the XML element RawCuffPressureWave is always a single text node then the simplest fix is base64String = nibpxml[[3, 1]]. Jul 21, 2020 at 5:45 • thank you for helping improve my understanding re nibpxml[[3]] being a list and the need to index to the first item to get the value with nibpxml[[3,1]]. Probably a bit obvious in retrospect, but eluded me a few days ago. I will remember! :-) Jul 23, 2020 at 11:09 ## 2 Answers We can use ImportString to decode the Base64 string and interpret the resulting bytes in a single step: ImportString["8ynnKdsp", {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1] (* {10739, 10727, 10715} *)  Explanation The documentation for the Base64 import format tells us: The import format and file encoding can be explicitly specified with Import["file", {"Base64", "format", elems1, elems2, ...]. In this case, format can be "Binary" which is documented to support many data representation elements including "UnsignedInteger16". Putting all this together with ImportString, we can perform the required conversion: $base64 = "8ynnKdspzinCKbcprSmhKZUpiCl8KW8pYylXKUkpQCk0KSop";

ImportString[$base64, {"Base64", "Binary", "UnsignedInteger16"}, ByteOrdering -> -1] (* {10739, 10727, 10715, 10702, 10690, 10679, 10669, 10657, 10645, 10632, 10620, 10607, 10595, 10583, 10569, 10560, 10548, 10538} *)  The documentation for Binary also tells us: For any data representation element type, Import["file", type] can be used as a shorthand for Import["file", {"Binary", type}]. So we can use this shorthand form to get the same result: ImportString[$base64, {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1]


The ByteOrdering option ensures that the unsigned integers are interpreted in "little-endian" format (low-byte first). This option can be omitted on a little-endian machine since it will be the default \$ByteOrdering. But even so it does not hurt to include it defensively if there is any possibility that our code might some day find its way onto a big-endian machine.

• Thank you for the detailed answer.. I am missing some understanding about strings as this approach only works if the string is hard coded into the script. When i read the "string" from an XML file, then it does not work (but the C1 approach with Partition does work!) See question in answer to @Roman Jul 21, 2020 at 1:05

There are many ways of doing this. The fastest I can think of is

Partition[Normal@BaseDecode[str], 2].{1, 256}


which takes less than half of the time that ImportString requires.

### Timing comparisons:

A = RandomInteger[255, 10^8] // ByteArray;
S = BaseEncode[A];

C1 = Partition[Normal@BaseDecode[S], 2].{1, 256}; // AbsoluteTiming // First
(*    4.11495    *)

C2 = ImportByteArray[BaseDecode[S], "UnsignedInteger16"]; // AbsoluteTiming // First
(*    5.98029    *)    (*    @ilian    *)

C3 = ImportString[S, {"Base64", "UnsignedInteger16"}, ByteOrdering -> -1]; // AbsoluteTiming // First
(*    9.64049    *)    (*    @WReach    *)

C1 == C2 == C3
(*    True    *)

• excellent answer with the performance metrics. i will ultimately be processing many of these files, so fast is helpful. I did try the C2 and C3 approached, but I could not get them to work and they still don't work for me, but the C1 approach with Partition works perfectly (nice it's also the fastest :-). There must be something I don't understand about stings and reading values from XML files. Jul 21, 2020 at 1:13
• Actually C1 and C2 work, but I cannot get C3 to work.... I will edit question so we can all learn. Jul 21, 2020 at 1:25