6
$\begingroup$

I need to compress a string into some format so that I can pass it along on the command line and avoid writing it to file (which would require locking files to prevent simultaneous writes so I prefer to pass it directly). I then need to decompress the string in Python. The standard Compress function seems like a good candidate as it indeed compresses strings into string without spaces. According to https://mathematica.stackexchange.com/a/104661/45020 I should be able to decode the string in python using zlib.

However, I cannot manage.

I tried with

import zlib
data = data if isinstance(data, bytes) else data.encode('utf-8')
data = base64.b64decode(data)
zlib.decompress(data)

where data is the string outputted by Compress in Mathematica with the leading 1: removed as suggested in the linked post.

It decompressed without any error in python, but seems to be a meaningless string nothing like the original string that I put in. I guess that the standard Compress is no good to me in this situation?

How can I best compress a string in Mathematica in such a way that I can decompress it in Python.


Compressing using "GZIP" does not seem to work at all. I get an error

"File \!\(\"/tmp/m0000213298301.gz\"\) not found during Import"

Sorry about requiring no spaces. That was not actually important.

$\endgroup$
8
  • 1
    $\begingroup$ Did you notice that both answers mention that base64 encoding is also involved? $\endgroup$
    – J. M.'s torpor
    Mar 1 at 11:12
  • 1
    $\begingroup$ If I do Compress["Hello, World!"] then remove the initial 1: with StringTake[Compress["Hello, World!"], 3 ;;] then I get eJxTTMoPCuZlYGDwSM3JyddRCM8vyklRBAA//AYO . From python I get b'!boRS\r\x00\x00\x00Hello, World!'. I suspect the initial junk is information Mathematica would use to determine the compressed object is a string when decompressing, since you can compress just about anything so it needs to know what type it is. The answer you linked actually mentions these mysterious bytes. $\endgroup$
    – flinty
    Mar 1 at 11:31
  • 2
    $\begingroup$ @flinty, thank you! When looking carefully I do see that some parts resemble the original. However in my case it is not as simple as it just being preceded by junk. The junk is everywhere. Things like \\012\\011. $\endgroup$
    – Kvothe
    Mar 1 at 11:39
  • 3
    $\begingroup$ Could you compress also with BinarySerialize (using PerformanceGoal -> "Size"), then in python use the WolframClientForPython to extract it? The thing is that BinarySerialize performs better than Compress. $\endgroup$
    – rhermans
    Mar 1 at 12:30
  • 1
    $\begingroup$ @MarcoB Yes. $\endgroup$
    – Alan
    Mar 1 at 14:32
6
$\begingroup$

I'm not familiar with your application nor your reason to remove spaces for passing arguments in the command-line interface. You will see two examples which in both of them a string successfully recived from Mathematica in python.

Without conversion:

Use python sys library to access command-line arguments passed to python with Mathematica RunProcess which will send values as argument without being worry about spaces:

message = "This is a sample text";

RunProcess[{"python", "-c", "import sys;print(sys.argv)", 
  message}, "StandardOutput"]

(*Out: "['-c', 'This is a sample text']" *)

In python use sys.argv to access the string you had sent as an argument.

With conversion:

I find out that Mathematica ExportString[message, "Base64"] will add \n to the end of the string, so before sending remove that:

message = "This is a sample text";

em = StringTake[ExportString[message, "Base64"], {1, -2}];

RunProcess[{"python", "-c", 
  "import base64;import sys;print(base64.b64decode(sys.argv[1]),end='')", 
  em}, "StandardOutput"]

RunProcess[{"python", "-c", 
  "import base64;print(base64.b64decode('" <> em <> 
   "'),end='')"}, "StandardOutput"]

(*Out 1: "b'This is a sample text'" *)

(*Out 2: "b'This is a sample text'" *)
```
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.