9
$\begingroup$

I find some ELMo-models on github.

I want to use the Simplified-Chinese version: http://pbmpb9h15.bkt.gdipper.com/zhs.model.tar.xz

But these files were saved as python pickles.

I want to import them as Mathematica List or Association.

Is there any extension to do this?

$\endgroup$
9
$\begingroup$

I found some tools which can translate python objects to wolfram expressions.

https://github.com/WolframResearch/WolframClientForPython

import pickle as pkl
import wolframclient.serializers as wxf
def pkl2wxf(path):
    file = open(path, 'rb')
    objs = []
    while True:
        try : objs.append(pkl.load(file))
        except EOFError : break
    file.close()
    print(objs)
    wxf.export(objs, path + '.wxf', target_format='wxf')

The function pkl2wxf can finish these works. Let's do some test:

f = open('objs.pkl', 'wb')
# Test basic types
testDict = {
    0: None,
    1: [1, 2, 3, 4],
    2: ('true', 'false'),
    3: {'yes': True, 'no': False}
}
pkl.dump(testDict, f)
f.close()
pkl2wxf('objs.pkl')

Now just use Import["objs.pkl.wxf"], all these python types are translated.

{<|
   0->None,
   1->{1,2,3,4},
   2->{"true","false"},
   3-><|"yes"->True,"no"->False|>
|>}
$\endgroup$
  • $\begingroup$ Thank you for posting this! $\endgroup$ – C. E. Oct 19 '18 at 6:41
  • $\begingroup$ In this vein, it should also be possible to use PJLink if you can get around the compilation obnoxiousness. $\endgroup$ – b3m2a1 Oct 19 '18 at 8:14
4
$\begingroup$

This pickle format is almost by definition, meant for Python and Python only. Your best bet is to load it in Python, and the export to some format that Mathematica can handle. For lists and dictionaries with only primitive datatypes in them, I'd use JSON. It is not clear that the data you linked is such.

If the data in this pickle is very simple, you could use the ExternalEvaluate to evaluate the Python data and transfer it to Mathematica. Automatic translation will only work for simple data types.


From the Python documentation page you linked:

The data format used by pickle is Python-specific. This has the advantage that there are no restrictions imposed by external standards such as JSON or XDR (which can’t represent pointer sharing); however it means that non-Python programs may not be able to reconstruct pickled Python objects.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.