Tag Info

15

After a bit of poking around, it looks like the binary format is pretty simple to parse. Mark Adler's answer is correct - the strings Compress[] returns are just zlib-compressed data. If you have Python installed, this function should take a compressed string and return the actual serialized bytes: pyDecompress[c_] := ...

15

It is using the zlib format followed by Base64 coding, and then preceding the resulting string with "1:". So to use it externally, you can strip the "1:", do Base64 decoding, and feed the result of that to a zlib decoder. However what you get out may not be immediately useful. I compressed the result of D[x^x, {x,9}], like one of the examples in the ...

13

This filters out your points by their EuclideanDistance. I think it makes a pretty good job preserving the curve's features with very few points: rx[n_] := Accumulate[Prepend[RandomVariate[ExponentialDistribution[1000], n], 0]] ry[n_] := Accumulate[Prepend[RandomVariate[NormalDistribution[0, .001], n], 0]] c = Transpose[{rx@#, ry@#}] &@100000; t = ...

12

Here is a modified version of belisarius' answer I ended up using. Instead of using the euclidean distance to find out if two points are close to each other I set a fixed rectangle in an area and collapse it into a line. For instance, consider the same example. rx[n_] := Accumulate[Prepend[RandomVariate[ExponentialDistribution[1000], n], 0]] ry[n_] := ...

12

To reduce the number of points, I'd use Interpolation: f = Interpolation[Transpose@{x,y}, InterpolationOrder->1]; Plot[f@x,{x,0,100}]

10

As I said in my comment, the Douglas-Peucker algorithm is a standard method for simplifying polygonal curves. This demonstration by Mark McClure contains a Mathematica implementation. Since he is a user on this site, I feel like he deserves the points for this answer. Nevertheless, in the interests of having an answer here, and hoping this falls under fair ...

9

The documentation is misleading here. On one hand, the only export option is "Append" which can be found under the Options tab. On the other hand, the general documentation reads I really wonder, why it is necessary to put Import only behind an option value when "DataEncoding" isn't an export option at all. Anyway, I have the same behaviour in MacOSX ...

8

This is not a proper answer to the question. I had forgotten what the proper syntax was for the Method option of Compress, but when I tried to find it again, I noticed that it seems like this is not documented anywhere. So, I just wanted to record this for posterity, as well as note that there are three working methods (as of version 9.0.1): test = ...

7

The Base64 string you provided as an example is not an encoding of a gzip stream (RFC 1952). It is an encoding of a zlib stream (RFC 1950). For background, those are different wrappers around the raw "deflate" compressed data format (RFC 1951), where the wrappers are headers and trailers proving information on the compressed data and integrity check ...

7

You can decompress on the Mathematica side easily. Compressed MySQL reply has the following format: first four bytes are size of uncompressed data (lowest byte first) the rest is the string compressed with deflate algorithm (zlib library) Here is an example of a reply: {10, 0, 0, 0, 120, 156, 243, 72, 205, 201, 201, 87, 240, 170, 112, 82, 4, 0, 19, ...

5

One workaround is to compress the HDF5 file after it has been exported from Mathematica, using the HDF5 command line tools. Note: on OS X the command line tools can be easily installed using MacPorts using port install h5utils. The command to recompress the data is h5repack -v -f GZIP=1 infile.h5 outfile.h5 This can indeed achieve a significant ...

5

In version 7 halirutan's export method does not produce a file that is recognized by Import. However, one can write: Export["matrix2.h5.gz", datapourrie, {"GZIP", "HDF5"}] And then: d2 = Import["matrix2.h5.gz", {"Datasets", "/Dataset1"}]; datapourrie == d2 True

4

Here is a way to only uncompress elements that have been compressed. Note that using Mathematica's Compress returns a string starting with <number>:, so we need to Uncompress only those elements that are strings and start with that. Clear@uncompress uncompress[s_String] /; StringMatchQ[s, DigitCharacter ~~ ":" ~~ ___] := Uncompress@s uncompress[x_] ...

4

This is a quite handy approach of encoding Mathematica-expressions, even complete notebooks, as grayscaled images, which can be posted as PNG-files (and decoded afterwards). Code for encoding and decoding is as follows: (* general encoding to grayscale data *) seEncode[expr_] := Block[{cc = ToCharacterCode[Compress[expr]], olen, a}, olen = ...

3

A new and mostly rewritten version of the original answer which was flawed. See edit history if interested. As any operation making $MaxNumber higher (more precisely: higher enough for its Precision to notice) results in an overflow, the Interval created here has the form Interval[{$MaxNumber - something small, Overflow[]}] The "something small" is ...

2

Create a matrix, and compress a column a = RandomInteger[10, {3, 3}] b = Transpose@{a[[1]], Compress /@ a[[2]], a[[3]]} Uncompress it: Quiet@Map[Check[Uncompress[#], #] &, a, {2}] Edit Instead of Quiet[] you could use Off[Uncompress::string] so other error messages are not hindered

2

Since Mathematica 8.0 we are in luck and there is a command called CreateArchive that can do this for us. Documentation: CreateArchive.

2

It's not clear exactly what you plan to do with the compressed data, but maybe this alternative approach will be useful. It uses ExportString and ImportString with a format that is especially suited for large data sets, HDF5. The compression step is very fast and efficient. I'm using the original data in the question: data = Range[30000000]; ...

1

It seems there is a undefined variable games in the compressed data. The code in the compressed data attempts to do a union on this games variable and some lists. So essentially, the error you get is the same one caused by Union[{}, games] To see it for yourself, try using Uncompress with two arguments. I suggest to use HoldComplete as a second argument. ...

Only top voted, non community-wiki answers of a minimum length are eligible