Questions tagged [compression]
The compression tag has no usage guidance.
43
questions
0
votes
0
answers
19
views
Temporary directories left behind by the CreateArchive function
The CreateArchive function sometimes leaves empty subdirectories of the form m_x in the destination directory, where x is a ten-digit alphanumeric identifier that ...
- 91
0
votes
0
answers
48
views
Is the data generated by Compress unique?
Is the data generated by the Compress function in Mathematica unique? I’m considering using it to identify duplicates because it reduces memory usage significantly, ...
- 723
3
votes
0
answers
57
views
How to improve Burrows-Wheeler Transform performance?
I have implemented the Burrows-Wheeler Transform as below:
...
- 25.6k
1
vote
1
answer
66
views
Compressing a file's name string
I was working with files where each name records a lot of information about what is in its respective file and wanted to compress it a little bit. So I wanted to change it's 'base'.
And I noticed that ...
- 114
1
vote
0
answers
61
views
Speed limitations when importing files from .zip archive
I have .m files with a total size of ~500 MB. Their import is really fast, within less than 1 second. In contrast, I may pack them into a .zip archive. The importing then is much slower, ~20 s. ...
- 5,923
3
votes
1
answer
133
views
Optimized GIF 20s for 85kb for Mathematica Code Demonstration
I’m new here this is my first post on Stack Exchange. In the thread below, I see that a GIF file is highly optimized (20s for 85kb) with no noticeable compromise in resolution. It appears to be the ...
- 33
0
votes
0
answers
34
views
Updating a compressed Boolean 1D array, without uncompressing
I iterate through a process, each time generating a new number between 1 and a very large number (10^15 or larger). I am waiting for any number to repeat.
Clearly a 1D Boolean array (starting out all ...
- 1,225
3
votes
0
answers
165
views
OpenRead with Method->"GZIP"
Aim
I'm trying to "Low level" read data from a large "GZIP" file as an InputStream.
Previous answeers
I ...
- 139
5
votes
1
answer
333
views
How can I compress DICOM images?
I have 193 dcm files in a folder. I will like to combine these images into .dcm.gz format (e.g. images.dcm.gz) and I was wondering how this could be done in Mathematica. Thank you in anticipation of ...
- 655
6
votes
1
answer
326
views
How to compress a string in Mathematica so that it can decompressed in python
I need to compress a string into some format so that I can pass it along on the command line and avoid writing it to file (which would require locking files to prevent simultaneous writes so I prefer ...
- 4,651
0
votes
1
answer
140
views
Import CSV data from internet to mathematica [closed]
This Link has data in CSV format for 2015 and other years. I would like to get the data for 2015 (by clicking on 2015) but the CSV data format has more rows than Excel can handle and therefore the ...
- 6,263
2
votes
1
answer
41
views
Compression results in unpacking of integers but not reals
Consider the following;
Pack some integers.
In[287]:= Clear[p5i, p5ic, p5icu, p5r, p5rc, p5rcu]
In[288]:= p5i = ToPackedArray@Range@5
Out[288]= {1, 2, 3, 4, 5}
...
- 790
5
votes
1
answer
177
views
Import ZIP file options broken in Mathematica 12.2.0
I am trying to read a zipped table, which has its entries separated by colons. In Mathematica 12.1.1 (and earlier) this worked well with the following command:
...
- 1,258
4
votes
2
answers
177
views
How to deal with zip files in the wolfram cloud?
I have a zip file in the cloud but I can't seem to unzip it!
...
- 31.7k
14
votes
0
answers
787
views
Is it possible to use better modern compression algorithm instead of zlib in Mathematica for efficient data saving and loading?
Some backgrouds of my problem:
Recently, I had a heavy data analysis work to do. Each day I got thousands of csv Files(colums with different datatype) with total several Gigabytes. I have to ...
- 17.3k
1
vote
0
answers
278
views
How to uncompress zip file fast with Mathematica?
What is the fastest possibilty to uncompress zip files with the following content?
It containes numbered ".png" files, corresponding ".header" files (theses are text files) and a file called "...
- 2,396
11
votes
2
answers
927
views
Efficient way to storage large lists in a file(s)
Let us assume that I need to storage in a file, or files, data with the following form
...
- 2,537
2
votes
1
answer
329
views
How to zip folders
I know that one can archive data with Mathematica using: CreateArchive[source,path]
I have a lot of folders (folder1, folder2,folder2 etc.) that I would like to ...
- 3,093
1
vote
1
answer
181
views
Compressing TIFF figure with constant resolution
I need to compress the TIFF figure "p5"(originally around 25 mb) in to less than 1 mb or as small as possible with constant resolution of 600 dpi.
...
- 111
6
votes
1
answer
327
views
Iconize a compressed image
I want to embed a compressed image in a notebook so that only an icon is visible (and not the whole image).
The following procedure seems to work :
...
- 18.9k
1
vote
0
answers
33
views
CreateArchive[] without directory copy [duplicate]
CreateArchive["C:\\temp out\\"]
The above creates a zip file containing directory temp out at level 1, but I require just the ...
- 31k
6
votes
0
answers
375
views
How to create TAR archive without top-level folder?
Mathematica has the handy CreateArchive function, which allows you to efficiently create a TAR archive from a directory. However, I see no way of specifying not to ...
- 2,055
12
votes
3
answers
218
views
Support for low-complexity (compressible) lists
By "low-complexity list" I mean a list like the following:
lcl = {17, 17, 17, 17, 17,
21, 21, 21, 21, 21, 21, 21, 21,
11, 11, 11, 11, 11, 11, 11}
...
- 11.7k
5
votes
1
answer
505
views
Decoding GZIP encoded Body, BodyBytes (ByteArray) and BodyBytesArray from URLRead
When calling the Stack Exchange API v2.2 using the new (Mma11) function URLRead we get a GZIP encoded Body in alternative formats: ...
- 37k
9
votes
1
answer
183
views
How to store data in a Mma SE question using hidden html-like markup?
Mathematica Stack Exchange accepts HTML in the questions, and therefore text such as
<undefined rubbish>
should be ignored when formatting the question for ...
- 37k
22
votes
1
answer
308
views
How to uncompress strings safely (without any evaluation)?
Compressed strings are often used to exchange Mathematica expressions on the Internet.
It is, however, not easy to see what such expression will do after decompression.
Resulting code can perform ...
- 7,876
27
votes
4
answers
1k
views
What's purpose of the new function BinarySerialize?
11.1 introduced a new function BinarySerialize, but I don't know what it can do better than the traditional method.Its behavior is very similar to ...
- 27k
7
votes
1
answer
402
views
How does - represent compressed data?
The Wolfram Blog article "The Semantic Representation of Pure Mathematics", by Eric Weisstein and Ian Ford (http://blog.wolfram.com/2016/12/22/the-semantic-representation-of-pure-mathematics/) has a ...
- 12k
9
votes
2
answers
332
views
Is there a way to create ZIP OutputStream ?
I want to write data to a file using compression. Is it possible to do that just in time (like in java ZipOutputStream). E.g. what I want:
...
0
votes
0
answers
214
views
Import MDB in ZIP as ASSOCIATION
I have a zip file which contains an access database file (mdb format). How do I extract the database and the headers and convert them to association format? I know the command for doing that ...
- 571
14
votes
1
answer
191
views
A weird issue with Interval[$MaxNumber]
Bug introduced in 9.0 or earlier and persisting through 11.0.1 or later
From the Interval documentation:
For approximate machine- or arbitrary-precision numbers ...
- 7,293
24
votes
2
answers
2k
views
What algorithm do the Compress and Uncompress functions use?
Compress and Uncompress are used internally by Mathematica to compress things like 3D data in notebook files - data that I'd ...
- 623
0
votes
1
answer
135
views
How to zip a folder? [duplicate]
This straightforward approach doesn't seem to work:
In[42]:= cmd = StringSplit@"zip -r ~/data.zip ~/test/*"
In[43]:= RunProcess@cmd
Related: How can I encrypt/...
- 31.7k
13
votes
2
answers
2k
views
How to correctly import data zipped with the "deflate" algorithm?
I'm writing an importer for the GIFTI file format. The details of the format are not particularly important, but the basic idea is that it is a relatively simple XML file which includes binary arrays ...
- 2,158
6
votes
1
answer
506
views
HOWTO encode/decode large chunks of Mathematica code/results e.g. for posting on sites?
At times, we as questioners would like to post huge Mathematica-expressions to websites (e.g. mathematica.stackexchange.com), or have an easy way to get such code from a site. Above a certain size, <...
- 3,753
8
votes
2
answers
319
views
Compress uses too much memory and sometimes crashes the Kernel
Using Compress is a good idea to save memory when working with large expressions. The problem is that Compressing large ...
- 2,445
4
votes
0
answers
154
views
Monitor progress of ZIP file extraction
Importing a ZIP file containing potentially many individual files in a (custom) format using
Import[filename, {{"*"}, "Gaussian"}]
takes quite a while. How can I ...
- 777
4
votes
0
answers
1k
views
Is there a way to use video codecs for avi files in Mathematica?
I would like to use Mathematica to create compressed .avi video files from image sequences. I read the documentation on .avi but couldn't find a solution. In the documentation it says:
Internal`$...
- 3,328
0
votes
1
answer
4k
views
Union::normal: Nonatomic expression expected when using Uncompress
I saved a expresion to a file like this
Export["file.dat",Compress@myExpression];
Then (i don't know why) when I import the file i get the compressed expresion ...
- 1,135
17
votes
4
answers
2k
views
Plotting data points: Optimizing size and visuals
The following problem is one that may have showed up to you while plotting a large data collection. Suppose you have a data set of say 1 million points. ListLinePlot...
- 6,510
15
votes
4
answers
1k
views
DataEncoding and compression in HDF5 format
I would like to export data in HDF5 format with Mathematica 9.
The Mathematica manual about Import and Export of this format can be read here.
As I'm dealing with large arrays, export often lead to ...
- 1,585
7
votes
2
answers
1k
views
DatabaseLink`: MySQL query with client/server compression
I'm trying to reduce transfer time when accessing rows of a mysql table over a network, where each row has a lot of data. Is there a way to enable client-server compression when accessing a mysql ...
- 5,141
3
votes
2
answers
168
views
How to uncompress only the compressed elements in a matrix
I have data in a rectangular structure – something like matrix={{a,b,c},{d,e,f},{g,h,i}}. The problem is that some of the columns consist of uncompressed data and ...
- 2,760