Questions tagged [packed-arrays]

Tag for questions about packed arrays. Packed Arrays are data structures that can be seen as monolithic blocks containing a single data type, particularly Integers and Reals. Many functions operate very efficient on packed arrays. Questions may deal with packed array generation, conversion, and usage.

3
votes
1answer
109 views

Possible speed improvement for numerical linear algebra?

I have a question about further optimising a numerical linear algebra calculation. I've trawled through many useful posts on this site and have ended up with something quite efficient - I was ...
2
votes
1answer
35 views

Clarification on how Total[] can be used on multi-dimensional array

I have a rank 5 tensor that ultimately I want to modify so that for the first 3 dimensions, each element is summed together. The result will be a rank 2 tensor whose elements are the summed totals ...
12
votes
2answers
263 views

Unexpected behavior of the procedure `Area` on the object 'Polygon'

Bug introduced in 11.3 or earlier and fixed on 12.0. Sometimes get a results, sometimes left unevaluated. For instance ...
19
votes
4answers
556 views

Is it possible to avoid unpacking when merging Association?

For example, if we have assoc = AssociationThread[{1, 2, 3} -> RandomReal[1., {3, 10}]]; and ...
7
votes
1answer
355 views

Avoiding unpacking an array when altering its dimension

I am trying to flatten a matrix from 3-dimensional to 2-dimensional one. The original one has dimension for example {10,120,120}: ...
12
votes
0answers
99 views

What changed in String handling between 11.0 and 11.3?

I was trying to optimize some String-handling code here using my standard trick of converting it to a list of bytes first and using basic list operations instead of ...
5
votes
0answers
63 views

Why can't (partially) packed arrays be used in this example?

Bug introduced in 11.3 or earlier and persisting through 12.0 or later I didn't expect any difference between these: ...
6
votes
1answer
140 views

How to send packed array from math kernel to 3rd party app via WSTP

Background I'm developing a small python wrapper PyWSTP. The wrapper is still in very early stage and lacking features. One important feature I'm considering is transferring dense array between math ...
5
votes
1answer
102 views

Packing and unpacking of arrays with Map/Table

How can I create function, that will not unpack packed arrays with Map/Table. For example I want read bytes from opened stream (this is my original goal): ...
3
votes
0answers
47 views

PackedArray version of Chop

I'm a big fan of Chop. It's fast and useful in pre-rounding transforms and things Unfortunately it also necessarily unpacks a real ...
3
votes
1answer
75 views

Strange unpacking of a packed array with Insert

In Mathematica 11.1.1, when a packed array is stored within another list, it gets unexpectedly unpacked when Inserting something into the outer list. Create a ...
16
votes
1answer
157 views

RandomChoice output isn't packed

Why isn't the output of RandomChoice a packed array? For instance: RandomChoice[{.5, .2, .3} -> {1, 2, 3}, 10] Developer`PackedArrayQ @ % {1, 2, 3, 1, 3, 1, ...
7
votes
2answers
132 views

Fast function to do multiple rectangle region checks

Suppose I have a set of "rectangles" and a set of "points": SeedRandom[5] rects = RandomReal[10, {5, 2, 2}] pts = RandomReal[10, {5, 2}] {{{0.00790584, 0....
2
votes
1answer
95 views

revisit Table unpacking issue

This post is related to my previous post, since that post is ended. I start a new here. Because there is still something I don't understand. define ...
1
vote
0answers
72 views

Is there a way to reduce size of complex number?

I would like to create huge matrices with hundreds of thousands of elements per side and I would like to use complex numbers in it for further processing by Mathematica functions. When I investigate ...
13
votes
3answers
203 views

Convert Real packed array of pairs to Complex packed array

I have a Real packed array arr. It may have arbitrary depth but Last@Dimensions[arr] == 2. ...
19
votes
2answers
776 views

How can I determine the type of a packed array?

Packed arrays can be of type Integer, Real or Complex. Knowing that an array ...
2
votes
0answers
115 views

Efficiently create an array of values of a function

I have a function of four arguments, and I would like to have its values on the four-dimensional grid in an array. What is the most efficient way to produce this array, and how do I make it packed? ...
8
votes
1answer
99 views

How to efficiently write into PackedArrays that are stored in Associations?

Using Association to generate complicated data types is very comfortable, in particular if one uses those tricks from How to make use of Associations?. However, I ...
5
votes
1answer
78 views

Unexpected result of Clip

A simple question. Here Clip works as documented: Clip[0.5, {0, 0}, {0,1}] (* 1 *) Values larger than 0 are replaced with ...
5
votes
3answers
217 views

Efficient way to replace a value in packed array of integers

When working with integer label matrices returned by such functions as MorphologicalComponents, ImageForestingComponents etc. it ...
2
votes
0answers
50 views

Memory optimization for exporting “VideoFrames” with many large Array plots

Suppose you have 5 arrays (normalized by the number of arrays - in this case 5): arrayList = Table[RandomChoice[Developer`ToPackedArray@{-1, 1}, {100, 100}]/5,{5}] ...
1
vote
1answer
100 views

Reap unpacks packed arrays

I noticed that Reap unpacks packed arrays. I've looked at previous answers but am still an unsure whether this a serious issue. E.g. consider Ex 47.3 from EIWL: ...
8
votes
1answer
137 views

RegionMeasure on packed array crashes Kernel in version 11.1

Bug introduced in 11.1.0 and fixed in 11.1.1 In version 11.1.0, applying RegionMeasure or Area to a region with packed ...
6
votes
1answer
150 views

PackedArray and Dot results in undesired unpacking

General problem I need to work with two vectors (List) of Real numbers that are quite large (each vector containing about $4\...
0
votes
1answer
49 views

Using Set without unpacking an array

I'm trying to optimise my code by making us of packed array. However, it appears that using Set in combination with Part ...
13
votes
0answers
106 views

Why does MapAt unpack individual elements?

(In case this is version specific I am using Mathematica 10.1 under Windows.) While responding to Partition array without unpacking I noticed something about MapAt ...
3
votes
2answers
172 views

Accelerating the construction of a ParallelTable

I wish to calculate a rather dense antisymmetric matrix quite fast based on a random distribution of points (and their radii) in the three dimensional space as follows: ...
13
votes
1answer
394 views

Why are these timings for Pick so different?

I construct two lists, to be used in Pick as the second argument. The first one is a -1, 1 list and the second one is identical, but with -1 replaced with 0. ...
1
vote
0answers
42 views

Looking for a package for data on (sparse) uniform grids

I have large amounts of sparse 2d and 3d "volumetric" data, i.e. fixed data associated with positions in a uniform (integer coordinate) grid. I think of it as partial function $$f: \mathbb Z^n \to D.$...
2
votes
3answers
178 views

Getting directly a packed array of repeated packed arrays

Consider this code: Needs["Developer`"]; N@Range[5]; PackedArrayQ@% Table[%%, 5]; PackedArrayQ@% which prints True, ...
13
votes
3answers
1k views

Fast way to export large amount of data in “Table” format

I have about a million 3D points, say data = Table[{1. x, 1. y, 0.}, {x, 1000}, {y, 10^3}]~Flatten~1; I want to visualize them, but ...
3
votes
1answer
78 views

Nested list can not be packed?

Why can't nested list be packed? In[252]:= ftest = {1., {3.}}; ftest = Developer`ToPackedArray[ftest]; Developer`PackedArrayQ[ftest] Out[254]= False
11
votes
1answer
299 views

Why does Outer sometimes return a packed array, and sometimes not?

For example In[342]:= test3 = Flatten[Outer[Plus, Range[1000], Range[1000]], 1]; // AbsoluteTiming Out[342]= {0.0198735, Null} returns PackedArray (...
10
votes
1answer
308 views

Speed up construction of a dense matrix

I wish to find the numerical eigenvalues of a relatively small (100 dimensional) Hermitian matrix. While the eigenvalue calculation itself is fast, the construction of the matrix is taking longer than ...
6
votes
1answer
148 views

Why are so many built-in numerical function doesn't check its argument is packed array or not? [closed]

This question is brought by this one. Outer[Times, RandomReal[1., 1000000], {1., 2.}]; is much slower than ...
0
votes
0answers
76 views

Handling really large Arrays (by possibly using lower precision) [duplicate]

I know there have been a few questions about efficient memory storage on mathematica.stackexchange, but I could not find a sufficient answer for my problem (except that it might not work at all). So ...
2
votes
0answers
109 views

Compilation of a function that needs to call Subsets

I'm working on a code that runs many (easy) operations on combinations (subsets) of different size (6, 7, 8, 9) taken from a big set of numbers (normally more that 40). So, this means the code has to ...
11
votes
2answers
415 views

Building sparse array from packed arrays

Suppose I have two large packed arrays a and b, that respectively contain the position of the elements of a square matrix (i.e. <...
5
votes
2answers
166 views

ReplaceAll performance problem: packed arrays on the LHS are unpacked when the RHS is too long

The following is one of the most mysterious performance problems I encountered and came up while extending the booleval function mentioned here. I am looking for a ...
5
votes
1answer
367 views

Isn't it possible to Pack a SparseArray?

I have the following matrix Va: ...
18
votes
1answer
487 views

Unpacked eigenvectors of complex matrices

Edit: This bug is fixed in Mathematica 10.0.0 I want to calculate eigenvectors of a Hermitian matrix. For example ...
4
votes
1answer
151 views

Memory usage of timeseries - lists of {Integer32, Real32}

A list of {Integer32, Real32} items in Mathematica seems to use 12-times as much memory than the C equivalent array of structs. ...
24
votes
1answer
594 views

Why does list assignment with a packed array result in unpacked values?

Bug introduced in 5.2 or earlier and fixed in 10.1.0 This is another question on the design choices in Mathematica. I understand that without direct reply from the developers it may not be possible ...
1
vote
1answer
123 views

Memory handling by ConstantArray

I noticed the following issue when using ConstantArray: a = ConstantArray[0., {10000, 100}]; ByteCount[a] (* 8000152 *) Then ...
7
votes
4answers
318 views

Why does array packing in Table behave like this?

Consider the following example, which generate two identical arrays. Why sometimes the array is packed and sometimes it doesn't? And why they perform differently? Here the arrays are packed: ...
7
votes
2answers
510 views

Fastest way to test if a numerical array has complex elements

I am looking for the fastest way to test whether an n-dimensional numerical (NumericQ) array (in the ArrayQ sense) is complex or ...
21
votes
2answers
1k views

Why does ImageData need four times more memory?

Background We are dealing with very large TIFF image files that are imported, processed and exported using Mathematica 9.0.1. Many of our algorithms only work with an array representation of the ...
5
votes
1answer
131 views

Discrepancy with TakeWhile between RandomInteger result and list on Linux

Bug introduced in 8.0.0 and fixed in 8.0.1 I have to following code that solves a problem for me. The example that is given to us is as follows: ...
2
votes
3answers
1k views

Use elements of Array inside function definition

I'm trying to define a function which modifies and sums over the elements of an array. Here is a very simplified version. myF[kmax_] = Sum[v[[k]]/k, {k, 1, kmax}] ...