Skip to main content

Questions tagged [packed-arrays]

Tag for questions about packed arrays. Packed Arrays are data structures that can be seen as monolithic blocks containing a single data type, particularly Integers and Reals. Many functions operate very efficient on packed arrays. Questions may deal with packed array generation, conversion, and usage.

Filter by
Sorted by
Tagged with
4 votes
1 answer
74 views

Is there a non-allocating way to modify elements of single-reference packed arrays?

I need to repeatedly increment cuboidal chunks of a 3D image. The image is large - on the order of 1G elements. The only way I know of doing it using built-in Image functions is: ...
Kuba hasn't forgotten Monica's user avatar
0 votes
3 answers
112 views

Generalize the code to more variables

I have this code it runs and gives me the solution. How can I make it more compact and If I want it to extend(generalize) it to more variables how can I do it. I had asked a similar question here (...
Learner's user avatar
  • 219
13 votes
1 answer
318 views

How to convert the ListVector into PackedArray in FunctionCompile

Cross post here I have a compile function cf to calculate the mean for a ragged list: ...
yode's user avatar
  • 26.8k
0 votes
0 answers
70 views

Packed/Unpacked array - what object/like 0 or 1?/how to check/ [duplicate]

In Mathematica, there is a concept called packed array. ...
imida k's user avatar
  • 4,295
2 votes
1 answer
66 views

What circumstances would cause memory to inflate while packing the sublists of a ragged matrix?

I'm working with a ragged matrix of considerable size (ByteCount = 47749114088; Length = 563529128). I had seen a suggestion mentioned in other questions (here on StackExchange), that one might be ...
Todd Allen's user avatar
  • 2,124
3 votes
0 answers
84 views

Fixing a step size with a variable causes array to be unpacked?

I would expect the following code to produce a packed 11x11x11 element array of 64 bit integers, but it doesn't: ...
Bruce Bartlett's user avatar
6 votes
1 answer
77 views

Unexpected unpacking of wrapped PackedArray

I'm confused as to why the following leads to unpacking of a PackedArray: ...
TimRias's user avatar
  • 3,180
5 votes
1 answer
61 views

Why is extracting values from an Association with 1 level slower than extracting values from a list of depth 1?

I am not looking for a solution or work-around, but an explanation. Here are six examples of doing a trivial operation on an association and a list. Initially, I was curious about the cost of calling ...
Craig Carter's user avatar
  • 4,510
1 vote
0 answers
90 views

Unpacking array message in Compile and Fold

Suppose, I have a nonlinear map, like this: ...
macros's user avatar
  • 99
5 votes
2 answers
153 views

Unexpected timing result of Total and Part performance of packed array

Define a = RandomReal[1., {2, 100, 100, 100}]; then Total[a, Infinity]; // RepeatedTiming {0.00183727, Null} and ...
matheorem's user avatar
  • 17.2k
2 votes
1 answer
41 views

Compression results in unpacking of integers but not reals

Consider the following; Pack some integers. In[287]:= Clear[p5i, p5ic, p5icu, p5r, p5rc, p5rcu] In[288]:= p5i = ToPackedArray@Range@5 Out[288]= {1, 2, 3, 4, 5} ...
RobertNathaniel's user avatar
1 vote
0 answers
37 views

spontaneously quitting kernel, assignment and packed arrays

I am having an issue where the kernel spontaneously quits, deep in a long code that I have used for quite some time without this issue. I've tracked it down to an assignment of the form: a[[All, All, ...
Jansen's user avatar
  • 1,223
4 votes
1 answer
63 views

Can't initialise packed array of rational numbers [closed]

If I evaluate the following code t = Developer`ToPackedArray[{0, 2/7, -(1/7)}]; Developer`PackedArrayQ[t] Mathematica returns ...
Gert's user avatar
  • 1,530
6 votes
1 answer
145 views

Why Does Subsets[...,{n}] not Output a Packed List, Even Though it Doesn't Unpack?

Assume list is packed. I expect Subsets[] is a structural operation because it depends on the number of elements, not on what ...
Just Some Old Man's user avatar
1 vote
1 answer
113 views

Excessive usage of memory while using ArrayFlatten or Flatten

I have 16 NxN matrices, where N can be anywhere between 1000-64000.. Some of them are real and some of them are complex. I want to Reshape then to form a 4Nx4N matrix. When I try to accomplish this ...
Roopayan Ghosh's user avatar
10 votes
0 answers
86 views

Different timing with large array assignment

Consider the following example n = 10^8; AbsoluteTiming[ A = ConstantArray[1., n]; // anyFunc; A[[2 ;; ;; 2]] = -1.; ] {0.388108, Null} ...
matrix42's user avatar
  • 7,108
3 votes
1 answer
178 views

Possible speed improvement for numerical linear algebra?

I have a question about further optimising a numerical linear algebra calculation. I've trawled through many useful posts on this site and have ended up with something quite efficient - I was ...
ala10's user avatar
  • 109
2 votes
1 answer
229 views

Clarification on how Total[] can be used on multi-dimensional array

I have a rank 5 tensor that ultimately I want to modify so that for the first 3 dimensions, each element is summed together. The result will be a rank 2 tensor whose elements are the summed totals ...
LooseyGoose's user avatar
12 votes
2 answers
301 views

Unexpected behavior of the procedure `Area` on the object 'Polygon'

Bug introduced in 11.3 or earlier and fixed on 12.0. Sometimes get a results, sometimes left unevaluated. For instance ...
Slepecky Mamut's user avatar
19 votes
4 answers
646 views

Is it possible to avoid unpacking when merging Association?

For example, if we have assoc = AssociationThread[{1, 2, 3} -> RandomReal[1., {3, 10}]]; and ...
matheorem's user avatar
  • 17.2k
7 votes
1 answer
369 views

Avoiding unpacking an array when altering its dimension

I am trying to flatten a matrix from 3-dimensional to 2-dimensional one. The original one has dimension for example {10,120,120}: ...
cj9435042's user avatar
  • 683
12 votes
0 answers
132 views

What changed in String handling between 11.0 and 11.3?

I was trying to optimize some String-handling code here using my standard trick of converting it to a list of bytes first and using basic list operations instead of ...
b3m2a1's user avatar
  • 46.9k
10 votes
1 answer
140 views

Why can't (partially) packed arrays be used in this example?

Bug introduced in 11.3 or earlier and persisting through 12.1 or later. Fixed in 13.0.0 or earlier I didn't expect any difference between these: ...
MeMyselfI's user avatar
  • 1,116
8 votes
1 answer
242 views

How to send packed array from math kernel to 3rd party app via WSTP

Background I'm developing a small python wrapper PyWSTP. The wrapper is still in very early stage and lacking features. One important feature I'm considering is transferring dense array between math ...
kh40tika's user avatar
  • 684
7 votes
1 answer
432 views

Packing and unpacking of arrays with Map/Table

How can I create function, that will not unpack packed arrays with Map/Table. For example I want read bytes from opened stream (this is my original goal): ...
Kirill Belov's user avatar
3 votes
0 answers
71 views

PackedArray version of Chop

I'm a big fan of Chop. It's fast and useful in pre-rounding transforms and things Unfortunately it also necessarily unpacks a real ...
b3m2a1's user avatar
  • 46.9k
4 votes
1 answer
110 views

Strange unpacking of a packed array with Insert

In Mathematica 11.1.1, when a packed array is stored within another list, it gets unexpectedly unpacked when Inserting something into the outer list. Create a ...
Theo Tiger's user avatar
  • 1,273
16 votes
1 answer
171 views

RandomChoice output isn't packed

Why isn't the output of RandomChoice a packed array? For instance: RandomChoice[{.5, .2, .3} -> {1, 2, 3}, 10] Developer`PackedArrayQ @ % {1, 2, 3, 1, 3, 1, ...
Carl Woll's user avatar
  • 131k
8 votes
2 answers
162 views

Fast function to do multiple rectangle region checks

Suppose I have a set of "rectangles" and a set of "points": SeedRandom[5] rects = RandomReal[10, {5, 2, 2}] pts = RandomReal[10, {5, 2}] {{{0.00790584, 0....
Carl Woll's user avatar
  • 131k
3 votes
1 answer
136 views

revisit Table unpacking issue

This post is related to my previous post, since that post is ended. I start a new here. Because there is still something I don't understand. define ...
matheorem's user avatar
  • 17.2k
1 vote
0 answers
178 views

Is there a way to reduce size of complex number?

I would like to create huge matrices with hundreds of thousands of elements per side and I would like to use complex numbers in it for further processing by Mathematica functions. When I investigate ...
StepanCZSK's user avatar
13 votes
3 answers
283 views

Convert Real packed array of pairs to Complex packed array

I have a Real packed array arr. It may have arbitrary depth but Last@Dimensions[arr] == 2. ...
Szabolcs's user avatar
  • 235k
20 votes
2 answers
869 views

How can I determine the type of a packed array?

Packed arrays can be of type Integer, Real or Complex. Knowing that an array ...
Szabolcs's user avatar
  • 235k
2 votes
0 answers
332 views

Efficiently create an array of values of a function

I have a function of four arguments, and I would like to have its values on the four-dimensional grid in an array. What is the most efficient way to produce this array, and how do I make it packed? ...
And R's user avatar
  • 542
10 votes
1 answer
127 views

How to efficiently write into PackedArrays that are stored in Associations?

Using Association to generate complicated data types is very comfortable, in particular if one uses those tricks from How to make use of Associations?. However, I ...
Henrik Schumacher's user avatar
6 votes
1 answer
93 views

Unexpected result of Clip

A simple question. Here Clip works as documented: Clip[0.5, {0, 0}, {0,1}] (* 1 *) Values larger than 0 are replaced with ...
Fred Simons's user avatar
  • 10.2k
5 votes
3 answers
360 views

Efficient way to replace a value in packed array of integers

When working with integer label matrices returned by such functions as MorphologicalComponents, ImageForestingComponents etc. it ...
Alexey Popkov's user avatar
2 votes
0 answers
91 views

Memory optimization for exporting "VideoFrames" with many large Array plots

Suppose you have 5 arrays (normalized by the number of arrays - in this case 5): arrayList = Table[RandomChoice[Developer`ToPackedArray@{-1, 1}, {100, 100}]/5,{5}] ...
Luca Pontiggia's user avatar
1 vote
1 answer
121 views

Reap unpacks packed arrays

I noticed that Reap unpacks packed arrays. I've looked at previous answers but am still an unsure whether this a serious issue. E.g. consider Ex 47.3 from EIWL: ...
TheDoctor's user avatar
  • 2,850
8 votes
1 answer
167 views

RegionMeasure on packed array crashes Kernel in version 11.1

Bug introduced in 11.1.0 and fixed in 11.1.1 In version 11.1.0, applying RegionMeasure or Area to a region with packed ...
Alexey Popkov's user avatar
6 votes
1 answer
169 views

PackedArray and Dot results in undesired unpacking

General problem I need to work with two vectors (List) of Real numbers that are quite large (each vector containing about $4\...
pbx's user avatar
  • 842
1 vote
1 answer
61 views

Using Set without unpacking an array

I'm trying to optimise my code by making us of packed array. However, it appears that using Set in combination with Part ...
user44136's user avatar
14 votes
0 answers
129 views

Why does MapAt unpack individual elements?

(In case this is version specific I am using Mathematica 10.1 under Windows.) While responding to Partition array without unpacking I noticed something about MapAt ...
Mr.Wizard's user avatar
  • 272k
3 votes
2 answers
245 views

Accelerating the construction of a ParallelTable

I wish to calculate a rather dense antisymmetric matrix quite fast based on a random distribution of points (and their radii) in the three dimensional space as follows: ...
M.J.2's user avatar
  • 491
13 votes
1 answer
401 views

Why are these timings for Pick so different?

I construct two lists, to be used in Pick as the second argument. The first one is a -1, 1 list and the second one is identical, but with -1 replaced with 0. ...
Fred Simons's user avatar
  • 10.2k
1 vote
0 answers
54 views

Looking for a package for data on (sparse) uniform grids

I have large amounts of sparse 2d and 3d "volumetric" data, i.e. fixed data associated with positions in a uniform (integer coordinate) grid. I think of it as partial function $$f: \mathbb Z^n \to D.$...
masterxilo's user avatar
  • 5,739
4 votes
3 answers
254 views

Getting directly a packed array of repeated packed arrays

Consider this code: Needs["Developer`"]; N@Range[5]; PackedArrayQ@% Table[%%, 5]; PackedArrayQ@% which prints True, ...
Lorenzo Pistone's user avatar
14 votes
3 answers
3k views

Fast way to export large amount of data in "Table" format

I have about a million 3D points, say data = Table[{1. x, 1. y, 0.}, {x, 1000}, {y, 10^3}]~Flatten~1; I want to visualize them, but ...
masterxilo's user avatar
  • 5,739
3 votes
1 answer
87 views

Nested list can not be packed?

Why can't nested list be packed? In[252]:= ftest = {1., {3.}}; ftest = Developer`ToPackedArray[ftest]; Developer`PackedArrayQ[ftest] Out[254]= False
Al Guy's user avatar
  • 1,610
13 votes
1 answer
349 views

Why does Outer sometimes return a packed array, and sometimes not?

For example In[342]:= test3 = Flatten[Outer[Plus, Range[1000], Range[1000]], 1]; // AbsoluteTiming Out[342]= {0.0198735, Null} returns PackedArray (...
matheorem's user avatar
  • 17.2k