Avoiding zeros in ArrayReshape

Question

Consider the list list={1,2,3,4,5,6,7,8} and imagine I wanted to change it to {{1,2,3},{4,5,6},{7,8}}. How can I do this?

Using ArrayReshape in the usual manner

list = {1, 2, 3, 4, 5, 6, 7, 8};
ArrayReshape[list, {3, 3}]

yields {{1, 2, 3}, {4, 5, 6}, {7, 8, 0}}. Is it possible to use ArrayReshape and avoid that extra 0 at the end? Essentially, I want to reshape my list according to its size.

Answer 1

list = {1, 2, 3, 4, 5, 6, 7, 8};
Partition[list, UpTo[3]]

{{1, 2, 3}, {4, 5, 6}, {7, 8}}

Answer 2

It seems that using ArrayReshape and modifying, if needed, the last entry to remove trailing zeros is faster than using Partition or using Inactive @ Nothing padding:

ClearAll[f1, f2, f3, f4]
f1 = Partition[#, UpTo[#2]] &; (* from Okkes's answer *)
f2 = Partition[#, #2, #2, 1, {}] &;
f3 = Activate @ ArrayReshape[#, 
  {Ceiling[Length[#]/#2], #2}, Inactive @ Nothing] &; (* from MarcoB's answer *)
f4 = Module[{p = ArrayReshape[#, {Floor[Length[#]/#2], #2}], m = Mod[Length@#, #2]},
      If[m == 0, p, Join @@ {p, {#[[-m ;;]]}}]] &;

Timings:

n = 999999;
k = 199;
list = Range[n];

First[RepeatedTiming[r1 = f1[list, k]]]

0.078

First[RepeatedTiming[r2 = f2[list, k]]]

0.075

First[RepeatedTiming[r3 = f3[list, k]]]

0.094

First[RepeatedTiming[r4 = f4[list, k]]]

0.012

r1 == r2 == r3 == r4

True

When n is divisible by k (so that ArrayReshape produces a rectangular array with no paddings):

k = 99;

First[RepeatedTiming[r1 = f1[list, k]]]

0.083

First[RepeatedTiming[r2 = f2[list, k]]]

0.075

First[RepeatedTiming[r3 = f3[list, k]]]

0.0072

First[RepeatedTiming[r4 = f4[list, k]]]

0.0071

r1 == r2 == r3 == r4

True

Answer 3

Using Partition with UpTo is clearly the most expedient answer here, as shown above. However, this got me wondering how one could force ArrayReshape into service anyway, just for fun. For instance, we could twist the use of the padding parameter in ArrayReshape to force it to pad with Nothing (or the vanishing function ##&[]), as in either of the following, which is then removed when evaluated:

ArrayReshape[Range[8], {3, 3}, Hold@Nothing] // ReleaseHold
ArrayReshape[Range[8], {3, 3}, Inactive@Nothing] // Activate

{{1, 2, 3}, {4, 5, 6}, {7, 8}}

---

After posting the answer, I refreshed the page to see that this was brought up in comments as well.

Answer 4

ArrayReshape is implemented using Internal`Deflatten (at least in version 10.1). This function requires $n \times m$ elements, or at least behaves strangely when this is not provided. I am guessing this relates to memory pointers in the low-level implementation.

Internal`Deflatten[Range@8, {3, 3}]
Internal`Deflatten[Range@8, {3, 3}]
Internal`Deflatten[Range@8, {3, 3}]
Internal`Deflatten[Range@8, {3, 3}]

{{1, 2, 3}, {4, 5, 6}, {7, 8, 281500746514433}}

{{1, 2, 3}, {4, 5, 6}, {7, 8, 300647710721}}

{{1, 2, 3}, {4, 5, 6}, {7, 8, 411062144}}

{{1, 2, 3}, {4, 5, 6}, {7, 8, 7164775599194924370}}

Note: If we unpack the list it doesn't work at all:

list = List @@ Range@8;
Developer`PackedArrayQ[list]
Internal`Deflatten[list, {3, 3}]

False

Internal`Deflatten[{1, 2, 3, 4, 5, 6, 7, 8}, {3, 3}]

This is ultimately why something like this also fails:

 ArrayReshape[Range@8, {3, 3}, Unevaluated @ Unevaluated[## &[]] ]  (* does not work *)

To avoid this any input to ArrayReshape is first padded with ArrayPad or trimmed to $length = n \times m$.

Partition drops in speed when using {} padding, at least in some cases, probably due to working around a similar low-level implementation. I used a method similar to kglr's but for Partition here:

• Reversibly merging sets of $k$ adjacent elements in an array
• Partition array without unpacking

Closely related:

• UnFlatten an array into a ragged matrix

---

Silly games

Just having fun using the above knowledge to break ArrayReshape:

ClearAll[foo]
foo /: {x__Integer, foo[] ..} := Developer`ToPackedArray[{x}]

ArrayReshape[{1, 2, 3, 4, 5}, {3, 3}, foo[]]

{{1, 2, 3}, {4, 5, 470483008}, {36571776, 8, 5100273664}}