5
$\begingroup$

I have a large array (arr) of size 22 X 225 X 225 and I want it to be reshaped to a size of 22 X 50625. ArrayReshape can do it perfectly well. However, it is too slow.

Is there any way to make it faster?

$\endgroup$
7
  • 2
    $\begingroup$ There is hardly any faster way than ArrayReshape. For the matrix A = RandomReal[{-1, 1}, {22, 225, 225}];, my machine runs B = ArrayReshape[A, {22, 50625}]; // RepeatedTiming in 0.00072 seconds. Try to use packed arrays. $\endgroup$ May 3, 2018 at 15:37
  • $\begingroup$ @HenrikSchumacher I was thinking to flatten each of the 22 arrays of 225 X 225. $\endgroup$
    – user36426
    May 3, 2018 at 15:41
  • $\begingroup$ That could be done with Flatten[A, {{1}, {2, 3}}] or Flatten /@ A, but is much slower, at least in the current version (11.3). $\endgroup$ May 3, 2018 at 15:43
  • $\begingroup$ Almost the same for me in 11.1. ArrayReshape: .0064, Join@@@A: .0064, Flatten/@A: .0048. Interestingly, ArrayReshape is the only one that does not unpack. $\endgroup$
    – Theo Tiger
    May 3, 2018 at 16:08
  • $\begingroup$ I get on order 0.01 sec for both ArrayReshape and Flatten/@A (Flatten marginally faster) with v11.1.1/linux. Something change w/ 11.3? ( I doubt your hardware is 10x faster than mine.. ) $\endgroup$
    – george2079
    May 3, 2018 at 16:11

2 Answers 2

7
$\begingroup$

Indeed, somebody must have put some effort in optimizing ArrayFlatten. That's good to see!

Here a benchmark for different methods and array sizes:

Needs["GeneralUtilities`"];
funs = <|
   "ArrayReshape[#,{...}]&" -> (ArrayReshape[#, {Dimensions[#][[1]], 
        Times @@ Dimensions[#][[2 ;; 3]]}] &),
   "Flatten[#,{{1},{2,3}}]&" -> (Flatten[#, {{1}, {2, 3}}] &),
   "Map[Flatten,#]&" -> (Map[Flatten, #] &),
   "Compile[...][#]&" -> (Compile[{{a, _Real, 2}},
        Flatten[a],
        RuntimeAttributes -> {Listable},
        Parallelization -> True,
        RuntimeOptions -> "Speed"
        ][#] &),
   "Partition[Flatten[#],...]&" -> (Partition[Flatten[#], 
       Times @@ Dimensions[#][[2 ;; 3]]] &)
   |>;
nlist = 2^Range[2, 9];
benchmark = Benchmark[Values[funs], RandomReal[{-1, 1}, {#, #, #}] &, 2^Range[2, 9]];

g = ListLogLogPlot[
  AssociationThread[
   Map[Style[#, Bold, FontFamily -> "Courier"] &, Keys[funs]],
   benchmark
   ],
  Joined -> True,
  PlotMarkers -> Automatic,
  PlotLabel -> Row[{"Timings on Mathematica ", $Version}],
  ImageSize -> Large,
  AxesLabel -> {"n", "Time (s)"},
  Ticks -> {nlist, Automatic}
  ]

enter image description here

enter image description here

Both performed on Intel 4980 HQ @ 2,8 GHz, 16 GB 1600 MHz DDR3.

$\endgroup$
5
$\begingroup$

This is a little too long for a comment. On my machine (MMA 11.2, Win7-64bit), there is a signifcant difference in performance between ArrayReshape and a direct Flatten application, with ArrayReshape being roughly 3x faster:

arr = RandomReal[1, {22, 225, 225}];

RepeatedTiming[ArrayReshape[arr, {22, 50625}];]    (* Out: {0.0065, Null} *)
RepeatedTiming[Flatten[arr, {{1}, {2, 3}}];]       (* Out: {0.019, Null}  *)

RepeatedTiming[Flatten /@ arr;]                    (* Out: {0.2, Null}    *)

Mapping Flatten is very slow in comparison.

The mapping method also unpacks the array:

Developer`PackedArrayQ@arr                                   (* Out: True  *)

Developer`PackedArrayQ@ArrayReshape[arr, {22, 50625}]        (* Out: True  *)

Developer`PackedArrayQ@Flatten[arr, {{1}, {2, 3}}]           (* Out: True  *)

Developer`PackedArrayQ@(Flatten /@ arr)                      (* Out: False *)
$\endgroup$
3
  • $\begingroup$ Yepp, using Map should not be encouraged as much as several years before. When I have to map over packed arrays, I try to do so with a Listable CompiledFunction`. $\endgroup$ May 3, 2018 at 16:28
  • $\begingroup$ @HenrikSchumacher oh no, that's not what i want to hear! a lot of my code is strongly based around mapping; i guess i'll be on the lookout for alternate methods in the future. $\endgroup$ May 3, 2018 at 17:35
  • $\begingroup$ @BenKalziqi Of course, you could do much worse than Mapping, e.g. with For. ;) Of course, you have to balance effort for prototyping, debugging, and maintaining vs. pure runtime. And Map is usually a good trade-off. $\endgroup$ May 3, 2018 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.