# Sporadic bugs in multidimensional SparseArray objects

Udate

This SparseArray[] bug is (apparently) fixed on version 10.0

$System//Print;$Version//Print;

Mac OS X x86 (64-bit)
10.0 for Mac OS X x86 (64-bit) (June 29, 2014)


Thanks, Wolfram!

Resolution

• WRI affirms that SparseArray[__] is broken (in 9.0 and 9.01)
• WRI's suggested workaround is SparseArray//Normal

My experience with this particular SparseArray bug has been that reliable coding requires that the Normal workaround be applied to every instance of SparseArray, because the failures of SparseArray are sporadic, occult, and associated to unknown preconditions.

Ouch!

My thanks are extended to all who commented.

Wolfram Technical Support
email: support@wolfram.com
Re: [CASE:646986]
Sporadic bugs (?) in multidimensional SparseArray objects

Hi John,

Thank you for getting back to us.

I understand that for now your temporary workaround
of aTest = aTest // Normal is holding the fort.

Our developers are working on this issue and
we anticipate that the resolution will appear
in the future release of Mathematica.

Sincerely,

<name redacted>

Wolfram Technology Group
Wolfram Research, Inc.


What appear to be sporadic bugs (?) in multidimensional SparseArray objects (?) are plaguing our code development efforts.

\$Version//Print;
9.0 for Mac OS X x86 (64-bit) (January 24, 2013)


The following code elicits wrong answers and/or kernel crashes:

Do[
(* aTest is a SparseArray[...] that crashes the kernel and/or
yields incorrect results unless wrapped in Normal[...] *)

aTest = RandomReal[NormalDistribution[0,1], {2,2,2,2}] +
I*RandomReal[NormalDistribution[0,1], {2,2,2,2}]//
SparseArray[
{
{i_,j_,k_,1 ,i_} :> #[[k,1,i,j]]*#[[k,2,1,j]],
{i_,j_,k_,2 ,j_} :> #[[k,1,i,j]]*#[[k,2,2,i]],
{i_,j_,k_,n_,j_} :> #[[k,1,i,j]]/;Equal[n-2,i]
},
{2,2,2,4,2},
Complex[0.0,0.0]
]&//Flatten[#,1]&//
Transpose[#,{4,1,2,3}]&//
Flatten[#,2]&//
Transpose;

(* enable the following line to "fix" aTest *)
(* aTest = aTest//Normal; *)

(* the following all should be 0.0 (to machine precision) *)

(aTest//Normal)-aTest//
Flatten//Abs//Max//Print["      (aTest//Normal)-aTest =?= 0 = ",#]&;

id = SparseArray[{i_,i_}\[Rule]1,{16,16},0];
(aTest//Normal).id-aTest.id//
Flatten//Abs//Max//Print["(aTest//Normal).id-aTest.id =?= 0 =",#]&;
,{10}];


Results (when the kernel doesn't crash immediately)

      (aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =7.43925
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =3.14861
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =1.27373
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =1.8666
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =2.03269
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =3.26388
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =1.30013
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =3.14138
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =0.953447
(aTest//Normal)-aTest =?= 0 = 0.
(aTest//Normal).id-aTest.id =?= 0 =0.841252


Remarks Variants of the above code crash/fail for Complex-valued and Integer-valued SparseArrays; apparently the multidimensionality is the essential trait.

Help, advice, and workarounds all are appreciated!

-
This is clearly a bug, present in version 9.0.1. I can reproduce the crash if your input is the very first thing I evaluate after a kernel restart. Otherwise I can reproduce the wrong results. You might want to report it to tech support. I'd like to note that version 8.0.4 doesn't have this bug. –  Szabolcs Mar 8 '14 at 16:33
I observe this behavior is also present in V9.0.1. Definitely should be reported to WRI. –  m_goldberg Mar 8 '14 at 18:13
@JohnSidles Write to support at wolfram.com. When you are certain that something is a bug, there's little use posting on this site, as this is not an official WRI site. The only thing people can do here is try to come up with workarounds. –  Szabolcs Mar 10 '14 at 0:43
@Szabolcs I have followed your suggestion. When you say: "The only thing people can do here [on Mathematica StackExchange] is try to come up with workarounds", that is not quite true! The StackExchange is cross-indexed, searchable, archival, publicly responsive, and rapidly verifying ... and these valuable qualities are much-appreciated. Thanks are extended to all who commented! –  John Sidles Mar 10 '14 at 3:32
Update: WRI is working on this bug: emails to "support at wolfram.com" with the subject "Re: [CASE:646986] Sporadic bugs (?) in multidimensional SparseArray objects" will be forwarded to the WRI development team. NOTE: there's no need to cc: me ... for my purposes the workaround of wrapping with "Normal" is acceptable. –  John Sidles Mar 10 '14 at 22:51

UPDATE

The bug is fixed in MMA 10.0 (see original question for details)

Answering my own question per Szabolcs' sugggestion:

• WRI affirms that SparseArray[__] is broken (in 9.0 and 9.01)
• WRI's suggested workaround is SparseArray//Normal
My experience with this particular SparseArray bug has been that reliable coding requires that the Normal workaround be applied to every instance of SparseArray, because the failures of SparseArray are sporadic, occult, and associated to unknown preconditions.