# Using dense matrix is faster than sparse matrix in LinearSolve [closed]

I'm doing FEM with Mathematica and have to deal with LinearSolve of a large matrix inside a Newton loop constantly. Hence I need to speed up the computation efficiency of LinearSolve. I'm aware of this post.

Similarly, I test with the following code. First I tried with dense matrix:

Clear["Global*"];
dim = 5000;
a = RandomReal[{1, 2}, {dim, dim}];
b = RandomReal[{1}, {dim}];
Print["Case=", i];
t = AbsoluteTime[];
LinearSolve[a, b];
time2 = AbsoluteTime[] - t;
Print["t(", i, ")=", time2];
Print["******"], {i, 4}];


The output is:

Case=1

t(1)=2.155357

******

Case=2

t(2)=1.279845

******

Case=3

t(3)=1.525075

******

Case=4

t(4)=1.246443

******


From the above test, it seems that the LinearSolve is automatically parallelized, which is good. My laptop has only 2 kennels hence no much improvement for 2,3, and 4 threads. Then I run it again with sparse matrix:

Clear["Global*"];
dim = 5000;
a = RandomReal[{1, 2}, {dim, dim}];
b = RandomReal[{1}, {dim}];
sa = SparseArray[a];
sb = SparseArray[b];
Print["Case=", i];
t = AbsoluteTime[];
LinearSolve[sa, sb];
time2 = AbsoluteTime[] - t;
Print["t(", i, ")=", time2];
Print["******"], {i, 4}];


However, it seems that LinearSolve needs much more time to deal with sparse matrix then with dense matrix.

Case=1

t(1)=11.346750

******

Case=2

t(2)=5.508803

******

Case=3

t(3)=5.551608

******

Case=4

t(4)=5.518430

******


This seems a little bit weird to me since I have experienced the improved efficiency by converting a dense matrix into sparse matrix in Matlab. Can anyone explain this? Is there any other way to speed up LinearSolve?

• Using a sparse array is effective when the array is actually sparse. You are converting a dense array into a sparse array that has a sparsity of 0, so I'm not surprised that sparse methods are slower. – Carl Woll Nov 8 '17 at 23:28
• If sparse matrix methods were faster than methods for dense matrices, when working with dense matrices, then the dense methods would disappear. Voting to close on grounds that there is no clear reason to expect behavior any different from what was observed. – Daniel Lichtblau Nov 9 '17 at 0:00
• As already noted, trying to use sparse methods on a demonstrably dense matrix is a fool's errand. You had chosen an especially poor example to present your premise. – J. M.'s ennui Nov 9 '17 at 0:55
• I'm voting to close this question as off-topic because the issued raised is not really a problem; it is arises from the OP's not having the necessary understanding of the Mathematica objects he is working with. – m_goldberg Nov 9 '17 at 2:09
• With only 2 kennels there is an upper bound how many dogs you can have running efficiently in parallell. – mathreadler Nov 9 '17 at 6:54

You cannot benefit from SparseArray if your matrix is not sparse! The data structure storing a SparseArray needs a certain amount of overhead which only pays off if your matrix is really, really sparse (density below a permille or so). Moreover, whenever you find yourself converting a large matrix to a sparse matrix you have actually done something wrong. SparseArrays have to be constructed in a sparse way by special constructor functions (the constructor functions are called SparseArray in Mathematica and sparse in MATLAB).
But admittedly, a point that is bugging me is that Mathematica uses UMFPACK for factorizations of SparseArrays (as backend of LinearSolve). While UMFPACK is quite robust, it is rather slow, extremely memory hungry, and not parallelized. However, we have the alternative to use the mostly undocumented option Method->"Pardiso" in LinearSolve which uses the Intel MKL Pardiso solver; and Pardiso is (partially) parallelized (back substitution is hardly parallelizable).
• Yes, that's what I say. Still, LinearSolve will perform magnitudes better for SparseArrays arising in FEM than for the according densified matrices. – Henrik Schumacher Nov 9 '17 at 0:12
• I think you may assume that the default parameters "ParallelOptions" are already optimized for your system. – Henrik Schumacher Nov 9 '17 at 0:13