Linear Algebra in Arbitrary Precision - SLOW

Question

I am trying to implement an arbitrary precision algorithm, but I am not very familiar with Mathematica or arbitrary precision arithmetic. I was able to implement it but surprised by how much slower it is than a machine precision implementation and worried not that I am missing something.

Basic arithmetic seems unreasonably slow in arbitrary precision compared to machine precision. Example:

a = RandomReal[{-10, 10}, {1000, 20}];
Timing[a.Transpose[a];]
Timing[SingularValueDecomposition[a, 20];]

The matrix multiplication takes 0.00657 seconds, the SVD 0.00414 seconds. Now if I would do the same for a number where I set the precision, even if the precision is lower than machine precision, I find:

b =  RandomReal[{-10, 10}, {1000, 20}, WorkingPrecision -> 5];
Timing[b.Transpose[b];]
Timing[SingularValueDecomposition[b, 20];]

where the multiplication takes 7.57129 seconds (!) and the SVD 0.574416 seconds. Is it possible that there are some basic operations that are efficient for arbitrary precision numbers and some are not? How could I make these operations faster? Are there packages available for arbitrary precision linear algebra? I apologise if I am missing something obvious - not a seasoned user.

Answer 1

Setting $MinPrecision and $MaxPrecision equal saves some time:

bb = RandomReal[{-10, 10}, {1000, 20}, WorkingPrecision -> 5];
Block[{$MinPrecision = 5, $MaxPrecision = 5},
 {AbsoluteTiming[bb.Transpose[bb];],
  AbsoluteTiming[SingularValueDecomposition[bb, 20];]}
 ]
(*  {{1.05731, Null}, {0.628201, Null}}  *)

In effect, the setting turns off precision-tracking. The result suggests that the precision-tracking computations might skipped in the arbitrary-precision computations.

As @user6014 points out, arbitrary-precision arithmetic is carried out by software, whereas machine-precision arithmetic is carried out by hardware. This accounts for most of the speed difference. (Some operations are further optimized for hardware arithmetic for packed arrays, in the MKL, in BLAS, and so forth.)

Some references:

Some Notes on Internal Implementation contains some notes about the use of machine-precision and arbitrary-precision numbers in linear algebra. They are not particularly enlightening is this case, though.

The tutorial Machine-Precision Numbers mentions the difference in speed and makes other comparisons of using machine-precision and arbitrary-precision numbers.