How can I create a large matrix without running into hangs or recursion errors?

Question

I'm trying to create a probability matrix, starting by creating a 100,000 by 100,000 identity matrix. However, when I try to create this using:

IdentityMatrix[100000]

The interpreter just stops without an error message or prompt, and the variables turn blue. Why does this happen and How can I get around it?

Answer 1

This happens because you asked Mathematica to allocate this amount of memory (unless told otherwise, Mathematica tries to allocate a packed array of 64 bit integers):

UnitConvert[100000^2 Quantity[64, "Bits"], "Gigabytes"]

Quantity[80, "Gigabytes"]

Might be a bit too much for your system.

Better use

IdentityMatrix[100000, SparseArray]

In general, I suggest to get in touch with SparseArray in order to learn how to construct sparse matrices efficiently.