**
Edit 5
**
Thank you all guys for your helpful comments. Based on what KAI and Oleksandr R. mentioned, and as far as I could understand, LinearSolve uses all capacity of the CPU cores to solve the equation involving large matrices. Consequently, it seems that if I want to solve some linear matrix equations for a few times, the best method is to solve each of them one by one in a LOOP to make it most efficient. In this way Mathemathica is able to use all capacity of CPU cores in each step and solve the equation in the most efficient way (in each step) and goes to the next step. But if you have a look at these 2 examples, apparently it is not like this and Parallelization forces Mathematica to solve the problem involving LinearSolve in a way that is likely more efficient and faster. You can check these examples on your computer. Based on the comments we had here, I am wondering how we can explain these examples.
Example 1:
Clear["Global`*"];
t = AbsoluteTime[];
NN = 8;
CC = Array[cc, NN];
For[i = 1, i < (NN + 1), i++,
Clear[a, b];
a = RandomReal[{i, i + 1}, {6000, 6000}];
b = RandomReal[{i}, {6000}];
CC[[i]] = LinearSolve[a, b];
];
time2 = AbsoluteTime[] - t
For example 1 CPU usage is 50% and time2=23.4
Example 2:
Clear["Global`*"];
t = AbsoluteTime[];
job1 = ParallelSubmit[a1 = RandomReal[{1, 2}, {6000, 6000}]; b1 = RandomReal[{1}, {6000}]; c1 = LinearSolve[a1, b1]];
job2 = ParallelSubmit[a2 = RandomReal[{2, 3}, {6000, 6000}]; b2 = RandomReal[{2}, {6000}]; c2 = LinearSolve[a2, b2]];
job3 = ParallelSubmit[a3 = RandomReal[{3, 4}, {6000, 6000}]; b3 = RandomReal[{3}, {6000}]; c3 = LinearSolve[a3, b3]];
job4 = ParallelSubmit[a4 = RandomReal[{4, 5}, {6000, 6000}]; b4 = RandomReal[{4}, {6000}]; c4 = LinearSolve[a4, b4]];
job5 = ParallelSubmit[a5 = RandomReal[{5, 6}, {6000, 6000}]; b5 = RandomReal[{5}, {6000}]; c5 = LinearSolve[a5, b5]];
job6 = ParallelSubmit[a6 = RandomReal[{6, 7}, {6000, 6000}]; b6 = RandomReal[{6}, {6000}]; c6 = LinearSolve[a6, b6]];
job7 = ParallelSubmit[a7 = RandomReal[{7, 8}, {6000, 6000}]; b7 = RandomReal[{7}, {6000}]; c7 = LinearSolve[a7, b7]];
job8 = ParallelSubmit[a8 = RandomReal[{8, 9}, {6000, 6000}]; b8 = RandomReal[{8}, {6000}]; c8 = LinearSolve[a8, b8]];
{R1, R2, R3, R4, R5, R6, R7, R8} = WaitAll[{job1, job2, job3, job4, job5, job6, job7, job8}];
time2 = AbsoluteTime[] - t
For example 2 CPU usage=100% and time2=19.8