$A$ is symmetric so you don't have to take the transpose of $A$.
A = {{a11, a12, a13},
{a12, a22, a23},
{a13, a23, a33}} 2/n;
B = {b1, b2, b3}/n
FullSimplify[Inverse[A.A].A.B]
(* {(a23^2 b1 - a22 a33 b1 + a12 a33 b2 + a13 a22 b3 - a23 (a13 b2 + a12 b3))/
(2 (a13^2 a22 - 2 a12 a13 a23 + a12^2 a33 + a11 (a23^2 - a22 a33))),
(a12 a33 b1 + a13^2 b2 - a11 a33 b2 + a11 a23 b3 - a13 (a23 b1 + a12 b3))/
(2 (a13^2 a22 - 2 a12 a13 a23 + a12^2 a33 + a11 (a23^2 - a22 a33))),
(a13 a22 b1 - a12 a23 b1 - a12 a13 b2 + a11 a23 b2 + a12^2 b3 - a11 a22 b3)/
(2 a13^2 a22 - 4 a12 a13 a23 + 2 a11 a23^2 + 2 a12^2 a33 - 2 a11 a22 a33)} *)
A more visually informative approach is the following:
A = 2 {{sxx, sxy, sxz},
{sxy, syy, syz},
{sxz, syz, szz}};
B = {bx, by, bz};
FullSimplify[Inverse[Transpose[A].A].Transpose[A].B]
(* {(bz sxz syy - bz sxy syz - by sxz syz + bx syz^2 + by sxy szz - bx syy szz)/
(2 sxz^2 syy - 4 sxy sxz syz + 2 sxx syz^2 + 2 sxy^2 szz - 2 sxx syy szz),
(-bz sxy sxz + by sxz^2 + bz sxx syz - bx sxz syz - by sxx szz + bx sxy szz)/
(2 (sxz^2 syy - 2 sxy sxz syz + sxy^2 szz + sxx (syz^2 - syy szz))),
(bz sxy^2 - by sxy sxz - bz sxx syy + bx sxz syy + by sxx syz - bx sxy syz)/
(2 sxz^2 syy - 4 sxy sxz syz + 2 sxx syz^2 + 2 sxy^2 szz - 2 sxx syy szz)} *)
Another approach is to keep the full sums. I don't recommend that as one can see from the output:
Here's the output: