For contest problems (and real mathematical work), brute-force application of Mathematica often does not suffice. The software is better used as a tool for discovery and understanding. This answer is intended to illustrate that process.
Like Mathematica itself, people make progress by identifying and trandforming patterns. In this problem several strongly patterned features are evident. There are several ways we could characterize and exploit them, but I think the core ideas common to any successful attack will include some aspect of these two:
On the left side of the equations we see four terms of the form $f(u) = \sqrt{u^2 + 1/u^2}$ where $u$ is variously equal to $\cos(x)$, $\cos(y)$, $\sin(x)$, and $\sin(y)$.
On the right side of the equation we see two terms which are square roots of the form $20\frac{x}{x+y}$. Notice that these are homogeneous: simultaneously rescaling $x$ and $y$ will not change either fraction. We ought therefore to think of these terms as function of the ratio $r = x/y$. Let's write $g(r) = \sqrt{20 \frac{1}{1+r}}$, so that the two right sides are $g(r)$ and $g(1/r)$.
Let's begin the Mathematica exploration, then, by implementing $f$ and $g$ and plotting their values so that we can understand how these functions behave.
f[u_] := Sqrt[u^2 + 1/u^2];
g[r_] := Sqrt[20 / (1 + r)];
Plot[f[u], {u, 0, 1}, AxesOrigin -> {0, 0}]
Plot[g[r], {r, -1, 3}, AxesOrigin -> {-1, 0}]
Because this is a system of equations, we should expect to have to recombine them somehow. One's first thoughts would range among adding, subtracting, squaring, multiplying, and dividing them. Addition, subtraction, and division promise to be simplest and to exhibit the most symmetry. Taking addition to be the easiest, let's try it first. The sum of the two equations is
$$\left(f(\sin x) + f(\cos y)\right) + \left(f(\sin y)+ f(\cos x)\right) = g(x/y) + g(y/x).$$
The left hand side clearly can be arranged to equal the sum of a function of $x$ and the same function of $y$:
$$\left(f(\sin x) + f(\cos x)\right) + \left(f(\sin y) + f(\cos y)\right) = g(x/y) + g(y/x).$$
This is progress, because it indicates we should be studying two univariate functions, $f_0(t) = f(\sin t) + f(\cos t)$ and $g_0(r) = g(r) + g(1/r)$. Let us again ask Mathematica for visual help in understanding them:
f0[t_] := f[Sin[t]] + f[Cos[t]];
g0[r_] := g[r] + g[1/r];
Plot[f0[t], {t, -\[Pi], \[Pi]}, AxesOrigin -> {-\[Pi], 0},
Ticks -> {Range[-\[Pi], \[Pi], \[Pi]/2]}]
Plot[g0[r], {r, -1, 3}, AxesOrigin -> {-1, 0}]
The really interesting and striking things that emerge from inspecting these plots are
$f_0$ is periodic with period $\pi/2$. In retrospect, that's obvious and easy to prove.
$f_0$ has a lower bound which, by virtue of the periodicity and symmetry of $f_0$, occurs at $\pi/4$ plus all integral multiples of $\pi/2$. It's easy to calculate, but let's confirm:
f0[\[Pi]/4]
$\sqrt{10}$
$g_0$ is defined only for non-negative values (which is trivial to show, now that we have seen it) and has an upper bound. We can find this using methods of calculus, but for now let's just consult Mathematica:
Maximize[g0[r], r]
$\left\{2 \sqrt{10},\{r\to 1\}\right\}$
It is now immediate that the left hand side of the sum of the equations cannot be any less than the minimum of $f_0(x)$ plus the minimum of $f_0(y)$; namely, $\sqrt{10} + \sqrt{10}$, and that the right hand side cannot be any greater than the maximum of $g_0(r)$; namely, $2 \sqrt{10}$. These values are equal!
This, the key insight, is the climax of the analysis. It establishes that the sum of the two equations can be true if and only if $f_0(x)$ and $f_0(y)$ are simultaneously at a minimum and $g_0(x/y)$ is at a maximum (which Mathematica has indicated occurs uniquely when $x/y=r=1$). Thus, remembering the periodicity of $f_0$,
$$x = y\ \text{ and }\ x = \pi/4 + n \pi/2 \quad (n \in \mathbb{Z})$$
are necessary conditions for any solution.
Plugging these values into the original equations shows that they both hold: we have found all the solutions.
Backtracking through this exploration will establish a program for proving the answer is correct. The only missing piece is finding the unique global minimum of $g_0$. This can be done in an elementary way (that is, without Calculus) by writing $t = \frac{1}{1+r}$, so that $g(r) = \sqrt{20}\sqrt{t}$ and $g_0(r) = \sqrt{20}\left(\sqrt{t} + \sqrt{1-t}\right)$. The common factor of $\sqrt{20}$ does not affect the position of the maximum. Concerning what's left, let $u = \sqrt{t}$ and $v = \sqrt{1-t}$. Then $u^2 + v^2 = 1$, showing that we need to maximize the linear form $u+v = (1,1)\cdot(u,v)$ along the unit circle. It is geometrically obvious (and easy to prove using a little vector arithmetic) that the maximum occurs when $(u,v)$ is parallel to $(1,1)$, whence $u=v$, implying $t=1/2$ and then $r=1$, QED.
ContourPlot) them out would help you gain some useful intuition? – Silvia Jan 11 at 16:35