Some DEs are more simple to solve for the dependent variable rather than the independent variable, for example
$$ \frac{dy}{dx} = y \quad\implies\quad \log(y)=x+c $$
from which you can obtain the solution for $y$ in terms of $x$ by using an inverse function, in this case $y=\exp(x+c)$.
Not all examples are this easy to invert, so Mathematica sometimes has to leave the solution written in terms of InverseFunction.
The # and & are part of Mathematica's pure (or anonymous) function notation. In particular & occurs at the end of a pure function and #=#1 represents the first slot of the function. For example
(#^2 + 1&)
is equivalent to
Function[{x}, x^2 + 1]
and acts upon its arguments like any other function
(#^2 + 1&)[t] == Function[{x}, x^2 + 1][t] == t^2 + 1
So, your DE must have yielded a complicated algebraic expression $f(y)=x+c$ that needs to be solved for the variable that you are interested in, $y=f^{(-1)}(x+c)$, which Mathematica can only perform symbolically using InverseFunction.