The answer to your question is system dependent. The value of
$InitialDirectory is very different on OS X, say, than on Windows or Linux. I recommend you evaluate
The result is an array of buttons, each -- when clicked on -- will give you a brief description of the Mathematica symbol it displays all of which will have some relation to directories. At the end of the brief description you see a
>>. This is a link. Clicking on it will bring up the documentation page for the given symbol.
$InitialDirectory will be displayed on one the buttons in the array along with many other system symbols Mathematica maintains for storing the full path names of directories that are important to it. You can learn a lot about how Mathematica relates to directories by exploring the button array,
On my system, which is OS X,
$InitialDirectory doesn't seem to be very important unless you are running Mathematica from the command line in Terminal.