Bug introduced in version 9.0 and persisting through 10.4
Graphics elements do not always line up perfectly in exported PDFs using "Save As..." method.
Take, for example:
Graphics[{Point[{0, 1}], Line[{{0, 0}, {0, 1}}]}, AspectRatio -> Automatic]
It looks like this on screen:
Notice that the dot isn't centred on the end of the line.
Now let's export to PDF (hoping that what we saw displayed in the Mathematica GUI is just a rendering problem). Right click the graphic, and choose Save As... -> PDF. The PDF looks like this when magnified in a viewer:
There is clearly a very serious misalignment. Is there a workaround?
Note: this seems to be a new problem in version 9. Version 7 and 8 are not affected.