A quick question, but which I don't believe has been asked here or at SO.

Does Mathematica have a simple way to just download a file from the web? i.e. if I have a list of PDF links (~ 2,000), can I use Mathematica to quickly take them and save them to my system. The obvious way that I've used in the past is to Import[] the data, but since we're talking large-ish PDFs I wonder if there's a way to skip that step.

I have used wget for this sort of thing in the past, but just seeing if there's an easy way to do it within Mathematica. The 'Web Operations' section of the documentation does not seem to have any obvious reference to this. If not, I will obviously just use the 'proper' tool.

-

Needs["UtilitiesURLTools"];
path = FetchURL[
get_obj=ifp2_cut.mesh.gz&acces=ifp2_cut", "ifp2_cut.mesh.gz"];
-
This works great. I've never used this Package, so will read up on it. –  programming_historian Feb 2 '12 at 18:03

As a quick update to this, Mathematica 9 added some additional functionality that I have been putting to use - it seems to work very well if you do text analysis, which I do.

Documentation for the new commands URLSave, URLFetch, and the really useful URLSaveAsynchronous are here. For example, I've been running commands like this:

URLSave["http://activehistory.ca","/users/MYPATH/activehistory.txt"];

Which seem to work perfectly.

-
This is the one I would recommend with V9 and beyond. The FetchURL function above now internally uses URLFetch and similar functions from the new HTTPClient` package. It supports proxies and lots of options. –  Joel Klein Jan 16 '13 at 16:49