Tell me more ×
Mathematica Stack Exchange is a question and answer site for users of Mathematica. It's 100% free, no registration required.
up vote 6 down vote favorite
3

I need M. to access an online public directory via some URL (fake in this case)

http://example.com/images

and perform operations similar to those M. can do on OS directories and files. Examples:

  1. Get all files names - like: {image1.png, image2.png, ...}

  2. Import files into M.

  3. Get various files info

What is an efficient way to do it if it's possible at all?

share|improve this question
If you know the files names already this is straight forward but it looks like you want to find out the file names ...right? I think you probably need to use a terminal command -- don't know which one but maybe wget or curl will let you do this -- with Run. – Mike Honeychurch Jan 22 '12 at 22:43
Thanks for the idea @MikeHoneychurch , I passed it as a comment to small discussion we have under the Brett's answer. Maybe David Zaslavsky can comment on this. – Vitaliy Kaurov Jan 22 '12 at 23:28
I use wget quite a lot which is why something like this seems possible but I am unsure if it can get you a list of files in a directory -- but if it cannot then there would be another terminal command that should be able to do it I suspect. – Mike Honeychurch Jan 22 '12 at 23:31
I use a GUI FTP but does the terminal ftp give you a list of files in a directory? – Mike Honeychurch Jan 22 '12 at 23:32
@Mike: FTP will do it, but only if you have FTP access to the site. See also the comment under Brett's answer. It might be wise to take this to the chat room though if the discussion is going to continue for long. – David Zaslavsky Jan 23 '12 at 0:16

1 Answer

up vote 7 down vote accepted

Here's one approach, though it's hard to say without knowing the site and what additional information you want for the files.

Import["http://kaurov.com", {"HTML", "Images"}]

enter image description here

There are several other items you can ask for (including what elements you can ask for!)

In[53]:= Import["http://kaurov.com", {"HTML", "Elements"}]

Out[53]= {"Data", "FullData", "Hyperlinks", "ImageLinks", "Images",
"Plaintext", "Source", "Title", "XMLObject"}

In[54]:= Import["http://kaurov.com", {"HTML", "ImageLinks"}]

Out[54]= {"http://kaurov.com/wordpress/wp-content/uploads/2011/10/masterimagelfss.jpg",            
   ...
   "http://kaurov.com/wordpress/wp-content/uploads/2009/11/life-death-spinner.gif"}
share|improve this answer
Haha, neat - a rather cunning reply. Is it possible to somehow get the file list and information? – Vitaliy Kaurov Jan 22 '12 at 21:12
Off topic, but: I wonder how you update all that information in the docked cell. Even if one uses Refresh, doesn't it stop updating after the kernel quits? What if you quit the v8 kernel and start a v7 kernel? --> stackoverflow.com/questions/8756565/… – Szabolcs Jan 22 '12 at 21:14
1  
@Vitaliy If the server gives you a file listing, it should be as easy as Import["http://kaurov.com", {"HTML", "Hyperlinks"}] If it doesn't, then perhaps impossible? – Szabolcs Jan 22 '12 at 21:16
4  
The HTTP protocol doesn't contain a method for listing files in a directory. In many cases URLs don't even correspond to actual files, so it may be impossible to tell what pages exist or not without actually trying to access them. So in general, it is impossible to produce a directory listing for a website. Some server administrators choose to publish such a listing as an index page, but you can't count on every website doing this, and the format used can vary from one site to another. – David Zaslavsky Jan 22 '12 at 21:23
2  
@Vitaliy: Regarding Mike's comment, no, using wget or curl won't help. It's a limitation of the underlying communication protocol, so you can't get around it by switching which program you use. This only applies to HTTP URLs, though. If you are accessing the server with an FTP client (whether command-line or GUI), then you can get a directory listing. But most servers do not allow FTP access (except by the person who maintains the website). – David Zaslavsky Jan 23 '12 at 0:15
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.