Questions about using Mathematica for accessing web sites and using such functions as URLFetch and URLSave.

learn more… | top users | synonyms

0
votes
0answers
28 views

Filtering bad urls from a list of image urls [on hold]

I have about 90k image urls and I'd like to Select only the ones that are real images. Map[Check[Import[#], Nothing] &, urls] would take about 6 hours. Any ...
0
votes
1answer
149 views

How to extract an answer on Mathematica.SE as an image [duplicate]

When I try to get this answer from Mathematica.SE by evaluating Import["http://mathematica.stackexchange.com/a/95725/21532"] I just get some text without any ...
6
votes
2answers
122 views

Web Browsing Automation: how to expand all comments under a blog post?

I wish to expand all the comments under a LiveJournal post for the purposes of saving the complete thread of comments. According to LiveJournal's FAQ when a post gets more than 50 comments they are ...
2
votes
0answers
45 views

Web crawler: want to ignore login windows [closed]

I'm trying to make web crawler with Mathematica. ...
3
votes
0answers
34 views

How do I use Mathematica to upload to Clyp?

Clyp is a service for hosting sound files that (as of now) doesn't require registration. I'm wondering if it's possible to write a function that'll upload a Sound ...
9
votes
3answers
533 views

How to make a picture grid of all Nobel Laureates in physics?

I want to make a picture grid of all Nobel Laureates in physics using mathematica. Later I wanna print this picture grid on a big wall! The first thought I came up with is to exploit the powerful ...
8
votes
2answers
233 views

How to use URLFetch for a https secure automated login?

(EDIT: Any example of a https secure login using certification keys using Mathematica will help.) (EDIT-2: Might this help?: Can anyone explain how to use Mathematica's OAuth package? ? ) With ...
3
votes
1answer
154 views

How to access a web API with HTTP POST and JSON

I want to get data using HTTPS from POST https://developers.betfair.com/api.betfair.com/exchange/betting/json-rpc/v1 HTTP/1.1 using JSON ...
1
vote
1answer
77 views

Uploading an image to a CloudDeploy Wolfram Language web app

I want to do image processing using the Wolfram Language on a web app hosted on the Wolfram Cloud. The user must be able to upload a PNG image to the Wolfram Cloud, how can I set this up? I've ...
1
vote
0answers
67 views

Two-step web data submission

I am trying to automate the process of downloading data from my univ. webpage. I need to submit the form in the URL below. http://202.88.252.21/CuPbhavan/res_newregentry.php?id=4789 An inspection ...
4
votes
1answer
96 views

How to change proxy settings programmatically?

I'm using version 10.2 on Linux, and need to access some web services via proxy server sometimes. Is there a way to change the proxy settings, which can be modified manually in the "Preferences" AFAIK,...
3
votes
1answer
91 views

Is there a library or API for interacting with WebSockets in Mathematica?

I'm trying to build a Mathematica model that can retrieve and send data to a service that uses a WebSocket protocol. Is there a library or set of APIs for MMA that interace to WebSockets?
0
votes
1answer
77 views

Getting error from CommunityGraphPlot

Situation I'd like analyse my web site with CommunityGraphPlot as follows: Step 1: Define a function to scrape the all the webpages from expected website. ...
1
vote
0answers
69 views

Import using POST method and web-page drill down

A response to a previous question about invoking the POST method via Mathematica was most informative in trying to further understand the intricacies of using the POST Method to obtain tabular data ...
10
votes
2answers
397 views

How do you submit an Input type of “file” using URLExecute?

I have to submit a form like as follows: <input type="file" name="jpgFile"> I can't seem to find any instructions on sending this in URLExecute. If I ...
6
votes
1answer
133 views

Use Mathematica to save all citations from Google Scholar Search

How can I save all citations from a Google Scholar search? For instance, in a search for "Radon transformation", there are about 35,500 results. I want to download all citations into a string - is ...
2
votes
1answer
100 views

How to download tables from a url?

I'm trying to import all the tables on a website, this one in particular: https://aws.amazon.com/ec2/pricing/ For local files that are already tabular, ...
3
votes
3answers
107 views

Why does Import[#, “ImageLinks”]& fail?

I'm having difficulty with Importing urls embedded in a site (with and without pagination). This command: ...
3
votes
1answer
55 views

How to access the string of the cell I am typing?

I'm trying to map a keyboard shortcut to google search the contents of the cell I am typing. Here is my code so far: ...
1
vote
1answer
330 views

How to scrape image search engines?

I'd like to download highest resolution images possible from the search results like these: ...
5
votes
1answer
344 views

How to download pictures from webpage automatically?

Today, I read a good web page about love which owns many beautiful pictures. It is fussy for me to download every picture hand-by-hand, so I'd like to use Mathematica to do this case automatically. I ...
4
votes
1answer
106 views

Import citation data for BibTeX using DOI?

I recently discovered that Digital Object Identifiers can be used for a lot more than just a reliable link to an article or dataset, and in some cases the DOI url's support semantic queries. This ...
2
votes
0answers
83 views

Retrieving a Portion of Webpage Code using URLFetch

The URLFetch command is very useful for pulling the entire html source of a given webpage, however, if one wanted to speed up the process, by only requesting a ...
24
votes
2answers
332 views

Reading from a socket stream

I noticed an undocumented socket package (SocketLink) in Mathematica (Using version 10, not sure when it was actually added) and wrote a barebones HTTP responder ...
1
vote
0answers
62 views

How to get the subdirectory of a ftp weblink?

For example, I can use FileNames["*",dir] to get all subdirectory under dir if dir is some directory of my local computer. Suppose that I want to get all ...
2
votes
1answer
248 views

How can I determine `Parameters` for use in `URLFetch`?

I'm trying to download data from website automatically using Mathematica, and URLFetch seems like it should be suited to the task. I'm interested in its "passing ...
8
votes
1answer
591 views

How to execute JavaScipt on a webpage and then import the result on OSX?

I would like to import data from this page through Mathematica, http://www.oddsportal.com/soccer/netherlands/eredivisie/ . I tried importing through the default Import function ...
6
votes
2answers
320 views

Lookup in Inverse Symbolic Calculator+ from Mathematica

There is a useful feature in WolframAlpha that allows to find a possible closed form for approximate numeric values. It can be used, for example, to guess the value of an integral that Mathematica ...
5
votes
1answer
411 views

More complete “MutipartData” POSTs using URLFetch

I'm trying to upload a file to a wiki using Mathematica's URLFetch. Some preliminary work is required - one needs to log in to the wiki, obtain an edit token, and exchange some cookies with the wiki ...
2
votes
1answer
456 views

Using a web-crawler to search website for occurrences of keywords

Given this list of cities: ...
2
votes
1answer
457 views

Importing web page data

I tried importing a webpage using Import: ...
10
votes
1answer
517 views

Parse website data that requires interaction with a form (POST)

I want to analyse Chilean elections data available in the official website. What I have returns the data for president (numbers are strings, to be parsed later). ...
2
votes
1answer
115 views

Problem with importing a JCAMP-DX IR spectrum file

I wrote a little function to import JCAMP-DX electron ionization or mass spectral files from the NIST webbook web site (e.g. mass spectrum of hydrogen chloride at http://webbook.nist.gov/cgi/cbook.cgi?...
10
votes
2answers
605 views

Look up info associated with a given CAS chemical identifier from the NIST webbook & NCI CADD Chemical Identifier Resolver

I would like to retrieve information associated with a given CAS registry number (Chemical Abstracts Service nr) from the NIST webbook web site in Mathematica, using the provided API (e.g. for ...
12
votes
4answers
537 views

How do you get a list of filenames from a URL?

How would I get a list of filenames available at a URL? For example, list the files at: http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/
3
votes
1answer
173 views

Use URLFetch in init.m

I'm not able to use URLFetch in my $UserBaseDirectory/Kernel/Init.m as it always returns ...
5
votes
1answer
751 views

How can I commit files to Github by Mathematica?

How can I publish Notebooks to Github via using Mathematica? I'm on windows 8.1 x64, Mathematica 9. How can I upload the current notebook to Github? My function expected is: ...
25
votes
2answers
3k views

Export to Google Spreadsheet with OAuthAuthentication

Context I'm trying to setup a connection between Mathematica and Google Docs. Specifically, I would like to be able to export data to a Google Spreadsheet (Note: if the spreadsheet is public it's ...
5
votes
1answer
524 views

How can I log in to my Wikipedia account and access my watch list?

Wikipedia's "Edit Watchlist" page (for logged in users) has a list of hyperlinks to the user's watchlist (here is an example from my account). I've tried to use ...
3
votes
1answer
88 views

Web deployment of document containing URLSave function

Is it possible to deploy a Mathematica notebook which contains a dynamic URLSave function? My code has the following structure: ...
3
votes
3answers
327 views

URLFetch functionality for a Persian website

I want to fetch the content of a Persian website. I tried URLFetch["http://www.tsetmc.com/"] Why is Mathematica unable to recode the characters in the results ...
41
votes
1answer
2k views

Web Browsing Automation with Mathematica

I want to build a bot to automate web browsing, this mean something like: filling forms press "submit" buttons find certain text inside pages and so on... How can I do this with Mathematica? The <...
42
votes
8answers
1k views

Is it possible to invoke the OEIS from Mathematica?

I had always wondered if there might be a way to write a function, which I'll call OEISData[], that more or less works as a curated data function for The On-Line ...