39

Here is a package which does what you want: https://github.com/arnoudbuzing/webunit Clone the repository from github, and place the WebUnit folder under $UserBaseDirectory/Applications To use it: Needs["WebUnit`"] InstallWebUnit[] (* launches chromedriver.exe *) StartWebSession[] (* launches Chrome web browser, assuming you have that installed *) ...


28

I built a service connection for this and blogged about it here: Playing with YouTube from Mathematica Here's an example of how this works. Connect to YouTube First install the paclet: PacletInstall["ServiceConnection_YouTube", "Site"-> "http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer/" ] Then connect: $so = ServiceConnect["...


23

Update If I were to do this today, I would probably use Google's SDK for Java and JavaLink, which would allow for a better solution. Some have asked me why I said that we have to involve the browser to do it in Mathematica, I was not clear on this point. The reason is that no ready made implementation of the required RSA SHA-256 encryption exists, so one ...


21

Here is one solution based on encoded package. To create a pack you have to create a .m file (in this case mailPack.m). There is one example using gmail configuration: BeginPackage["mailPack`"] sendMail::usage="sendMail[subject, body, to, file] send mail using myMail"; Begin["`Private`"] sendMail[subject_,body_,to_,file_:None]:= SendMail[ "To"->to,...


20

I'm Riccardo, current developer of URLRead in WL and I have some experience working with encoding in WL. I would like to inform you that this is not a bug. In modern versions of mathematica we have ByteArray, and this is a representation of bytes. But for decades strings have been both bytes and "unicode" at the same time. The problem here is that all ...


20

tl;dr This is really quite easy with a toolkit I've been building; this is an example of a GitBook I built with these tools. Preface This answer is gonna build off some stuff I've been developing off-and-on for the past few months. Everything is packaged up and most of it is in a palette, so you'll be able to do this with minimal effort. It started with ...


19

Use URLFetch in Mathematica 9.0.1.


19

Does this do the job? links = Import[ "http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/", "Hyperlinks"] {"http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/?C=N;O=D", "http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/2005/?C=M;O=A", "http://www1.ncdc.noaa.gov/pub/data/cmb/drought/weekly-palmers/...


19

Tracing the evaluation (Mathematica 11.1.1) shows that the string is passed to Developer`ReadRawJSONStream which actually produces the messages: Developer`ReadRawJSONStream[StringToStream@string, "IssueMessagesAs" -> Import] General::jsonoutofrangeunicode: Out of range unicode code point encountered. Import::jsoninvalidtoken: Invalid token ...


15

I would use RawJSON to import the information: bingJSON = Import[ "http://www.bing.com/HPImageArchive.aspx?format=js&idx=0&n=10&mkt=en-US", "RawJSON" ]; Then, create a template: URLTemplate="https://bing.com/`urlBase`_`resolution`.jpg"; Finally, Import the template with the given data: Table[ Import @ TemplateApply[ ...


14

As it turns out. Yes. And there's an easy way to check (on Mac). Check out this answer: https://stackoverflow.com/questions/49510/how-do-you-set-your-cocoa-application-as-the-default-web-browser/49512#49512 All we need to do is go to our application's info.plist and search for CFBundleURLName. And what do you know? There's stuff there! This is what I ...


14

As far as I am aware, HTTP doesn't support requesting the <title> element specifically. The browser can either GET the entire page, or request only the HTTP header information with HEAD (which notably does not include the HTML headers, where <title> is contained). It is also sometimes possible, as explained in WReach's answer, to request only ...


13

The easiest way is to modify $Path to include the directory you placed your packages into. Suppose your packages directory is in the Dropbox\Documents\myPackages folder inside your home directory; then you can add it to your path like this: AppendTo[$Path, ToFileName[{$HomeDirectory, "Dropbox", "Documents", "myPackages"}]] (EDIT: amended according to ...


13

GoogleDataLink from Lauschke Consulting does exactly that. With GDL you can seemlessly retrieve/query, upload, update, filter, etc. (in short: exchange) data between M and Google Spreadsheets. You can exchange data on a cell level ("atomic" expressions in M) as well as entire (rectangular) cell ranges as M 2-dim tables. You can query/filter and retrieve ...


13

In this case it would be far more reliable to request the information from the source directly. You can find the source using your web browser's developer tool, in Google Chrome you find this information under "Network." It should look something like: Importing that data as JSON gives us a nested list of rules containing all the data: rawData = Import["...


13

This is the best use of SocketConnect that I can think for now. Dynamic[Framed@TextCell[ Text[Style[ txt, LineSpacing -> {0.5, 0}, FontFamily -> "Courier", TextJustification -> 0]], PageWidth -> 180 , Background -> RGBColor[0, 0, Rational[8, 27]], FontColor -> White ]] socket = SocketConnect[{"towel.blinkenlights.nl", ...


13

The HTTP protocol defines the Range header that can be used to limit the number of bytes returned in a response. Web sites are not required to support this header -- but if the target web site does then we may be able to use it to good effect. We can tell if a web site claims to support the feature by looking for the Accept-Ranges response header from a ...


13

Since the page is generated asynchronously, you can use the same data source that the page itself does. Using the network inspector in my browser, I discovered that the page loaded its data from this url. It's some API that delivers JSON, so we can use that data. The first thing we can do is parse the URL, so get an idea of the various parameters we can ...


13

Create the new index test and put the data in it: MapIndexed[ URLExecute@HTTPRequest[ <| "Domain" -> "localhost", "Port" -> 9200, "Path" -> {"test", "external", ToString[First@#2]} |>, <| Method -> "PUT", "Body" -> ExportString[<|"Text" -> #1|>, "PythonExpression", CharacterEncoding -> "...


12

Another option for individual files and stuff is to use the GitHub API with a custom plug-in to the built-in service connection framework. Here's a service connection I wrote for it: PacletInstall["ServiceConnection_GitHub", "Site"->"http://www.wolframcloud.com/objects/b3m2a1.paclets/PacletServer" ] Then we load it like this: $gh = ServiceConnect@"...


12

As pointed out yesterday by ZachB in answer to another question, there is a hack that makes URLFetch add a filename to the Content-Disposition header as required by RFC 7578 multipart/form-data section 4.2: URLFetch[url, "Method"->"POST", "MultipartElements"->{ {"file\"; filename=\"test.jpg","image/jpeg"}-> Import["test....


12

Mapbox works. It requires an API key which can be retrieved for free, for information about pricing read this. Mapbox has many beautiful map styles, and also the ability for you to create your own map styles or modify existing ones with their browser based tool. Here is a list of default styles that everyone has access to: styles = {"streets", "light", "...


12

So after being wrong about Bing being free, we'll pursue my other suggestion, which was to use the ServiceConnect framework. I demonstrate how to use it in general here. For the purposes of this, though, here's what all you need to do. Get the paclet by running this: (* If you've already installed, replace this with PacletUpdate to get the newest ...


12

I haven't used these submarined functions for EC2, but I have found the S3 functionalities in this package to be extremely useful. Since there is currently no documentation in v11.3, I thought at the very least I could share a brief guide to the most important operations. Setup AWS CLI AWSLink uses the AWS command line tool under the hood, so you need ...


12

You can set $AllowInternet to False (or just turn off your WiFi or unplug the cable).


11

That website delivers the URL with a content-encoding of gzip. Apparently, URLFetch does not automatically handle decompressing this. You can decode it manually: tempFile = URLSave["http://www.tsetmc.com/", Close[OpenTemporary[]]]; data = Import[tempFile, {"GZIP", "Text"}]; DeleteFile[tempFile]


11

This is a bug due to the fact that URLFetch is dealing incorrectly with line breaks. Here's a workaround: in1 = URLFetch[ "http://blog.wolfram.com/wp-content/uploads/2008/06/se-30.jpg", "ContentData" ]; ImportString[FromCharacterCode[in1]] You can compare this technique to the other as follows: in2 = ToCharacterCode[URLFetch[ "http://blog.wolfram....


11

This is a hack, but it works. UploadFile[url_, filePath_, urlParams___] := With[ { bytes = Import[filePath, "Byte"], filename = StringJoin[FileBaseName[filePath], ".", FileExtension[filePath]] }, URLExecute[ url, urlParams, "Method" -> "POST", "MultipartElements" -> { {"file\"; filename=\"" <> filename, "...


11

Based on this article: kylen314.com/archives/1647 st = Import[ "http://mp.weixin.qq.com/s?__biz=MzA5NDY3OTAyNA==&mid=205311623&\ idx=1&sn=7b565a6ed5789732f698d5a6b4c5c652&scene=2&from=timeline&\ isappinstalled=0#rd", "XMLObject"]; PicAddress = Cases[st, XMLElement["img", {"data-src" -> src_, ___}, {}] :> src, {0, ...


11

Here's something to get you started down to path of scraping the somewhat larger individual pictures from the Nobel website: links = Import[ "https://www.nobelprize.org/nobel_prizes/physics/laureates/index.html?images=yes", "Hyperlinks"]; individualpagelinks = Select[ links, StringMatchQ[ "https://www.nobelprize.org/nobel_prizes/physics/...


Only top voted, non community-wiki answers of a minimum length are eligible