Hot answers tagged

14

We know Mathematica supports communication with different programming languages via external connectors. I will show a way with Python here. Main idea is to import the YouTube streaming link using an external python library (pytube). I assume OP has Python installed in the system and it can be accessed via functions like ExternalEvaluatein Mathematica. Now ...


14

As far as I am aware, HTTP doesn't support requesting the <title> element specifically. The browser can either GET the entire page, or request only the HTTP header information with HEAD (which notably does not include the HTML headers, where <title> is contained). It is also sometimes possible, as explained in WReach's answer, to request only ...


13

The HTTP protocol defines the Range header that can be used to limit the number of bytes returned in a response. Web sites are not required to support this header -- but if the target web site does then we may be able to use it to good effect. We can tell if a web site claims to support the feature by looking for the Accept-Ranges response header from a ...


11

Import["http://wolfram.com", "Title"] "Wolfram: Computation Meets Knowledge" Import["https://mathematica.stackexchange.com/questions", "Title"] "Newest Questions - Mathematica Stack Exchange"


10

Here is an example of how to build a download manager. We start by defining a function that uses URLDownloadSubmit to initiate a download: manifest = <||>; SetAttributes[taskProgress, HoldFirst] taskProgress[manifest_][event_] := manifest = <|manifest, event["Task"] -> event|> SetAttributes[taskFinished, HoldFirst] taskFinished[manifest_]...


9

I think the issue might be with the nature of the content in this URL. It's not that you are not been able to access the URL, as you can check yourself with running either URLRead["https://www.youtube.com/embed/HQefO4iLM0g"] which should return 200 OK, or Import["https://www.youtube.com/embed/HQefO4iLM0g","Text"] which will ...


7

This is what it looks like for me when I try to export the grid as HTML: These are just the first couple of errors. It goes on like that. On the other hand, ExportString[exp, "HTMLFragment"] seems to work. A solution could then be to do something like this: html = ExportString[exp, "HTMLFragment"] SystemOpen@Export[ FileNameJoin[{$TemporaryDirectory, "...


6

Here's another way to do this, based on extracting the "File" handler key parallelDownload[things_] := DynamicModule[ {jobs, results}, Dynamic[ Internal`LoadingPanel@ Grid@ If[AllTrue[Values@results, # =!= None &], Append[ { Button[ "Get Result", NotebookWrite[ ...


5

You may use the HandlerFunctions option of URLDownloadSubmit to track a counter updated in "TaskFinished" event. With jobs as defined in OP. completedCount = 0; tasks = URLDownloadSubmit[#1, #2, HandlerFunctions -> <|"TaskFinished" -> (completedCount++ &)|> ] & @@@ jobs; ProgressIndicator[Dynamic[completedCount], {0, Length@...


4

"Body" -> ExportString[<|"userName" -> "administrator", "password" -> "xxx"|>, "JSON"] Thanks to Kuba!


4

In addition, using a modification of this regex: (StringCases[#, RegularExpression["[^/]+$"]]&/@list)//Flatten {0208009v1, 0905.0227v1, 0907.5143v2, 0509348v1, 0608711v2, 0002018v2} where list = {{"http://arxiv.org/abs/math/0208009v1"}, {"http://arxiv.org/abs/0905.0227v1"}, {"http://arxiv.org/abs/0907.5143v2"}, ...


4

What I would do is run URLDownloadSubmit once on each URL that you want to download. That will make all those files start to download asynchronously in the background as a result. When downloading that many images, it's not worth it to keep track of how many bytes have been downloaded of each image. It's better to just count the number of files that have ...


3

The problem seems to be that the site in question has (pretty basic) defence against bots, and does not let you download it programmatically for a default type of request that Import is making. You can set HTTP headers to pretend that you send a request from a browser, to overcome this: url = "https://es.investing.com/indices/germany-30-futures-...


3

Does something like this work for you? With[ { links = WebImageSearch["dog", "ImageHyperlinks", MaxItems->10], dir = CreateDirectory[] }, Monitor[ Table[URLDownload[links[[i]], dir], {i, Length[links]}], ProgressIndicator[i / Length[links]] ] ] Your original question wasn't clear to me that the ...


3

This works as expected in version 9 (Windows 10 64bit): Export["test.html", Notebook[{Cell[BoxData[ToBoxes[exp]], "Output"]}], "HTML"] How the exported file looks in Chrome:


3

list = {{"http://arxiv.org/abs/math/0208009v1"}, {"http://arxiv.org/abs/0905.0227v1"}, {"http://arxiv.org/abs/0907.5143v2"}, {"http://arxiv.org/abs/math/0509348v1"}, {"http://arxiv.org/abs/math/0608711v2"}, {"http://arxiv.org/abs/math-ph/0002018v2"}}; In addition to StringSplit suggested ...


2

Here I create a different list of asynchronous tasks , but should be the same with your downloads. tasklist = Table[SessionSubmit[ScheduledTask[counter += 1, {m}]], {m, 1, 20}]; Dynamic[ ProgressIndicator[ Count[Through@tasklist["TaskStatus"], "Removed"] , {0, Length[tasklist]} ], UpdateInterval -> 1]


1

The problem is that the Wikipedia tables have <th> tags on the first row whereas this webpage uses <td> tags for the header line. For this reason, in the xmlTableToDataset function you need to replace headers = Cases[ headerRow, d : XMLElement["th", _, _] :> (... etc ...), Infinity ]; with headers = ...


1

If you only want to scrap tables, here is another alternative: ClearAll[scrapTables, scrapTablesCore]; scrapTables[url_String] := scrapTablesCore[URLRead[url]["Body"]] scrapTables[path_File] := scrapTablesCore[ReadString[path]] scrapTablesCore[text_String] := DeleteCases[#, {}, Infinity] &@ StringCases[text, Shortest[ ...


1

This is not a full answer As @b3m2a1 suggested, I looked into the paclet and tried updating the functions: p = PacletFind["Service*_*Fl*"] p[[1]]["Location"] Which looks like this: Then I added this to Flickr.m: flickrdata["RawFaves"] := { "URL" -> "https://api.flickr.com/services/rest?method=flickr.favorites.getlist", "...


1

I could paste text in a ghetto way. On your virtual Wolfram keyboard press the double upward arrow, top left corner. Now press something that looks like an equal in a box with two lines connecting to it, top right corner of the virtual keyboard, which should insert a box. I was able to paste into the box using my smartphones default long tap and paste. It ...


Only top voted, non community-wiki answers of a minimum length are eligible