0
$\begingroup$

I need to fetch some journals issues, in example like this: www.sciencedirect.com/science/journal/00963003/124/1

You may check this part to be constant:

string = "https://www.sciencedirect.com/science/journal/00963003"

I read here that I can do something like src=Import["https://www.sciencedirect.com/science/journal/00963003/124/1","HTML"]

But this is only for one issue. In this example there are as many as 310 volumes and each volume may have none, 1, 2, 2-3,3 or 4 issues. Is it possible to write a script in matematica to fech all HTML pages of issues at once, or at least at minimal number of fetching trials? Sorry, I cannot make more than 2 links, so some links are without https://

$\endgroup$

closed as off-topic by yohbs, happy fish, MarcoB, István Zachar, b.gates.you.know.what May 28 '17 at 19:49

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "The question is out of scope for this site. The answer to this question requires either advice from Wolfram support or the services of a professional consultant." – MarcoB, István Zachar, b.gates.you.know.what
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ A simple Import states that I'm "Not entitled to full text". I don't know how to incorporate permissions in such requests. $\endgroup$ – David G. Stork May 3 '17 at 18:41
  • 11
    $\begingroup$ Keep in mind that most journals have software that will stop access (for you and often your institution) if you do this systematically. $\endgroup$ – Dunlop May 3 '17 at 18:48
  • $\begingroup$ @Dunlop, thank you; probably they do. But what if I need just abstracts or, say, I will r script with random pause to search at random... $\endgroup$ – iot May 4 '17 at 4:13
  • $\begingroup$ Sorry I don't know, as it depends a lot on the publisher and their terms of use. This you will have to check out yourself for each publisher. Alternatively if you are only looking for abstracts, try checking databases like pubmed which should be searchable through Mathematica via open APIs $\endgroup$ – Dunlop May 4 '17 at 6:09
  • 2
    $\begingroup$ Import the website above your known issue (e.g., sciencedirect.com/science/journal/00963003/124 or sciencedirect.com/science/journal/00963003) and parse it in order to determine which issues lie below it. Then use those results to import the desired volumes. $\endgroup$ – Jack LaVigne May 4 '17 at 12:56
2
$\begingroup$

Is it what you are after?

issues = Flatten@
 Table[(ToString[i] <> "/" <> #) & /@ {"", "1", "2", "2-3", "4"}, {i, 1, 310}]

Then one could StringJoin, smth like:

links = StringJoin[{string, "/", #}] & /@ issues;

"XMLObject" is a better option. Fetch first five objects:

Import[#, "XMLObject"] & /@ Take[links, 5] // AbsoluteTiming

{32.3833, {...1...}}

It took me half of a minute for five pages fetching. (BTW, from PC that has no subscription.)

Proof of work:

% [[2]] // Length
(* 5 *)

Disclaimer I would not recommend to use it from institute network.

In no event will we be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising out of, or in connection with, the use of this code. ;-))

$\endgroup$
  • 1
    $\begingroup$ I already solved it but thanks anyway. the question is not well-accepted here $\endgroup$ – iot May 20 '17 at 5:53
3
$\begingroup$

If the sequences can be enumerated in a Table, then try this:

Table[
 Import[
"http://www.sciencedirect.com/science/journal/00963003/124/" <> ToString[mycounter], 
  "HTML"], 
{mycounter, 1, 3}]

or if a list of texts...

... {mylist, {"myjournal1/1","myjournal1/2","myjournal1/May1999","myjournal2/August2011""}

etc.

$\endgroup$
  • $\begingroup$ I have no idea what "like Descartes 310 on 5" means. If there is a list 1,2,3,5,6 (missing 4), then simply place // Quiet at the end of the function call and any errors will be "silent." $\endgroup$ – David G. Stork May 4 '17 at 15:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.