I need to fetch some journals issues, in example like this: www.sciencedirect.com/science/journal/00963003/124/1

You may check this part to be constant:

string = "https://www.sciencedirect.com/science/journal/00963003"

I read here that I can do something like src=Import["https://www.sciencedirect.com/science/journal/00963003/124/1","HTML"]

But this is only for one issue. In this example there are as many as 310 volumes and each volume may have none, 1, 2, 2-3,3 or 4 issues. Is it possible to write a script in matematica to fech all HTML pages of issues at once, or at least at minimal number of fetching trials? Sorry, I cannot make more than 2 links, so some links are without https://

  • $\begingroup$ A simple Import states that I'm "Not entitled to full text". I don't know how to incorporate permissions in such requests. $\endgroup$ May 3, 2017 at 18:41
  • 11
    $\begingroup$ Keep in mind that most journals have software that will stop access (for you and often your institution) if you do this systematically. $\endgroup$
    – Dunlop
    May 3, 2017 at 18:48
  • $\begingroup$ @Dunlop, thank you; probably they do. But what if I need just abstracts or, say, I will r script with random pause to search at random... $\endgroup$
    – iot
    May 4, 2017 at 4:13
  • $\begingroup$ Sorry I don't know, as it depends a lot on the publisher and their terms of use. This you will have to check out yourself for each publisher. Alternatively if you are only looking for abstracts, try checking databases like pubmed which should be searchable through Mathematica via open APIs $\endgroup$
    – Dunlop
    May 4, 2017 at 6:09
  • 2
    $\begingroup$ Import the website above your known issue (e.g., sciencedirect.com/science/journal/00963003/124 or sciencedirect.com/science/journal/00963003) and parse it in order to determine which issues lie below it. Then use those results to import the desired volumes. $\endgroup$ May 4, 2017 at 12:56

2 Answers 2


Is it what you are after?

issues = Flatten@
 Table[(ToString[i] <> "/" <> #) & /@ {"", "1", "2", "2-3", "4"}, {i, 1, 310}]

Then one could StringJoin, smth like:

links = StringJoin[{string, "/", #}] & /@ issues;

"XMLObject" is a better option. Fetch first five objects:

Import[#, "XMLObject"] & /@ Take[links, 5] // AbsoluteTiming

{32.3833, {...1...}}

It took me half of a minute for five pages fetching. (BTW, from PC that has no subscription.)

Proof of work:

% [[2]] // Length
(* 5 *)

Disclaimer I would not recommend to use it from institute network.

In no event will we be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising out of, or in connection with, the use of this code. ;-))

  • 1
    $\begingroup$ I already solved it but thanks anyway. the question is not well-accepted here $\endgroup$
    – iot
    May 20, 2017 at 5:53

If the sequences can be enumerated in a Table, then try this:

"http://www.sciencedirect.com/science/journal/00963003/124/" <> ToString[mycounter], 
{mycounter, 1, 3}]

or if a list of texts...

... {mylist, {"myjournal1/1","myjournal1/2","myjournal1/May1999","myjournal2/August2011""}


  • $\begingroup$ I have no idea what "like Descartes 310 on 5" means. If there is a list 1,2,3,5,6 (missing 4), then simply place // Quiet at the end of the function call and any errors will be "silent." $\endgroup$ May 4, 2017 at 15:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.