2
$\begingroup$

I am trying to import data from multiple web pages hosted by a single online source. The data is posted by the source as one data set per web page for each week of the year. I would like to import the data for all 52 weeks in a year at the same time rather than modifying my code for each of the weeks and importing one at a time.

Here is my code to import one week's data:

week012012 = 
  Import["http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=01&\p=.htm", "Data"]

If it is of interest or relevance here is the further processing I do with the data after it has been imported:

week012012B = Cases[week012012, {_, _, _, _, _, _, _, _, _, _, _, _?NumericQ}, ∞];

Grid[week012012B, Frame -> All]

The site uses a consistent naming scheme for each week of the year and indeed from year-to-year as well. If I were looking to get just a few weeks data I could manually change the URL for the weekend from 01 to 02, 03, 04..., but I want all 52 weeks for 2012. The approach I have been playing with is to use string manipulation to modify the URL and then import and save the data for each of the weeks. Any suggestions?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ The site seems to have an RSS feed. So this is probably a duplicate of How to scrape the headlines from New York Times and Wall Street Journal $\endgroup$
    – Jens
    Commented Mar 21, 2013 at 3:28
  • $\begingroup$ Is the question actually just how to generate a list of URL strings? I'm not sure what you're asking. The parsing of the Import is not your concern, or is it? $\endgroup$
    – Jens
    Commented Mar 21, 2013 at 3:32
  • $\begingroup$ Not looking for the RSS feed. Looking to import the data from 52 pages hosted on the site sequentially with a single command. $\endgroup$
    – Nguyen Van Falk
    Commented Mar 21, 2013 at 3:54
  • $\begingroup$ Then you should probably remove the "Reconstitution" from the title. $\endgroup$
    – Jens
    Commented Mar 21, 2013 at 4:01
  • $\begingroup$ Disagree with the suggestion to alter the title, but I am not going to object that someone edited it. The reason is that the import is reconstituting data tables found on the 52 web pages. $\endgroup$
    – Nguyen Van Falk
    Commented Mar 30, 2013 at 21:27

1 Answer 1

Reset to default
5
$\begingroup$ 
wkstrngs = StringJoin /@ Map[ToString, PadLeft[IntegerDigits /@ Range[52]], {-1}]; 
wkurls = Quiet["http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=" ~~ 
  # ~~ "&\p=.htm" & /@ wkstrngs[[;; 5]]] (* remove [[;;5]] for all 52 weeks *)
(* {"http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=01&\\p=.htm",
    "http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=02&\\p=.htm",
    "http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=03&\\p=.htm",
    "http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=04&\\p=.htm",
    "http://www.boxofficemojo.com/weekend/chart/?yr=2012&wknd=05&\\p=.htm"} *)
fiveweeks = Import[#, "Data"] & /@ wkurls;
data5wks = Cases[#, {__,_?NumericQ}, Infinity] & /@fiveweeks; (*thanks: Mike Honeychurch*)
Grid[#, Frame -> All] & /@ data5wks
Improve this answer
$\endgroup$
3
  • $\begingroup$ This is exactly what I was looking to accomplish. Thanks. $\endgroup$
    – Nguyen Van Falk
    Commented Mar 21, 2013 at 4:02
  • $\begingroup$ if you're testing for a numeric last element presumably {__,_?NumericQ} would suffice as the pattern? $\endgroup$
    – Mike Honeychurch
    Commented Mar 21, 2013 at 6:39
  • $\begingroup$ @Mike, you are right; thank you. $\endgroup$
    – kglr
    Commented Mar 21, 2013 at 7:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.