Skip to main content
Mathematica

Return to Question

Post Closed as "Duplicate" by C. E.♦, m_goldberg, MarcoB, Carl Lange, Alex Trounev
deleted 42 characters in body
Source Link
rhermans
rhermans
  • 37.4k
  • 4
  • 61
  • 152

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer to the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the process.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer to the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the process.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

I prefer to the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the process.

Tweeted twitter.com/StackMma/status/764143532001165313
added 2 characters in body
Source Link
yode
yode
  • 27.2k
  • 4
  • 66
  • 174

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer to the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the solutionprocess.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the solution.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer to the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the process.

edited body
Source Link
yode
yode
  • 27.2k
  • 4
  • 66
  • 174

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the items favorite items and its number of voteits number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-20-10.pnghttp://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the solution.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-20-10.png

I prefer the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the solution.

In version 11,if we want to crawl public website like wiki,we can do it like this:

URLRead["https://en.wikipedia.org/wiki/Wiki", "Body"]

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-09-26.png

But I don't know how to crawl some website needed username and password,such as Stack Exchange,our Facebook homepage or our educational management system.Of course some webs providing some API can help us to do this.but I don't want to use method of API to do this.Anyone can give a example to do it?For example,how to crewl the Stack Exchange of Mathematica.I want to get all of the favorite items and its number of vote in my homepage like following

http://o8aucf9ny.bkt.clouddn.com/2016-08-12-19-36-37.png

I prefer the answer use Mathemtica of version 11,because it start to support cookie(FindCookies),which maybe help to simplify the solution.

Source Link
yode
yode
  • 27.2k
  • 4
  • 66
  • 174