Extract information from webpage

If you browse this page for instance, you'll notice at the bottom in gray, a line with

Contributor: Name < email >

Certainly for security reasons, the email does not appear in the source code. Instead, one can see decryptMail("puveng.nyrknaqer@tznvy.pbz", "link5a34220e387d1", ""). For that reason, Mathematica cannot get the email address from Import["https://hal.archives-ouvertes.fr/hal-..."]. I was wondering if it was possible to manage another way: in the end, a human can read the email address (either directly, or by using an page inspector), so there must be a way for MMA to get it. Question: how?

I tried using GUIScreenShot and then TextRecognize; it works provided the resolution is good enough, but it is not very robust (the position depends on the length of the text above, what if the page takes time loading, etc.).

• See WebUnit from Arnold Buzing. That could do it. – b3m2a1 Dec 16 '17 at 0:22

As suggested by b3m2a1, WebUnit does the job.

StartWebSession[]
OpenWebPage["https://hal.archives-ouvertes.fr/hal-01665363"]
str = GetPageHtml[]


Then it suffices to get the info from str, typically with something like

StringCases[str, Shortest["mailto:" ~~ x__ ~~ "\""] -> x]

• Might be worth doing a Q/A on how to fix WebUnit for Linux based on what you put in chat and then linking to that here. – b3m2a1 Dec 16 '17 at 4:30
• @b3m2a1 Yes but that's not something general, it depends on the distribution, its version, etc. Also my problem was more due to chromedriver than WebUnit. I'll try on another PC and will write a Q/A if relevant. – anderstood Dec 16 '17 at 13:48