"Accept" header in HTTP request for DOI Content Negotiation

Question

I am trying to get the .bib info from a DOI, following the guide here.

Basically, I wish to run

curl -LH "Accept: application/x-bibtex" http://dx.doi.org/10.1016/j.algal.2015.04.001

to define an "Accept" header with content "application/x-bibtex".

With Mathematica. I try

 URLRead[HTTPRequest[URL["https://doi.org/10.1007/s00209-015-1612-7"],
  <| "Method" -> "GET",
   "Headers" -> {"Accept: application/x-bibtex"},
   "ContentType" -> "application/x-bibtex"|>]]

but this fails, with error message

"Message text not found -- ({Accept: application/x-bibtex}"

The output I expect is a string, with the contents:

@article{Kouhia_2015,
    doi = {10.1016/j.algal.2015.04.001},
    url = {http://dx.doi.org/10.1016/j.algal.2015.04.001},
    year = 2015,
    month = {jul},
    publisher = {Elsevier {BV}},
    volume = {10},
    pages = {41--47},
    author = {Mikko Kouhia and Henrik Holmberg and Pekka Ahtila},
    title = {Microalgae-utilizing biorefinery concept for pulp and paper industry: Converting secondary streams into value-added products},
    journal = {Algal Research}

EDIT: I also want this to work with other journals. curl can handle that as well, as there seem to be a standard to request this particular format.

curl -LH "Accept: application/x-bibtex" https://doi.org/10.1090/s0002-9947-96-01558-9

Answer 1

It seems you are trying to profit from DOI Content Negotiation described here.

Content negotiation allows a user to request a particular representation of a web resource. DOI resolvers use content negotiation to provide different representations of metadata associated with DOIs.

A content negotiated request to a DOI resolver is much like a standard HTTP request, except server-driven negotiation will take place based on the list of acceptable content types a client provides.

Making a content negotiated request requires the use of a HTTP header, "Accept".

After @Kuba's comment this works

With[{url = URL["https://doi.org/10.1007/s00209-015-1612-7"]},
 URLRead[
  HTTPRequest[
   url
   , <|
    "Method"  -> "GET",
    "Headers" -> {"Accept" -> "application/x-bibtex"}
    |>
   ]
  , "Body"
  ]
 ]

Other content formats defined by crosscite.org are:

{
 "RDF XML" -> "application/rdf+xml",
 "RDF Turtle" -> "text/turtle", 
 "Citeproc JSON " -> "application/vnd.citationstyles.csl+json", 
 "Schema.org in JSON-LD " -> "application/vnd.schemaorg.ld+json", 
 "Formatted text citation " -> "text/x-bibliography", 
 "RIS " -> "application/x-research-info-systems", 
 "BibTeX " -> "application/x-bibtex", 
 "CrossRef Unixref XML " -> "application/vnd.crossref.unixref+xml", 
 "DataCite XML " -> "application/vnd.datacite.datacite+xml", 
 "ONIX for DOI " -> "application/vnd.medra.onixdoi+xml"

}

---

I wasn't aware of that capability, so my first approach (A bit convoluted) was to parse the HTML of the journal website.

With[{url = URL["https://doi.org/10.1007/s00209-015-1612-7"]},
 Import[
  Echo@StringJoin[
    "https:",
    First@Select[
      ImportString[
       URLRead[HTTPRequest[url], "Body"]
       , {"HTML", "Hyperlinks"}
       ], StringContainsQ[#, "format=bibtex"] &, 1]]
  , "String"
  ]
 ]

This obviously will only work when the DOI link redirects to a journal website with a particular link structure. Notice that the BibTeX content is different.

Answer 2

request = 
 HTTPRequest[
  URL["https://doi.org/10.1007/s00209-015-1612-7"], <|
   "Method" -> "GET", 
   "Headers" -> {"Accept" -> "application/x-bibtex"}, 
   "ContentType" -> "application/x-bibtex"|>];

URLRead[request, "Body"]

(*
    @article{Blasiak_2016,
        doi = {10.1007/s00209-015-1612-7},
        url = {https://doi.org/10.1007%2Fs00209-015-1612-7},
        year = 2016,
        month = {jan},
        publisher = {Springer Science and Business Media {LLC}},
        volume = {283},
        number = {1-2},
        pages = {601--628},
        author = {Jonah Blasiak},
        title = {Haglund's conjecture on 3-column Macdonald polynomials},
        journal = {Mathematische Zeitschrift}
    }
*)

request = 
 HTTPRequest[
  URL["https://doi.org/10.1090/s0002-9947-96-01558-9"], <|
   "Method" -> "GET", 
   "Headers" -> {"Accept" -> "application/x-bibtex"}, 
   "ContentType" -> "application/x-bibtex"|>];

URLRead[request, "Body"]
(*
    @article{Fomin_1996,
        doi = {10.1090/s0002-9947-96-01558-9},
        url = {https://doi.org/10.1090%2Fs0002-9947-96-01558-9},
        year = 1996,
        month = {sep},
        publisher = {American Mathematical Society ({AMS})},
        volume = {348},
        number = {09},
        pages = {3591--3621},
        author = {Sergey Fomin and Anatol N. Kirillov},
        journal = {Transactions of the American Mathematical Society}
    }
* )