# Difficulties with Importing PDFs in Mathematica

I am trying to import a group of PDFs that I have downloaded (it is in the realm of 1,000+). They are a group of doctoral dissertations that I'm trying to automatically extract bibliographic information from. They open perfectly in Preview and Adobe Acrobat Reader. Unfortunately, when I go to Import[] them, I receive the following errors like so:

Import["1997-01.pdf","Elements"]

Import::general: Expected cross reference table >>
Import::general: Expected cross reference table >>
Import::general: Could not find document trailer >>
General::stop: Further output of Import::general will be suppressed during this calculation. >>


I have never encountered this with Elements before, which is normally my first step in figuring out how to tackle a file. The same holds for Plaintext, etc. If I Import["1997-01.pdf","Text"] it generates a file akin to this:

%PDF-1.2
%BHIL-SC:0014503155,00391,00004,00008,00399,00790,00004,00004,00009,00400,00791,00004,00004,00010,00401,00792,00004 ... etc. etc.


The file itself is downloaded from an online repository, however, but is of high quality.

My end goal is to try TextRecognize[] on it, but I cannot even get to that stage. Does Mathematica offer any workarounds for PDFs like this? FWIW, the PDF claims that I have full permissions to edit, alter, etc., but there is no built-in text layer.

--- Edited to Clarify ---

The PDF is version 1.2. It is an image layer, not a text layer. Hope that this helps!

-
Out of curiosity, have you worked this out? –  CHM Mar 30 '12 at 1:51
@CHM I have. I'm using Automator to take each PDF, split it into individual files, and then converting it to a JPEG. I then use Mathematica to import the JPEG, TextRecognize[] it, and have begun crunching the information that way. It's a painstakingly slow process in terms of computing time, but all pretty automated. –  programming_historian Mar 30 '12 at 3:31
@ianmilligan I suggest to you GIF 600 DPI. I've had better results with that than with JPG 600 DPI and TIFF 600 DPI as well as smaller filesizes than TIFF. Mind going to the chatroom? –  CHM Mar 30 '12 at 3:43
I suspect the file(s) may benefit from being processed by Adobe Acrobat Pro, specifically, its Recognize Text option. –  murray Apr 19 at 3:51

If it's really scanned images, then you could try this:

pages = Import["yourfile.pdf", {"PDF", "Images"}]


Otherwise, I'd suggest running the file through ghostscript or another distiller to clean up the potentially malformed PDF code first. The command would look like this:

gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=newfile.pdf badfile.pdf


Edit

Since you mentioned that you're on Mac OS X and also appear to be able to view the PDF file in Preview.app, there is an even simpler way:

If you're on Lion: Open the PDF in Preview, and export it as multipage TIFF. This can be done under the File > Export menu. The resulting file can be imported in Mathematica and yields a list of images.

Another possible approach that works for all OS X versions is to open the PDF in Preview and print it to a file. With that method, you could also select only the pages you really need by highlighting them in the Thumbnail view, and then choosing File > Print Selected Pages.... The PDF-printed file would hopefully have been processed to get rid of the errors.

-
Thanks very much for your help. The command returns the same error Import::general : Expected cross reference table >>. I will look into ghostscript, I have never used it before. –  programming_historian Mar 10 '12 at 16:30
OK - since it looks like you need it, I've added more specific info on ghostscript. I'm using it on Mac OS X, but it's cross-platform. –  Jens Mar 10 '12 at 16:41
Thanks, Jens. This is great and will save me hours upon hours of work. I'm also on OS X so this would be perfect. –  programming_historian Mar 10 '12 at 16:44
@Ian, I realized that as a Mac user you have some more "manual" options that could be useful, see the edit above. –  Jens Mar 10 '12 at 18:12
These are cool. This will motivate me to learn Automator or something else to make this a quicker process. –  programming_historian Mar 10 '12 at 19:47

In the command line on linux (I am using fedora 20), try:

pdftotext PDFFILE.pdf NEWFILE.txt


Then in Mathematica:

variable = Import[NEWFILE.txt, #] & /@
{"Element1", "Element2", "Element3", "Element4", "Element5"};


where "Element1..." are the elements obtained by

Import [NEWFILE.txt, "Elements"]


Then you can examine the variable using part:

var[[   SOME INDEX OPERATION  ]]

-
on fedora if you dont have pdftotext try: sudo yum install pdftotext –  Schopenhauer Apr 19 at 3:00
after you do this there we go you have your data ready for string manipulation using regular expressions to find references or just use Perl –  Schopenhauer Apr 19 at 3:02