Pasting external PDF vector graphics without losing ability to export them

Question

As in previous versions, Mathematica 9 on Mac OS X (10.7.5 here) still has a problem with pasted PDF graphics that were copied from external applications such as Preview.app or Skim.app (I usually use the latter).

To see the problem (I don't know if this also happens on other platforms than Mac), try the following:

Export["t.pdf", "Hello"];

SystemOpen["t.pdf"]

In the PDF viewer, activate the select tool, select all or part of the image and copy it to the clipboard (this is automatically copied in PDF format).

Now in Mathematica, type the line

Export["t1.pdf",  ];

and insert the copied PDF directly in the slot for the second argument by performing a paste from the clipboard.

Evaluate the cell and then look at the result:

First@Import["t1.pdf"]

The result is empty. One can also observe that the pasted PDF in the above example looks like it's been rasterized and is being displayed at the wrong resolution. It actually isn't rasterized, but the display looks ugly and the export of such pasted PDF simply doesn't work. To fix this, I had written the following palette some time ago:

CreatePalette[Column[{Button["Convert PDF on clipboard",
     Module[{pdf, out = "/tmp/MathematicaOutput.pdf",
       prog = "from AppKit import *\nboard=NSPasteboard.generalPasteboard()\nresult = board.dataForType_(NSPDFPboardType)\nif result:\n\tresult.writeToFile_atomically_('**FILENAME**',1)\n\" | /usr/bin/python"}, 
            Run["printf \"" <> 
        StringReplace[prog, {"**FILENAME**" -> out}]];
      CopyToClipboard[If[FileExistsQ[out], pdf = Import[out, "PDF"];
        (*DeleteFile[out]*);
        pdf[[1]], "No PDF"]];
      ], Appearance -> "Palette"],
    "Click before pasting externally generated PDF"}], 
  WindowTitle -> "Convert External PDF"];

After evaluating this, a palette is created that you have to click right after copying a PDF from an external application. If you then do the paste operation again in the following line:

Export["t2.pdf",   ];

First@Import["t2.pdf"]

$\mathtt{Hell}$

The result is the expected PDF. There is another issue related to cropping of the PDF. The above results are obtained using Skim, but when using Preview the crop box is lost and the original bounding box is used.

My question is: Is there a simpler way to get pasted PDF from external applications on Mac to display properly in the notebook and then export correctly?

Answer 1

This isn't exactly what you want, but if you File > Print > Save as PDF then the content will appear correctly. It's only Export where it doesn't appear.

Answer 2

This is a possible solution that I came up with after looking at another answer by @Ragfield: Encoding format used by GraphicsData?

I had to modify the function decodePICT in that post to cut out the leading zero bytes:

DecodePDF[data_String] := 
  Module[{slash, backslash, zero, LF, CR, decode, codes, len, 
    i}, {slash, backslash, zero, LF, CR} = ToCharacterCode["/\\0\n\r"];
   decode[char_] := If[char == slash, backslash - zero, char - zero];
   len = Length[codes = ToCharacterCode[data]];
   i = 1;
   Last@Last@
     Reap@While[i <= len - 1, 
       Which[codes[[i]] == LF || codes[[i]] == CR, i++, 
        codes[[i]] == backslash, i += 4, True, 
        Sow@BitAnd[
          BitOr[BitShiftLeft[decode[codes[[i]]], 2], 
           BitShiftRight[decode[codes[[i + 1]]], 4]], 255];
        i++;
        If[i <= len - 1, 
         Sow@BitAnd[
           BitOr[BitShiftLeft[decode[codes[[i]]], 4], 
            BitShiftRight[decode[codes[[i + 1]]], 2]], 255]];
        i++;
        If[i <= len - 1, 
         Sow@BitAnd[
           BitOr[BitShiftLeft[decode[codes[[i]]], 6], 
            BitShiftRight[decode[codes[[i + 1]]], 0]], 255]];
        i += 2;]]];

convertPDF[badPDF_RawBoxes] := Module[
  {
   str = First@
     Cases[InputForm[badPDF], GraphicsData["PDF", dat_] :> dat, 
      Infinity],
   out = "/tmp/MathematicaOutput.pdf"
   },
  Export[out, DecodePDF[str], "Binary"];
  First@Import[out]
  ]

Now if you type

convertPDF[   ]

and paste the externally generated PDF into the space above, then the result will be the "fixed" PDF which satisfies both requirements: it looks smooth and it can be exported as usual, with no apparent problem.

The next step is to put this into a Palette again. I'll do that after testing this approach some more (unless someone comes up with a better idea).

Edit

The above was exposition to explain what I did; you don't need to copy that code

Here is the final result: a Palette that implements the above conversion for a PDF on the clipboard. It's a drop-in replacement for the Palette I posted in the question, but it doesn't require python and it doesn't run any external commands. It's also faster than the python-based conversion:

CreatePalette[Column[{Button["Convert PDF on clipboard",
     Module[
      {
       str,
       out = "/tmp/MathematicaOutput.pdf",
       decodePDF
       },
      decodePDF[data_String] := 
       Module[{slash, backslash, zero, LF, CR, decode, codes, len, 
         i}, {slash, backslash, zero, LF, CR} = 
         ToCharacterCode["/\\0\n\r"];
        decode[char_] := 
         If[char == slash, backslash - zero, char - zero];
        len = Length[codes = ToCharacterCode[data]];
        i = 1;
        Last@
         Last@Reap@
           While[i <= len - 1, 
            Which[codes[[i]] == LF || codes[[i]] == CR, i++, 
             codes[[i]] == backslash, i += 4, True, 
             Sow@BitAnd[
               BitOr[BitShiftLeft[decode[codes[[i]]], 2], 
                BitShiftRight[decode[codes[[i + 1]]], 4]], 255];
             i++;

             If[i <= len - 1, 
              Sow@BitAnd[
                BitOr[BitShiftLeft[decode[codes[[i]]], 4], 
                 BitShiftRight[decode[codes[[i + 1]]], 2]], 255]];
             i++;

             If[i <= len - 1, 
              Sow@BitAnd[
                BitOr[BitShiftLeft[decode[codes[[i]]], 6], 
                 BitShiftRight[decode[codes[[i + 1]]], 0]], 255]];
             i += 2;]]];
      str = First@Append[
         Cases[NotebookGet[
                     ClipboardNotebook[]], 
          GraphicsData["PDF", dat_] :> dat, Infinity], ""]
      ;
      CopyToClipboard[
       If[str =!= "",
        Export[out, decodePDF[str], "Binary"];
        First@Import[out],
        "No PDF"
        ]
       ];
      ], Appearance -> "Palette"],
    "Click before pasting externally generated PDF"}], 
  WindowTitle -> "Convert External PDF"];

There are some things that don't work correctly with this PDF conversion, but that was the same in my original approach: when copying from multipage PDF documents, some color and background information can be lost. However, the main reason why PDF copy and paste on the Mac is nice (as opposed to bitmaps which are always a viable alternative) is that it produces resolution independent outlines for fonts, lines and shapes - and those properties are definitely preserved with this Palette.

Edit 2

The remaining problems mentioned above imply that this solution doesn't produce a better outcome than my original one, but this answer does run faster because it doesn't have to initialize python and the AppKit module every time.

To fix the problems that remain, I have always been using an additional intermediate step that uses ghostscript. The Mac Application for this is called FontBegone and can be downloaded from my web site. It requires ghostscript, which can be obtained conveniently as part of a $\TeX$ installation.

With this, my personal workflow for copying PDF is: highlight the desired crop area in Skim and copy. Tap the FontBegone icon, then tap the button for the Palette in this answer. Now paste into Mathematica, and you're done.

Having this ability to paste PDF into a notebook completes the circle in a workflow that is quite unique to Mac OS X. The other half of the circle is the ability to copy as PDF from Mathematica into other applications such as LyX to create $\LaTeX$ documents with vector graphics produced in Mathematica (see this answer). We can move PDF graphics around freely in both directions via the clipboard.

More discussion (currently based on my original solution) can be found here.