# How to “Copy as Unicode” from a Notebook?

I spent some time manually editing a post replacing Mathematica ASCII \[Alpha] with Unicode α. I did this by laboriously choosing Copy as LaTeX, pasting into the edit box, and then copying the Unicode symbol from the preview below. This made me realize I am lacking a "Copy as Unicode string" function in Mathematica.

How can I most easily copy an expression such as:

In Unicode:

αβ + Mod[δΨ, 2 ⁢ρ^2]

-

Here's how to add a permanent menu item and keyboard shortcut for Silvia's solution. (This can be easily adapted to Jens' solution for Mac OS.)

Copy Silvia's first code block that starts with Needs["NETLink"] to a file and save it as UniCodeCopy.m in one of the directories included in $Path. ### 2. Initialize the Front End Save the following to the file $UserBaseDirectory/Autoload/FrontEnd/init.m (create it if needed):

Needs["UniCodeCopy"]

CopySelectionAsUnicode[] :=
Module[{selection, code},
FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
selection = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
code = StringReplace[selection, {"\\\n" -> "",
"\n" ~~ space : " " ... :> StringJoin["\n", ConstantArray[space, 4]]}];
UniCodeCopyWriteToClipboardUnicode[code];]

FrontEndExecute[
FrontEndExecute[CopySelectionAsUnicode[]],
(* Alt + U *)
MenuKey["u", Modifiers -> {"Command"}],
SystemMenuEvaluator -> Automatic,
Method -> "Queued"]}]]


For key bindings, the possible modifiers are "Shift", "Control", "Command" (Alt) and "Option" (Alt).

(It took me a few hours to figure this out. I hope this proves useful for other struggling novices.)

-
Very nice! Thanks for sharing :) –  Silvia Aug 10 '14 at 6:46
Are you saying that $UserBaseDirectory/Autoload/FrontEnd/init.m should be created, because there is no such file on my Windows 8.1 computer? There is, however, the file $UserBaseDirectory/FrontEnd/init.m on my computer, but appending your code to the end of that file caused Mathematica 10.0.2.0 to crash during initialization. Thanks in advance for your assistance. –  bbgodfrey Mar 8 at 22:40
Yes, just create it. I've added that to my post. –  ens Mar 8 at 22:53

Here is a function that copies a Unicode string to the clipboard using JLink:

Needs["JLink"];
InstallJava[];
LoadJavaClass["java.awt.Toolkit", AllowShortContext -> False];

uniclip[s_String] :=
JavaBlock[
javaawtToolkitgetDefaultToolkit[]@getSystemClipboard[]@setContents[#, #]& @
JavaNew["java.awt.datatransfer.StringSelection", s]
];


Example usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

uniclip @ ToString[expr, InputForm]


After evaluating these expressions, the clipboard contains αβ + Mod[δΨ, 2*ρ^2].

Update

In the comments, a request was made to include code to invoke this functionality from a context menu. I was unable to fulfill this request. If someone wants to take up the work, here is a packaged form of the function which will save them the trouble of dealing with the finicky auto-load requirements of anything that uses JLink:

BeginPackage["Uniclip", {"JLink"}];

CopyUnicodeToClipboard::usage =
"CopyUnicodeToClipboard[s] copies s to the clipboard using Unicode characters";

CopySelectionAsUnicode::usage =
"CopySelectionAsUnicode[] copies the currently selected text to the clipboard using Unicode characters";

Begin["Private"];

retry:_CopyUnicodeToClipboard :=
( InstallJava[]
; LoadJavaClass["java.awt.Toolkit", AllowShortContext -> False]
; CopyUnicodeToClipboard[s_String] :=
JavaBlock[
javaawtToolkitgetDefaultToolkit[] @
getSystemClipboard[]@
setContents[#, #]& @
JavaNew["java.awt.datatransfer.StringSelection", s]
]
; retry
)

CopySelectionAsUnicode[] :=
Module[{text}
, FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]]
; text = NotebookGet[ClipboardNotebook[]][[1, 1, 1]]
; CopyUnicodeToClipboard @ text
]

End[];

EndPackage[];


WARNING The function CopySelectionAsUnicode functions erratically when called from a menu item. It involves a re-entrant call to the front-end which introduces a lengthy delay.

The auto-load declaration to go into the kernel's init.m looks like this:

DeclarePackage["Uniclip", {"CopyUnicodeToClipboard", "CopySelectionAsUnicode"}];


The package code must go into a file named Uniclip.m somewhere in the $Path. The context menu declaration (for ContextMenus.tr) looks like this: MenuItem["&Unicode", KernelExecute[ToExpression["UniclipCopySelectionAsUnicode[]"]], MenuEvaluator -> Automatic],  I had added it under all of the Copy As menu items for the Text, Input, Output, Plugin, CellBracket styles. - This is great! I suppose it should work on all platforms? – Silvia Sep 7 '14 at 20:43 Great - I see no problems on Mac OS X 10.7.5 with Mathematica 10. – Jens Sep 7 '14 at 21:33 @Silvia I can only vouch for it working on Windows, but by rights it should work on all platforms. – WReach Sep 7 '14 at 22:20 I believe this is the first multiplatform solution. Provide complete self-contained package code to add this as a menu item and shortcut key (in the context menu if you can manage it), and I'll Accept it so long as no failures appear. Thanks! – Mr.Wizard Sep 8 '14 at 6:55 @Mr.Wizard I spent a couple of hours today trying to fulfill the request, but failed. I kept hitting race conditions in re-entrant front-end calls while trying to hook up the context menu. This is because the only way I know how recover the text selection involves a front-end call, and the menu item invocation is already a front-end call. I can't really spend more time on it right now, so if anyone wants to incorporate uniclip into another answer that adds menu items and shortcut keys, feel free. – WReach Sep 20 '14 at 18:33 Edit 2: A new version of the Mac solution with button is listed below Fixed problem with pasting into textarea In some applications on Mac, copying as Unicode from Mathematica already works without having to do any postprocessing. However, it doesn't work in textarea fields in web browsers. Nevertheless, if you're willing to do a few additional mouse clicks, the Unicode forms can be brought into the browser by taking the detour through one of those applications that do support Mathematica's characters. One convenient choice is to open a new window in TextEdit and paste your original Mathematica code into it first. In the default RichText mode, TextEdit displays the special characters as Unicode glyphs. Then you can simply copy whatever you just pasted back from TextEdit and paste it into the browser. It seems that as soon as TextEdit is recognized as the provider of the Unicode text on the pasteboard, there is no conversion back to the Mathematica representation, so you get the correct appearance in the browser. In the following, I use the same idea in order to automate the conversion. Instead of TextEdit, I go through a temporary file in RichText (RTF) format on the provider side. The Mac pasteboard does support RTF as a format for data on the pasteboard, but this isn't accepted by the browsers I've recently tried when attempting to paste into textarea. Therefore, I have to add one additional step: using the textutil tool which is built into OS X, I convert the RTF file to a regular text file with Unicode characters first. Then I read the text file back in and put it on the pasteboard. Of course, this means it's only going to work on Mac OS X because it uses Cocoa bindings in the built-in Python interpreter: copyAsUnicode[t_] := Module[{out = FileNameJoin[{$TemporaryDirectory,
"MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]},
Export[out <> ".rtf", t];
Run["textutil -convert txt " <> out <> ".rtf -output " <> out <>
".txt"];
Run["printf \"from AppKit import *\n\
board=NSPasteboard.generalPasteboard()\n\
content=NSData.dataWithContentsOfFile_('" <> out <>
".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\
board.setData_forType_(content, NSStringPboardType)\n\
\" | /usr/bin/python"];
DeleteFile[{out <> ".txt", out <> ".rtf"}]]


The idea is to export to RTF and read the result to the clipboard outside of Mathematica. The function is invoked for example as copyAsUnicode["αβ+Mod[δΨ+ρ2]"]. This example itself was copied that way, too, i.e., I typed copyAsUnicode["copyAsUnicode[\"αβ+Mod[δΨ+ρ2]\"]"], which I again copied the same way... OK, I think you get the idea.

Of course the next step would be to make this into a Palette that acts on the NotebookSelection, but the above is the main step. Maybe someone else knows how to do something like this in other operating systems (I don't).

Installing this function as a button:

Responding to the comment, here I'm just listing the same function as above, but wrapped in Silvia's code to make it into a button. It requires no package loading because I inlined everything into the button code:

Button["UniCode Copy", Module[{codestr},
FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
codestr = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
Module[{out =
FileNameJoin[{$TemporaryDirectory, "MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]}, Export[out <> ".rtf", codestr]; Run["textutil -convert txt " <> out <> ".rtf -output " <> out <> ".txt"]; Run["printf \"from AppKit import *\n\ board=NSPasteboard.generalPasteboard()\n\ content=NSData.dataWithContentsOfFile_('" <> out <> ".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\ board.setData_forType_(content, NSStringPboardType)\n\ \" | /usr/bin/python"]; DeleteFile[{out <> ".txt", out <> ".rtf"}]]], Method -> "Queued"] SelectionMove[EvaluationNotebook[], Previous, Cell]; FrontEndExecute[FrontEndToken["GeneratePalette"]]; FrontEndExecute[FrontEndToken["Clear"]];  Now you can also install this permanently by going to Palettes > Install Palette and selecting a name for the palette. - This looks good, but since I cannot test it on Windows I will leave it to Mac users to upvote. Thank you for your answer. – Mr.Wizard Feb 2 '12 at 4:27 Hi, I posted one question with little difference from directly copy Input Expression to SE and need your attention. mathematica.stackexchange.com/questions/27289/… – HyperGroups Jun 20 '13 at 1:47 @Jens, this is a nice workaround (and it still works). Any idea if there are plans for this to be integrated into a Palette as you suggested? – Rico Picone May 13 '14 at 18:30 @RicoPicone Thanks, I added the button code, hopefully that should put it on equal footing with the windows solution (except for potential shortcomings of the Mac Unicode handling, which I have no control over...). – Jens May 13 '14 at 19:33 @Jens, thanks for your work on this. I can't get it to work. I get the message (copy from clipboard option) that the clipboard contents are invalid. I'll try to save it as a file, next. – Rico Picone May 14 '14 at 14:32 ## Declaration: This method for Windows is based on the .NET code from Todd Gayley's this wonderful answer. My .NET knowledge is absolutely ZERO, all credit goes to Todd. ## Code: The main idea is to extract the "Input"-style code string, convert it to the UTF-16 little endian form, which is the standard byte order in Windows, feed the bytes to system clipboard by using .NET API. The main functions are placed in a package: Needs["NETLink"] InstallNET[]; BeginPackage["UniCodeCopy"] mmaUnicodeToUTF16LE::usage = ToString[Row[{Style["mmaUnicodeToUTF16LE[_Integer]", Bold], " convert integer list obtained from ", Style[Row[{"ToCharacterCode[", Style["string", Italic], ", \"Unicode\"]"}], Bold], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm]; stringToUTF16LE::usage = ToString[Row[{Style["stringToUTF16LE[_String]", Bold], " convert ", Style["string", Italic], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm]; WriteToClipboardUnicode::usage = ToString[Row[{Style["WriteToClipboardUnicode[_String]", Bold], " write ", Style["string", Italic], " to Windows clipboard using the UTF‐16 Little Endian encoding."}], StandardForm]; Begin["Private"] Clear[mmaUnicodeToUTF16LE] mmaUnicodeToUTF16LE[bytecode_Integer] := PadRight[#, 2 Ceiling[Length@#/2]] &@ Join[Most[#] - Rest[# 2^8], #[[{-1}]]] &@ DeleteCases[FixedPointList[BitShiftRight[#, 8] &, bytecode], 0] Clear[stringToUTF16LE] stringToUTF16LE[str_String] := mmaUnicodeToUTF16LE /@ ToCharacterCode[str, "Unicode"] // Flatten // Join[#, {0, 0}] & Clear[WriteToClipboardUnicode] WriteToClipboardUnicode[str_String] := Module[{bytecode, strm, dataObject}, bytecode = stringToUTF16LE[str]; NETLinkNETBlock[ strm = NETLinkNETNew["System.IO.MemoryStream", bytecode]; dataObject = NETLinkNETNew["System.Windows.Forms.DataObject"]; dataObject@NETLinkSetData["Text", strm]; NETLinkLoadNETType["System.Windows.Forms.Clipboard"]; SystemWindowsFormsClipboardSetDataObject[dataObject] ] ] End[] EndPackage[]  Then execute the following code, which will generate a palette with a UniCode Copy button: Button["UniCode Copy", Module[{codestr}, timestamp = {}; FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]]; codestr = NotebookGet[ClipboardNotebook[]][[1, 1, 1]] // StringReplace[#, {"\\\n" -> "", "\n" ~~ space : " " ... :> StringJoin["\n", ConstantArray[space, 4]]}] &; UniCodeCopyWriteToClipboardUnicode[codestr] ], Method -> "Queued" ] SelectionMove[EvaluationNotebook[], Previous, Cell]; FrontEndExecute[FrontEndToken["GeneratePalette"]]; FrontEndExecute[FrontEndToken["Clear"]];  To use it, select the Cells you want to copy as input text code, then press the button. The package can be put in a file and be auto-loaded when MMA start, the palette can be installed to system menu. ## Examples: Unicode expressions in Mathematica Notebook: Text generated directly by the above code: "αβγδϵζηθχϕϡΔΥϖϒϠíãéõÂÆŁØÝ∟△♆♇√÷∇■○▫✶✓¥¶†︵│︷中文汉字日文コンピュータ" α β + Mod[δΨ, 2 ρ^2] - Very nice. Thanks! – Mr.Wizard Apr 1 '14 at 22:10 @Mr.Wizard It's my honour :) Needed this tool myself for a loong time. And there are still problems, such as some characters from private area get copied, e.g. a. Maybe a whitelist is needed. – Silvia Apr 1 '14 at 22:16 Arguably copy of characters such as \[Transpose] as  is acceptable as they can still be pasted into a Notebook, although it would be nice to also have the option to copy the expanded FullForm of such private letters. – Mr.Wizard Apr 2 '14 at 2:23 @Mr.Wizard One possible way came into my mind: Read the Boxes, identify the private characters (especially those used as functions like \[Transpose]), convert the corresponding parts into InputForm(how?), then make the copy. – Silvia Apr 2 '14 at 20:41 @Mr.Wizard Wow thank you for your bounty! :D – Silvia Apr 7 '14 at 23:59 I use a small web application for when there are too many to convert by hand. - Very handy. Now if we could just get this as a button in the editor. – Mr.Wizard Jun 19 '13 at 16:09 On Mac, I can get the same result by simply pasting the Mathematica code into the TextEdit application and immediately copying the same passage from there. So there's no need to go to the web. – Jens Jun 19 '13 at 17:15 @Jens very true. One day everyone will use Macs... :) (typed on a iPad) – cormullion Jun 19 '13 at 17:27 @Jens Hmm.. did not work for me - MacBook Pro OSX - all standard things. Copied from notebook - pasted into TextEdit - had all the \[Alpha] etc. Some special settings? – Vitaliy Kaurov Apr 3 '14 at 4:48 @VitaliyKaurov Works for me. Did you set the document format to Plain Text perhaps? Then it won't work of course. – Jens Apr 3 '14 at 5:37 Since a native method is not forthcoming, I shall post my file based circumvention, for Windows. You will need to have this utility in the command path (it apparently is stock with Windows 7). copyUnicode[expr_] := Run["clip <", Export["$Clipboard.temp", ToString[expr, InputForm],
"Text", CharacterEncoding -> "Unicode"] ];


Usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

copyUnicode[expr]


This leaves the following text in the Windows Clipboard:

αβ + Mod[δΨ, 2*ρ^2]


Here is a version of the function that holds (does not evaluate) the expression:

SetAttributes[copyUnicode, HoldFirst]

copyUnicode[expr_, form_: InputForm] :=
Run["clip <",
Export["\$Clipboard.temp", ToString[Unevaluated@expr, form], "Text",
CharacterEncoding -> "Unicode"]];


Now:

Plot[\[Alpha], {\[Alpha], 0, 10}] // copyUnicode


Puts in the Windows Clipboard:

Plot[α, {α, 0, 10}]

-
I think clip is native in at least win-7. Perhaps you should note that this places the output in the system clipboard. The output of your copyUnicode function is just a 0 (at least, on my PC). You have to do a ctrl-v paste afterwards. A MMA Paste doesn't seem to work. –  Sjoerd C. de Vries Feb 2 '12 at 21:10
Very useful function! Why not make it available for any code sample? (Now it works only for single expression…) –  xzczd Nov 23 '12 at 14:09
@xzczd I'll see if I can improve it. –  Mr.Wizard Nov 23 '12 at 23:56
There's a built-in CopyToClipboard function, but it doesn't seem to work, at least on Linux. –  Mechanical snail May 21 '13 at 1:46
Does it possible to convert string to Plot[\[Alpha], {\[Alpha], 0, 10}] or Cell[BoxData["\<\"Plot[\\\\[Alpha], {\\\\[Alpha], 0, 10}]\"\>"], "Output"}] to Unicode Form? Because I use one function something like InputCell2SE@cellExpression//CopyToClipboard copy InputCell to SE which turns out Plot[\[Alpha], {\[Alpha], 0, 10}], and I used FrontEnd..."InputText" –  HyperGroups Jun 19 '13 at 15:08

Here's a version that doesn't require a temporary file.

# Linux (needs xclip)

SetAttributes[copyUnicode, HoldAll];
copyUnicode[expr_] := With[{
stream = OpenWrite["!xclip -in -selection clipboard", CharacterEncoding -> "UTF-8"]
},
WriteString[stream, ToString[Unevaluated@expr, InputForm]];
Close@stream;
];


Example: executing the cell

Cell[BoxData[
RowBox[{"copyUnicode", "[",
RowBox[{
RowBox[{
RowBox[{
SuperscriptBox["x", "2"], "\[SmallCircle]",
RowBox[{"{",
RowBox[{
RowBox[{"\[LeftFloor]", "\[Alpha]", "\[RightFloor]"}], ",",
"\"\<\[LeftFloor]\[Alpha]\[RightFloor]\>\""}], "}"}]}],
"\[PlusMinus]",
RowBox[{
SqrtBox["5"], "\[CirclePlus]", "\[HappySmiley]"}]}],
"<=",
"\"\<\[Integral]\[PartialD]\[RightArrow]\[Union]\[Sum]\[Infinity]\
\[Element]\>\""}], "]"}]], "Input"]


gives x^2 ∘ {Floor[α], "⌊α⌋"} ± Sqrt[5] ⊕ ☺ <= "∫∂→⋃∑∞∈".

# Windows

Not tested, but it should work if you use "!clip"instead of"!xclip -in -selection clipboard". You might have to change the encoding to UTF-16.

# Caveats

• Note that certain characters get ASCIIfied anyway in InputForm when not inside a string.
• Mathematica uses non-standard private-use code points for some characters like U+211D ℝ, even when a standard code point exists, so the output will be wrong if the input contains such characters.
-
Your caveats also apply to Mac OS X, so there's more work to be done before it becomes 100% reliable... (+1) –  Jens Jun 19 '13 at 20:21
Another caveat to add: some special characters, such as \[UpEquilibrium], get converted incorrectly or not at all, no matter what I try. So I don't think it's possible to get a perfect translation to Unicode in general. –  Jens Jun 19 '13 at 20:46