# Importing text generates whitespaces between every character

I have a TSV file with string data in it that I want to import. While the TSV format is correctly parsed into a Mathematica table format using Import[<file>,"Table"], the actual text content is oddly formated.

A minimum example that reproduces the problem is this:

I have a text file called text.txt. Its only content is test.

When I use either of the commands ReadList["text.txt"] or Import["text.txt"] I obtain as output:

ÿþt e s t  (visually)

When I copy paste the above visual output into a text editor, it reads as:

"ÿþt\.00e\.00s\.00t\.00"

How is it possible to obtain the expected output: test ?

This is Mathematica 11.2 running on Win 10.

## 1 Answer

Your text file is saved with the byte order mark (BOM) (specifically the UTF-16 (LE) encoding).

Use the following:

Import["test.txt", CharacterEncoding -> "Unicode"]
(* test *)


or change the encoding in your text file (for example with Notepad++) to UTF-8:

• Thanks. It is strange that I have never encountered this issue in 10+ years of Mathematica usage on multiple OS and computers. Jan 3 at 12:01
• How did you generate your text file? With Notepad on Win10? Be careful on the selected encoding in the Save dialog. Jan 3 at 12:05
• The file was generated from another tool, so this can be the origin. However, I then opened the file with Notepad (normal version) and edited it to contain test. And saved it under a new name. Seems that Notepad still preserved the original encoding of that other software. When importing and saving with Excel, Excel destroys the original encoding and removes the Mathematica import issues. However, your solution with CharacterEncoding -> "Unicode" is more elegant because I can import directly from the other software. Jan 3 at 12:08
• @tobalt Serves you right! For using other tools, not just Mathematica... Jan 3 at 12:24