4
$\begingroup$

When I want to import data (.txt or .dat) for plotting, I can just use the Import. Problem is that, sometimes the data in the files are separated by comma, sometimes they don't. Sometimes they are arranged in columns nicely, sometimes they are just in one line. My usual approach is to 'beautify' the data manually, by making sure that each pair of data occupies one line, and the coordinates are separated by the tab key without any comma. This works for a small number of data, and will be impossible for large amount of data.

So my question is, are they any ways to import data regardless of how they are arranged in the data file?

|improve this question
$\endgroup$
4
$\begingroup$

Import and SemanticImport can handle many different formats, and each has several options to customize the import.

For a start, check out Table, TSV and CSV.

Look at documented "Table" options such as "FieldSeparators", "IgnoreEmptyLines", "RepeatedSeparators", "HeaderLines", etc. These give you quite a but of flexibility. It is possible to simultaneously allow multiple different field and record separators, e.g. let either tab or comma separate entries within the same file.

SemanticImport can additionally take a type specification for each column you are importing. This helps with certain data formats. It also has a "Delimiters" option. Its return type can be controlled using its third argument (format), which is described in the Details section of its documentation page. SemanticImport is unfortunately slower than Import.

There is also Read and ReadList. ReadList is very fast, takes type specifications and allows specifying the RecordSeparators.

If none of this helps with your format, please ask a separate, more specific question.

Finally: be sure to read the documentation of all of these, including the Details and Options sections!.

|improve this answer
$\endgroup$
  • $\begingroup$ SemanticImport actually manages to load the data in a nice table format, but is it possible to convert it to a matrix? The main problem I am having is that if I use Import, very often it can represents the data in matrix form $\endgroup$ – Physicist Sep 1 '16 at 10:40
  • $\begingroup$ @Physicist Ah, I assume you mean that it returns a Dataset by default. Please look up Dataset. Please check under the Details section of the SemanticImport documentation where it is explained how to use its third argument ("form") to choose a return format (e.g. list of rows like Import). $\endgroup$ – Szabolcs Sep 1 '16 at 10:42
  • $\begingroup$ @Physicist You can use Normal to convert a Dataset to nested lists and associations. It may actually be ds[All, Values] or similar what you need, depending on the type of Dataset. But better just use the third argument of SemanticImport. $\endgroup$ – Szabolcs Sep 1 '16 at 10:51
  • $\begingroup$ Normal works perfectly for me in this case. thx $\endgroup$ – Physicist Sep 1 '16 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.