# Import tabular data while skipping field delimiters within text delimiters

I have data of the form:

X 1.213 'String Field With Lots of Spaces' 8.123
Y 2.43  'String Field With Lots of Spaces' 8.123
Z 4.83  'String Field With Lots of Spaces' 8.123


I'm trying to import it using the builtin "Table" format (or anything else that would work). Problem is, my field delimiter, a space, can occur within my text delimiters, as the above shows. Setting "FieldDelimiters"->" " and "TextDelimiters"->"'" does not do the trick. Is this possible or do I have to roll my own importer?

The data is a 2D array. The problem in reality is a little more complicated, in that I have two types of text delimiters, ' and ", and one can contain the other. And so a string can be of the form 'String "With" Lots" of Quotes' or "String 'With' Lots' of Quotes". I don't a priori know the type of entries in a column, their ordering, or their widths. I.e. everything has to be inferred on the fly.

Note I am not asking for code to do this from scratch: I can do it myself. I'm just wondering if the builtin importers support this.

• you should show a fully "complicated" example. Do you know the structure? (symbol,number,string,number) – george2079 Nov 8 '16 at 0:18
• I updated it with some additional comments. I don't know the structure a priori. It has to be inferred, but the "Table" importer can usually do that. – Mohammed AlQuraishi Nov 8 '16 at 13:06
• Import[file, "Table", "TextDelimiters" -> "'"] works for the first example.. – george2079 Nov 8 '16 at 15:12

You can Import as "Text". Then use ImportString and RegularExpression to parse. Finally, ToExpression to convert the numbers. It is bit of a run around. I guess this could be considered rolling your own importer.

With

str = "X 1.213 'String \"Field\" With \"Lots of\" Spaces' 8.123
Y 2.43 \"String Field 'With' Lots 'of Spaces'\" 8.123
Z 4.83 'String Field With Lots of Spaces' 8.123
A 4.83 \"String Field With Lots of Spaces\" 8.123";


Updated for not knowing the types in each column.

StringCases[
RegularExpression["((\"|')[^\\2]+?\\2)|([[:alpha:]])|(\\d+(\\.\\d+)?)"] :>
If["$4" != "", ToExpression@"$0", "\$0"]]@StringSplit[str, "\n"]

{{"X", 1.213, "'String \"Field\" With \"Lots of\" Spaces'", 8.123},
{"Y", 2.43, "\"String Field 'With' Lots 'of Spaces'\"", 8.123},
{"Z", 4.83, "'String Field With Lots of Spaces'", 8.123},
{"A", 4.83, "\"String Field With Lots of Spaces\"", 8.123}}


Hope this helps.

This gets 2 out of three lines right:

ImportString[

"X 1.213 'String Field \"With\" Lots of Spaces' 8.123
Y 2.43  \"String Field 'With' Lots of Spaces\" 8.123
Y 2.43  \"string 1 ' \" 'string 2 \" '"

, "Table", "TextDelimiters" -> {"'", "\""}] // MatrixForm


as you see if you are really general you end up with ambiguities. Most likely you need to import as "Lines" and work with string expressions.