Questions tagged [csv-format]

Questions about the Text based file format CSV (Comma Seperated Values). Questions include, but are not restricted to, importing from and exporting to this file format.

Filter by
Sorted by
Tagged with
37
votes
5answers
9k views

How can I import a huge CSV file quickly?

I have two CSV files, each one is around 1 GB of data. When I use Import["file.csv"], it takes a very, very, long time to import the data. So how can I accelerate ...
20
votes
2answers
7k views

Speeding up Import and Export in CSV format

I am handling large numerical data in Mathematica. In smaller problems everything worked fine using Export and Import with the ...
13
votes
2answers
1k views

Parsing a CSV file using Interpreter

The new Interpreter functionality in version 10 looks like it has the potential to make parsing custom data formats very easy. I'm trying to create a CSV parser. ...
13
votes
4answers
3k views

Import[ ] drops leading zeros from data when importing a CSV file

I need to load data from the CSV file shown below. Import[ ] drops the leading zeros in column 3 when it imports the CSV file, but they are security identifiers, and I need them. ReadList[ ] ...
12
votes
5answers
2k views

Export a simple DataSet with key-value associations to CSV in Mathematica 10

I'm trying to Export a simple dataset with key-value pairs to a CSV Export["n.csv", MyDataset, "CSV"] This results in a single-line serialization: ...
11
votes
2answers
2k views

Stream CSV or TSV files

RAM limitations make import impractical for large CSV files. Is it possible to Stream CSV correctly? str = OpenRead["EU.csv"]; Read[str, "CSV"] Read::readn: ...
9
votes
3answers
1k views

Mathematica 11.2 Import Statement Produces Errors & Truncates CSV Data Created in 11.1

Bug introduced in 11.2.0 TL;DR Issue 1 (bug): Import of CSV files with column specification fails when "HeaderLines" or ...
8
votes
1answer
7k views

Reading csv/xlsx data into Mathematica with column headings as list names

It's old question. However, I could not find satisfactory answer. Decided to post this question. I have a .csv file with column heading. How can I import this ...
8
votes
1answer
1k views

Reading specific entries from a CSV file

This question is related to the following: Speeding up Import and Export in CSV format Stream CSV or TSV files Reading periodic elements from a large file I have a set of very large CSV files (a ...
7
votes
1answer
371 views

Incorrect result while using SemanticImport for CSV file import

I might use this incorrectly. I want to use a .csv file that (as list) looks like: ...
7
votes
1answer
1k views

Working with a CSV data file with 100 millions rows (and 30 columns)

I am new to working with large datasets.. Currently I am struggling to upload a "big data" file into Mathematica. The file is in the CSV format and has 14GB. There are 100 millions of rows and 30 ...
6
votes
2answers
6k views

Plot CSV-data; make statistical statements in Mathematica / WA

First, I'm quite new to Mathematica so sorry in advance if some questions seem to be trivial. I have a CSV-file which looks like this: ...
6
votes
2answers
2k views

Exporting strings without the quotes

I'm trying to export the following string "sep = ," to a CSV-file, as to be able to define the separation symbol. However, when using the following command: <...
6
votes
2answers
127 views

Importing a string such as “3d456” from a CSV file causes trouble

I have a CSV file I'm importing with Import and it consistently crashes Mathematica. It turns out the culprit is one particular value of an ID field that was ...
6
votes
1answer
1k views

Efficiently reading CSV with ReadList

I have a large CSV file (~1gb) with mixed data. Basically with strings and integers. It takes very long to import this whole dataset with Import[] so I would like ...
6
votes
2answers
229 views

SemanticImport fails with CSV when last column is ALL empty

(MMA v11.1.1) I have many empty fields in my CSV files and I couldn't understand why sometimes SemanticImport just failed to recognize the headers. I just find out ...
5
votes
1answer
360 views

Huge CSV file with numeric and text data

I try to load a huge CSV file (~ 4 Gbyte), the data is both numeric and textual. So, I can't load the data as a string. ...
5
votes
2answers
798 views

Can Mathematica understand Excel dates when in number format?

I have a very large Excel file with dates in the format 2013/01/17 08:02:00 PM which is a custom date format. I have to convert this file to CSV in order to import it into Mathematica because I can't ...
5
votes
0answers
130 views

How to change Export default in 11.2?

To achieve the same CSV output as from previous versions Export requires the option "TextDelimiters" -> "". Presumably the ...
4
votes
3answers
2k views

Importing CSV with multiline text fields

Bug introduced in 8 or earlier and fixed in 11.2.0 Here is a valid CSV: a1,b1,c1,d1,"e1 e1 e1" a2,b2,"c2 c2 c2",d2,e2 It has two rows and five columns, and ...
4
votes
1answer
480 views

Import specific columns from a large amount of CSV files

I'm looking to import the 4th column of data from a few hundred CSV files that I have stored in various folders. I have been able to import all CSV files from a given directory through use of: ...
4
votes
1answer
716 views

Importing data from a CSV file — a string is being converted to a number

I'm very much a newbie, so please forgive any errors or stupidity. I have a csv file named csvfile. Its, contents confirmed by me, by looking at the contents in a ...
4
votes
2answers
812 views

Excluding Specific Rows Using Import[…]

As the question states, I'm trying to import a .csv file but I would like to exclude certain rows (points which I know are bad data). This is easy when the points ...
4
votes
1answer
780 views

How to make Import[…, “Table”] not convert strings to numbers and not convert numbers to strings?

Bug introduced in 8 or earlier and persisting through 11.2.0 I have to to use r = Import["foo.txt", "Table", "FieldSeparators" -> {","}] since my data is Table ...
4
votes
0answers
266 views

How to speed up import of large csv table? [duplicate]

Loading less than 2 gB of csv data took 5.29 hours: ...
3
votes
3answers
1k views

Why does a certain CSV file import properly into LibreOffice but not into Mathematica 10.0?

I'm working on a project involving data manipulation and it involves working with CSV files. I know that the files are properly formatted as when I import them into LibreOffice, all of the columns ...
3
votes
1answer
393 views

Writing CSV or alternative

I've tried to export a list to a CSV file. I shut it down after about 20 minutes of processing. The list has 261,000 rows and 30 columns. I finally exported it to a TXT file and its size was 111 MB....
3
votes
3answers
212 views

How can I import a CSV file into Mathematica? [closed]

I have a data.csv file in the Downloads folder in my Macbook. I am trying to import the data into a Mathematica notebook using ...
3
votes
1answer
707 views

Appending to a .csv file [duplicate]

I have a selection of Lists of data which are in the same format, and if I Export them all to CSV at the same time I get a nice ...
3
votes
1answer
2k views

How do I read a CSV file and remove comments from it

I have very recently started using Mathematica 9.0 Home Edition and am trying to understand it as best I can so please be tolerant ;). I have a file of data (sugar.csv) which contains blood sugar ...
3
votes
2answers
2k views

Interpreting the interpolating function and saving data to plot with external program

So far I have been solving non-linear pdes with NDSolve and then plotting the result with the in-built Plot3D and ...
3
votes
1answer
5k views

How to export CSV from Mathematica?

I have previously been using logf[x_, y_] := f[x, y] /. v_ :>(PutAppend[Unevaluated[f[x, y] = v;], "out.txt"];); which produces output like ...
3
votes
2answers
545 views

I have a bigdata CSV file as a single table of five columns and almost 10 million rows

I need to examine the bigdata file with a database/DBMS and have used MS Office 365/Access successfully so to do, import and query. I have also tried, many times, to open a MMA 10.4 database link ...
3
votes
1answer
1k views

Removing quotes from fields after import of table [closed]

I would like to remove quotes around fields when importing a csv table. It should be simple, but I cannot find any answer. My data is in the format ...
3
votes
1answer
982 views

StringSplit on comma but ignore commas in quotes [closed]

I'm importing CSV files and some of the fields include quoted text with commas as part of the field, rather than as a field separator. What is the syntax (or alternative method entirely) to import ...
3
votes
2answers
667 views

Troubles importing date and time from CSV file

I'm trying to import CSV file with the following format: 20150803 00:00:01.649,1.09675,1.09678 using this code: ...
3
votes
1answer
2k views

How to properly export string columns in csv?

How can I make sure that certain string columns of a matrix I want to export get properly formatted? That is, all quotes escaped, and every string wrapped in quotes. Here's what happens now: ...
3
votes
0answers
110 views

SemanticImport extremely slow with “Number” format

With the following definition: col = "num\n" <> StringRiffle[Table[i, {i, 1000}], "\n"]; I do not understand why the ...
3
votes
0answers
127 views

Exposing .csv via Wolfram Cloud API

I'm sure there's a better way to accomplish this, but my goal is to generate a .csv file from a running notebook, and have it accessible by API. In the main program, I'm exporting the data like so: <...
2
votes
1answer
556 views

Empty field “\”\“” In CSV Import Problem

I have a huge CSV data, with the format like this: "5";"8";"2803"; "";"6";"2793"; I import the data with this code: ...
2
votes
1answer
116 views

How do I tell Mathematica to write numbers on a file without using scientific notation?

I need to write numbers out to a file without using the scientific notation. How do I tell Mathematica to do that? This is the code I am using: ...
2
votes
1answer
530 views

Convert spreadsheet (Excel) to *.csv during import

I have long lists of data in spreadsheets and have the problem that Mathematica crashes regularly during import of the .xlsx files. Is it possible within Mathematica to take a certain tab of a .xlsx ...
2
votes
1answer
189 views

How can empty fields in a csv file be set to Missing when importing as a Dataset?

When importing a csv file with empty fields as a Dataset, the empty fields are set to zero-length strings (""). There is an option, ...
2
votes
2answers
83 views

Convert CSV data for use with CandleStickChart

can anyone show me an example of how to import a CSV file and produce and CandlestickcChart using the imported data. I have tried ...
2
votes
2answers
430 views

Exporting files while running .m file from command line

I have the following code in a .m file which is called test.m: Export["test.csv",0]; Exit[] I would like to run this .m file from the command line. I found that I can do that with the command "math -...
2
votes
1answer
1k views

Export CSV from tables

I am reading my csv which looks like: float,float,float,integer,boolean I read it with: ...
2
votes
2answers
933 views

Plotting data imported from CSV file

So I have some data in a csv file that I want to plot. The csv file has several columns. I want to plot one of the columns as the x axis and all the other columns on the y axis. I have used the Import ...
2
votes
1answer
96 views

CSV import with delimiters not working for certain characters (curly quotation marks)

Here's an example CSV file (Example 1): "Text","”","False","False","False","False","0","1","24","0","9","450","450" and the code to import it (Code 1): ...
2
votes
1answer
68 views

Importing a csv file yields weird first character

Whenever I import a CSV file, I get this weird character at the beginning of the file ,I am not sure why. How can I avoid this?
2
votes
1answer
122 views

Importing integer codes with leading zeros as strings from CSV [closed]

Importing CSV seems to automatically cast to integer what are supposed to be integer codes. This causes problems when some codes have a prefix of a variable number of "0" and have to be matched ...