Questions tagged [dataset]

Questions on the Dataset data type and related functionality introduced in version 10. Do not use for questions on data in more generic list format or data-manipulation questions generally.

Filter by
Sorted by
Tagged with
12
votes
5answers
2k views

Export a simple DataSet with key-value associations to CSV in Mathematica 10

I'm trying to Export a simple dataset with key-value pairs to a CSV Export["n.csv", MyDataset, "CSV"] This results in a single-line serialization: ...
12
votes
5answers
299 views

Best way to convert numerical columns of a dataset with missing values into a matrix

What is the best way (simplest, fastest) to convert the numerical columns of a dataset into a matrix? The numerical columns can have missing values. For example for this dataset: ...
12
votes
5answers
488 views

Dataset Processing: efficient ways to clean and merge sets for Life Sciences

Dataset Processing (for Life Sciences) Note: a related, but distinct task is posted here ID Swapping: Efficient use of a reference table to convert ID values. A common task, at least for me, ...
12
votes
3answers
400 views

Create a new dataset column based on the row key

I am wondering if there is a straightforward way to add a new column to a dataset if the column needs to be computed from the key values rather than other columns. Here is a simple example. ...
12
votes
1answer
371 views

How to make datasets display correctly?

NB: The problem illustrated below is truly ubiquitous. I hope the examples given below are sufficiently different to demonstrate this fact, and to discourage answers that hinge on the details of the ...
12
votes
1answer
156 views

Using Select on Dataset with missing keys

I have a very large dataset with about a million entries with up to 75 keys each. Across the whole dataset about half of the keys are blank. In order to reduce filesize, I have used the following ...
11
votes
4answers
3k views

Join two datasets on keys

I want to be able to join two datasets by keys, pretty much like I'd do in SQL. I thought the JoinAcross function might do it, but it doesn't operate the way I thought. The example in the ...
11
votes
5answers
319 views

Select only first n elements in operator form?

Is there any way to make Select take a second argument (to select only the first n elems) while in operator form? It's too ...
11
votes
2answers
238 views

Missing: KeyGroupBy?

Especially in tree-structured data, keys and values, being two sides of the same coin, frequently interchange roles. The ability to aggregate by value but not by key upsets the balance. How to ...
11
votes
2answers
615 views

Add column heads to a dataset

I have a dataset which I group an then calculate the maximum of an item. When done I get the desired result, but have no column heads. Here the example association (subset of my data): ...
11
votes
1answer
285 views

10.0.2 breaks a recursive “trie” query

Bug introduced in 10.0.2 and fixed in 10.2 The 10.0.2 Dataset type system complication breaks a recursive trie constructor that worked in 0.1: ...
11
votes
2answers
217 views

Lookup with default broken in Dataset

Given an Association, ...
11
votes
1answer
548 views

Select From List of Nested Associations

I have an association: ...
11
votes
2answers
197 views

Large Dataset query fails if column contains Missing

In version 11.0 (Win 8.1 Pro) a query on a large Dataset for a column that contains Missing fails most unceremoniously. With <...
11
votes
1answer
870 views

A comprehensive list of all correct input formats for [[experimental]] Neural Net functions?

Question: correct formats for Mathematica's NetTrain function Explanation of the problem Background So Mathematica 11 was released earlier this month, and while there are many improvements and they ...
11
votes
1answer
98 views

Nondeterministic $Failed when mapping over a Dataset

Consider the following simplified example (on Mathematica 12): ds = Dataset@{<|"hell" -> "o"|>, <|"hell" -> "no"|>}; ...
11
votes
1answer
166 views

Possible Bug in ProbitModelFit when used in a Dataset

Since upgrading to Mathematica 10 on Mac OSX I have come across a number of instances of this error, which occurs using Probit and Logit model fit. A bit of googling show this is something to do with ...
11
votes
2answers
794 views

How to merge Datasets representing matrices with named rows/columns?

I am working with square (non-symmetric) matrices with named rows and columns. The data I receive is sometimes messy and rows/columns may come in various permutations. I usually need to semi-...
11
votes
1answer
131 views

Display bug in Dataset's UnitConvert handling

Bug introduced in 11.0.1 or earlier and fixed in 11.1.0 Is this yet another annoying regression in v11 Dataset type handling? The display form seems to be a ...
11
votes
1answer
358 views

Getting data from FRED

Mathematica V10.4 supports a service for connection to FRED. If you create a connection with fred = ServiceConnect["FederalReserveEconomicData"] (you need to ...
11
votes
1answer
180 views

Odd StringTake problem in Dataset

Bug introduced in 10.0 and fixed in 10.4 Dataset is new in 10.0 10.2 Win64 Works fine... ...
11
votes
1answer
134 views

Dataset interferes with SeedRandom

Bug introduced in 11.0.1 or earlier and fixed in 11.1 This seems like a pretty awful bug: SeedRandom[1]; RandomInteger[10, 10] Out: {1,4,0,7,0,0,8,6,0,4} While:...
10
votes
2answers
597 views

Unexpected behavior when using SortBy as a dataset filter

I am still experimenting with datasets. Recently I tried to query the Titanic example to get info on the oldest passengers. Here is what I tried ...
10
votes
2answers
404 views

Why can I not replace a Dataset using a rule?

Simple replacement of a Dataset object does not seem to follow the expected behavior... Dataset[{}] /. _Dataset->1 Returns the empty Dataset not 1 as I would ...
10
votes
2answers
261 views

Does Dataset round? [duplicate]

I'm importing the following Excel file: My code and its results are: ...
10
votes
2answers
180 views

A subtlety in Query

Let us consider a simple Dataset: ds=Dataset[{<|"a"->12,"b"->4|>,<|"a"->7,"b"->8|>,<|"b"->3|>}] Column "b" is easy to find: <...
10
votes
1answer
1k views

Converting structured data to a dataset [duplicate]

A Dataset seems perfect to represent structured data. Has anyone already thought about how to convert as efficiently/elegantly as possible an XML or JSON file to a Dataset in Mathematica 10 ?
10
votes
1answer
200 views

Dataset function and related gadgets not ideal for large datasets

There's a performance bottleneck when using these types of functions. If I had a large CSV file and were to import it as a regular 'unstructured' CSV, I save almost half the time importing the file, ...
10
votes
2answers
199 views

Using Merge[Identity] and Transpose on a Dataset

While experimenting with this answer, I ran across the following: ...
10
votes
1answer
192 views

Problems with Dataset's querying on a fresh Kernel

Bug introduced in V10.4.1 or earlier and fixed in V11.3 A support case with the identification [CASE:3710757] [...] I checked and reproduced the issue at my end and have reported the issue to ...
10
votes
1answer
166 views

Using MapIndexed[List] on a dataset fails — why?

The following works: MapIndexed[List]@{1, 2, 3} (* {{1, {1}}, {2, {2}}, {3, {3}}} *) This works too: ...
10
votes
0answers
773 views

Dataset in Mathematica 10.0.1

Wolfram pages "Mathematica Quick Revision History" in Mathematica 10.0.1 release says: "New operators and compositions of operators enabled for associations functions such as Dataset and Query" ...
9
votes
4answers
207 views

Dataset query for keys of upper levels

Say I have a Dataset like this: dat=Dataset[<|"a" -> 2, "b" -> "y", "c" -> {2, 3}|>] I want to get the keys of upper levels as elements in the ...
9
votes
3answers
903 views

How to pass options to ListPlot when using a DataSet?

Still learning the new DataSet feature, I would like to create a ListPlot where the color of the points is to be determined by a ...
9
votes
3answers
2k views

Converting datasets to nested lists

Imagine the Shipments relation: ...
9
votes
3answers
289 views

Problem constructing Association tables with some operators

fixed in 10.0.2 First, set up the example Dataset from the documentation: titanic = ExampleData[{"Dataset", "Titanic"}]; Why ...
9
votes
1answer
286 views

Why is Dataset upset by division by zero?

Consider this example: ...
9
votes
3answers
264 views

Applying functions to some columns in a Dataset and keeping only those

Take the following example Dataset: data = Table[Association["a" -> i, "b" -> i^2, "c" -> i^3], {i, 4}] // Dataset ...
9
votes
2answers
258 views

How to insert zero counts uniformly with Dataset Counts query

Missing[] is generated when an association does not contain the specified key. ...
9
votes
1answer
193 views

Dataset - append row with a value which depends on two other rows

I have go through the forum but did not see this question. Hope I don't miss it. I want to add a new colum "C" to a dataset with existing columns named "A" and "B". The value of "C" shall be the ...
9
votes
1answer
187 views

How to FoldList over Dataset column without destroying/rebuilding Dataset?

I have a Dataset in the following form but with many more rows and a few other columns. ...
9
votes
1answer
181 views

Operator syntax for Merge-like Query

Bug introduced in 10.0.0 and fixed in 10.0.2 Is it possible to take advantage of Dataset's Query operator syntax to Merge parallel queries thereby avoiding Function's that are otherwise required? ...
9
votes
1answer
407 views

How do I 'Join' two Structured Datasets?

Consider the following Datasets dsCompetitions with fields {"id","name","eventTypeId"} dsEventTypes with fields {"id", "name"} I want to join the datasets as ...
9
votes
1answer
111 views

Export[…, “tsv”] drops minus signs from Dataset [horrific bug?]

Toy example: First, create a Dataset: ...
9
votes
1answer
280 views

Did 10.0.2 break some forms for named slots?

Bug introduced in 10.0.2 and fixed in 10.2.0 Since 10.0.2 some code that previously worked using named slots fail in very particular circumstances. Attchd screenshot shows the issue w/ a particular ...
9
votes
1answer
119 views

How can weighted averages be calculated for a Dataset summarized with GroupBy

I would like to compute market-value weighted stock prices grouped by sector from a Dataset but can't figure out how to apply ...
9
votes
1answer
2k views

Converting a Table to a Dataset

I feel like this is a simple task, and there may be a built in method to doing this (but I can't find it if there is). I would like to be able to take a table and convert it into a structured Dataset. ...
9
votes
1answer
193 views

Applying functions to irregularly distributed Keys in Association lists

This Q is related to this one, but now refer to the situation as irregularly distributed Keys in lists of Associations: ...
9
votes
1answer
103 views

JoinAcross datasets based on Equal and not on SameQ

JoinAcross will join keys if values are the same (SameQ==True) but not if there are equal. For example these two associations are not joined : ...
9
votes
1answer
563 views

Group dataset on two keys (GroupBy)

I would like to group the elements of row c of the following dataset according to the values in the columns a and b. That is Go from here To here And then I would like to print this on a grid, ...