Hot answers tagged

39

Official Statement In case some of you missed it: There was an official reply from one of the WRI devs recently in our chat Hi. This is Jose, from Wolfram. We are aware of an unacceptable slowdown in some Dataset expressions, due to a bad dynamic interaction with the summary boxes of some objects. Both the TimeSeries objects of the coronavirus datasets ...


26

Dataset was restructured in the 12.1 release in order to support expanded formatting options and interactivity such as hiding and sorting. As a result, some Dataset outputs showed a slowdown due to inefficiencies in the dynamic output structures they produce. Because the code of Dataset is automatically field upgradable, we have released an update to its ...


13

To obtain a tabular rendering for a dataset, all rows must have the same number of columns, with the same set of keys, in the same order. But in our case the last association has fewer elements than the rest and the keys are in different orders in each row. Assuming that $ds contains the dataset: $ds[Values /* (PadRight[#, Automatic, ""] &), ...


11

dr = DateObject /@ DateRange[{2020, 1, 1}, DatePlus[{2020, 1, 1}, Quantity[15, "Days"]]]; SeedRandom[1] data = Transpose[{RandomChoice[dr, 30], Round[ RandomReal[10, 30], .1], RandomChoice[{"app", "graphics", "research"}, 30]}]; ds = Dataset[AssociationThread[{"Date", "Hours", "Task"}, #] & /@ data]; ds[GroupBy["Date"], All, {"Hours", "Task"}]...


11

You can use KeyDrop: picture = Import["https://i.stack.imgur.com/p7ax0.jpg"]; table = Transpose[Partition[TextRecognize[picture, "Word"], 9] /. "Long_" -> "Long"]; ds = Dataset[AssociationThread[First @ table, #] & /@ Rest[table]] KeyDrop[{"FIPS", "Admin2"}] @ ds We also get the same ...


10

titanic[KeyValueMap[Append[#, "counts" -> #2] &]@*Counts, {"class", "sex"}]


9

data = {{1, 190}, {1, 200}, {1, 210}, {1, 180}, {1.5, 175}, {1.5, 200}, {1.5, 190}, {2, 100}, {2, 150}} ; data2 = KeyValueMap[List]@ GroupBy[data, First -> Last, Around] (*thanks: Lukas Lang*) ListPlot[data2] ListPlot[data2, PlotStyle -> Directive[Blue, PointSize[Large]], PlotRangePadding -> Scaled[.15], IntervalMarkersStyle -> <|...


9

Use StandardForm in the second argument of ToString: "some string " <> ToString[Tooltip["abbreviation", "expanded form"], StandardForm] <> " string continued"


8

I think what you need is more like this: dat=Dataset[MapThread[<|"Country" -> #1,"Population" -> #2|> &, {Countries, populations}]] The structure you are looking for is one association per row, with the keys equal to the column headings.


8

You need a good staring estimate for NonlinearModelFit. I would first look at your data using Fourier to get the frequencies. Thus ft = Fourier[data[[All, 2]], FourierParameters -> {-1, -1}]; nn = Length@data; freq = Table[(n - 1)/(nn 128.), {n, nn}]; ListLinePlot[Transpose[{freq, Abs[ft]}], PlotRange -> {{0, 0.002}, {0, 0.2}}] The plot shows a ...


8

SetOptions[Dataset, HeaderBackground -> Black, HeaderStyle -> White, ItemStyle -> Red]; ds = Dataset[{<|"a" -> 1, "b" -> 3|>, <|"a" -> 2, "b" -> 4|>}] ds[1] A cumbersome way to inject the options is to wrap dataset objects with Dataset[#, Options[Dataset]]&: {Dataset[#,Options[...


7

This is the simplest way I could think off: ds = Dataset[{ <|"Name" -> "Steven","Born" -> 1980, "Year" -> 2017, "Score" -> 115|>, <|"Name" -> "Steven", "Born" -> 1980, "Year" -> 2018, "Score" -> 230|>, <|"Name&...


7

data = {<|"letter" -> "a", "foo" -> 1, "bar" -> 2|>, <| "letter" -> "b", "foo" -> 3, "bar" -> 4|>, <|"letter" -> "c", "foo" -> 5, "bar" -> 6|>}; dataDS = Dataset[data]; ClearAll[f]; f[...


7

Select[Counts[data[[All, 1]]]@#[[1]] == 1 &] @ data Join @@ Select[Length@# == 1 &] @ GatherBy[data, First] Values @ GroupBy[data, First, If[Length @ # > 1, Nothing, #[[1]]] &] data[[Flatten @ Select[Length @ # == 1 &] @ GatherBy[Range @ Length @ data, data[[#, 1]] &]]] FixedPoint[SequenceReplace[#, b : {{a_, _}, ___, {a_, _}} :...


7

ds1 = {<|"date" -> 2008, "origine" -> "Terre-Neuve-et-Labrador", "destination" -> "Terre-Neuve-et-Labrador", "type" -> "produits de base", "valeur" -> 28452.|>, <| "date" -> 2009, "origine" -> "Terre-Neuve-et-...


7

Assuming that you intend to use a Dataset with named rows (and being aware that this means that each row must have a unique name), one way is: (*define the dataset*) data = Dataset[<|"Sue" -> <|"Day" -> "Tue", "Rate" -> 13|>, "Paul" -> <|"Day" -> "Wed", "...


6

Join[Dataset[Association /@ Thread["ID" -> Id]], ds, 2] Also MapIndexed[Prepend[#, "ID" -> #2[[1]]] &, ds]


6

Here is a way that produces output that conforms to the first option: val = asc // Query[ All , All , <| "x" -> Pick[#x, Positive[#y-#c]], KeyDrop[#, {"x", "y"}] |> & ]; val[[1, 1]] (* <|"x" -> {10, 5, 6, 2, 7, 3, 2, 7, 10, 7}, "c" -> 5, "d" -> "bide"|> *)


6

If the conditions are column independent, perhaps having separately defined function with multiple downvalue patterns would be the shortest syntax: dataset = ExampleData[{"Dataset","Titanic"}][[;;20]] f[v_, {p_ /; p > 10, "age", ___}, _] := If[v > 30, LightYellow] f["male", {p_ /; p > 10, "sex", ___}, _] := LightBlue Dataset[dataset, Background -&...


6

The exhibited expression does not work because the function argument a is an association. Query syntax is only recognized when it appears within the arguments of a Dataset or Query expression. We can use Query to fix up the function: data[{All -> Function[a, a // Query[{"a" -> {"B" -> (# - a["b", "X"] &)}}]]}] As an aside... the following ...


6

While kglr's approach is correct, there is a more succinct one: ds[GroupBy["Date"],Total,"Hours"] The reason it works is because ds[GroupBy["Date"],Total] produces the total for every column in the grouped result, and "Hours" selects just the one we need.


6

I'll bite. Let's play with the "Planets" dataset, as it offers a great example of nesting. planets = ExampleData[{"Dataset", "Planets"}] Descending operators Part operators such as All and "key" are descending operators, which means that the next operator will be applied at a nested level. Compare: planets[Length] (* 8 *) as we have 8 rows in our ...


6

Update Dataset doesn't seem to give the kind of control over formatting that you want, but Grid does, so I recommend using it for display. You can still use the dataset to store data. ALso, since the key "Earth planet and Moon" only function is to serve as a label, I would remove the 1st level association from you dataset—reverting it back to the ...


6

With a little help your attempt works, it's only necessary to limit omega (remember Nyquist–Shannon sampling theorem) ! Try fitFunction =NonlinearModelFit[data , {a*Sin[omega*x + phi] + cost, (2 Pi)/3000 2 > omega > 0 }, {a, omega, phi, cost}, x , Method -> NMinimize ] Show[{ Plot[ fitFunction[x ] , {x, 0, data[[-1, 1]]}], ListPlot[data]} ]


6

As suggested by @J.M.~~___, here is one way to do it: dataset[ListLinePlot, Callout[{#a, #b}, #c] &] Additional options can be added to ListLinePlot like this: dataset[ ListLinePlot[#, ScalingFunctions->"Reverse", PlotTheme->"Detailed"]& , Callout[{#a, #b}, #c] & ]


5

ds = { <|"assigned_ts" -> "2017-04-20 12:09:39", "closed_ts" -> "2017-04-20 13:13:01"|>, <|"assigned_ts" -> "2017-04-20 12:10:30", "closed_ts" -> "2017-04-20 12:28:29"|>} // Dataset ds[All, <|#, "difference_ts" -> DateDifference[DateObject[#"assigned_ts"], DateObject[#"closed_ts"], "Second"]|> &]


5

Import[myfile.ndjson,"JSON"] doesn't work because the file as a whole is not valid JSON. But each line is valid JSON and so we can treat the file as a stream and read it one line at a time. The simplest way to do this would be stream = OpenRead@"~/test.ndjson"; ClearAll[line]; result = {}; While[line =!= EndOfFile, line = ReadLine[...


5

Here is a way that generates the columns in the order that they occur in the original dataset: dsAll[ Select[#class==="apples"&] /* KeyUnion , <| "Name" -> #name, #name -> #value& /@ #params |>& ] If the exact order of the columns is important, an additional re-ordering stage can be added: dsAll[ Select[#class=...


5

Let data be your list of associations, then lengths = Length /@ SplitBy[data, #["Order Date"] &]; colors = Flatten@Riffle[ ConstantArray[White, #] & /@ lengths[[1 ;; ;; 2]], ConstantArray[LightBlue, #] & /@ lengths[[2 ;; ;; 2]] ]; Dataset[data, Background -> {colors}, MaxItems -> All]


5

You can use the package "DataReshape.m" as shown below. (It is in my ToDo list to submit the corresponding functions to Wolfram Function Repository very soon.) Load the package: Import["https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/DataReshape.m"] Here is the original dataset: dsData = Dataset[{<|"...


Only top voted, non community-wiki answers of a minimum length are eligible