Questions on using or implementing more or less common data structures such as linked lists, structs, trees, tries, heaps etc. in Mathematica.

learn more… | top users | synonyms

82
votes
1answer
4k views

File-backed lists/variables for handling large data

Background Currently I am working with some large data (most of it generated by Mathematica itself). I usually find it a hassle to do this. For example, I just exported a large amount of data to ...
81
votes
7answers
8k views

Struct equivalent in Mathematica?

I really miss having something like a struct in Mathematica. I know of (and regularly use) a couple of programming techniques which feel like a ...
73
votes
3answers
3k views

How to make use of Associations?

I have been curious about it for long. Now that Mathematica 10 arrives, I think it's time to ask the question: How the new Association data structure would be used ...
55
votes
2answers
2k views

What is a Mathematica packed array?

A simple sounding question with a few sub questions: What is the difference between unpacked vs packed array? Are packed arrays more space efficent, how much so? Are packed arrays more time ...
48
votes
2answers
10k views

How can I use Mathematica's graph functions to cheat at Boggle?

Boggle is a word game played with 16 dice and a 4x4 tray. This question is inspired by a Stack Overflow question about Boggle that I decided to solve using Mathematica. In addition to Mathematica, I ...
31
votes
2answers
1k views

How can I add a column into a existing Dataset?

I'm very excited about the brand new Dataset function. I have played with it in Wolfram Cloud, and haven't figured out how can I add a new column into an existing ...
28
votes
2answers
937 views

Can a Trie be implemented efficiently?

What is the most space and time efficient way to implement a Trie in Mathematica? Will it be practically faster than what is natively available in appropriate cases?
26
votes
4answers
2k views

How to create strong types?

In a program I'm writing, I create a list of lists, that looks something like this: {{100, {1,2,3,4,5}}, {105, {2,4,6,8}}, {42, {42,39,56}}} I then pass this ...
24
votes
3answers
841 views

Error changing Dataset using Part

bug present as of 10.1.0 bug present as of 10.0.2 In the new guide Computation With Structured Datasets we can find this part, on how to change a Dataset But if we create a ...
22
votes
3answers
901 views

Emulating R data frame getters with UpValues

What's the best way to emulate R's data frames functionality? This includes the ability to select rows and columns in a 2-dimensional table by the string identifiers positioned typically in the first ...
21
votes
5answers
652 views

How can a big table be treated as a database?

I often work with big tables that I want to treat kind of like a database. Here's an example table. ...
20
votes
2answers
973 views

How can one define a custom data object?

In Mathematica there are different objects like InterpolatingFunction or SparseArray? How can I define a custom data object with ...
20
votes
4answers
855 views

Efficient circular buffer?

I wish to create an efficient circular buffer. That is, I wish to keep a fixed length list while appending a new value and dropping the oldest, repeatedly. As we know lists in Mathematica are ...
19
votes
1answer
257 views

Make my data-structure atomic

Lately, and by lately I mean since version 7 or so, the number of atomic expressions in Mathematica constantly grew. In former times only the native types like integers and optimised arrays were ...
18
votes
1answer
588 views

Mouse motion heat map - live visualization and analysis

Mouse motion heat map is an great way to study spatial attention distribution, styles of movement, reaction, etc. I am trying to design a code that visualizes such heat map. The requirements: 1) It ...
16
votes
3answers
491 views

How to get complete Documentation Center graph of guide pages?

On the very last image below you can see a typical path of walking through Documentation Center guide pages. What is the best way to get the graph data and visualize the whole structure of these ...
16
votes
2answers
555 views

Efficient priority queues?

I'm trying to figure out how to transfer the concept of a priority queue to the functional world. Searches have turned up some implementations that use Append and other expense list copying ...
15
votes
1answer
152 views

Why do lookup times in Dataset and Association differ by orders of magnitude?

Consider the following three data structures: a list of rules, an association, and a dataset: ...
15
votes
1answer
536 views

Creating a R DataFrame like construct in Mathematica

I kept getting my data mixed up so I decided to see if I could construct a package adding a R DataFrame like construct to Mathematica. I managed to cobble something together that sort of works, but ...
14
votes
2answers
686 views

Question about designing a particular data structure

Background: I store polygons with a color assigned at the face level. In some occasions I want to use vertex colors but I want to store data efficiently, i.e. don't want to duplicate data to the ...
14
votes
1answer
549 views

Efficient code for tallying entries in very large lists

I am dealing with data sets containing tens of millions of (hashable) entries and simply using the Tally function to count the frequency of each unique list element ...
13
votes
4answers
397 views

Short syntax for accessing System`Utilities`HashTableAdd and System`Utilities`HashTableGet

I have a need for a struct analogue in Mathematica. After reading this post I came to the conclusion that System`Utilities`HashTable is one of the best options. But ...
13
votes
1answer
188 views

Why won't SparseArray let me store values with the head List?

When you attempt to define a SparseArray with elements that have the head List it complains: ...
12
votes
3answers
3k views

Excel Headers --> Mathematica

I have a large Excel .xls file. Miniature version shown below: (formatting should appear as two columns...) ...
12
votes
3answers
457 views

Using Memoization with a Mutable Object

In looking for a solution to this question, I ran across some old binary tree code by Daniel Lichtblau, reproduced below: ...
11
votes
3answers
3k views

Implementing a dictionary data structure

As one learns from a course on data structures, hash maps or dictionaries can be efficient when applied to appropriate tasks. I need a hash map in Mathematica and I've never found it. I'm scratching ...
10
votes
1answer
214 views

Is there a method to enumerate the keys/values of System`Utilities`HashTable

Lacking from the simple API of System`Utilities`HashTable is a method to enumerate the keys/values. ...
10
votes
1answer
275 views

Efficient way to solve equal sums $x_1^k+x_2^k+\dots+x_5^k=y_1^k+y_2^k+\dots+y_5^k$ with Mathematica?

I need to solve the system of equations, call it $S_1$, in the integers $$x_1x_2x_3x_4x_5 = y_1y_2y_3y_4y_5$$ $$x_1^k+x_2^k+\dots+x_5^k=y_1^k+y_2^k+\dots+y_5^k,\;\; k= 2,4,6$$ I used a very ...
9
votes
0answers
293 views

Learning Resources for Data Science in the Wolfram Language

I'm delivering an upcoming course on Data Science with the programming/analysis done predominantly in the Wolfram Language. I created the following table to leverage students' familiarity with Lists ...
8
votes
2answers
425 views

Converting hierarchies of rules to associations

The impossibility of MatchQ-ing inside and Association MatchQ-ing Associations (MMA 10) explains a problem I have. I would like ...
8
votes
2answers
426 views

How to get the list of defined values for an indexed variable?

I would like a table with rational indexes - thus it would be practical to use a dictionary, which, in Mathematica are implemented with the indexed variables. I would like to be able to do: ...
8
votes
2answers
268 views

Cases[data,Colon[key,_]] vs. Cases[data,key:_] toward XPath, XQuery

Exploring methods to index and search tree-structured key:value pairs via named-entity index (key paths) as opposed to Position-based indexing. Using Rule as ...
8
votes
2answers
237 views

Distribute and analyze large datasets

I need to analyze very large datasets, and they obviously don't fit in a computer memory. I need to produce some statistics on these datasets, and this brings me here: How can I make some statistical ...
8
votes
1answer
311 views

Is it possible to construct a function to make a poor man's Mathematica database?

I have a function that is set up to write to a database with 10-20 standard variables with 10 fields each, but the application is going to a place where database access is not available. Can ...
6
votes
1answer
264 views

Return a Part by Reference? (For accessing hierarchical / structured data)

Is there a way to return a part of a list by reference, for reading and writing? I want to do this so I can easily manipulate "rule-styled structured", i.e., hierarchical data that is loaded from a ...
6
votes
2answers
379 views
6
votes
1answer
151 views

Return struct from function (with DownValues)

In Mathematica there is a way to create struct-like data type: struct[myField]=value I want to return myStruct from a function: ...
5
votes
2answers
336 views

DownValue “associative array” to list? [duplicate]

I have a "sparse array" like this arr[1.5]=0.4, arr[3.5]=0.7, arr[7]=0.3 which is not really a sparse array since the keys are not all integers. This is like a ...
5
votes
1answer
184 views

Error in ARIMAProcess example

I am trying to compile the sample of ARIMAProcess of MMA here .It doesn't work.What is wrong?Could you please help?I am using exact same code. ...
5
votes
1answer
176 views

What is the fastest way to maintain a large set of expressions?

In my algorithm I need to maintain a set (an unordered list of distinct elements) of expressions supporting two operations: Test an expression for membership in the set Adding a new expression to ...
5
votes
1answer
260 views

Processing and storing a graph with a large number of vertices s.t. k-hop neighborhoods are efficiently retrievable

I have an unweighted, undirected graph $G$ with $(v_1, ..., v_N) \in V$ vertices, where each vertex in the lattice represents a string of some length ($\leq 200$ characters), and I need to construct ...
4
votes
2answers
266 views

Access pairs of vectors, rather than the matrices, in partitioned data structure at a levelspec

I have some old code to analyze correlation between 2 time series by looking at windows of different lengths shifting across the time series data. The code: gets 2 sets of time series data (in ...
4
votes
1answer
219 views

Accessing the Data of an RLink RDataFrame

I am trying to extract the data table itself from an RLink-generated RDataFrame: ...
4
votes
1answer
83 views

Filtering Query Nested Data (Dataset and Association)

I am trying to filter a nested Dataset. I have some data that looks like this (curated for this example): ...
4
votes
2answers
573 views

What is the Mathematica way of joining two tables?

Is there a special Mathematica way of joining tables of data together, so that I would get similar functionality as if I was running a database with sql? For example, I have one particular table ...
4
votes
1answer
95 views

Parsing string with nested information

I am trying to parse a string, but I have no idea how to solve the following problem ...
4
votes
2answers
128 views

How would I programmatically add a row to a Dataset?

I would like to use a Dataset to track the process of a lengthy calculation over a large volume of material. Dataset appeals to me because of the power of the queries and it also looks like it may ...
4
votes
1answer
208 views

What's the most intelligent way to store the information of a selfmade graph?

I have a very long list of element, say $a1,a2, a3$,..., and they are all connected in that each is the child of many parents. E.g. $a74$ is associated with $a2,a55,a71$, and also contains ...
3
votes
3answers
169 views

Breaking up combined data into several smaller data sets

I have one combined data with first number as the x-value and the remaining as different y-axis values (y1,y2,y3...). How do I break this combined list, data= (x,y1,y2,y3...) into smaller lists, such ...
3
votes
2answers
218 views

Mathematica data structures for analysis [closed]

I'm starting to learn Mathematica and one thing that has been bugging me is the apparent lack of a good structure for keeping data and metadata for analysis. Something along R's ...