This is an offshoot from my previous question about dealing with large datasets in Mathematica. I think it might provide more value to more people as a separate question.
I'd like to know how people are dealing with data wrangling of messy data in Mathematica.
I'm dealing with data that mostly has the same columns, but can be shifted about a bit, sometimes have extra columns added. Mathematica, doesn't seem to have a record or structure type, only lists and matrices. How are people dealing with this? I've taken to loading my data into an SQL database, but even this is suboptimal - for my classifier I have to make sure I select the same fields, in the same order, in the same offset in a list. Difficult as sometimes my data is in similar, but slightly different forms.
Example. Notice there are additions, transpositions, and a missing field in the second. This is the same data, but in a different form.
One dataset: col1, col2, col3, col4, col5.
another: cola, colb, col2, col3, col5, col1.
EDIT: For the sort of work I'm doing the trouble I run into is matching up the columns/fields from source1 to the same columns in source2. In different sources of data the columns move about, some are some deletions/additions.
Mathematica works mostly with lists, not records or structures so I can't say somefunc[samp1.tempdata, samp2.tempdata]. Records, and the ability to iterate over the members of a record would honestly be a godsend.
