Efficient Data Entry / Manipulation with Many Datasets

Question

I currently have a simple task of graphing many different datasets (26 here, with many different instances where I'll need to do, say, 26 different ones from the same template). I'm trying to minimize the amount of "name changing" I'll have to do manually, as the treatment of each dataset is virtually identical aside from the names and (obviously) individual data points being graphed.

For example, I have a large table (specifically imported using ExcelRead with ExcelLink) with:

data = {{DataSetName1, m1, m2, m3,m4},{DataSetName2, m1, m2, m3, m4},...}

etc. for all 26 datasets and their measurements 1 - 4 as a function of time.

Next I have to extract each datasets information individually using

time=[5, 10, 20, 1440] for the time intervals my data is taken at,

DataSet1raw = Rest[Part[data, 2, 1;;5] and then DataSet1 = Transpose[{time, DataSet1raw}] DataSetList = {"DataSet1Name","DataSet2Name","...", "DataSet26Name"}

to finally yield the time-series {{t1, m1},{t2, m2}, ...} form and a list of the names of each set.

Combining the list of time-series (& names respectively) to graph in thirds for ease of viewing,

AllSets = {DataSet1, DataSet2, ..., DataSet26} FirstCollection = Take[AllSets, 1;;9] SecondCollection = Take[AllSets, 10;;18] FinalCollection = Take[AllSets, 19;;26]

After doing this individually for 26 numbered data sets (such as NB1,NB2,NB3, etc., all of which contain name-specific components previously used), it is finally in the form where I can graph the datasets easily using:

ListLogLinearPlot[FirstCollection, Legending, etc.]

So, my question is a general one of how to optimize this process. Currently I'm leaning towards generalizing the terms like I've done above, and trying to use a simple process of "Specific name = generic name," but suspect my lack of general organizational Mathematica familiarity (and coding in general) has me complicating much of this process.

Thank you for taking the time to respond!

Update:

After investigating further as to how to possibly simplify the process, I've at least found in the documentation a means of possibly doing the renaming of each drug iteration (26 times here) by a series of ToString and ToExpression functions, in case anyone else is facing a similar dilemma.

Answer 1

I am not certain where your trouble lies, but since you mention ToString and ToExpression you're surely making this harder than it needs to be. Let me show you a couple of methods to format your data in the way I think you intend. Sample data:

dat = {{"cat", 8, 18, 9, 1}, {"dog", 19, 15, 15, 7}, {"fox", 7, 10, 9, 20}, 
       {"bird", 17, 10, 1, 5}, {"lynx", 19, 9, 7, 1}, {"snail", 3, 12, 17, 2}};

time = {5, 10, 20, 1440};

We may write a function such as:

f[time_][{name_, val___}] := Prepend[{time, {val}}\[Transpose], name]

This uses a SubValues pattern to simplify application while preventing implicit reliance on global values, by which I mean using time directly in the function rather than as a parameter. Also, this function does no argument testing, only Destructuring. To be robust you may wish to check that time and {val} have the same Length, etc.

Now:

f[time] /@ dat

{{"cat", {5, 8}, {10, 18}, {20, 9}, {1440, 1}},
 {"dog", {5, 19}, {10, 15}, {20, 15}, {1440, 7}},
 {"fox", {5, 7}, {10, 10}, {20, 9}, {1440, 20}},
 {"bird", {5, 17}, {10, 10}, {20, 1}, {1440, 5}},
 {"lynx", {5, 19}, {10, 9}, {20, 7}, {1440, 1}},
 {"snail", {5, 3}, {10, 12}, {20, 17}, {1440, 2}}}

Alternatively you could use an assignment on Part, either on a copy of dat or on dat itself for in-place modification.

dat2 = dat;
dat2[[All, 2 ;; 5]] = {time, #}\[Transpose] & /@ dat[[All, 2 ;; 5]];

(* dat2 will be the same as the output above *)

For splitting your data into thirds I recommend using Partition, e.g.:

thirds[lst_List] := Partition[lst, #, #, 1, {}] & @ Ceiling[Length@lst/3]

Range@26 // thirds

{{1, 2, 3, 4, 5, 6, 7, 8, 9},
 {10, 11, 12, 13, 14, 15, 16, 17, 18},
 {19, 20, 21, 22, 23, 24, 25, 26}}

If you need to assign these chunks to symbols you can use e.g.:

{one, two, three} = Range@26 // thirds;

two

{10, 11, 12, 13, 14, 15, 16, 17, 18}

In general however I recommend not using a separate symbol for each part, but instead using Part, or indexed objects, or replacement rules.

Answer 2

I'd recommend rethinking your data structure a bit. How about defining lists like:

data[[i]] = {{DataSetName1, m1, m2, m3,m4},{DataSetName2, m1, m2, m3, m4},...}

for i=1 to however many pieces of data you have. Then the corresponding

time[[i]] =[5, 10, 20, 1440]

and etc for all the different items. Now you can just index through the i, instead of renaming everything at each step.