# How to transform the data in the following way?

Consider the following data:

datatest =
Join[RandomReal[{2, 3}, {5, 2}],
Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i,
1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];


The description of its columns is given by

meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};


I would like to multiply the columns containing "val_" by 0.01, the columns containing "mom_" by 0.001, and leave unchanged the other columns. In reality, the true data contains hundreds of such columns and thousands of rows, so it would be good to have an automatic way of doing this using patterns. I may define the transformation rule, say, If[StringContainsQ[str,"val_"]==True,0.01], but I do not know how to apply this rule to the list in the form transformation&/@datatest. Could you please help me?

• {0.01, 0.01, 1, 1, 0.001} # & /@ datatest ? or {#1/100, #2/100, #3, #4, #5/1000} & @@@ datatest
– Syed
Oct 10, 2023 at 12:37
• @Syed : this rule would be very inconvenient if datatest is made of hundreds of columns. I need a pattern of the transformation. Oct 10, 2023 at 12:41
• Thanks for the clarification. I see what you are trying to do now.
– Syed
Oct 10, 2023 at 12:45
• f = If[StringContainsQ[#, "val_"], 0.01, If[StringContainsQ[#, "mom_"], 0.001, 1]] & and (f /@ meanings) # & /@ datatest ?
– Syed
Oct 10, 2023 at 14:12

## 4 Answers

With the data:

datatest =
Join[RandomReal[{2, 3}, {5, 2}],
Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i,
1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];
meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};


We first need to determine in which columns "val_" and "mom_" appear:

posval = Position[meanings, s_String /; StringMatchQ[s, "val_" ~~ __]];
posmom = Position[meanings, s_String /; StringMatchQ[s, "mom_" ~~ __]];


Now we can multiply the corresponding columns with the matching factors:

datatest[[All, Flatten[posval]]] *= 0.01;
datatest[[All, Flatten[posmom]]] *= 0.001;


This results in:

datatest // MatrixForm


Taking the idea from @Syed's comment:

datatest = Join[RandomReal[{2, 3}, {5, 2}],
Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2]
meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};

transf = Table[Which[
StringContainsQ[col, "val_"], 0.01,
StringContainsQ[col, "mom_"], 0.001,
True, 1
], {col, meanings}]
result = transf # & /@ datatest

result // MatrixForm


OP's example input:

datatest =
Join[RandomReal[{2, 3}, {5, 2}],
Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i,
1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];
meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};


TLDR: You can use Dataset to construct a dataset DatatestSet and then specify a list of transformations for the columns you want to transform and then use DatatestSet[All,transformations]

Note: I am not an expert in datasets, I just noticed that the solution below seems to work and that the syntax seems to be coherent with the documentation on Dataset.

Your datatest list and meanings seem similar to datasets in Mathematica so I wanted to add a solution that uses Dataset.

First, consider constructing the dataset

DatatestSet=Dataset[AssociationThread[meanings,#]&/@datatest];


Then, specify the list of columns that will be transformed

valColumns=Select[StringMatchQ["val_"~~___]]@meanings;

momColumns=Select[StringMatchQ["mom_"~~___]]@meanings;


Define a multiplication function for convenience (optional)

mulFun[p_][x_]:=p*x;


Define the list of transformations that will be performed on the specified columns.

transformations=Join[Thread[valColumns->mulFun[0.01]],Thread[momColumns->mulFun[0.001]]]


Displaying the original dataset in order to compare with the transformed dataset below

The transformed dataset :

DatatestSet[All,transformations]


I would like to show a solution similar to userrand's:

datatest =
Join[RandomReal[{2, 3}, {5, 2}],
Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];

meanings = {"val_x", "val_y", "Something", "Another", "mom_x"};

asc = AssociationThread[meanings, #] & /@ datatest;

pv = Flatten @ Position[meanings, _String?(StringMatchQ[#, "val*"] &)];
pm = Flatten @ Position[meanings, _String?(StringMatchQ[#, "mom*"] &)];

res =
Query[All,
Join[
Thread[pv -> (# * 0.01 &)],
Thread[pm -> (# * 0.001 &)]]] @ asc;

res // Dataset


Or, alternatively,

Values[res] // MatrixForm