2
$\begingroup$

Consider the following data:

datatest = 
  Join[RandomReal[{2, 3}, {5, 2}], 
   Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 
     1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];

The description of its columns is given by

meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};

I would like to multiply the columns containing "val_" by 0.01, the columns containing "mom_" by 0.001, and leave unchanged the other columns. In reality, the true data contains hundreds of such columns and thousands of rows, so it would be good to have an automatic way of doing this using patterns. I may define the transformation rule, say, If[StringContainsQ[str,"val_"]==True,0.01], but I do not know how to apply this rule to the list in the form transformation&/@datatest. Could you please help me?

$\endgroup$
4
  • 2
    $\begingroup$ {0.01, 0.01, 1, 1, 0.001} # & /@ datatest ? or {#1/100, #2/100, #3, #4, #5/1000} & @@@ datatest $\endgroup$
    – Syed
    Oct 10, 2023 at 12:37
  • $\begingroup$ @Syed : this rule would be very inconvenient if datatest is made of hundreds of columns. I need a pattern of the transformation. $\endgroup$ Oct 10, 2023 at 12:41
  • $\begingroup$ Thanks for the clarification. I see what you are trying to do now. $\endgroup$
    – Syed
    Oct 10, 2023 at 12:45
  • 1
    $\begingroup$ f = If[StringContainsQ[#, "val_"], 0.01, If[StringContainsQ[#, "mom_"], 0.001, 1]] & and (f /@ meanings) # & /@ datatest ? $\endgroup$
    – Syed
    Oct 10, 2023 at 14:12

4 Answers 4

4
$\begingroup$

With the data:

datatest = 
  Join[RandomReal[{2, 3}, {5, 2}], 
   Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 
     1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];
meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};

We first need to determine in which columns "val_" and "mom_" appear:

posval = Position[meanings, s_String /; StringMatchQ[s, "val_" ~~ __]];
posmom = Position[meanings, s_String /; StringMatchQ[s, "mom_" ~~ __]];

Now we can multiply the corresponding columns with the matching factors:

datatest[[All, Flatten[posval]]] *= 0.01;
datatest[[All, Flatten[posmom]]] *= 0.001;

This results in:

datatest // MatrixForm

enter image description here

$\endgroup$
3
$\begingroup$

Taking the idea from @Syed's comment:

datatest = Join[RandomReal[{2, 3}, {5, 2}], 
  Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2]
meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};
transf = Table[Which[
   StringContainsQ[col, "val_"], 0.01,
   StringContainsQ[col, "mom_"], 0.001,
   True, 1
   ], {col, meanings}]
result = transf # & /@ datatest

result // MatrixForm

enter image description here

$\endgroup$
3
$\begingroup$

OP's example input:

datatest = 
  Join[RandomReal[{2, 3}, {5, 2}], 
   Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 
     1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];
 meanings = {"val_x", "val_y", "Something", "Something", "mom_x"};

TLDR: You can use Dataset to construct a dataset DatatestSet and then specify a list of transformations for the columns you want to transform and then use DatatestSet[All,transformations]


Note: I am not an expert in datasets, I just noticed that the solution below seems to work and that the syntax seems to be coherent with the documentation on Dataset.


Your datatest list and meanings seem similar to datasets in Mathematica so I wanted to add a solution that uses Dataset.

First, consider constructing the dataset

DatatestSet=Dataset[AssociationThread[meanings,#]&/@datatest];

Then, specify the list of columns that will be transformed

valColumns=Select[StringMatchQ["val_"~~___]]@meanings;

momColumns=Select[StringMatchQ["mom_"~~___]]@meanings;

Define a multiplication function for convenience (optional)

mulFun[p_][x_]:=p*x;

Define the list of transformations that will be performed on the specified columns.

transformations=Join[Thread[valColumns->mulFun[0.01]],Thread[momColumns->mulFun[0.001]]]

Displaying the original dataset in order to compare with the transformed dataset below

input

The transformed dataset :

DatatestSet[All,transformations]

result

$\endgroup$
2
$\begingroup$

I would like to show a solution similar to userrand's:

datatest =
  Join[RandomReal[{2, 3}, {5, 2}], 
   Table[{"something" <> ToString[i], "another" <> ToString[i]}, {i, 1, 5, 1}], RandomReal[{0, 1}, {5, 1}], 2];

meanings = {"val_x", "val_y", "Something", "Another", "mom_x"};

asc = AssociationThread[meanings, #] & /@ datatest;

pv = Flatten @ Position[meanings, _String?(StringMatchQ[#, "val*"] &)];
pm = Flatten @ Position[meanings, _String?(StringMatchQ[#, "mom*"] &)];

res =
  Query[All,
    Join[
     Thread[pv -> (# * 0.01 &)],
     Thread[pm -> (# * 0.001 &)]]] @ asc;


res // Dataset

enter image description here

Or, alternatively,

Values[res] // MatrixForm

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.