3
$\begingroup$

I have a dataset that looks like this and I sorted them by ts since that corresponds to the values of when the data was taken. After reading the primer on associations and keys, and this question, I wanted to operate within the data. So for example, I would like to divide the values of "lamb" by "integration_time" from the same ts association and do the same for "filter" on "spectrum", and then ideally plot them all together in the same plot.

I tried to break it down and see if I can even access the nested "parameters" with:

MapAt[f, a1, {All, "parameters", "A"}]

a1[All, {"parameters" -> {"A" -> f}}]

But it returned error,"Part[All, "parameters", "filter"} of Association [<<228>>] does not exist."

I tried again with this code from a related question:

filter = Select[#["parameters", "A"] &];
Query[A]@a1

I tried to specify a range, but same problem. Can someone tell me what am I doing wrong here exactly? Also, when I try to KeyDrop "time" for example, it wouldn't even work. I would also appreciate any more efficient approaches instead of (sorting data>perform operation> update dataset>plot).

Sample data, for First[a1]:

{<|"time" -> "202301222_202041", 
  "parameters" -> <|"A" -> 40., 
    "B" -> 8., 
    "C" -> 0|>|>, <|"time" -> "202301222_202041", 
  "X" -> {121.076, 121.085, 121.094, 121.103, 121.112, 121.121, 121.13, 121.139, 121.148, 121.157, 121.167, 121.176, 121.185, 121.194, 121.203, 121.212, 121.221, 121.23, 121.239, 121.248, 121.257, 121.266, 121.275, 121.284, 121.293, 121.302, 121.311, 121.32, 121.329, 121.338, 121.347, 121.356, 121.365, 121.374, 121.383, 121.392, 121.401, 121.41, 121.419, 121.428, 121.437, 121.446, 121.455, 121.464, 121.473, 121.482, 121.491, 121.499, 121.508, 121.517, 121.526, 121.535, 121.544, 121.553, 121.562}, 
  "Y" -> {121.076, 121.085, 121.094, 121.103, 121.112, 121.121, 121.13, 121.139, 121.148, 121.157, 121.167, 121.176, 121.185, 121.194, 121.203, 121.212, 121.221, 121.23, 121.239, 121.248, 121.257, 121.266, 121.275, 121.284, 121.293, 121.302, 121.311, 121.32, 121.329, 121.338, 121.347, 121.356, 121.365, 121.374, 121.383, 121.392, 121.401, 121.41, 121.419, 121.428, 121.437, 121.446, 121.455, 121.464, 121.473, 121.482, 121.491, 121.499, 121.508, 121.517, 121.526, 121.535, 121.544, 121.553, 121.562}|>}
$\endgroup$
9
  • $\begingroup$ I see that you included an attachment, but could you make your question self-contained by including at least a few representative entries from your data? $\endgroup$
    – MarcoB
    Aug 5, 2023 at 13:23
  • $\begingroup$ Thank you for your reply, updated. $\endgroup$ Aug 5, 2023 at 13:38
  • $\begingroup$ @atomic-muclei The "sample data" is not valid WL. $\endgroup$
    – Alan
    Aug 5, 2023 at 15:11
  • $\begingroup$ @Alan Thank you, just updated. $\endgroup$ Aug 5, 2023 at 15:20
  • $\begingroup$ @atomic-muclei So calling your example object lst, do you want something like <|"new" -> #"lamb"/#"spectrum"|> & /@ Join @@@ GatherBy[lst, #"ts" &]? $\endgroup$
    – Alan
    Aug 5, 2023 at 15:42

2 Answers 2

5
$\begingroup$

Let lst be the example list in the question. The

unify = Join @@@ GatherBy[lst, #"time" &]
result = <|"new" -> #"X"/(1 + #"parameters"["A"])|> & /@ unify
$\endgroup$
3
  • $\begingroup$ Thank you so much! I just tried it and it works perfectly. May I ask how would I identify them later when I plot them all together? So for example will I be able to remove a certain datapoint (a ts association)? $\endgroup$ Aug 5, 2023 at 17:23
  • 1
    $\begingroup$ @atomic-muclei You can keep ts as well by changing the function to <|"ts" -> #"ts", "new" -> #"lamb"/(1 + #"parameters"["nd_filter"])|> &. Is that what you mean? $\endgroup$
    – Alan
    Aug 5, 2023 at 18:27
  • 1
    $\begingroup$ Hi @Alan, sorry but I just changed the variable names so the question can be clearer. nd_filter -> A, ts -> time, lamb -> X. $\endgroup$ Aug 7, 2023 at 13:04
4
$\begingroup$
new = 
  With[{as = <|lst|>}, 
    {<|"new" -> as["X"]/(1 + as["parameters", "A"])|>}];

new == result

(* True *)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.