Skip to main content
replaced http://mathematica.stackexchange.com/ with https://mathematica.stackexchange.com/
Source Link

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown herehere by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|> //Dataset

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown here by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|> //Dataset

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown here by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|> //Dataset
Tweeted twitter.com/#!/StackMma/status/554210528139431937
edited title
Link
alancalvitti
  • 15.2k
  • 3
  • 27
  • 93

Select query with (path) named slot lookups?

added 10 characters in body
Source Link
alancalvitti
  • 15.2k
  • 3
  • 27
  • 93

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown here by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|> //Dataset

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown here by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|>

Given this Dataset fragment (copy-paste friendly version at bottom)

enter image description here

Can this query be simplified by referencing named slots directly in the Select logic?

data[All, Values /* Thread][All, All, Apply@Join][All, Select[#VisitStartTime <= #"WEBCAM_TIME" <= #VisitEndTime &]]

enter image description here

Although functional, limitations include: (1) Creates intermediate Datasets; (2) threads VisitStartTime and VisitEndTime, which must then be dropped; (3) Values removes the Key "Morae/..." which must then be reinstered in each row; (4) overall complexity makes it difficult to modify to similar datasets.

An approach based on named slot syntax shown here by WReach looks applicable, but the naive version fails:

data[All, {"Morae/Manual/Emr/Coder1Data" -> 
   Select[#["Master", "VisitStartTime"] <=  #"WEBCAM_TIME" <= #[
       "Master", "VisitEndTime"] &]}]

enter image description here

Even though it is possible to reference the slots in the other branch like this:

data[All, {"Morae/Manual/Emr/Coder1Data" -> {#["Master", 
      "VisitStartTime"], #["Master", "VisitEndTime"]} &}]

enter image description here

Is it possible to combine these path-lookup methods?

INPUT DATA

data = <|"Q001" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 12, 9.4}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 10.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 13.1}], 
      "TASK" -> "F1"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 12, 27.9}], 
      "TASK" -> "F1"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 12, 15.}], 
     "VisitEndTime" -> TimeObject[{0, 38, 49.}]|>|>, 
 "Q002" -> <|"Morae/Manual/Emr/Coder1Data" -> {<|"WEBCAM_TIME" -> 
       TimeObject[{0, 0, 1.9}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 3.5}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 9.7}], 
      "TASK" -> "F25"|>, <|"WEBCAM_TIME" -> TimeObject[{0, 0, 10.8}], 
      "TASK" -> "F25"|>}, 
   "Master" -> <|"VisitStartTime" -> TimeObject[{0, 13, 0.}], 
     "VisitEndTime" -> TimeObject[{0, 22, 27.}]|>|>|> //Dataset
Source Link
alancalvitti
  • 15.2k
  • 3
  • 27
  • 93
Loading