This query segments a time-series and filters out dropout frames.
Can it be simplified - ie, made more readable?
segmentationQuery =
Query[SplitBy[#, NumberQ[#"POR_X"] &] & ] /*
Query[Merge[Join] /* Key[True], GroupBy[NumberQ[#"POR_X"] &]]
(Note there's no SplitBy
operator form in 10.0.0. hence using functional form.)
The Slot
"POR_X" is used twice: in SplitBy
and GroupBy
- this is not pleasing.
Using only GroupBy
in the top-most subquery yields only 2 groups of data (signal and blank) but all the numeric frames are merged (not the intended behavior).
This fragment, extracted from eye-tracking dataset, illustrates droputs of variable duration and number of rows:
data = {<|"Time" -> 0.033452, "POR_X" -> 613.76,
"POR_Y" -> 424.74|>, <|"Time" -> 0.066747, "POR_X" -> 611.84,
"POR_Y" -> 424.84|>, <|"Time" -> 0.100124, "POR_X" -> 610.53,
"POR_Y" -> 420.11|>, <|"Time" -> 0.133515, "POR_X" -> 605.95,
"POR_Y" -> 421.18|>, <|"Time" -> 2.01096, "POR_X" -> "",
"POR_Y" -> ""|>, <|"Time" -> 2.04425, "POR_X" -> "",
"POR_Y" -> ""|>, <|"Time" -> 2.07772, "POR_X" -> "",
"POR_Y" -> ""|>, <|"Time" -> 2.24451, "POR_X" -> 412.47,
"POR_Y" -> 805.92|>, <|"Time" -> 2.27792, "POR_X" -> 518.11,
"POR_Y" -> 652.38|>} //Dataset;
data[segmentationQuery][All, Dataset] // Normal