1
$\begingroup$

I have a Dataset that requires some cleaning. Let's consider this data:

data = Dataset[{<|"ID" -> "1", "Name" -> "Bob", 
    "Hobbies" -> "cycling, soccer, cooking"|>, <|"ID" -> "2", 
    "Name" -> "Bill", "Hobbies" -> "television, gaming, tech"|>, <|
    "ID" -> "3", "Name" -> "Bob", 
    "Hobbies" -> "reading, writing, juggling"|>}]

Most of it is relatively easy such as turning strings into integers. However, some of columns require multiple functions that may use multiple arguments. If I want to change the ID field into an integer, I can simply write this:

reviseddata = data[All, {"ID" -> ToExpression}]

However, this won't work when I need to use multiple arguments for a function such as StringTrim.

I originally tried this, but I get a failure:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#, ","]] &}]

This code will perform the functions correctly, but I ended up with a list of rules instead of the full dataset:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]] &}]

I tried following the advice in this thread, but still ran into issues.

I know I can pull the column, do the functions on all elements, then replace the column, but I was hoping to work within this same syntax for simplicity since I am actually dealing with several similarly formatted datasets.

I am sure it is something obvious that I have missed. Thanks in advance.

$\endgroup$
2
  • 1
    $\begingroup$ Have you tried: reviseddata = data[All, <|#, "Hobbies" -> StringTrim[StringSplit[#Hobbies, ","]]|> &] ? I think this may do what you are hoping. $\endgroup$
    – Mark R
    Jun 27, 2019 at 20:13
  • $\begingroup$ @MarkR, That is perfect. Syntax changes a bit, but I can run all changes in one line of code, which is what I was looking for. reviseddata = data[All, <|#, {"ID" -> ToString[#"ID"], "Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]]}|> &] $\endgroup$
    – kickert
    Jun 27, 2019 at 20:32

2 Answers 2

3
$\begingroup$

I think you're just discoverring that & has very low precedence. Consider:

a -> b& //FullForm

Function[Rule[a,b]]

The precedence of & is lower than the precedence of "->". So, to fix things, include parentheses:

reviseddata = data[All, {"Hobbies" -> (StringTrim[StringSplit[#, ","]] &)}]

enter image description here

$\endgroup$
1
  • $\begingroup$ Thanks @Carl Woll, I could have sworn I tried that syntax, but apparently not. $\endgroup$
    – kickert
    Jun 28, 2019 at 0:21
1
$\begingroup$

Having used Carl's suggestion in my initial work, I eventually found that creating a defined function was the most efficient option (especially if you are going to use it more than once).

stringToList[string_]:=StringTrim[StringSplit[string]]

reviseddata = 
 data[All, {"Hobbies" -> stringToList}]

While Carl's answer certainly answers my question directly, for future readers, I would recommend the defined function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.