I have a Dataset that requires some cleaning. Let's consider this data:

data = Dataset[{<|"ID" -> "1", "Name" -> "Bob", 
    "Hobbies" -> "cycling, soccer, cooking"|>, <|"ID" -> "2", 
    "Name" -> "Bill", "Hobbies" -> "television, gaming, tech"|>, <|
    "ID" -> "3", "Name" -> "Bob", 
    "Hobbies" -> "reading, writing, juggling"|>}]

Most of it is relatively easy such as turning strings into integers. However, some of columns require multiple functions that may use multiple arguments. If I want to change the ID field into an integer, I can simply write this:

reviseddata = data[All, {"ID" -> ToExpression}]

However, this won't work when I need to use multiple arguments for a function such as StringTrim.

I originally tried this, but I get a failure:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#, ","]] &}]

This code will perform the functions correctly, but I ended up with a list of rules instead of the full dataset:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]] &}]

I tried following the advice in this thread, but still ran into issues.

I know I can pull the column, do the functions on all elements, then replace the column, but I was hoping to work within this same syntax for simplicity since I am actually dealing with several similarly formatted datasets.

I am sure it is something obvious that I have missed. Thanks in advance.

  • 1
    $\begingroup$ Have you tried: reviseddata = data[All, <|#, "Hobbies" -> StringTrim[StringSplit[#Hobbies, ","]]|> &] ? I think this may do what you are hoping. $\endgroup$ – Mark R Jun 27 '19 at 20:13
  • $\begingroup$ @MarkR, That is perfect. Syntax changes a bit, but I can run all changes in one line of code, which is what I was looking for. reviseddata = data[All, <|#, {"ID" -> ToString[#"ID"], "Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]]}|> &] $\endgroup$ – kickert Jun 27 '19 at 20:32

I think you're just discoverring that & has very low precedence. Consider:

a -> b& //FullForm


The precedence of & is lower than the precedence of "->". So, to fix things, include parentheses:

reviseddata = data[All, {"Hobbies" -> (StringTrim[StringSplit[#, ","]] &)}]

enter image description here

  • $\begingroup$ Thanks @Carl Woll, I could have sworn I tried that syntax, but apparently not. $\endgroup$ – kickert Jun 28 '19 at 0:21

Having used Carl's suggestion in my initial work, I eventually found that creating a defined function was the most efficient option (especially if you are going to use it more than once).


reviseddata = 
 data[All, {"Hobbies" -> stringToList}]

While Carl's answer certainly answers my question directly, for future readers, I would recommend the defined function.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.