0
$\begingroup$

I have a Dataset that requires some cleaning. Let's consider this data:

data = Dataset[{<|"ID" -> "1", "Name" -> "Bob", 
    "Hobbies" -> "cycling, soccer, cooking"|>, <|"ID" -> "2", 
    "Name" -> "Bill", "Hobbies" -> "television, gaming, tech"|>, <|
    "ID" -> "3", "Name" -> "Bob", 
    "Hobbies" -> "reading, writing, juggling"|>}]

Most of it is relatively easy such as turning strings into integers. However, some of columns require multiple functions that may use multiple arguments. If I want to change the ID field into an integer, I can simply write this:

reviseddata = data[All, {"ID" -> ToExpression}]

However, this won't work when I need to use multiple arguments for a function such as StringTrim.

I originally tried this, but I get a failure:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#, ","]] &}]

This code will perform the functions correctly, but I ended up with a list of rules instead of the full dataset:

reviseddata = 
 data[All, {"Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]] &}]

I tried following the advice in this thread, but still ran into issues.

I know I can pull the column, do the functions on all elements, then replace the column, but I was hoping to work within this same syntax for simplicity since I am actually dealing with several similarly formatted datasets.

I am sure it is something obvious that I have missed. Thanks in advance.

$\endgroup$
  • 1
    $\begingroup$ Have you tried: reviseddata = data[All, <|#, "Hobbies" -> StringTrim[StringSplit[#Hobbies, ","]]|> &] ? I think this may do what you are hoping. $\endgroup$ – Mark R Jun 27 at 20:13
  • $\begingroup$ @MarkR, That is perfect. Syntax changes a bit, but I can run all changes in one line of code, which is what I was looking for. reviseddata = data[All, <|#, {"ID" -> ToString[#"ID"], "Hobbies" -> StringTrim[StringSplit[#"Hobbies", ","]]}|> &] $\endgroup$ – kickert Jun 27 at 20:32
2
$\begingroup$

I think you're just discoverring that & has very low precedence. Consider:

a -> b& //FullForm

Function[Rule[a,b]]

The precedence of & is lower than the precedence of "->". So, to fix things, include parentheses:

reviseddata = data[All, {"Hobbies" -> (StringTrim[StringSplit[#, ","]] &)}]

enter image description here

$\endgroup$
  • $\begingroup$ Thanks @Carl Woll, I could have sworn I tried that syntax, but apparently not. $\endgroup$ – kickert Jun 28 at 0:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.