# Applying a function to a Dataset column conditional on another slot's value

How to apply a function to a column conditional on another slot's value?

The Dataset documentation show only how to apply a function to named slots independently. The {key-> foo} operator can only access values associated to key

Given input dataset:

    data = <|"data1" -> <|"PLATFORM" -> "GoogleDrive", "LOCATION" -> "path1"|>,
"data2" -> <|"PLATFORM" -> "Dropbox", "LOCATION" -> "path2"|>,
"data3" -> <|"PLATFORM" -> "GoogleDrive", "LOCATION" -> "path3"|>|> //Dataset;


Want to apply foo to "LOCATION" if "PLATFORM"=="Dropbox" is True, yielding the modified data:

This workaround solution using GroupBy and Merge is ugly, rearranges data's rows (undesirable), and forces the user to define Identity functions:

platformFunctions = <| "GoogleDrive" -> Identity, "Dropbox" -> foo|> ;

{data [GroupBy[Key@"PLATFORM"]] // Normal, platformFunctions} //
Merge[Normal[
Dataset[First[#]][All, {"LOCATION" -> Last[#]}]] &] //
Values  // Normal // Flatten // Association // Dataset


Is there more convenient method?

Here is one way:

data[All, MapAt[foo, #, #PLATFORM /. {"Dropbox" -> "LOCATION", _ -> {}}] &]


Or, if one prefers If:

data[All, MapAt[foo, #, If[#PLATFORM == "Dropbox", "LOCATION", {}]] &]


Edit

The preceding expressions use MapAt with a fixed function (foo) and a conditional set of parts (either "LOCATION" or an empty list). As @alancalvitti notes in the comments, MapAt can also be used with a conditional function (either foo or Identity) against a fixed part ("LOCATION"):

data[All, MapAt[If[#PLATFORM == "Dropbox", foo, Identity], #, "LOCATION"] &]


This is essentially the same as @LeonidShifrin's solution.

Edit 2 - Dynamic Query

We could also apply a dynamic query operator:

data[All, Query[If[#PLATFORM == "Dropbox", {"LOCATION" -> foo}, All]]@# &]

• Thanks WReach. This variation also works: data[All, MapAt[If[#PLATFORM == "Dropbox", foo, Identity], #, "LOCATION"] &] . As of 10.0.1 MapAt operator form seems broken though. – alancalvitti Sep 17 '14 at 17:54
• @alancalvitti I added your variation to my answer, thanks. I haven't got 10.0.1 yet, but I am quite disappointed if it has introduced new bugs into MapAt. – WReach Sep 17 '14 at 18:18
• Other bugs besides op form? Broken in 10.0.0. too. – alancalvitti Sep 17 '14 at 18:20
• @alancalvitti Okay, if we are talking about the known 10.0.0 bugs then I'm still disappointed but not quite as much :) – WReach Sep 17 '14 at 18:22
• I wonder how they keep record of the increasing number of functions as various issues emerge. A giant screen w/ all Mathematica symbols layed out & color mapped... or scroll like stock exchange. – alancalvitti Sep 17 '14 at 18:30

Perhaps, there is a better one, but here is one that comes to mind:

ClearAll[transform];
transform["Dropbox"] := foo;
transform[_] := Identity;


and then

data[All, With[{tr = transform[#PLATFORM]}, MapAt[tr, #, {Key["LOCATION"]}]] &]

• Well I didn't want to bloat the example but the actual data has many columns. Can you modify with something similar to this? (Append[ Complement[data[1, Keys] // Normal, {"LOCATION"}] // Normal // AssociationMap[Identity], {"LOCATION" -> (If[#PLATFORM === "Dropbox", foo[#LOCATION], #LOCATION]&)}] – alancalvitti Sep 16 '14 at 0:07
• @alancalvitti See my edit for a better approach. – Leonid Shifrin Sep 16 '14 at 0:16

First, we define a new operator:

ApplyIf[f_, g_, x_] := If[TrueQ[f[x]], g[x], x];
ApplyIf[f_, g_][x_] := ApplyIf[f, g, x];


Then we do the query:

data[All, ApplyIf[#PLATFORM == "Dropbox"&, MapAt[foo, "LOCATION"]]]


I think it's pretty clear -- we're using composition to nicely separate the condition from the action.

ApplyIf should probably exist in the core language. The particular compound Map[ApplyIf[test, f]] is extremely common, I find.

Here is another relatively straightforward approach:

 data[All, <|#, "LOCATION" -> If[#PLATFORM == "Dropbox", foo[#LOCATION], #LOCATION]|> &]


Or using Mr.Wizard's suggestion:

data[All, <|#, "LOCATION" -> If[#PLATFORM == "Dropbox", foo, # &][#LOCATION]|> &]

• Nice. I believe it could also be written: data[All, <|#, "LOCATION" -> If[#PLATFORM == "Dropbox", foo, # &][#LOCATION]|> &] – Mr.Wizard Sep 16 '14 at 7:12
• @RunnyKine, the difference is that this retains the existing columns and presumably their order too. – Simplex Sep 16 '14 at 11:39
• @Simplex, that's great. That's the form I'll use - wish I'd waited to accept. – alancalvitti Sep 16 '14 at 16:19
• To Taliesin or others involved with Dataset dev, note despite the simplicity of Simplex's query, "LOCATION" appears 3 times. Can you implement a more compact operator? – alancalvitti Sep 16 '14 at 16:23
• @alancalvitti, conceptually, can one do better than having one "LOCATION" for where the result should be stored and another "LOCATION" for calculating the value? Mr.Wizard's approach works for me in that the final value with your function is "base/path2" - and it only uses "LOCATION" twice. Are you getting or expecting a different result? – Simplex Sep 17 '14 at 6:14

A not nice alternative could be

data[GroupBy@Key@"PLATFORM"][
{"Dropbox" -> Query[All, { "LOCATION" -> foo}]}][
Apply[Join]]


Hopefully soon we'll be able to use data on the lhs, and/or have an advanced MapAt.

• Interesting use of Apply, but prefer WReach and my variation using MapAt. But no op form in 10.0.1, is that what you mean by advanced? – alancalvitti Sep 17 '14 at 17:57
• Yes. I meant I hoped something like this could eventually work data[MapAt[foo, {Select[#PLATFORM == "Dropbox" &], "LOCATION"}]] – Rojo Sep 17 '14 at 23:04
• But without any changes to MapAt I would hope this could work toodata[{Select[#PLATFORM == "Dropbox" &] -> {"LOCATION" -> foo}}] – Rojo Sep 17 '14 at 23:06