Making a service connection for a local process

Question

I know I can customize my service connections and change how I can import data and all that.

I also know that a local process that I'd access via command line (e.g. git) acts a lot like a web API, just the call structure is a little different. So can I make a service connection to a local process like that?

Answer 1

Happily there's only really one complication here, but it's quickly taken care of.

This is that we can't pass argument lists as the URL in our raw import functions, they need to be proper lists otherwise there's a Throw[$Failed] that gets invoked. So we'll just make sure to pass our argument lists encoded via URLBuild and unspool with URLParse

Otherwise we'll build in pretty much the same way as we do here, with slight modification.

---

<ServiceName>Functions.m

First off we need to add our import functions to our <ServiceName>Functions package.

GitProcessArgumentsFormat

This will just reformat the passed URL and "Parameters" option so that we can pass options in a normal way

GitProcessArgumentsFormat =
 Function[
  With[{
    base =
     If[StringMatchQ[#, "`" ~~ __ ~~ "`"],
        Key[Replace[StringTrim[#, "`"], "*" -> All]],
        #
        ] & /@ Flatten@{#},
    kwargs =
     Replace[
      FilterRules[#2,
       Except[Alternatives @@ (First /@ Options[RunProcess])]
       ], {
       (k_ -> True) :>
        (k -> If[StringLength@k > 1, "--" <> k, "-" <> k]),
       (k_ -> s_) :>
        (k -> 
          If[StringLength@k > 1, "--" <> k <> "=", "-" <> k <> " "] <>

                      If[StringQ@s, s, ToString[s, InputForm]])
       },
      1]
    },
   With[{keys = Cases[base, Key[k_] :> k]},
    Flatten@{
      Replace[base, {
        Key[All] :> Values@kwargs,
        Key[k_] :> Lookup[kwargs, k]
        },
       1],
      FilterRules[kwargs,
       Except@Alternatives @@

        Flatten@Replace[keys, All -> Keys@kwargs, 1]
       ]
      }
    ]
   ]
  ]

GitProcessParseResponse

This parses the <| "StandardOutput" -> _, "StandardError"-> _ |> association we get from RunProcess and formats it appropriately

GitProcessParseResponse =
 Function[
  If[AssociationQ@#,
   If[StringLength@Lookup[#, "StandardError", ""] > 0,
    Message[ServiceExecute::apierr, Lookup[#, "StandardError", ""]]
    ];
   If[StringLength@Lookup[#, "StandardOutput", ""] > 0,
    Lookup[#, "StandardOutput", ""]
    ]
   ]
  ]

GitProcessBaseImport

This is the function that

GitProcessBaseImport =
 With[{
   getArgs = GitProcessArgumentsFormat,
   parse = GitProcessParseResponse
   },
  Function[
   With[{
     cmd = URLParse[#, "Path"],
     params = Lookup[{##2}, "Parameters"],
     },
    With[{e =
       RunProcess[
        getArgs[cmd, params],
        FilterRules[params,
         Alternatives @@ (First /@ Options[RunProcess])
         ]
        ]
      },
     If[AssociationQ@e,
      parse@e,
      $Failed
      ]
     ]
    ]
   ]
  ]

GitProcessImport

Then we have the function that will actually get called on the argument list:

GitProcessImport =
 Function[
  With[{
    dir =
     Replace[Lookup[Lookup[{##2}, "Parameters"], "repo"], {
       d_String?DirectoryQ :> d,
       d_String?(DirectoryQ@*DirectoryName) :> DirectoryName@d,
       _ :> Directory[]
       }],
    params =
     Replace[{##2}, {
       ("Parameters" -> r_) :>
        ("Parameters" -> FilterRules[r, Except["repo"]])
       },
      1],
    cur = Directory[]
    },
   SetDirectory[dir];
   (If[dir =!= cur, ResetDirectory[]]; #) &@
    CheckAbort[
     GitProcessBaseImport[#, Sequence @@ params],
     ResetDirectory[];
     ]
   ]
  ]

---

<ServiceName>.m

We then have very slight changes that we need to apply to the primary file.

The first one is that we'll add these two parameters to the base data:

{
  "ClientInfo" -> {},
  "URLFetchFun" -> GitProcessImport"
  }

The "ClientInfo" means we use no auth process (because local -- although we could imagine doing this via SSH in which case we'd want the auth info back). And the "URLFetchFun" is the function that we defined earlier.

Then every function we define will look like this:

{
 "URL" -> URLBuild[ {"git", "init"}],
 "Parameters" -> {"repo"},
 "RequiredParameters" -> {"repo"}
 }

Although for some we'll have a function for that "URL" parameter.

---

Testing

We'll then give it a test:

$so = ServiceConnect["Git"]

Then make some random dict and initialize it as a repo:

In[903]:= $dir = CreateDirectory[];

In[904]:= $so["Init", "repo" -> $dir]

Out[904]= "Initialized empty Git repository in \
/private/var/folders/kr/mjjwvjr91fz4zpqfyd0p7bmc000_8h/T/\
m000005840871/.git/
"

Then we'll add a file and track it:

In[906]:= Export[FileNameJoin@{$dir, "fil.m"}, "bleb"];
$so["Add", "files" -> All, "repo" -> $dir]

In[908]:= $so["GetStatus", "repo" -> $dir]

Out[908]= "On branch master

Initial commit

Changes to be committed:
  (use \"git rm --cached <file>...\" to unstage)

    new file:   fil.m

"

---

Code

Here's the paclet:

PacletInstall@"http://www.wolframcloud.com/objects/user-e4d1d43a-267f-4924-934a-2ba2321519a9/paclets/ServiceConnection_Git/Paclets/ServiceConnection_Git-1.0.0.paclet"

And here's the template notebook with all of this in it:

https://www.wolframcloud.com/objects/user-affd7b1c-ecb6-4ccc-8cc4-4d107e2bf04a/Git_service_template.nb