How can I spelunk CloudObjects?

Question

I want to dig through the structure of a cloud directory to enumerate what's in it and figure out what objects I can work with.

How can I do this?

Answer 1

I just wrote a little chunk of code to do this in a semi-clumsy fashion:

Options[getCloudObjectPath] =
  {
   "Username" -> None
   };
getCloudObjectPath[path_String, ops : OptionsPattern[]] :=

  URLBuild@Flatten@{
     Replace[OptionValue["Username"],
      {
       s_String?(StringLength[#] > 0 &) :>
        "user:" <> StringSplit[s, "@"][[1]],
       _ -> ""
       }
      ],
     URLParse[path, "Path"]
     };
Options[getCloudObjectList] =
  {
   "Iterations" -> None
   };
getCloudObjectList[copath_, ops : OptionsPattern[]] :=

  If[IntegerQ@OptionValue["Iterations"],
   Nest[
    Sort@DeleteDuplicates@
       Flatten[{Map[Quiet@Check[CloudObjects@#, {}] &, #], #}] &, 
    CloudObjects[copath],
    OptionValue["Iterations"]
    ],
   Quiet@Check[CloudObjects@copath, $Failed]
   ];
formatCloudObjectDS[cloudObjData_Association] :=

  Association@
    KeyValueMap[
     StringReplace[First@#, 
        "https://www.wolframcloud.com/objects/" -> "user:"] ->
       Association@Prepend[#2, CloudObject -> #] &,
     cloudObjData
     ] // Dataset;
Options[getCloudObjectDS] =
  Join[
   {
    "CheckPrivate" -> False
    },
   Options[getCloudObjectList],
   Options[getCloudObjectPath]
   ];
getCloudObjectDS[cpath_String, ops : OptionsPattern[]] :=
  Replace[
   getCloudObjectList[
    getCloudObjectPath[cpath, 
     FilterRules[{ops}, Options[getCloudObjectPath]]], 
    FilterRules[{ops}, Options[getCloudObjectList]]
    ],
   l_List :>
    formatCloudObjectDS[
     l // If[TrueQ@OptionValue["CheckPrivate"],
         Map[# -> Quiet@Check[Options[#], $Failed] &],
         Map[# -> {} &]
         ] // Association // Select[Not@*FailureQ]
     ]
   ];
getCloudObjectDS[ops : OptionsPattern[]] :=
 getCloudObjectDS["", ops]

We can use it like:

getCloudObjectDS["Username" -> "b3m2a1", "Iterations" -> 1]

to look at some of the stuff I have lying around up there.

Or we can plumb a cloud account that I saw in a URL in one of those Live CEOing videos:

getCloudObjectDS["Username" -> "ResourceSystem"]

If we want to only get the non-private resources and their options (note that this adds a bunch of time to the requests) relating to the DataRepository resources:

getCloudObjectDS["published/DataRepository", "Username" -> "ResourceSystem", 
 "CheckPrivate" -> True]

Hopefully this makes working with the cloud a bit easier.