# How to cache results of WebImageSearch?

Every call to WebImageSearch costs money (wolfram service credits), thus, I would like to page through results without having to redownload them.

Given the following search how would one retrieve items 6-10 (without repurchasing the first 5)?

WebImageSearch["desert plains", "Thumbnails", MaxItems -> 5]


Update

Response to @kuba's question in the comments:

Update 2

@b3m2a1's solution doesn't run for me:

vgBlock[fn_, HoldPattern[SetDelayed][lhs_, rhs_], expr_] := Module[
{$recurse}, InternalInheritedBlock[{fn}, Unprotect[fn]; (lhs /; ! TrueQ[$recurse] := Block[{$recurse = True}, rhs]); DownValues[fn] = SortBy[DownValues[fn], FreeQ[$recurse]];
expr
]
];
vgBlock~SetAttributes~HoldAllComplete;
vgBlock[GCSFormatRequestParameters, GCSFormatRequestParameters[e___] := GCSFormatRequestParameters[e] /. ("start" -> "1") -> ("start" -> ToString[OptionValue["StartingFrom"]]),
WebImageSearch[params,  Sequence @@ FilterRules[{Method -> "Google",  ops}, Options @ WebImageSearch]
]
]


When I run googleImageSearch2["sweet dee", "MaxItems" -> 4], the progress bar starts working but then fails and the symbols turn red:

Ah ok, WebImageSearch stopped working because I ran out of service credits:

Probably should have warned me!

• Can you add a code which shows $CloudCreditsAvailable changing? I can't reproduce it. And can you confirm that those results are not cached, e.g. when you call MaxItems->10 next time, does it take doubled amount or? – Kuba May 11 '18 at 13:31 • @Kuba it's not cloud but service credits - both cost money. – M.R. May 11 '18 at 14:10 • Try with less items so not to draw down credits $ServiceCreditsAvailable Put[WebImageSearch["desert plains", "Thumbnails", MaxItems -> 5], LocalObject["myimages"]] $ServiceCreditsAvailable Then Get[LocalObject["myimages"]] $ServiceCreditsAvailable credits remain the same as previous call. – Hans May 15 '18 at 2:46
• The only idea I have for getting the next 5 would be to use Except[q]. But the documentation is of no help there the only metadata I could see of use is ByteCount. So Map[ByteCount[#] &, a] where a is the result from Get[...] But I found no good examples. – Hans May 15 '18 at 3:27
• The function that implements this (for Google search) is WebSearchPrivateGCSPaginationCalls and it looks like it's terribly designed. The result count is hard coded. The start point can't be set via parameter. Etc. – b3m2a1 May 18 '18 at 7:06

So it looks like the only real way to do this is by hacking the mainframe package that underlies this. Everything seems to be clumsy and hard coded, so I had to dig a little. The following works for Google, but it is easily adaptable to Bing.

The main function of interest is GCSFormatRequestParameters which formats request parameters to Google Custom Search. When you use this it hard codes a "start"->"1". We want to specify our own starting point so we will simply hack this function to do our bidding before it passes the parameters to the API call.

First a function to implement a single Villegas-Gayley redefinition in a general manner:

vgBlock[fn_,
HoldPattern[SetDelayed][lhs_, rhs_],
expr_
] :=
Module[{$recurse}, InternalInheritedBlock[{fn}, Unprotect[fn]; (lhs /; ! TrueQ[$recurse] :=
Block[{$recurse = True}, rhs]); DownValues[fn] = SortBy[DownValues[fn], FreeQ[$recurse]];
expr
]
];
vgBlock~SetAttributes~HoldAllComplete


Then we'll use this to override the old results:

Options[googleImageSearch2] =
Join[
Options[WebImageSearch],
{
"StartingFrom" -> 1
}
];
ops : OptionsPattern[]] :=
vgBlock[GoogleCustomSearchFunctionsGCSFormatRequestParameters,

GoogleCustomSearchFunctionsGCSFormatRequestParameters[e] /. ("start" -> "1") -> ("start" ->
ToString[OptionValue["StartingFrom"]]),
WebImageSearch[params,
Sequence @@
]
]


And finally we can test that it works:

googleImageSearch2["doggo", "MaxItems" -> 4]


googleImageSearch2["doggo", "MaxItems" -> 4, "StartingFrom" -> 3]


One useful note when digging make sure wrap any calls to WebImageSearch in:

Block[
{
WebSearchPrivateGCSPaginationCalls,
WebSearchPrivateBSPaginationCalls
},
doop doop
]


• @M.R. which Method to WebImageSearch are you using? It looks like it defaults to Bing. Another possibility is that the definition reordering I attempted to impose isn't working. – b3m2a1 May 20 '18 at 19:50
• See my answer to (106068) for another implementation of something similar to your vgBlock`. – Carl Woll May 20 '18 at 20:07