I'm trying to parallelize some tasks which involve fetching data from a URL and processing it. This is the code:

keyId = 1;
SetSharedVariable[keyId];
Clear[keyLock];

EncodeURL[str_String] :=
StringReplace[str,
x : Except[Alternatives @@ Characters@";/?:@=&$-_.+!*'()"] :> ExternalServiceEncodeString[x]] ApiKeys = { "key1", ... ... "lastKey" }; Api[urlNoKey_] := TimeConstrained[ApiInternal[urlNoKey], 5, Api[urlNoKey]] ApiInternal[urlNoKey_] := Block[{content, code, url = EncodeURL[ baseURL <> urlNoKey <> "?api_key=" <> ApiKeys[[keyId]]]}, {content, code} = Check[URLFetch[url, {"Content", "StatusCode"}], {$Failed, $Failed}]; PrintTemporary[url]; CriticalSection[{keyLock}, If[keyId == Length@ApiKeys, keyId = 1, keyId++]; ]; Switch[code, 200, ImportString[content, "JSON"], 404, {}, 429 , LolApiInternal[urlNoKey], (* Rate time limit exceeded *)$Failed, Print["Api::error -> URLFetech returned \$Failed"]; {},
code, Print["Api::error -> Unkown code: ", code,
".Content was: ", content]; {}
]
]


I am using a 4 core machine but since the computing is small and URLFetch is involved, I am using 8 kernels (using LaunchKernels[8])

I am trying to distribute the processing of a list of about 10.000 urls:

ParallelMap[Process@Api[#] &, urlList];
`

I get the following errors (and more) during the computation (usually, near the end)

Any idea why and how to solve it? Note that I am using FetchURL and ImportString (I think this last one uses a Java link)

-