Intuitive explanation of RemoteBatchSubmit[] in AWS for total novices?

Question

When I read Wolfram's post about the new features in MMA 12.2 I was intrigued by RemoteBatchSubmit[]. He wrote:

I have only ever used MMA from my laptop, and never used any other cloud computing or high-performance computing service. I've always thought of these as a service for really heavy computations fancy scientists (or companies) do.

Now, looking into the documentation, the user guide to set up AWS (Amazon Web Services), confused users in Wolfram community, and the hundreds of product versions/tiers/pricings for AWS computing services, not to mention that apparently one also needs something called "Wolfram Service Credits" to do this!??...

I wonder: could anybody here who has experience on the matter and a good understanding of the nuances, explain in simple terms (Edits: I've been adding a couple of additional questions to this list, plus some more "extra" thoughts/doubts at the end.):

1. What is needed to send a computation to RemoteBatchSubmit[]? (Besides my code, internet, and the basic free AWS account) I.e.: Do I need some paid AWS tier or the "free tier" services (I think is 750 computing hours per month in something called EC2) is "enough" for some decent computing speed-up? I know this depends obviously on the specific task, but think of some program that takes overnight in a "regular" laptop: would it be worth to send this to AWS batch? Or is it more cost-hassle effective only with some seriously crunchy computations (CERN-level folks? haha).

2. What are Wolfram Service Credits and how do they work in this context? I.e. Would I need to know beforehand how many credits to buy for a specific computation? In the possible issues section of the documentation, it seems that one possible issue is that such credits can run-off. How am I supposed to know how many to buy beforehand? How are these credits related to the credits or payments to AWS services?

3. How I am supposed to know, if I've never done something like this, how many "nodes", "cores", "region", whether to request vCPUs, GPUs, and other seemingly important details, when sending my batch job?

4. If you could write some example(s), step by step, but in plain language, that would be great.

5. Once you submit a job to the cloud, I imagine you can turn your computer off or whatever. How do you retrieve the results later? Say, I quit Mathematica: from a new (blank) notebook, can I retrieve the results from a computation I sent from another notebook a previous day, or do I need the notebook where I sent the job from? Can you get a notification when it is done or you have to check the status over time?

6. From a single interactive session in a notebook, can I launch many different jobs? If so, do I need to make a new environment for each one, or just create independent jobs using the same environment? (e.g., using a single env as the environment, can I then create job1 = RemoteBatchSubmit[env, code1], job2 = RemoteBatchSubmit[env, code2], etc?

7. Are the variables, data, functions, and other definitions of the local notebook sent/carried onto the job submission, or do I have to define everything inside the job RemoteBatchSubmit[] environment?

For context, I make a bunch of computations on a decent desktop, but some of these take overnight; some even multiple days. The computations themselves are nothing fancy: some computation on large networks. The "output" is a .mx file of about 100MB, so nothing that requires too much storage-memory either.

Wolfram's blog seemed to suggest it is quite simple to just send these jobs for batch processing, is it really that easy? Consider folks like me, who ignore many of the technical terms of cloud/high-performance computing. The other questions I've seen are more specific than what I'm looking for; I'm looking for an intuitive, introductory, yet complete explanation on how to get started. Let's unleash the power of MMA's high-performance computing to the masses! XD

P.S. Edit 1: If you do give some example or share your experience with some project(s) you've done, it would be great if you could share some estimate on how much ($) did you spend, whether with AWS or Wolfram Credits (I still don't know whether I'd need both types of "credits").

P.S. Edit 2: I know that people somehow have set up Wolfram Engine running in Jupyter, to then use AWS for cluster/batch computing. But here, I'm specifically asking how to use RemoteBatchSubmit[] from your run-of-the-mill Wolfram notebook.

P.S. Edit 3: Interested to know whether any type of code would benefit from sending it to the cloud. For instance: is it a big Table[]-like worth sending to the cloud, or only things that can be run in parallel (e.g. ParallelTable[]) would be faster/more convenient. Related to this: when sending batch jobs to the cloud, should I code things differently than what I do in my laptop? (if not using GPU/parallelization). Finally: how do we install packages in the cloud service? I'm using IGraphM a lot in my computations, and as I understand AWS will run on the Wolfram Engine; how do I get to install/use a specific package there? Do I need to install a package every time I make an environment/new-session?

P.S. Edit 4: I was consistently getting the error mentioned here. It seems one has to "Switching locations on AWS, then creating a new BatchComputeEnvironment, solved the issue".

Thank you!

Answer 1

There are quite a few questions in this post; I'll try to answer each of them in sequence and provide examples and references where relevant. Let me know if I missed something or left anything unclear.

What is needed to send a computation to RemoteBatchSubmit[]?

The Set Up the AWS Batch Computation Provider documentation workflow should cover the steps involved in configuring your AWS account and Mathematica session to submit jobs to AWS Batch. In brief, these are:

1. Create an AWS account
2. Create API access credentials (access key/secret) for the account (it's easiest to create these for a new account's default "root" user, but security best practice is to use an IAM user)
3. Create a remote batch submission environment CloudFormation stack using the template provided in the documentation (if you're the admin of your own AWS account, you can safely leave all the template parameters at their default settings here)
4. Copy the RemoteBatchSubmissionEnvironment expression generated by the CloudFormation stack into a Mathematica notebook and assign it to a variable (if this is your first time doing so, you will be prompted to choose an AWS authentication method; the quickest option is to choose "Manual" mode and enter the API credentials you created in step #2)
5. Submit a job using RemoteBatchSubmit with the RemoteBatchSubmissionEnvironment expression from step #4.

Do I need some paid AWS tier or [are] the "free tier" services ["enough"]?

Unfortunately, the AWS Free Tier covers only t2.micro and t3.micro EC2 instance types, which are not compatible with AWS Batch. But the name "Free Tier" is somewhat misleading, in that there isn't a distinct "paid tier". Almost all AWS services (including those involved in using RemoteBatchSubmit) are pay-as-you-go, so the amount of money you spend is directly linked to amount of compute resources you use - which is in turn linked the parameters and scale of jobs you submit. The lineup and pricing of EC2 instances can definitely be overwhelming at first, so here's a quick rule of thumb: the C5 instance family, which is ideal for CPU-intensive workloads, has various instance types with different numbers of vCPUs (virtual CPUs), but these types are all priced at a fixed rate of around 5¢ per vCPU per hour (it varies slightly by region). So, I could run a 4-hour job on 16 vCPUs (total 64 vCPU-hours) for about $3.20 in EC2 costs (not including licensing costs; I'll address that next).

What are Wolfram Service Credits and how do they work in this context?

Wolfram Service Credits are used to pay for a handful of different "integrated services" accessible through the Wolfram Language, but the one that's relevant here is a service called "on-demand licensing". On-demand licensing is how you pay Wolfram for the right to use the Wolfram Engine kernel in a remote batch job - it's the pay-as-you-go alternative to the node-locked annual or perpetual Mathematica (or Wolfram Desktop etc.) license you use on your own computer. With on-demand licensing, you pay a fixed rate of 2* Service Credits per kernel per hour. The real-world pricing of Service Credits varies depending on the volume of credits you buy, but right now it's between 0.4¢ and 0.6¢ per credit.

Going back to my example job from earlier: assume I run a parallel computation using my 16 vCPUs. My job will launch one control kernel plus 16 parallel subkernels (17 kernels total), so I'll pay (17 kernels) * (4 hours) * (2 credits per kernel/hour) = 136 Service Credits = 82¢ for the job.

In the possible issues section of the documentation, it seems that one possible issue is that such credits can run-off.

You'll see in that section that RemoteBatchSubmit tries to prevent you from submitting a job that you don't have enough credits to run. But you can manually override this, and RemoteBatchSubmit can't prevent you from submitting e.g. two consecutive jobs that together will run you over your balance. If this happens, your Service Credits balance will go negative, but any kernels running at that time will continue running. (The example showing a job that runs out of credits mid-computation is outdated.)

How are [Service Credits] related to the credits or payments to AWS services?

They are not related - you pay AWS for infrastructure costs, and separately pay Wolfram (via Service Credits) for product licensing costs.

How I am supposed to know, if I've never done something like this, how many "nodes", "cores", "region", whether to request vCPUs, GPUs, and other seemingly important details, when sending my batch job?

• Nodes: Off the top of my head I can't think of where you saw this word; "nodes" is not a configurable parameter in RemoteBatchSubmit jobs.
• Region: In general I suggest using the AWS region closest to your physical location for the fastest upload/download speeds. Nothing's going to break if you use e.g. the us-east-2 region while you're physically in Europe or Asia; it's just likely to be a bit slower submitting and retrieving jobs than if you pick an eu- or ap- region.
• Cores/vCPUs: This depends a lot on the nature of your code, and the reasoning involved is not specific to RemoteBatchSubmit. If your code is parallelized (e.g. with ParallelTable), adding more cores will likely make your code run faster. If your code is not parallelized, the performance gain of adding more cores depends on if your code makes use of internally multithreaded functions. In general, you can get an idea of whether more cores are likely to speed up your program by checking your operating system's CPU monitor (Task Manager on Windows; Activity Monitor on macOS; top on Linux/Unix) while your code runs. If the WolframKernel process(es) is/are maxing out all of your machine's cores, then you're likely to see an improvement in performance (although not necessarily linear) if you add more cores. If it's only maxing out a single core, then adding more cores probably won't magically make it faster.
• GPUs: These are only relevant if your code is using GPU computation features like CUDALink or NetTrain. If you're not sure, then it probably is not.

Once you submit a job to the cloud, I imagine you can turn your computer off or whatever. How do you retrieve the results later? Can you get a notification when it is done or you have to check the status over time?

I like to use SendMail in my jobs to get an email notification when they finish, but this only works if the job succeeds (as opposed to crashing or timing out). To be certain about the job's status you have to check it with the "JobStatus" property.

From a single interactive session in a notebook, can I launch many different jobs? If so, do I need to make a new environment for each one, or just create independent jobs using the same environment?

You can submit as many jobs as you like to a single environment. If an environment doesn't have resources to service a job immediately (e.g. it's already at its maximum vCPU limit), the job will get queued up and will run as soon as resources are available. (A job waiting in the queue will have the "Runnable" status.) You only need to create a new environment if you want to change environment settings, like the maximum vCPU limit.

One use case of multiple submission environments is a company or team with multiple users. The account administrator could create an environment for each user and grant each user permission to access only their environment and no others.

Are the variables, data, functions, and other definitions of the local notebook sent/carried onto the job submission, or do I have to define everything inside the job RemoteBatchSubmit[] environment?

By default, definitions referenced by code inside RemoteBatchSubmit[] are captured automatically and recursively, in the same way as with other functions such as ParallelEvaluate and CloudDeploy. This behavior is controllable with the IncludeDefinitions option. In fact, even definitions referenced from loaded packages are captured, although this may not work well for complex packages or packages with native code so IMO it's safer to install/load the package explicitly.

Interested to know whether any type of code would benefit from sending it to the cloud. For instance: is it a big Table[]-like worth sending to the cloud, or only things that can be run in parallel (e.g. ParallelTable[]) would be faster/more convenient.

Depends on what's "worth" it to you. As I explained previously, only parallelized or multithreaded computations really benefit from the kind of scale you can achieve in the cloud. But even if my code uses only a single core, I find that I often appreciate the convenience of running it in a remote batch job rather than on my local machine. I'm usually happy to spend a few pennies for the luxury of not locking up my own computer with a long-running computation. (If the CPU specs of the EC2 instance type you use are superior to those of your local computer then you might see an appreciable speedup even on single-core jobs, but this is usually marginal compared to the performance gain of adding more cores to a properly parallelized program.)

How do we install packages in the cloud service? Do I need to install a package every time I make an environment/new-session?

In the case of IGraphM, you should be able to run the installation instructions on the page you linked inside RemoteBatchSubmit, like:

job = RemoteBatchSubmit[env, (
  Get["https://raw.githubusercontent.com/szhorvat/IGraphM/master/IGInstaller.m"];
  <<IGraphM`;
  (your code)
)]

If the package is a paclet, you can PacletInstall it like normal.

You do need to install the package in every job you submit, because each new job runs in an isolated environment with no access to state set by previous jobs.

_{* You'll notice on the Service Credits page that this is the "discounted" rate. The meaning of this is admittedly not explained very well there - the only time you don't get the discounted rate is if you're submitting a job from within a Mathematica trial license or a notebook in a free Wolfram Cloud subscription. A quick way to check is to evaluate CreateLicenseEntitlement[]; the rate is shown in the summary box next to "Standard kernel limit".}